Can't boot spi-installer on pinebook pro

I'm trying to install NixOS on my PBP and I've built both Tow-Boot and your NixOS sdimage (the new feature branch). I can't seem to boot the spi-installer.img though; after writing it to the sd card the PBP never boots from it. The sdimage produced by your overlay works fine otoh, so I'm guessing it's an issue with the spi-installer.img. Since my PBP boots currently, can I just write the SPI image directly on the PBP instead of booting from the image?

I can't seem to boot the spi-installer.img though

What do you mean from that? On a Pinebook Pro, the initial boot firmware selected will be, in order, the one in SPI, the one on eMMC or the one on SD. So if you have U-Boot on SPI, the IBF installed on the SD card will never itself boot.
But, that doesn't mean the SD installer can't work. If you select, somehow, to boot from SD, it should be able to do something with it. Hopefully present you with the bootmenu-based menu to erase/install. Though maybe there's a bug... The spi-installer.img as released might not work correctly depending on what U-Boot booted it. Currently it has been mainly tested against the same revision Tow-Boot build. It is something to work on, actually test and use the SPI installer outside of Tow-Boot so a user can upgrade to Tow-Boot.

Now for other methods of installing:
Do you have a U-Boot installed to the SPI Flash? You could erase it so the Tow-Boot on SD is used, which in turn should be able to "boot itself".
Alternatively, you can "just" write firmware.spiflash.bin to the SPI Flash. I don't know how, haven't investigated yet, but it is possible on Linux.
Otherwise, with U-Boot the important commands are those:
https://github.com/Tow-Boot/Tow-Boot/blob/a4a624473c59b1c3c6d16ed19de11fb8baeac343/support/builders/spi-installer/default.nix#L23-L26
Where mainly, it loads the firmware binary, the if ensures the script fails if loading fails.
Then, sf probe makes the SPI flash device available, sf erase is more of a formality, I'm not 100% positive it is strictly required. Finally sf write well... it writes the firmware! But if you were to just run those commands, I would invite you to instead just sf erase and then boot the SD card SPI installer. This way you can ensure the Tow-Boot version there should work.

Thanks, with those pointers I managed to boot from the SD after erasing the SPI. Tow-Boot installed fine and can boot itself too. Thanks for the help!

I ran into this issue as well with the December 2021 U-boot release. I ended up using nandwrite to write the binaries/Tow-Boot.spi.bin file from the 2021.10-004 release. For anyone who was just as worried as I was, it did flash successfully after writing 379 blocks.

Same problem here. The SD card also wouldn't boot, but I could run sudo nandwrite -p /dev/mtd0 Tow-Boot.spi.bin from Linux, and after writing 379 blocks Tow-Boot was installed.
On Debian/Ubuntu, nandwrite is part of the mtd-utils package.
Maybe this should become an official instruciton?

I installed nandwrite, but I don't have a /dev/mtd0 device.  How do I enable that?

I've just been going through this too. on an original october 2019 pbp there does appear to be something in the spi from the factory, and the original mr fixit debian doesn't expose /dev/mtd0 even with mtd-utils  installed. what I've done is to install current manjaro to an sd card and boot from that. manjaro does show /dev/mtd0 so you can install mtd-utils  (with sudo pacman -S mtd-utils) and do sudo nandwrite -p /dev/mtd0 Tow-Boot.spi.bin from there. Unfortunately for me I now can't boot anything but the emmc since the keyboard in my pbp is broken and the tow-boot I installed is defaulted to boot emmc first doesn't seem to recognise usb keyboards.

/dev/mtd0 is provided by the kernel. If it is not showing up, then you are missing a kernel configuration. I am not 100% sure, but I think the config needed is CONFIG_SPI_ROCKCHIP=y for the rk3399.

Just so others know, the direct writing way worked for me too;
dd if=Tow-boot.spi.bin of=/dev/mtd0
(/dev/mtd0 may not be listed in lsblk, but you prolly still have it in /dev....) Thanks to the original poster mentioning this route. Tow Boot on Pinebook Pro worked fine.

Can't boot from SD spi installer either for RockPro64 here.


🛡 Redact Sensitive AWS Information in Terraform Plan Output

User Need
As a Operations engineering team member
I want to redact sensitive AWS information such as Access Key IDs and account numbers from the Terraform plan output
so that we prevent security risks associated with exposing sensitive data.
Value
Prevents accidental exposure of sensitive AWS credentials and account details, reducing the risk of misuse or malicious attacks.
Functional Requirements (What):

[ ] Implement a script to redact AWS Access Key IDs, Secret Access Keys, and Account Numbers from the Terraform plan output.
[ ] Ensure that the script runs automatically as part of the Terraform workflow to prevent any sensitive data from being outputted.
[ ] Validate that the redacted output does not contain any sensitive information.
[ ] Update the workflow to recycle any exposed AWS access keys to maintain security.

Non-Functional Requirements (How):

[ ] The redaction script should be efficient and not introduce significant latency to the Terraform workflow.
[ ] The script should be robust enough to handle various formats of sensitive information that may appear in the Terraform plan output.
[ ] The solution should be easily maintainable and adaptable for future changes in Terraform or AWS configurations.
[ ] Ensure the solution is compatible with the existing Terraform reusable workflow and does not require significant changes to the workflow structure.

Acceptance Criteria:

[ ] All sensitive AWS information (Access Key IDs, Secret Access Keys, Account Numbers) is redacted from Terraform plan outputs.
[ ] The solution is implemented and tested in the current Terraform workflow without causing any disruptions.
[ ] Confirm that exposed AWS access keys are recycled immediately upon detection to maintain security.
[ ] Validate the redaction script's performance and ensure it does not significantly slow down the workflow.
[ ] Document the changes in the Terraform workflow and provide instructions for handling potential future issues.

Notes:

The redaction script will be based on the example provided by the Ministry of Justice's modernisation platform, which uses sed commands to redact sensitive data from output logs: https://github.com/ministryofjustice/modernisation-platform/blob/main/scripts/redact-output.sh

Consider adding logging or alerting mechanisms to notify the team if sensitive information is detected in the output.

Github security-Slack integration: https://github.com/advanced-security/secret-scanning-notifications


[GSoC'21]Livestreaming support

PR Checklist
If you are contributing to Javis.jl, please make sure you are able to check off each item on this list:

[ ] Did I update CHANGELOG.md with whatever changes/features I added with this PR?
[ ] Did I make sure to only change the part of the file where I introduced a new change/feature?
[ ] Did I cover all corner cases to be close to 100% test coverage (if applicable)?
[ ] Did I properly add Javis dependencies to the Project.toml + set an upper bound of the dependency (if applicable)?
[ ] Did I properly add test dependencies to the test directory (if applicable)?
[ ] Did I check relevant tutorials that may be affected by changes in this PR?
[ ] Did I clearly articulate why this PR was made the way it was and how it was made?

Link to relevant issue(s)
Closes #91
How did you address these issues with this PR? What methods did you use?
Streaming is done via ffmpeg. I have used @async for the streaming to happen asynchronously and free up the julia shell,
For now the streaming can be terminated by manually killing the ffmpeg process.

To view the livestream on can use ffplay udp://<IP_ADDRESS>:8080 or add the address as input to a media source in OBS.

One thing that we need here is be able to specify whether we directly stream to twitch or to a local ip/port. What can be the possible syntax for that along with livestream=true flag?

We might want to just have livestreamto and then options nothing as default, twitch and local or something.

That is pretty neat @Sov-trotter
As you mentioned it's currently a hassle to stop the stream.
I was wondering whether it's possible to not show the output of ffmpeg such that one can easily start a new stream but you probably had the same idea already.
Another thing is it would be great to start the stream before rendering is completed if that is possible.

Killing the ffmpeg process also doesn't kill the stream directly on obs which I find weird but I also don't know how it works exactly in the background whether it's really "live" or a bit cached.

I was wondering whether it's possible to not show the output of ffmpeg such that one can easily start a new stream but you probably had the same idea already.

Yeah that's fixed in the latest commit(had to let the loglevel flag).

Another thing is it would be great to start the stream before rendering is completed if that is possible.

I do want to try that out, and would love to have that as a future milestone. Ref : https://trac.ffmpeg.org/wiki/StreamingGuide#Pointtopointstreaming Saving a file and Streaming at the same time section.

Some kind of a bug;I get this in the console and sometimes the stream goes dark and after a couple of seconds it's back to normal.

It's related to the encoder which skips some frames then,I guess we can go around this by tuning some parameters?

Killing the ffmpeg process also doesn't kill the stream directly on obs which I find weird but I also don't know how it works exactly in the background whether it's really "live" or a bit cached.

Yeah OBS has been unfriendly in that regard and has held this PR for a few days after which I finally decided to get things right at Javis's side first. We obviously expect OBS to update the preview automatically, but unfortunately there's a 1-2 minute delay before it does that. I discussed it with @TheCedarPrince yesterday. I did file an issue at obs https://github.com/obsproject/obs-studio/issues/4753

Is it possible to get the id of the ffmpeg process through julia and kill that one before starting a new stream?
Currently one can't run it twice, i.e. after making a change of the code.

Yeah. That's the plan I currently have. The tricky thing is we need to know the PID of the ffmpeg process. @schedule doesn't do that though. We can do something like pidof ffmpeg and kill <pid>, but that will be confusing in the case where the user has multiple ffmpeg processes running.

Yes maybe we can grep for the pathname as well to be sure that it's actually the one from Javis.

ps aux | grep ffmpeg | grep test.gif | awk '{print $2}'

I think rather than pathname we can use some other parameter to check for the correct process since pathname can be different eg: stream_loop is a good parameter

Yeah but as we know the pathname it should be easy, right?
Another thing that might exist is to add some info field in the command which does nothing which we can use to find the command.
Not sure whether ffmpeg supports things like that.
Also yes stream_loop is probably enough.

Yeah but as we know the pathname it should be easy, right?

Yeah. But I am also thinking from the perspective of another cancel_stream() function which will be independent of the render function, which shouldn't take in the pathname as it won't be userfriendly

@Wikunia @TheCedarPrince  I think I have been able to solve the problem! The latest commit includes the fix. I would like if you guys can try it out and give feedback when we meet tomorrow?

Thanks @Sov-trotter
Feels like I still have to cancel the stream manually and get an infinite amount of error messages otherwise.

Are the error messages similar to the ones you shared above? Apparently I don't get any.
Could you also share the example you're trying?

using Javis

function ground(args...)
    background("white") # canvas background
    sethue("black") # pen color
end

function object(p = O, color = "black")
    sethue(color)
    circle(p, 25, :fill)
    return p
end

function path!(points, pos, color)
    sethue(color)
    push!(points, pos) # add pos to points
    circle.(points, 2, :fill) # draws a circle for each point using broadcasting
end

function connector(p1, p2, color)
    sethue(color)
    line(p1, p2, :stroke)
end

myvideo = Video(1000, 1000)

path_of_red = Point[]
path_of_blue = Point[]

Background(1:70, ground)
red_ball = Object(1:70, (args...) -> object(O, "red"), Point(100, 0))
act!(red_ball, Action(anim_rotate_around(2π, O)))
blue_ball =
    Object(:same, (args...) -> object(O, "yellow"), Point(200, 80))
act!(blue_ball, Action(anim_rotate_around(2π, 0.0, red_ball)))
Object(1:70, (args...) -> connector(pos(red_ball), pos(blue_ball), "black"))
Object(1:70, (args...) -> path!(path_of_red, pos(red_ball), "red"))
Object(1:70, (args...) -> path!(path_of_blue, pos(blue_ball), "blue"))

streamconfig = setup_stream(:local)
render(myvideo; pathname = "tutorial_1.mp4", streamconfig)

if you run that two or sometimes three times (after changing a color or something but that is probably irrelevant) I get these errors:
[h264 @ 0x87ec00] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x89b840] Invalid NAL unit size<PHONE_NUMBER> > 1442).
[h264 @ 0x89b840] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x8b84c0] Invalid NAL unit size (-1644236136 > 626).
[h264 @ 0x8b84c0] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x8d5140] Invalid NAL unit size<PHONE_NUMBER> > 394).
[h264 @ 0x8d5140] Error splitting the input into NAL units.
Error while decoding stream #0:0: Invalid data found when processing input
[h264 @ 0x8f1dc0] Invalid NAL unit size (-1424218937 > 407).
[h264 @ 0x8f1dc0] Error splitting the input into NAL units.

in a never ending stream. Everything works nicely when I run cancel_stream() in between each time though.

That works!

@Wikunia I am able to reproduce the error.
Not cancelling the stream before creating the new one was creating a race condition.

Oh I will not be able to stream two things at the same time via different ports, right?

We can tweak the cancel_stream() method to support that by declaring a data structure that stores features like(address, filename, port etc.) and only kill the process if they match?

Yeah maybe we keep this so for now to have a working version for it. I'm currently testing it for a video I want to make.
Might be a good way to have an instant updated preview :smile:

How to set this up in OBS's side(for people trying this out)

Have you tested streaming directly to twitch yet @TheCedarPrince ?

@Sov-trotter what was the reason to change the default port and address?
I get a error writing trailer of udp://<IP_ADDRESS>:8081: Permission denied now

Yeah sorry I didn't mention this before committing.
I am actually trying look for a default ip/port which has low probability of being used(we can put a check in the code if the port is taken but I still want the default to be the lesser used one)
Eg : I created jupyter notebook session and tried streaming to the same ip and port and it did work as long as the streaming protocol was not tcp. If it was tcp, it ended up crashing even the jupyter kernel and that's something I really want to avoid.
Also <IP_ADDRESS>:8080 is used quite frequently imo.

How about 14015 for the closest approximation to "Javis" I can think of :smile:
But I think using <IP_ADDRESS> makes sense as it's often used for this, right?

I am not sure. But now I think leaving it to <IP_ADDRESS> is actually better than <IP_ADDRESS> since it's used quite often

av_interleaved_write_frame(): Permission denied
error writing trailer of udp://<IP_ADDRESS>:8081: Permission denied

Do you have something running at that IP/port?

Also @TheCedarPrince @Wikunia it'd be nice if you can also review the workflow docs.

av_interleaved_write_frame(): Permission denied
error writing trailer of udp://<IP_ADDRESS>:8081: Permission denied

Do you have something running at that IP/port because this works for me.

Nope I think it might be more the fact that it's <IP_ADDRESS> rather than <IP_ADDRESS>

@Sov-trotter - just let me know when you want me to take a look at this again and I can do a final pass and bounce it back to @Wikunia again for final signoff for merging. :)

I have committed the changes requested by @Wikunia, so yeah you can share your feedback! :)

Have you seen this error before @Sov-trotter :
[gif @ 0x10cfd80] seek_frame_generic failed as this stream seems to contain no keyframes after the target timestamp, 1002 non keyframes found

It happened to me when I changed from 1000 frames to 1200.

@Wikunia I can reproduce this. However the error takes place at the last frame(or after the last frame).
REPRO:

Remove the -loglevel error flag
set the number of frames to 1200

Also I didn't notice any flicker or frame drop when the error was thrown.
Same for mp4 [h264 @ 0x1b758c0] Invalid NAL unit 8, skipping.

Should be ready for merging provided no formatting issues. Feel free to review @Wikunia , merge, and either you or me can tag a release. Great work @Sov-trotter :raised_hands: :tada: :tada:

The changelog section seems to be missing


Module gets "404: Not Found" while trying to send a message immediately after Webhook object creation

Whenever I try to send a message using Hook.info() (or whichever method) right after creating a new Webhook object, as in:
var Webhook = require("webhook-discord")
var Hook = new Webhook("WEBHOOK_URL")
Hook.err('Test','test')

I get this error:
(node:11404) UnhandledPromiseRejectionWarning: {"code": 0, "message": "404: Not Found"}
(node:11404) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:11404) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

After debugging the module code, I found out this.endpoint was not correctly formed at the moment of Hook.err() call, since the module relies on a Promise to fetch webhook URL properties (like id and token). As a console.log() shows:
...
sendPayload(payload) {
        console.log(this.endpoint);
        // logs "https://discordapp.com/api/v6/webhooks///slack"
        return new Promise((resolve, reject) => {
            snekfetch.post(this.endpoint)
                .send(payload)
...

I already made a little fix/workaround myself, but I'm not confident enough to send a pull request 😅 so I created an issue instead.
Gist with my workaround: https://gist.github.com/liddack/758e14c7161d814e74fa0a49c05ab585

Hey! This is a known issue but I just never got around to addressing a fix. I'll try get around to implementing your fix (which is greatly appreciated) into the library, however if you want feel free to open a pull request so I can look over and give it more of a test. Will leave this open for now :)

This is fixed in version 2.2.2, see this commit


[BUG] Input events like Ctrl-P are buffered inside Docker.

Describe the bug
When running inside Docker, Ctrl-P (and probably other shortcuts) aren't immediately registered by the application, but buffered until the next input event.
To Reproduce
Enter a directory with the cursive 0.16 source code. Then run
docker run --it --rm -v $(pwd):/cursive ubuntu:focal /bin/bash
# The following happens inside Docker
cd cursive
apt update && apt install build-essential curl libncursesw5-dev
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env
cargo run --example key_codes

Press Ctrl-P. Nothing happens. Now press Ctrl-P again, or any other key. This time, Ctrl-P is registered, as well as the second input.
Expected behavior
Ctrl-P is received by the application without buffering.
Environment

Operating system used: Ubuntu 20.04 Docker container running on an Ubuntu 18.04 host.
Backend used: I tried ncurses and crossterm
Current locale: Happens both with en_US.UTF-8 and the default POSIX in the Docker container
Cursive version: 0.16
Terminal emulator: Gnome-terminal

Additional context
This problem doesn't happen outside Docker.
I'm not sure if this is a problem with cursive or Docker. However, I didn't find anything interesting when Googling Docker problems with terminal raw mode.

Sorry, I didn't search thoroughly enough: https://stackoverflow.com/questions/20828657/docker-change-ctrlp-to-something-else explains why this happens.


RFI reader

This PR adds s1-reader an ability to load Radio Frequency Interferences (RFI) information, which is available starting from IPF version 3.40.

@gshiroma @vbrancat This PR will be used to extract RFI information to be populated in the RTC and CSLC product.
Would you mind taking a look at this PR? Thanks

@seongsujeong, any progress on this PR?

@vbrancat @gshiroma Would you mind taking another look at the PR? Thanks!


Add support for JEDI-NEPTUNE

Description
Add support for jedi-neptune-env:

library versions for NEPTUNE dependencies
module environment variables for NEPTUNE dependencies
add jedi-neptune-env to unified-dev and skylab-dev template

Also: Make skylab-dev template look like unified-dev template.
Dependencies
Waiting on https://github.com/NOAA-EMC/spack/pull/264
Issues
Fixes #498
Testing
Tested on Narwhal with Intel and on macOS with apple-clang.

@ulmononian @AlexanderRichert-NOAA Can I ping you for a review if you have time? Thanks ...


Support PyPI trove classifiers

Basically this https://github.com/sdispater/poet/issues/5
It'd be nice to detect as many things automatically, but somewhere in pyproject.toml to manually add them would also be useful.

I agree. For now, only Python trove classifiers are set automatically.
And while I can set the license ones automatically as well, having a section where developers can set them is actually a good idea.


Kubernetes Documentation

Test

/triage needs-information

Hi @rajar0201
Was there a suggestion you wanted to make about the Kubernetes documentation?

Yes I want
On Sun, 22 Aug 2021 at 3:35 PM, Tim Bannister @.***>
wrote:

Hi @rajar0201 https://github.com/rajar0201
Was there a suggestion you wanted to make about the Kubernetes
documentation?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/website/issues/29453#issuecomment-903244144,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AUB3ANOOYF5QLM2X6J7Y4NLT6DDVNANCNFSM5CLRIKFQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

@rajar0201 can you please elaborate the issue?

i wanted to make about the Kubernetes documentation

/remove-lifecycle stale

/remove-lifecycle stale

/remove-lifecycle stale


Aligned change request UI with updated Figma designs

Context
With the MQ designs some of the UI has been changed and the Cases UI is now inconsistent.
Changes proposed in this pull request
Changes to "Case UI" + tests
Guidance to review
Renamed Case to Change request as part of aligning to updated designs
Checklist

[ ] Attach to Trello card
[ ] Rebased master
[ ] Cleaned commit history
[ ] Tested by running locally

 Change of name/DOB snags


Add a constructor with a FunctionRegistry parameter to JacksonAdapter

This enables using custom functions with the JacksonAdapter.

I've extended the PR to include constructors for JcfAdapter.

I'm closing this since it's been superseded by #7 & #8