refactor ModelChain. add SingleDiode modelchain

Continuing from #151...
I refactored ModelChain into a parent ModelChain class with subclasses SAPM and SingleDiode. In doing so, I moved some solar position, irradiance, and weather prep work into ModelChain.prepare_inputs. prepare_inputs is called during run_model.
I also rebased this code on the changes that added support for SingleAxisTracker and voltage/current/power scaling.
I think that this is close to ready, but would appreciate testing and comments.

@jforbess here's the latest PR that supports the CEC modules. I hope it's easier to pull and test. Note that you'll still need to manually add a few values to the module_parameters.

closing in favor of #194. I'll let the branch live on for a little bit longer, though.


Various workflow ajustement

Add zlib dependency update
Fix (tentative) for CI bot's comment on contribution

:arrow_right: Commit 1648945aa199f19767a34eec6a09a5a5e675a2b0
:clipboard: Logs
:mag: Linting results:
:heavy_check_mark: Python Linting
:heavy_check_mark: C++ Linting
:hammer_and_wrench: Building result:
:heavy_check_mark: Building
:package: Tarballs:
If the building step was successfull, you can download the appropriate tarball in the artifacts section here.
Unzip and then execute sudo tar -C / --no-same-owner -h -xzf linux-enable-ir-emitter*.tar.gz.


Enhanced image polishment

Set canvas background color to white.
Support DPF elemental field by maintaining pick data tracking in enhanced image module instead of DPF.
Make the enhanced image base on the global mesh instead of the field-specific mesh.
More to add

Resubmit a new PR


Scrollbar missing in the sidebar

My Setup

Linux Mint 18

Version 1.3.1

[x] I have tested with the latest application version

[x] I can simulate the issue easily

Description
When viewing direct messages, I am unable to scroll if they run off the screen..
Current Behavior
Unable to see all direct messages easily..arrow pointing to the issue.

Expected Behavior
Should have a scrollbar as with the constraints being the red box in the screenshot

Update: I can scroll with my mouse scrollwheel but there should be a scrollbar...

DOH! Nevermind! There is one! -- it's just the same color as the background -- ignore this.


feat(logging): add a sample for list logs

Description
Fixes b/281691466
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist

[X] I have followed Contributing Guidelines from CONTRIBUTING.MD
[X] Tests pass:   go test -v ./.. (see Testing)
[X] Code formatted:   gofmt (see Formatting)
[X] Vetting pass:   go vet (see Formatting)
[X] Please merge this PR for me once it is approved

@yucentao Please do not provide links to internal Google resources in response to reviewer comments.


Cannot Create items

type: may not be undefined

A new update was pushed today, could you check if this helped? If not, I'd need a better error description of what you're trying to do 🙏

@Lockbox313 Thank you, just took a look at that and the answer is simple: There are no items in the broken compass system. And Foundry does not allow items without a type to be created 🙏


Do not update field if blame is null

This one can produce exception when blame is unable to determine:
EntityManager#persist() expects parameter 1 to be an entity object, NULL given.
Scenario: updating entity through CRON tasks.

solved by other commit


How to make a breadcrumb navigation bar?

I want to create a flat breadcrumb navigation system like an Android system, Can you teach me something? thanks

Breadcrumb implementation doesn't exist currently and it's not included in the backlog right now.
I'm not sure implementing it inside UraniumUI, there is already a good implementation for MAUI.
You can try this: https://github.com/IeuanWalker/Maui.Breadcrumb

Okay, thank you. Let me take a look


Make aliases for rails notification API

We can create aliases for rails notification API for easily switching and using for developers

we have a same alredy


adds Jane Adams Junior high and sd54 school district.

@snorkobilov Please provide us with:

the school official website URL
the school street address, including city and country
an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course
a proof, which shows that the school recognizes the domain you are submitting as an official email domain for the students.

@snorkobilov You might miss my previous comment. Please react.

@snorkobilov I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for your understanding!


Add Vue context to computed options type (fix #10878)

Vue passes the Vue object as the first parameter, which is useful for arrow functions. This change reflects that in the types.
Fixes #10878

What kind of change does this PR introduce? (check at least one)

[x] Bugfix
[ ] Feature
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change? (check one)

[ ] Yes
[x] No

If yes, please describe the impact and migration path for existing applications:
The PR fulfills these requirements:

[x] It's submitted to the dev branch for v2.x (or to a previous version branch), not the master branch
[x] When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)
[ ] All tests are passing: https://github.com/vuejs/vue/blob/dev/.github/CONTRIBUTING.md#development-setup
[ ] New/updated tests are included

If adding a new feature, the PR's description includes:

[ ] A convincing reason for adding this feature (to avoid wasting your time, it's best to open a suggestion issue first and wait for approval before working on it)

Other information:

Thank you but this is not exactly what should be passed. What should be passed is the type of the component to get access to other data and computed properties. Note it might not be possible to support fully so the issue might not be really fixable.


[Enhancement]: Review Storage mechanics PLUS!

Hello!
Firstly, I would like to commend you on the work you've done here. It's precisely what I need for a very important project for my city as well.
I am creating a fork of the repository now to begin diving into the codebase and attempt to determine what may be happening on our end, so please know I plan on rolling up my sleeves and helping contribute to a solution, NOT just log the bug :P
Anyhow, please see below:
ISSUE: Many hours after starting PM2 service for NVRJS, the timeline does not show video segments on the timeline.
Context:
We are looking to use NVRJS for the camera systems we've built utilizing Raspberry Pi's, PoE cams + switch, and USB harddrives.
We are so very close thanks to your work here. But after testing for roughly a week, we see timeline issues.
We DO see that it is properly restarting the ffmpeg processes and the files are logging to disk.
So everything seems to be working (ffmpeg, UI), except for some reason the segments stored to disk.
Also, NVRJS runs rock solid (haven't seen it rebooting over and over or anything for days at a time).
Once I DO end up restarting NVRJS, the timeline begins working normally, though is missing files that are definitely on disk.
I'll log here if I make progress on it!
Thank you for any insight you can help with though :)

Hi @KeithHanson,
It's way passed my bed time where I am (2AM) but wanted to touch base before I hit sleep.
Thanks for the kind words btw :)
I must admit, this was a DIY project, but would welcome any help in making it more accessible.
On to the issue.
My initial thoughts are the way it stores the clip metadata.
It uses SQLITE which I'm not overly a fan of - I wonder if a different storage mechanism can be used?
To stop multiple writes happening, all writes are queued (FIFO), and an interval is happening to check for anything in the queue with the Commit() function (every 10s).
it's based on setTimeout, and if a write happens to error out, it may not reach calling setTimeout again for the next 10s, meaning there is lots in queue, but the Commit function is not being executed, as it crashed before setting the next 10s - the crash may have been due to some SQLITE lock issue, or file IO error, hence why it am not overly happy with my choice in using SQLite.

Thank you for responding so quickly! DIY is my approach to handling a fleet of pole cameras for our municipality. We have a decentralized, edge-based, DIY approach. So far, our Real Time Crime Center analysts like the interface :) And it fits my approach (if it can run decentralized straight from the pole, that's the best outcome).
Your response above all makes sense and what I was suspecting - likely some weird failure on the HD that gets taken care of by our other services on the Pi (decrypts and re-mounts the volume).
I will make a branch littered with debug statements to try to catch this (though, as you know, we will need to wait some time for whatever failure to occur).
Have you considered ditching the SQLite DB entirely? It's almost perfect as a use case to just look at the file system and draw the timeline since I only care about showing the user what's on disk. But you may have explored this already, which might lead me down the wrong track if you've already tried it.

Also, one last thought for now :P Converting from SQLite to something like PostgreSQL or MySQL might be pretty straightforward. I'm trying to keep as few moving parts on my setups as possible (since there's already so many), and keep processor utilization down as much as possible, so I liked the SQLite DB at first sight.
I think I will spend some time on the error not re-queueing the timeout. I'm not the best nodejs dev but I'm sure we're not the only ones who have had this issue with critical functions not being called / queued back up.

To draw clips on the timeline, one could look to use ffinfo, along with the timestamp of the file being created/last modified, to accurately place them in time, the missing piece will be to associate any event data, and associate them and store the event data somewhere (other than SQLite :))

I think I will spend some time on the error

Makes perfect sense at this stage, might be a simple overlook in my part 👍

To draw clips on the timeline, one could look to use ffinfo, along with the timestamp of the file being created/last modified, to accurately place them in time, the missing piece will be to associate any event data, and associate them and store the event data somewhere (other than SQLite :))

Yes - that is pretty much the implementation I was thinking.
Ah - true true. I forgot about the events timeline (which was one of the reasons I chose this over others - that is very interesting down the road to me, and a simple API for it makes a lot of sense).
Perhaps it might be worth exploring an option for that. Since the timeline of segments is critical for our uses, it would make sense to store event data in an optional database. If you're not using that feature (like us), then you can disable it.
If you DO want that feature, event data is likely considered as non-critical compared to the video on disk itself.
So using a more "fail-proof" mechanism ("this is what is on disk and that's all I know") makes sense to me if it does to you.

I think I will spend some time on the error

Makes perfect sense at this stage, might be a simple overlook on my part +1

Quick research suggests the "retry" module might be the ticket to patch it quickly :) Will give it a go in a branch.
I also think I will hit this issue again pretty quickly - I'm testing this on several Pi's on actual poles right now so... one of them is bound to trip up (something is causing problems regularly on our side - which is good for you and I :D )

Progress: I've got debug statements running on a camera system right now via this fork/branch:
https://github.com/city-of-shreveport/nvr-js.git/(debug-statements)
Working on setting up a way to cause failure for the SQLite DB and test the retry module in another branch.

Ok! I found it. Some error catching and logs went a long way. I am sorry I had to dirty up your clean code :D
But the problem, I think, is more basic than I initially thought. Since the FIFO dequeue wasn't checking for an error on the run, it just drops those segments. I added an enqueue and reset the timeout. I tested this by simply mving the .db file and moving it back.
I'll submit a pull request once I do more live testing on our setup. I've pushed my changes here and re-deployed on two test and live systems. Will report back with any findings :)

Wow! You don't waste time 😅
Would you like to test MKV?
I did take a small look at the other PR - but your environment might be a more substantial test.

can you view live footage
can you view recorded segments

All whilst using mkv.
It's online 449 - swap out the mp4 extension

Ok - you have twisted my arm 😅
I need a break from other OSS projects
I have created a branch (2.1.0) - I will use this to merge any enhancements being worked on - so any PR's, address them to this branch for the time being.
I think we can use the video file(s) as a way to provide the timeline content, and do away from any DB based index of segments.
this way even MKV files still being written to will be in the timeline.
I'll wait to hear back from the MKV tests, before I venture down that road, but if the MKV files prove to work in the browser, this should be a smooth change - even metadata could  be made extremely simple also - a flat JSON file per event (that contains the linked segment file) 😎

Awesome! I love the idea of a flat file metadata setup.
One thing I will need to is to checksum the video upon creation, for example, so as to provide proof that the file was not tampered with (if used in court in the U.S., chain of custody is a big deal when you can't prove authenticity of a file; with a checksum on creation, none of that matters much).
So that change could be interesting. Perhaps a video file + a metadata/events file along with each file on disk? IE: for any events coming in, they'd get appended to a file named the same as the video, but with a json extension.
This way, handing all details related to a clip is just two file downloads.
Re: 2.1.0 branch - will do!

So that change could be interesting. Perhaps a video file + a metadata/events file along with each file on disk? IE: for any events coming in, they'd get appended to a file named the same as the video, but with a json extension.

That's the ticket!
A metadata file per segment.
The metadata file will contain the checksum +basic info + any events that occurred in that segment
{
  "segment": {
    "cameraId": "52e5b562-1a1c-4ae8-88a9-c92dc94b497b",
    "fileName": "52e5b562-1a1c-4ae8-88a9-c92dc94b497b.1660052517.mkv",
    "startTime":<PHONE_NUMBER>,
    "endTime":<PHONE_NUMBER>,
    "checksum": "sha256:xxxxxxxxxxxxxx"
  },
  "events": [
    {
      "event": "Building Car Park Gates Opened",
      "sensorId": "bf9d2a78-7425-4249-a559-39bc9ff85f6c",
      "timestamp":<PHONE_NUMBER>
    },
    {
      "event": "Building Car Park Gates Opened",
      "sensorId": "bf9d2a78-7425-4249-a559-39bc9ff85f6c",
      "timestamp":<PHONE_NUMBER>
    }
  ]
}

PERFECT (for my use cases at least)!
I am happy to let you know that everything has been working perfectly with my branch's patch to re-enqueuing the writes to the SQLDB for over 12 hours now. I'll let it keep running for another 12 before I update that box to using MKV's.
I'm going to deploy this to another system now after making the MKV adjustment.
Question for my current branch, though - I do not submit pull requests often (ever), so I'm not all that versed on how best to handle this.
I have a pending branch that re-queues the SQLDB write. I actually need this in production atm, so am curious if you would:

Like me to submit a pull request for just that patch now that it's been fairly well tested in the wild?
Would you like another branch modifying the MP4 output to MKV from current main? Or should I spin up another branch based off of my current branch?

Thanks for the guidance!

Hey @KeithHanson,
Believe it or not. - I have made massive progress, in removing the need for SQLite altogether (I also don't waste time 😄)
This is what will be coming (have written the logic for).....

When a new video file is about to be created

A new metafile for it will be created (as above - give or take)

events created during this segment will be added to that file also.

at the end of the segment, its checksum is added to that metafile

a new video file is created + a new metafile

Rinse and repeat

File name examples:
 {CamerID}.{StartUnixTimstamp}.mkv
 {CamerID}.{StartUnixTimstamp}.json

These files will be used to drive the UI.
This should remove a truck load of surface area for problems to occur.
I think at this stage

Stick to your "remix of the logic" - for lack of a better term
Once you patch the files to use mkv (in place of mp4)

let me know of the experience

A PR is not likely worth it at this stage

Once I am happy, you can migrate to the latest version.
does that sounds like a plan?
I have not touched this project in a few weeks - so thanks for getting me excited about it again 😄

Absolutely! And I'm excited you're excited :)
Agreed about the PR! Sounds like my changes won't be worth much since this will now be flat-file based.
Patching in and deploying the MKV now - will let you know after a couple hours of footage :)

By all means, once the new (better) enhancements are in place, feel free to rebrand it - Its OSS after all 😉

I do think making that configurable would be a great update :) But I also don't mind proudly using open-source and the branding that comes with :)
So! I just patched in MKV files, deployed them to another system, and watched two segments hit the timeline.

Live Streaming Works (Chrome/Firefox)

I can view recorded segments on the timeline (Chrome/Firefox) but can only view the video from the timeline in Chrome

All works in Chrome (linux), but playback of a recorded segment DOES NOT work in Firefox with the following error:
Uncaught (in promise) DOMException: The media resource indicated by the src attribute or assigned media provider object was not suitable.

A quick google seems to indicate that MKV support is just not in Firefox and from what I can see, not coming :/
Thoughts? This isn't a deal breaker for us here since we use Chrome anyways.

Live Streaming Works (Chrome/Firefox)

That make sense, since live streaming uses the MP4 container (well fMP4 - with trickery using websockets)
One of the benefits with NVRJS - is that one FFMPEG instance caters for both the segment and live streaming.
See, I split the one input (camera) to 2 outputs [Live, File] - using one FFMPEG instance per camera,
IMO - The benefit to be had with MKV outweighs the use of Firefox, so at this point I can look past that 😅
Firefox prob doesn't work as it doesnt like MKV - but it works for Live as that is using the MP4 container, where as the segments use MKV.
I could test Fragmented MKV - but will likely be pushing my luck 😄

My side of the world will be sleeping soon.
update here (for my Morning), to ensure MKV is still a go ahead, and I'll continue to work on the enhancements 👍

Ok. I can say that: it works reliably on both test machines. BUT...
For whatever reason, loading MP4 segments are significantly faster than loading MKV segments.
I have both in my timeline and recorded a test for you to see the dramatic difference.
https://cityofshreveport.app.box.com/s/pjjd2y3chp8iepxloy3qn4rh8zw8hyh4 (1 minute .webm file)
Basically, MKV took about 4-5 seconds to load PER SEGMENT (with 3-minute segments - very small compared to regular operation of 15-minute segments), and MP4 took about half a second, reliably.
I have tested it during multiple periods today, and the behavior has been the same. MKV is significantly slower to load than MP4. I have no idea why, though :/

Ok,
I have pin pointed why MKV is slower.
As part of the recording chain, I move each files internal metadata to the start of the file (its usually at the end by default)
MP4 allows you to move this metadata to the start (and I do so)

This allows quicker loading of the file into the Browser (by only loading a small portion and start playing whilst it still downloading), and guess what, MKV does not allow this, so Evan though I add the flag to ffmpeg chain, it has no affect to MKV files.
Meaning the browser has to grab the enter file (to get to the meta info at the end of the file, before it can start playing)
Turns out also, that MKV is really only supported by Chrome, I'm going to opt out of using MKV, but if one wanted to, there is a value you can change in code.

I'm almost done with using flat files - so far its working great!

Ahhh that makes sense!
I think I am with you. Snappier is better for realtime/in-the-moment lookups. I set the segments small enough that I don't think we will need to worry about that.
That is looking really good! I'm excited to begin testing it 😁
One thing I've noticed on our machines, but PM2 registers the NVRJS.js as errored after some time. When I check PM2 logs NVRJS, I don't see anything that would indicate it errored.
I'll try to dig in when I can today, but if you have some insight there as well, we'd appreciate it 😁
I can also open another issue on it to track if you'd like. But that is really the only problem I'm bumping into with my requeue patch in place while waiting for the flat file updates 😁

Mmmm - I'm not sure how PM2 identifies something as errored, it might be picking up some FFMPEG debug output, and freaking out maybe.
I don't use PM2 myself these days (well... I did when I created this, hence the example start up 😅)
I mainly use systemd now.
One thing to note about the upcoming version (3.0.0)
Since it now uses a different storage mechanism, it won't pick up on previous recordings, as the DB has gone.
Unless you manually create the JSON files for each segment 😬
I should be able to push a test version out a bit later.
but I will also push it to the branch I created (2.1.0 - which I'll rename to 3.0.0 once uploaded)

@KeithHanson
Overhaul complete.
https://github.com/marcus-j-davies/nvr-js/tree/v3.0.0
https://github.com/marcus-j-davies/nvr-js/blob/v3.0.0/CHANGELOG.md
Its not published yet - but you should be able to pull down the branch and use it.

Going to test this immediately :) THANK YOU! :)
I will keep digging into the PM2 issue. I love it due to PM2.io's interface, custom actions, metrics, etc. It's pretty nice when managing our 20+ camera systems that will eventually be 100+.
One question:

Since it now uses a different storage mechanism, it won't pick up on previous recordings, as the DB has gone.
Can you tell me more here? Is it because we have to generate the metadata file to know it exists? Perhaps we could create an "import" or a "backlog" sort of script that does that work with what's on disk?

Our storage is pretty much temp storage anyways - the important parts are the live monitoring and clip retrieval in the moment. So not having it show up in the timeline for previous isn't a huge deal breaker, though I may just blow it away and start fresh.
Not a huge deal, but could become important in the future to have some sort of "Catch up" script. Most important thing is the video is on disk and files get deleted over time when space or retention dictates.

Yup - Im hoping the prospect of losing previous recordings is extremely minimal when updating.
providing this new way is successful, I cant see it changing anytime soon. 👍

One thing I should change.
The timestamp of an event being created from live view is captured, ONLY after the Prompt.
so if a user takes 5 minutes to title the event - the timestamp is 5 minutes out.
easy change - Capture timestamp, then ask for event title.

Correct....
NVRJS now has an index of JSON files (lives in memory during runtime) - it scans the NVRJS_SYSTEM folder for the json files at start up, then updates it during the process lifetime.
such index looks like this.
Contract
{
  "cameraID": {
    "timestamp": "timestamp.json"
  }
}

Example
{
  "055d5242-b403-42fb-acfc-8b478ae5cebd": {
    "454354353534": "454354353534.json",
    "545656464566": "545656464566.json"
  },
  "055d5242-b403-42fb-acfc-8b478ae5cebd": {
    "454354353534": "454354353534.json",
    "545656464566": "545656464566.json"
  }
}

When the timeline in the UI is scrolled, it queries this index, pulling all JSON files between timeline start and timeline end that is currently in view providing the keys for the files falls with that period of time.
These JSON files, contain the meta - that is used to load clips and events in the UI.
{
  "segment": {
    "metaFileName": "1660154880.json",
    "cameraId": "66e39d21-72c4-405c-a838-05a8e8fe0742",
    "fileName": "1660154880.mp4",
    "startTime":<PHONE_NUMBER>,
    "endTime":<PHONE_NUMBER>,
    "checksum": "sha256:xxxxxxxxxxxxxxxxxxxxxx",
    "segmentId": "ef633a07-b720-4fd0-a15a-33ef0b8a346f"
  },
  "events": [
    {
      "eventId": "8d720aad-f7e6-49a4-824a-4b0289731a58",
      "event": "Agent 463 - Car theft",
      "sensorId": "LIVE-VIEW-EVENT",
      "timestamp":<PHONE_NUMBER>
    }
  ]
}

Brilliant :) If you don't mind, I'd like to keep this open for the next 24 hours while I have my systems testing it?
But this is fantastic. Tested the event capture too. FANTASTIC.
One tip I'd like to share - I am telling my team to generally only use the live view sparingly because the live view is the full quality stream - which will chew up bandwidth (not a huge worry, but once we enable "buddy system backups" where a pole backs up to another pole in the city, we'll be using the bandwidth at full tilt all the time).
It would be amazing if I could config the low quality stream for live view, and limit the HQ viewing to the timeline.
I also have a few other additions I'd like to make - a "maximize" button on the floating dialog boxes, a config option to always show full controls on the video embed tag (we really need this - being able to go full screen and back quickly, hopping around in the video, etc), and a config option to disable login (we have device based access over an SDWAN called ZeroTier - if someone is on that, it's because I personally clicked the button to allow them to be) - we are not concerned with logins so it would be nice to skip it.
Would you like me to make separate enhancement issues like this one for each of those "wants"?

One more question - apologies.
When cleaning out old files, is it ONLY date based? Or does low disk space also trigger the clean as well?

It would be amazing if I could config the low quality stream for live view, and limit the HQ viewing to the timeline.

Already Possible, by updating the stream config section 😎
https://github.com/marcus-j-davies/nvr-js/blob/ab9e40289bb3ce9845cd637a72eec83e58b7c41a/nvrjs.config.example.js#L48
By default it uses copy as the encoder = no transcoding (next to no CPU usage),
will use what ever bitrate is offered by the incoming stream.
if you wanted to reduce it down, it needs to be transcoded (where you have control of encoding)
but that will use CPU.
Example of lowering the quality for live stream (using libx264)
streamConfig: {
  an: '',
  vcodec: 'libx264',
  'b:v': '256k',
  f: 'mp4',
  movflags: '+frag_keyframe+empty_moov+default_base_moof',
  reset_timestamps: '1'
}

There are various encoders available (some hardware ones also - I think it depends on the system that it is running on)
above is using a software renderer (libx264) at a bitrate of 256k

When cleaning out old files, is it ONLY date based? Or does low disk space also trigger the clean as well?

Its purely based on the number of days old the footage is continuousDays (14 days by default)

Ah - so, to clarify, I have 1 RTSP stream coming from the camera that is low quality, and another separate RTSP stream coming in for HQ.
I'd like the live view to use one RTSP stream, and the storage/timeline view to depend on the HQ RTSP Stream? Hope that makes sense.

Its purely based on the number of days old the footage is continuousDays (14 days by default)
Eek. Ok. I'm not 100% on the number of days so I'll play with it. It's another edge case, but bad things will likely happen if we hit 100% :P I'd feel much better if it took that into account.

If I stub my toe on this enough, I'll work on a small patch to keep a buffer of space available on top of days of retention.
Thank you! For everything :)
PS - how would you like me to handle my wishlist above? Enhancement issue + pull request?

So far so good. I've deployed this to several systems (~10 atm) and have recorded roughly 6 hours of footage.
Analysts are kicking the tires on it right now as well :)

Ah - so, to clarify, I have 1 RTSP stream coming from the camera that is low quality, and another separate RTSP stream coming in for HQ.
I'd like the live view to use one RTSP stream, and the storage/timeline view to depend on the HQ RTSP Stream? Hope that makes sense.

NVRJS uses 1 FFMPEG Process per Camera, it then conjures 2 output streams from the one input.
I sketched the below, this is per camera.
To pull in 2 streams from each camera, will require 2 FFMPEG instances.
so having just 3 cameras, will require 6 FFMPEG processes, which could start to see performance issues.
As below, the live stream can be tailored to suite, i.e to reduce bandwidth, but it will require transcoding (i.e using libx264, instead of  copy for vcodec) - nothing stopping you using a HW encoder i.e h264_v4l2m2m (if your system has one,  I know most recent  RPIs do) but the HW encoder h264_v4l2m2m has a bug, in that it produces a green tint, so you need to build FFMPEG from source, to get the latest build.
moving away from copy for the live stream, means you can set a bitrate for the video (b:v)

If I stub my toe on this enough, I'll work on a small patch to keep a buffer of space available on top of days of retention.

The way it works in NVRJS is a combination of continuousPurgeIntervalHours and continuousDays
Given continuousDays set to 7 and continuousPurgeIntervalHours set to 12.
it will purge any footage that is 7 days old every 12hours.
Maybe a version of this can live along side the purge schedule, with the ability to pass in the number of days, and triggered, based on a drive space poll. but the drive space poll, needs to be OS independent.
https://github.com/marcus-j-davies/nvr-js/blob/a11239feb418ad94d3228c3057777d7e9630eea3/NVRJS.js#L647

Just want to report in - so far so good. I've deployed this to several systems (~10 atm) and have recorded roughly 6 hours of footage.
Analysts are kicking the tires on it right now as well :) Several excited reactions when I demo'd creating an event from the live view :D

That's encouraging to hear 😄

PS - how would you like me to handle my wishlist above? Enhancement issue + pull request?

I would say a PR, raising issues is fine, but may take me longer to get time to work on them 😅

Hm. I just want to utilize the already encoded low-quality stream. But I will handle it another way (like spin up a second nvr-js to a backup drive).
I do have an issue I've identified on two of the ten systems.
I don't know what causes it (logs look good), but I suspect it may be similar to before.

As you can see, we're doing great up to<PHONE_NUMBER>.mp4, then the json files stop being created.
Any ideas? Assume all things can happen on the pole, because we've seen it plenty - power sag drops USB disk and things have to recover, for instance.

As a quick fix, I'm going to hack together an "import" job that will generate the missing json files.
Other than these two systems' and this one issue, the rest have been rock solid.

Mmm, nothing obvious is coming to mind - was it one camera or a whole unit?
I wonder if a small IO error caused the NodeJS FileSystem Watcher to halt.
See, I use Nodes FS Watcher API to monitor the directory, for new MP4s  - as that is the trigger to create a new JSON file.

I could move that logic to listen for FFMPEG instead. - as I receive the segment entry name at the end of each segment.
but it currently does nothing
https://github.com/marcus-j-davies/nvr-js/blob/03cb3e53857676cebdb5db0d9c5b918c3b25a8cb/NVRJS.js#L620
The above will receive the file name after each segment - so In theory, I cooked use it instead of monitoring for files (if IO errors can stop the watcher

I would much prefer attaching to ffmpeg since that is working so well. We def have IO errors for about 10 seconds sometimes. Everything recovers within about that timeframe but the json unfortunately.
Waiting until the segment is finished is pretty typical of most NVRs so.... Not a critical need (to me).

Ok,
I do believe this is the FS Watcher not recovering from an IO error in that case.
I'll attach it directly to FFMPEG.
I'll patch, and push the code - this will also include the new Ability to disable login (I added it today 🤫 )

Fantastic! I'll tune in here and redeploy everywhere once ready!
(Analysts will love disabling user/pass 😁)

Ok, Have done some research....
Moving to FFMPEG to monitor the start of segments isn't straight forward currently, for a few reasons.

There is no event to tell us (only on completetion of a segment)
Having it based on the start up of an FFMPEG process, does not guarantee the start timestamp, is the true timestamp recording started (it can be different over serval seconds - Hey I'm a perfectionist 😅 )
Events occurring during a segment cant be written to disc (no segment meta file will exist, to write to and there is quite a lot of logic to rewrite to work around it)

With that said, the watcher does have an error event.

With this, I can restart the watcher, if it had errored due to an IO error (like a power drop say),
what I will do is, wait 10s before restarting it, if I encounter this event.
I think that will be my best approach avoiding the potential to introduce problems with the logic rewrite.
and should, in theory allow the watcher to recover.

Yes - that would be fine - as long as we're able to recover automatically we are happy :)

Just an update - I am moving the Meta Creation Logic over to listen for FFMPEG directly after all.
its WIP - so don't pull down any changes just yet.

Roger that! Thanks for the update!

@marcus-j-davies Any updates? :) :) :)
Current metadata bug is affecting us - I'm solving it with a cron-based restart of the service for now.
If you think it will be more than a few more days, I'll go ahead and knock out the backfill task for us and have that on a periodic run & reboot service along with that every hour and I don't think we'll see any further metadata writing issues.
If I can help in any way further than that, please let me know! I don't want to step on your toes, though as I'm certain you'll produce a better outcome than I could :P
Again, many thanks for such a great tool and all the collaboration with us!

Hi Keith,
Only a couple of lines left really.
I'll see if I can knock it out this evening.

@KeithHanson
The patch is ready to test.
To Disable Login:
add this to your config file.
module.exports = {
	/* System Settings */
	system: {
		/* Disable Security - Know what your doing before changing this! */
		disableUISecurity: false,
		.....
        }
}

THANK YOOOOOU!!!! Will deploy tonight!

Got excited. Deployed to one of our systems for testing and so far things are working :P
I've got about 30+ systems I'll deploy to tonight to verify.

Ok - I've got this running on 10 of our live deployments as of about 20 minutes ago :)
Going to let this run for 24 hours or so and report back :)

Oooh, as a bonus - I moved the files from one folder to the renamed camera folders and bam - everything was picked up. I did have to restart the service for it to show in the timeline, but suuuuper convenient!

Ok, I have deployed to all systems we have under control. ~30.
Something else I thought of that is a win because of this is the way we want to backup our data.
I'm planning on having a second NVRJS running on 81, but it will pull the low quality streams to disk. We're designing an algorithm that will pick the most appropriate "buddy" to back up to and from and kick off some kind of copy process.
Because of these changes, we can easily obtain at least low quality footage by simply using the interface and update the config to display the backed up folder <3
This lets us focus on the hard/critical part for us (choosing the right pole to back up to with a variety of factors) while getting a UI for free! :) :) :)

You are excited 😆
There are APIs built in to fetch segments and current system utilisation, if you wanted to create a dashboard to monitor all instances (not footage, more health) - i have not spent a great deal documenting them but they do exist.
Let me know if the recent patch fixes the missing meta files (I'm hoping they have)

30 total systems - We'll get to 80 before end of year :)
Excellent - we will definitely tap into that - we have a heartbeat service that checks all kinds of things relevant to us (disks, encryption status, temps, software versions, etc).

Yeah - a try catch should be added in a couple of places really.
I am more interested in the root problem.
What did<PHONE_NUMBER>.json &<PHONE_NUMBER>.json look like?

Almost undoubtedly disk error - we have to deal with it on our crappy power grid and recover automatically (sad, I know - drink one for me in commiseration :P).
Both files were empty.

If you want I can add in this try catch for now and submit a pull request. I've patched the two systems that failed manually for now.
If you're already off to the races on the patch I can hang back ofc :)

Cool - well not, but you know 😅
I'll add a try catch to any reading of a file (and writing for good measure).
I cant do anything special here, other than backing out of the current operation  - in favour of not bringing down the instance in its entirety.

Awesome :) And that is fine - I'm not sure if there is any magical code that could solve that lol. But that's also a reason I reduce things to 3 minute chunks (faster streaming, faster downloading, less risk of missing important things because of gremlins/dragons like this).

Just did a count of all the systems. Deployed most recent commit to 41 raspberry pi's with 2TB and 4TB drives attached :)
I tested it on the previously failing system, and everything went smoothly.
Thank you! :) I'll report in if I find anything.

Rock solid :D

Nice!
This is with the recent patch to stop reading Corrupted  JSON files?

Correct! I deployed it after I saw the update.
I see some gaps here and there, but that's from whatever failures happened.
But the only problem I've had is HDD space filling up at this point :D Just means I need to tune the retention.
I've spot checked about 10 of the 41 systems and those that didn't fill up their drive have a full 2 days of history!
Brilliant! :)

Assuming all is well?
I am aiming to publish 3.0 soon.

i think this is a great project, and it gives me a good idea to make a NVR system. thank you very much.
i tested v2.0 and v3.0 version.
every .mp4 file record video for one minute.  the record video part works well.
after i  record video for 16 hours,  if i scroll the mouse wheel many times to zoom in/out timeline quickly,
and then scroll the timeline from right to left a few seconds. it will crash, although i press the refresh button, it also doesn't work.
two versions all happened the same issue.

i tested the vis-timeline using the link: https://visjs.github.io/vis-timeline/examples/graph2d/08_performance.html, the issue didn't happen.
record video for 5 minutes(not many hours), the issue didn't happen.

i think when i scroll the mouse wheel, the timeline.on('rangechanged') will be called many times,
query SQLite or ReadMetaFile maybe have some delay.

Hi @trc-turing,
Version 3 has removed SQLIte entirely, and its now based on a JSON file per segment, this seems to have removed a lot of problems, v3 is currently being used in a very large installation, that seems to be quite stable.
As for rangechanged - yes! this is what causes the query (previously SQLite), but now uses the file mech.
rangechanged will only continue if rangechange is not triggered within 500ms.
https://github.com/marcus-j-davies/nvr-js/blob/5639de6b9d71b975a18ffc7a9c73df6731618316/web/static/js/scripts.js#L78
in other other words, the scroll/zoom must be static for 500ms - try messing with this value.

thanks for your reply.
yes, you are right. if i change the settimout duration from 500 to 1000 or 3000.
it doesn't crash. but after i did it, when i click the timeline,
because the duration it will have a delay, or click event doesn't work.

Yup,

Yup, the timeline needs to load data based on the time span in view +- 2 hours.
As the NVR can be running for weeks/months/years at a time, I don't load everything - as the browser could be overloaded with data, can you imagine 1 min segments over a week? That's 10,080 events in the timeline!
I therefore need to cap it based on the current view.
I load what timespan is in view (plus 2 hours)
You can override the 2 hour buffer by changing SearchTimeBufferHours in the scripts view.
At the moment, I don't have a method (or time)  to stop the unnecessary loading.
I can probably improve it, but I need to find the time todo so.

你好@trc-turing,
版本 3 完全删除了 SQLIte，现在它基于每个段的 JSON 文件，这似乎消除了很多问题，v3 目前正在一个非常大的安装中使用，这似乎相当稳定。
请参阅 v3 更改日志 https://github.com/marcus-j-davies/nvr-js/blob/v3.0.0/CHANGELOG.md
至于rangechanged——是的！这就是导致查询的原因（以前是 SQLite），但现在使用文件 mech. 只有在 500 毫秒内未触发 rangechanged时才会继续。rangechange
https://github.com/marcus-j-davies/nvr-js/blob/5639de6b9d71b975a18ffc7a9c73df6731618316/web/static/js/scripts.js#L78
换句话说，滚动/缩放必须是静态的 500 毫秒 - 尝试弄乱这个值。

published yet to NPM

wait published  to npm

@baozi510
The 3.0 Pull Request is ready -> https://github.com/marcus-j-davies/nvr-js/pull/13
I'll publish in a day or so.


PT-1131 | Fix event pagination error and improve tests

Description :sparkles:
Fix event pagination error and improve tests
Issues :bug:
Closes :no_good_woman:
PT-1131:
Related :handshake:
Testing :alembic:
Automated tests :gear:
Manual testing :construction_worker_man:
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #208 (46d3dc2) into master (ff08b81) will increase coverage by 0.26%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #208      +/-   ##
==========================================
+ Coverage   82.76%   83.03%   +0.26%
==========================================
  Files         208      208
  Lines        3708     3708
  Branches     1109     1109
==========================================
+ Hits         3069     3079      +10
+ Misses        611      601      -10
  Partials       28       28

Impacted Files
Coverage Δ

src/domain/events/EventsPage.tsx
90.16% <ø> (+4.91%)
:arrow_up:

src/domain/events/utils.ts
100.00% <100.00%> (+38.88%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ff08b81...46d3dc2. Read the comment docs.


Omnisharp Roslyn not starting/detected

It looks like vim doesn't detect Omnisharp Roslyn. When I open a CS file inside vim it starts logging Omnisharp info which makes one think it's running properly but :echo OmniSharp#ServerIsRunning() always print zero and of course auto-completion doesn't work. My project has a .sln file just in case.
OS: Ubuntu 18.04 (x64)
Mono: v<IP_ADDRESS>
Python: v2.7.15rc1 (:echo has('python') == 1)
Neovim: v0.2.2
My ~/.config/nvim/init.vim file:
execute 'source' fnamemodify(expand('<sfile>'), ':h').'/config/main.vim'

call plug#begin()
Plug 'arcticicestudio/nord-vim'
Plug 'derekwyatt/vim-scala'
Plug 'elmcast/elm-vim'
Plug 'valloric/youcompleteme', { 'do': './install.py' }
Plug 'OmniSharp/omnisharp-vim'
Plug 'oranget/vim-csharp'
Plug 'fsharp/vim-fsharp', { 'for': 'fsharp', 'do':  'make fsautocomplete' }
Plug 'ensime/ensime-vim', { 'do': ':UpdateRemotePlugins' }
call plug#end()

imap fd <Esc>

autocmd BufWritePost *.scala silent :EnTypeCheck

let g:omnisharp_proc_debug = 1
let g:OmniSharp_server_path = '/home/elon3m/Downloads/omnisharp-mono/OmniSharp.exe'
let g:OmniSharp_server_use_mono = 1

let g:elm_detailed_complete = 1
autocmd FileType elm let g:deoplete#disable_auto_complete = 1
let g:spacevim_enable_ycm = 1
let g:ycm_semantic_triggers = { 'elm' : ['.'], 'cs' : ['.'] }

I have tried many different things like changing server_path and some other settings. Nothing seems to work. I ran UpdateRemotePlugins and also checkhealth which gives me no error.
C-x C-o logs the following:
Error detected while processing function OmniSharp#Complete[11]..OmniSharp#GetCompletions[1]..provider#python#Call:
line   18:
error caught in request handler 'python_eval ("getCompletions('')",)':
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/neovim/plugin/script_host.py", line 150, in python_eval
    return eval(expr, self.module.__dict__)
  File "<string>", line 1, in <module>
  File "/home/elon3m/.SpaceVim.d/plugged/omnisharp-vim/python/omnisharp/OmniSharp.py", line 118, in getCompletions
    response = json.loads(getResponse('/autocomplete', parameters))
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
Press ENTER or type command to continue
Error detected while processing function OmniSharp#Complete[11]..OmniSharp#GetCompletions[1]..provider#python#Call:
line   18:
ValueError: No JSON object could be decoded
Press ENTER or type command to continue
Error detected while processing function OmniSharp#Complete[11]..OmniSharp#GetCompletions:
line    3:
E714: List required

I tried both on Emacs and VS Code and it works perfectly. Just in case I changed the server_path to point to Emacs server but got the same result.

Are you using the Http version of the omnisharp-roslyn server?
Both Emacs and VSCode use stdio to communicate with the server, whereas vim users HTTP. So vim requires a different omnisharp-roslyn download.

I would also probably try working without YouCompleteMe as that complicates matters somewhat (at least it used to)

@nickspoons thank you, that was my mistake. I'm having a different issue right now (https://github.com/OmniSharp/omnisharp-vim/issues/347), I will try to solve these permissions issues.


Extend OpenAIChat to track token usage

This change supports the use of get_openai_callback, as in https://langchain.readthedocs.io/en/latest/modules/llms/examples/token_usage_tracking.html, for tracking token usage.

the formatting issues should be fixed now


command not found

Hi,
I was trying to run nnDetection. However, while running the command
nndet_prep 500 --full_check
it says, /bin/bash: nndet_prep: command not found.
I suspect it might be an issue by setting up the environmental variables which I set using
os.environ['det_data'] = os.path.join(nnDetDir,'nnDet_raw_data')
os.environ['det_models'] = os.path.join(nnDetDir,'nnDet_trained_models')
os.environ['OMP_NUM_THREADS']="1"
where "nnDet_raw_data" is the directory where the data (imagesTr, labelsTr, imagesTs and labelsTs) is located and 'nnDet_trained_models' is the directory where the trained models will be saved.
Could you please help me figure out what might be causing an issue?

Hi, the installation probably failed, please check the FAQ to rule out common mistakes and provide the installation log.
Best,
Michael

Hi Michael,
here is the system information i am getting after running scripts/utils.py
----- PyTorch Information -----
PyTorch Version: 1.11.0+cu113
PyTorch Debug: False
PyTorch CUDA: 11.3
PyTorch Backend cudnn: 8200
PyTorch CUDA Arch List: ['sm_37', 'sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86']
PyTorch Current Device Capability: (7, 5)
PyTorch CUDA available: True
----- System Information -----
System NVCC: nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Mon_May__3_19:15:13_PDT_2021
Cuda compilation tools, release 11.3, V11.3.109
Build cuda_11.3.r11.3/compiler.29920130_0
System Arch List: None
System OMP_NUM_THREADS: 1
System CUDA_HOME is None: True
System CPU Count: 8
Python Version: 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:53)
[GCC 9.4.0]
----- nnDetection Information -----
det_num_threads 6
det_data is set True
det_models is set True
Is there anything that look susppicious? I am still getting the command not found error. Is there any particular directory that I need to run the command from?

nnDetection requires python 3.8 or up, could you try with that?

Hi Michael,
Thanks. I am currently using Google cloud which is unfortunately pre-built using python 3.7 and I can not change it. Does nnDetection not support python 3.7 at all?

There are some things like the walrus operator which are only supported under python 3.8+, so 3.7 unfortunately won't work

Hi Michael,
I was able to install it through Docker in WSL-Ubuntu environment. However, while running, it was showing CUDA out-of-memory error. My workstation has nvidia quadro GV100 32 GB. I checked the training with nndet_train XXX -o augment_cfg.multiprocessing=False and it's not showing the error although the training is very slow. Attached is the screenshot of nndet_env. Also, I noticed that low resolution plan was triggered. Is there any way I can do training with low resolution plan?

Dear @sovanlal ,
did you run the planning with the V100 as well?
If yes, it might make sense to increase the offset (linked below) significantly, e.g. to 18GB or similar, so the resulting plan would roughly correspond to a 2080ti plan. Those run pretty nicely and you could scale up the batch size if you want to take full advantage of the GPU VRAM.
(I need to run some additional testing with the offset, that is why it is not the default right now)
https://github.com/MIC-DKFZ/nnDetection/blob/d637c5e2da16e0fe7cf8a5b860907eb57e60d4fe/nndet/planning/estimator.py#L49

I have made the necessary changes. However, the training is stuck after sometime. I am not sure what is the exact reason. Will appreciate any help.

@mibaumgartner ,
Interestingly, first two epochs ran successfully while it was stuck at 3rd epoch. The training time was really fast for first two epochs (~15 mins). Any clues?

Can you check your VRAM consumption (this might be interesting for the quick training)?
Regarding getting stuck: Please double check that the training is not running our of memory (normal RAM), decreasing the number of threads would help with this. Also make sure to the OMP_NUM_THREADS environment variable.

@mibaumgartner ,
the number of threads I am using is 6 although I tried with 12 and 24 (maximum thread of my CPU) as well. OMP_NUM_THREADS=1 as is shown below by nndet_env. I am running it using docker container but the issue still persists

Not quite sure what is causing the problem.

Could you try to increase "--shm-size=24gb" to 36 or even higher, depending on how much memory is available.
Also manually monitor CPU, GPU and RAM usage during the training please. I think something exceeds the available resources and thus it crashes.
I'm also wondering why you metrics are negative, this should only happen if not all classes are present during the online validation phase. Is it some kind of toy dataset?

@mibaumgartner ,
This might be a CUDA driver related issue that I am now trying to fix.
Also, for assigning class I am following the instruction for creating data set and label.json file. My dataset consists of 921 tumor segmentation (in .nii.gz) files,. So, I created the label.json file like below

My dataset do not have multiple tumors per patient. So it has only one instance (class 0) per patient. Is there anyting woring with this approach?

The label.json looks fine, did you enter more than one class in your dataset json?
Alright, a CUDA driver problem might indeed result in crashes.

@mibaumgartner ,
my dataset has only one class i.e. tumor and I have changed my dataset.json like below
{
"dim": 3,

"labels": {

    "0": "tumor"

},

"modalities": {

    "0": "CT"

},

"name": "tumor detection",

"target_class": 0,

"task": "Task500_TumorDetection",

"test_labels": true

}
Does it look good or need any modifications?

Something seems to be off from the logs and the dataset json you provided. The logs explicitly mention that no objects of class 1 were found for evaluation, but from your dataset.json and explanations there shouldn't be a class 1. This results in the negative validation metrics.

@mibaumgartner
It looks like I managed to start the training without interruption. It's still running and I hope it does so for the rest of the training :-). I think the CUDA driver was an issue. I had CUDA driver 12.0 and eventually had to downgrade it. I also fixed the negative metrics issue.

@mibaumgartner ,
the metrics that are calculated while training, are those for training or validation data?

The metrics are computed on the validation data

Hi @mibaumgartner ,
I wanted to see the bounding box predictions as image (in .nii).
I saw in one of the threads that nndet_box2nii is the required command. Could you please let me know what should be the exact input that is required for the command?

Is there anything specific you are missing? You can also retrieve additional information via the -h flag. All the info is also in the script here: https://github.com/MIC-DKFZ/nnDetection/blob/d637c5e2da16e0fe7cf8a5b860907eb57e60d4fe/scripts/utils.py#L33-L46

All of the results are in the repository, you can find them here: https://github.com/MIC-DKFZ/nnDetection/blob/main/docs/results/nnDetectionV001.md

Hi @mibaumgartner ,
I was running Task007_Pancreas dataset. The val_results that I got from fold0 is quite low e.g<EMAIL_ADDRESS>is 0.30 and<EMAIL_ADDRESS>is 0.19. Although I am yet to run all the folds , this does not seem to be right. Could you please help figure out what went wrong?

Indeed, the values are significantly too low, how does the training log look like?

Dear @sovanlal ,
the training log performance look rather suspicious, since the performance should be smaller during online validation should be smaller and the drop for the online validation should be smaller.
Given the extremely small patch size I think that you are using a rather small GPU which causes the problem -> during online validation objects are extracted from the patches and thus if the object is "large", the performance during training is good (since if basically always fills the entire patch) but during inference the the objects are "suddenly" not detected anymore which results in very bad performance. Training with a RTX2080ti should result in a patch size of [40, 224, 224] which is significantly larger than [10, 56, 56] from your training log.

Sorry, I lost scope of the previous conversations. I think you bumped the Tensor block size way to high which leads to the tiny patch size (in the estimator). Which value did you enter there?
Did you perform any other changes? Could you provide the prepare/planning log as well to double check that there are not other problems witch the GPU?
(The configured patch size is not suitable for your GPU VRAM)

@mibaumgartner ,
I used 10 GB as offset in these two places
"RTX2080TI": int(mb2b(10000))
offset: int = mb2b(10000),
These were the only changes I made.

Alright, than I would recommend to decrease the offset until you in a similar range than the above mentioned patch size (40, 224, 224)

@mibaumgartner ,
Can [32, 192, 192] be a good choice?

Definitely better than the old one but I'm still unsure why you get significantly reduced patch sizes out with a 32GB VRAM GPU. I got the patch size of 40, 224, 224 on a RTX 2080ti with 11GB and patch size for the same dataset should only depend on the available VRAM of the GPU.

I was able to get (40, 224, 224) but now it's showing CUDA out of memory error :-). I am not quite sure what;'s going on.
Logs are attached.
train.log
logging.log

I rechecked the planning stage and noticed that I already took care of memory estimation for bigger card which I forgot about. Even if more memory is available, it will only plan for a 11GB VRAM GPU => so the configuration should consume around 11gb as was also determined on my GPU.
This makes the training error, "CUDA out of memory" even more dubious though, since there is definitely enough memory available. I'm not aware of any Issues which reported out of memory on GV100 GPUs. I've trained several models on V100 GPUs in our cluster without any problems though. Can you manually check the memory consumption when you start the training? It seem rather odd that it would run out of memory.

The offset should be kept at the default setting.
nnDetection requires more CPU and IO resources than nnU-Net so they can not be compared directly. The FAQ includes an section on debugging bottlenecks, maybe there is something interesting in there? Slowly increasing the num_threads could help with a CPU bottleneck (maybe sure not run out of RAM)


release instrumentation-openai 0.4.0

Fix the release workflow to install deps and compile before published,
otherwise a mostly empty package is published.

release workflow: https://github.com/elastic/elastic-otel-node/actions/runs/12303052688


Add preprocessing tool to the smoke tests.

Codecov Report
Patch and project coverage have no change.

Comparison is base (399bb7d) 93.13% compared to head (792c5e8) 93.13%.

:exclamation: Current head 792c5e8 differs from pull request most recent head 3cb0fb5. Consider uploading reports for the commit 3cb0fb5 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1900   +/-   ##
=======================================
  Coverage   93.13%   93.13%
=======================================
  Files         355      355
  Lines       25843    25843
=======================================
  Hits        24069    24069
  Misses       1774     1774

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


ipad magic keyboard cannot be used

I am using the testdrive on ipados 14.6 and the magic keyboard or any external device is not working for that matter. Would be great to see that feature.

Hmm, I've been using a Magic Keyboard (via Bluetooth) on my iPad and it works fine in Lagrange. Although, I'm running iPadOS 15 Beta.

any external device is not working

Which external device(s) did you try using?

Magic Keyboard and Touchpad. But strangely enough. I justed tested it and it does work now. Hmm

Well, if you find the circumstances under which it doesn't work, we can reopen the issue.


no resource plugin tls-v2.4.0 found in the workspace or on your $PATH

Not sure if this bug belongs here or in a different pulumi project.
I updated Pulumi and all plugins and Python dependencies to the latest versions. After doing that I'm now getting an error that the tls plugin isn't on version 2.4.0.. which obviously doesn't exist since it looks like 2.3.0 is the latest.
     └─ pulumi:providers:tls                                    default_2_4_0                                   1 error

Diagnostics:
  pulumi:providers:tls (default_2_4_0):
    error: no resource plugin 'tls-v2.4.0' found in the workspace or on your $PATH, install the plugin using `pulumi plugin install resource tls v2.4.0`

It seems I have the correct version installed:
# ls /usr/local/lib/python3.7/site-packages/ | grep pulumi_tls
pulumi_tls
pulumi_tls-2.3.0.dist-info

# pulumi version; ls -w 1 /root/.pulumi/plugins/
v2.10.0
resource-aws-v2.11.0
resource-aws-v2.13.1
resource-consul-v2.2.1
resource-consul-v2.3.0
resource-docker-v2.2.3
resource-f5bigip-v2.3.0
resource-kubernetes-v2.3.1
resource-kubernetes-v2.4.0
resource-kubernetes-v2.4.3
resource-kubernetes-v2.5.0
resource-rancher2-v2.2.0
resource-random-v2.1.3
resource-random-v2.3.0
resource-random-v2.3.1
resource-terraform-v2.3.0
resource-tls-v2.1.3
resource-tls-v2.2.0
resource-tls-v2.3.0
resource-vault-v2.2.1

Hi @iridian-ks
Are you saying that when you updated an existing plugin that it tried to install pulumi-tls 2.4.0?
Paul

@stack72 no, I am running pulumi up and it's failing because tls-v2.4.0 is not installed.

But it occurred when you upgraded the versions of the providers?

The error message occurred when I ran pulumi up, but yes the error only started after upgrading both Pulumi from 2.9.0 to 2.10.0 as well as upgrading all of the providers to their latest versions.
I just tested trying to downgrade the pulumi_tls package as well as Pulumi back to their original versions, but I'm getting the same error. I do have other providers installed that I did not test downgrading back to their original versions.

It seems to resolve if I do this:
tls_provider=tls.Provider("my-tls-provider")
tls_opts=pulumi.ResourceOptions(provider=tls_provider)
tls.PrivateKey(
  ...
  opts = tls_opts,
)

It seems the issue is when not feeding an explicit provider. Is this expected?

I'm running Pulumi through Docker and I was able to find the previous docker image with the old working versions of the providers and Pulumi. I can reproduce by rebuilding the image, but in order to get work done, I'm stuck on this old image.

Closing: pulumi-tls is at 4.x at this point, so marking won't fix.


Password Protected LNDConnect

I propose an update of lndconnect that allows a user to add a password before outputting the QRCode or string. Then the QRCode data or string data is encrypted using the user's password.
The reasons for this are:

The data is no longer sensitive
The data will not be so long, resulting in easier to scan QRCodes

I think we should add an additional field such as password=true to keep backwards compatibility. When clients parse the lndconnect strings, they should parse the password field, knowing whether this lndconnect data is hashed with a password or not. Then clients can prompt users for their password and decrypt the lndconnect string.

I still don't see a threats that can be avoided by encrypting the lndconnect uri. (that might be just because i'm lacking imagination).

If you want to share your code over the internet you still have to share the password over a secure channel. Instead could just share the lndconnect uri over that secure channel and you get the same level of security.
If you need anything more sophisticated you're also sophisticated enough to just encrypt the uri yourself. (lndconnect -j | rot13 😉)

I would expect encrypting the data would rather increase the size of the uri than shortening it. At least if we're using a popular block cipher like AES.

 Especially those you have to enter on mobile devices. Everybody will just use 12341234.


Incompatible with immer-yjs

Love this project. Unfortunately, I tried installing this on a project at work and it collided with immer-yjs:
https://github.com/sep2/immer-yjs/blob/main/packages/immer-yjs/src/immer-yjs.ts#L39C78-L39C78
../node_modules/immer-yjs/src/immer-yjs.ts:39:64 - error TS2345: Argument of type '(v: JSONValue | YMap<any> | YArray<any>) => JSONValue' is not assignable to parameter of type '(value: unknown, index: number, array: unknown[]) => JSONValue'.
  Types of parameters 'v' and 'value' are incompatible.
    Type 'unknown' is not assignable to type 'JSONValue | YMap<any> | YArray<any>'.

39                     arr.splice(retain, 0, ...change.insert.map(toPlainValue))
                                                                  ~~~~~~~~~~~~

Found 1 error in ../node_modules/immer-yjs/src/immer-yjs.ts:39

I have "skipLibCheck": true, in my tsConfig and "exclude": [ ..., "node_modules", "../node_modules" ...]
Is there something I'm missing here. For now I can't use it as is, which is a pity since it's super great!

I'm having a similar problem with reduxjs. Even with "skipLibCheck": true, I'm getting tsc errors from inside my node_modules/ folder.

@tylerlaprade In the case of immer it's because immer releases typescript directly and so 'skipping' the lib check isn't really an option... I think.


Fix up links in spec

Now that multiple pages have been squished into one, all the links must be updated.

Closing, as we'll replace the spec doc here with a doc coming from CF


How to set nvim_buf_set_extmark opts?

Now can only set virt_text, but it's guibg is strange when cursorline, how can solve this?

Check :h nvim_buf_set_extmark, anything in opts can be passed here.
Could you show what you have right now?

Check :h nvim_buf_set_extmark, anything in opts can be passed here. Could you show what you have right now?

Thanks, I added hl_model = "combine", it works


Multi-attach block PVC not supported

I created this PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc1
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  volumeMode: Block
  storageClassName: ontap-san
  resources:
    requests:
      storage: 1Gi

and it never bound to a PV. Multi-attach raw block support is needed for a variety of use cases, and there's no technical reason (outside of Trident) not to support it.
Fixing trident to support this case requires teaching the middle-end the difference between "filesystem" and "block" volume modes. Currently the middle-end uses the terms "filesystem" and "block" as synonyms for NFS and iSCSI respectively, which is confusing, because the concepts are orthogonal. The following 4 combinations are possible:
Filesystem on NFS (this works today, with RWX support)
Filesystem on iSCSI (this works today, with RWO support, due to platform limitations)
Blocks on NFS (not implemented)
Blocks on iSCSI (this works today, with RWO support, but should support RWX)

I have a large customer in need of this to support rolling upgrades of database containers.

@forgosh If your customer is actually using PVCs with volumeMode: Filesystem, even if it's iSCSI this bug doesn't apply to them. This bug is specifically about "raw block" volumes without file systems

@forgosh @bswartz Wouldn't you risk of corrupting your container data as 2 distinct container would access the same backend disk with write operations?
One typically requires a clustered file system for that , in order to handle arbitration on who may read or write (scsi reservations) at any point in time. This is probably the reason you can do a RWO attachment and not a RWX for SAN based protocols (iSCSI) while you can for NFS (read/write arbitration is done by NFS locking).
May be you could create a clone and then connect the new storage instantiation to the container and eventually split the clone if use case is a successful rolling upgrade.

@sdes1480 you are correct. I was hoping this was a solution to that problem. However, as @bswartz points out, this is only for RAW disks.

@forgosh @bswartz Wouldn't you risk of corrupting your container data as 2 distinct container would access the same backend disk with write operations?
One typically requires a clustered file system for that , in order to handle arbitration on who may read or write (scsi reservations) at any point in time. This is probably the reason you can do a RWO attachment and not a RWX for SAN based protocols (iSCSI) while you can for NFS (read/write arbitration is done by NFS locking).

With raw blocks, SCSI reservations are possible from inside the container. You just need the SYS_RAWIO security capability for your container. In this way multiple containers can arbitrate access to a single volume. Alternatively, the containers can establish some other protocol to control access, either using the disk as a mailbox, or by using network communications between the containers.
All of these things are at least possible with raw block volumes. With filesystem volumes, the kernel is performing the I/O and it becomes impossible to avoid corruption unless the filesystem itself is cluster-aware (OCFS2).

Fixed in 19.10


feat: add kafka consumer server

Proposal

Add Kafka Server
Add config kafka get_consumer

Links

https://kafka.apache.org/
https://github.com/fede1024/rust-rdkafka

Codecov Report

Merging #16 (750903c) into main (cf8bf9c) will decrease coverage by 3.73%.
The diff coverage is 20.51%.

@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
- Coverage   77.73%   74.00%   -3.74%
==========================================
  Files          28       31       +3
  Lines         566      604      +38
==========================================
+ Hits          440      447       +7
- Misses        126      157      +31

Impacted Files
Coverage Δ

src/bin/kafka.rs
0.00% <0.00%> (ø)

src/config/kafka.rs
0.00% <0.00%> (ø)

src/presentation/kafka/mod.rs
0.00% <0.00%> (ø)

src/config/env_var.rs
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


New global property are no longer writable

Likely introduced by 9a3a082225277ff65857ac046fa09edb51702367
/cc @lexs

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at<EMAIL_ADDRESS>Thanks!

By analyzing the blame information on this pull request, we identified @lexs and @javache to be potential reviewers.

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

Thanks @marclaval, I see that there is another PR that tries to fix this problem and is getting merged atm. Let's close this one in favor of #7704.


Cope with blank feature flags

Context
Prod died because of an empty string in a feature flag.
Changes proposed in this pull request
Anything except trim('true') (case insensitive) is now false

 Make Search UI feature flag code more robust and better tested

 Allow mc-support to grant access requests through mc-be


Example code mismatch in readme

The example code shown in the png at the top of the readme (coming from file example-syntax-highlighting.png) shows different syntax than the example code under the "Pages in Pushup" section. Namely, "Pages in Pushup" code has "^else" while the image is missing the ^ before the else. Which is correct? Does either work?

Good catch, the screenshot is wrong and else ought to be ^else


Kubernetes operator

Consider adding support for deploying an operator
https://coreos.com/operators/

I got to love kooper for this.

At this moment there is no standard, although there are some projects like rook operator kit or Giantswarm operator kit that are trying to create it.
Spotahome studied these projects before developing Kooper and they didn't fit the requirements:
...
https://github.com/spotahome/kooper#motivation

Just my two cents, though 😃

I played around with building an operator myself, and lately I've been looking at this - https://github.com/kubernetes-incubator/apiserver-builder


PyPi using 1.3.9 but latest release on official repository is 1.3.7?

It seems like PyPi's latest OpenEXR version is 1.3.9 but the latest release on the official repository is 1.3.7?  I do not see 1.3.9 referenced anywhere outside PyPi.  How is that possible?
Installing 1.3.9 I am getting an import error, seemingly because of a version mismatch.  However 1.3.7 will not install via pip, I get numerous errors including a "legacy-install-failure".  How can I fix?
Building on an M1 Max Macbook Pro

The version of the Pypi package and the OpenEXR are unlinked, mainly because Pypi enforces to.
I made and uploaded autobuilds for all platforms. just do pip install OpenEXR==1.3.9 and the problem shall be auotmagically fixed


Sema: Allow default arguments to access @usableFromInline decls

The restriction that default arguments be disallowed from accessing @usableFromInline decls is overbearing for library developers who need to write non-trivial code to compute a default value, since it forces them to either write a verbose closure inline in the function signature or expose a public helper function which unnecessarily expands API surface. A @usableFromInline function a more reasonable way to encapsulate a verbose default value computation.
This reverses the semantic changes included in https://github.com/apple/swift/pull/15666.
Resolves rdar://112093794.

@swift-ci please smoke test

I've updated the changes to also remove FragileFunctionKind.allowUsableFromInline since it's always true now.

@swift-ci please smoke test


Made gradle file permission script platform independent

Tested locally on my Window's machine.  Hopefully this will be a fix for: https://github.com/openid/OpenYOLO-Android/issues/41

Codecov Report

Merging #47 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master     #47   +/-   ##
========================================
  Coverage      69.7%   69.7%
  Complexity      415     415
========================================
  Files            58      58
  Lines          2334    2334
  Branches        239     239
========================================
  Hits           1627    1627
  Misses          637     637
  Partials         70      70

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b494d56...814a999. Read the comment docs.


Notes, BPM, ms_per_tick

I have added an alias Notes = Vector{Note}. It feels very natural for me plus I was using it all the time for research. I think it is nice, plus it is only something optional.
I have also added two functions: BPM(midi) and ms_per_tick(midi) which were super important for me, and I think fit perfectly on this package.
Soon I will upload a new package about manipulating music (using MIDI). I had these 2 there but I feel they fit much more here.
Please let me know if you agree, and if yes go ahead and merge!

@JoelHobson okay that is everything. I improved significantly the README as well, I think.

Those sound like excellent utility functions to include. I'm happy to merge your changes, I just have one nitpicking comment.
I'm also really happy that you're still using this library and contributing to it!

@JoelHobson I added an explanation.
 You told me to use them over one and a half years ago when I started using this library!

@Datseris Hahaha, no worries, I just meant an explanation of what 'tttttt' represented. Your explanation is fine. I'll merge this in. Thanks again!


Some visuals like the skybox are glitching

Most of the visuals are glitching out, no clue whats wrong but it seemed like yesterday they were working perfectly fine. Anyone else having this issue?

Idk i use onetap for legit

@HowToMakeFaceitAB
Not helpful at all.

🤔🤔

Elaborate.
Sent with GitHawk

Never mind, it appeared that only in Dust2 it would glitch, the sky boxes wouldent change and generally some skins were taking longer to load than other maps. Could anyone explain why?

Dust 2 uses 3d skybox, which needs to be disabled to change skybox.


krew lists version 0.1.0 while 0.2.1 is available?

A PR is missing?

Up, krew still listing old version, which doesn't show anything on the webview besides the header.


handshake on multiplex subscription

Fixes #567, which extended to all read queries: none of them would cause a handshake, so but they would block on the handshake, so they'd have to wait until a write query or an explicit .subscribe was called. Now we explicitly send the handshake on each request unless we've already sent it.

This change is

In that case, shouldn't we just always send the handshake lazily before the first query instead of also attempting it when the socket is opened?

... and then we could just make hz.connect = () => socket.sendHandshake()?

Good point, I'll make that change. Though we still have a potential problem when subscribing to the socket it may not actually send the handshake. There aren't a lot of good reasons to do that though, and we don't expose the socket externally. When we make the jump to typescript it'll definitely be private.

Ok, I made the fix and then reverted it. It doesn't properly emit the disconnected event when there's an error when you do this.


Binder link does not start

I get an error on Binder when starting the notebooks.
The screenshot of the error is below

Thanks. I guess the problem is that the channels are ccpi/label/dev rather than the stable one.

The environment of binder is updated and the link works.


Support multiple function clauses in anonymous functions

This adds support for anymous functions with multiple clauses.
Before this change, this code would fail to compile:
argument fn
  "foo" -> "bar"
  _ -> "default"
end

Contributor checklist

[x] Bug fixes include regression tests
[ ] Chores
[ ] Documentation changes
[x] Features include unit/acceptance tests
[ ] Refactoring
[ ] Update dependencies

🚀Thank you for your contribution🚀


Renamed Pokemons go back to default when in PC

Putting a renamed pokemon into a PC reverts it back to default name. I have tried it in singleplayer and this doesnt happen, therefore I believe its a server issue (?)

Closing the issue as it seems to be solved.


Updated withings_sleep to match garmin standards

Updated both withings sleep and scanwatch to grain standards. Fixed adherence simulation issues where heart rates were randomly removed instead of blocks being removed. Please note that since we do not have withings sleep extractor, we used the data from the ScanWatch as this was how it was in the original notebook. The data should be the same as the watch since they measure similar sleep metrics.


Camel 13126 for v2.23 swagger

JIRA CAMEL-13126, applied to v 2.23.x

Make this PR agains camel-2.x branch


Optimized oldBehaviors(for:), newBehaviors(for:) and shouldInvalidate…

Optimized oldBehaviors(for:), newBehaviors(for:) and shouldInvalidateLayout(forBoundsChange:)
While chaining multiple map, flatMap and filter calls makes the code very readable, a class that is meant to be used as a library should strive for performance. I have taken the liberty to optimise the aforementioned methods so they don't perform superfluous full passes over collections.
Also, now shouldInvalidateLayout(forBoundsChange:) returns the result of comparing the current bounds.width with the newBounds.width, so the layout can adapt to interface orientation or size class changes.

Some good improvements here. Thank you for this contribution! I didn't really agree with your style of indentation, hope you don't mind. 😉

Oh not at all @roberthein ; in fact I should have "rolled" with your indentation style in the first place :)


Fetch only root certiifcate from cert chain

As we'll be adding an intermediate CA certificate to Fulcio's
certificate chain, we want to only include root certificates in the CT
log's cert bundle.
Signed-off-by: Hayden Blauzvern<EMAIL_ADDRESS>
Summary

Ticket Link

Fixes
Release Note

cc @nsmith5


Always render fenced code blocks

✔ Right now, it's possible to render some highlighted fenced code blocks with this syntax:
```js
doCodeBlock()=>{
console.log('code block');
}
```
  doCodeBlock()=>{
  	console.log('code block');
  }

✔ It works also without specifying the language:
```
this is code block
```
this is code block

❌ But this one doesn't work, the result is the text on one line unformatted:
```bash
git commit -m "feat: code block"
```
git commit -m "feat: code block"
Also this obviously lead to an unformatted line:
```abc
This should be some code block
```
This should be some code block

I haven't looked at whether it's possible to configure the languages, but I think it would be wise that as soon as quotes are used, then the code block is always rendered by default.

I have checked your issue with the current version of the plugin (0.7.3).
Everything seems to work fine, even with the "bash" language hint:

I just tried on a clean install on another computer, and indeed, it works correctly.
I will investigate this evening or tomorrow why my installation poses this strange problem and keep you informed.

Indeed everything is OK on v0.7.3
For some reason, the version I installed was on an old version v0.7
Although I installed the plugin last week, my package manager was not updated on my side.
I should have investigated better from the start 🙄
This ticket can be closed.
Thank you and sorry for inconvenience.


Consider using MEMORY

One possible use is to avoid using __romPos: https://discord.com/channels/710646040331681844/813939516385525790/1173330150226657381

GNU LD docs: https://sourceware.org/binutils/docs/ld/MEMORY.html

A repo using MEMORY: https://github.com/decompals/iQuePlayer-SecureKernel/blob/ddfbfc542a17df6843520060364654698a1dfe78/sk.lcf (https://github.com/decompals/iQuePlayer-SecureKernel/pull/15/files)


TN-1210 Fix assessment status i18n

Explicitly add assessment status strings for extraction
Add translation call to assessment status in /patients list

Hello @ivan-c! Thanks for submitting the PR.

In the file portal/models/i18n.py, following are the PEP8 issues :

Line 91:80: E501 line too long (80 > 79 characters)
Line 104:5: E722 do not use bare except'
Line 106:80: E501 line too long (82 > 79 characters)


design a way for users to request an invite to a project

users might find out that a project already exists at the project creation page. We want to enable a way for them to request to join a project.
maybe depends on #576

related to #253

This will be fixed by #979, but when Kevin and I discussed the design, we decided to use the new feature of the AddProjectMember modal where it can take a user ID, and have the URL to approve the project join request simply be the project page, with URL parameters that will pull up and populate the AddProjectMember modal. That does mean, however, that this issue is now blocked behind the completion of #986, which is where that feature is being added.
So I'm marking this issue as Blocked for now, and moving on to work on something else until #986 is finished (which should be pretty soon).


Make controllers optional

Games usually make their own camera controllers. So I would suggest to put controllers under a feature which is enabled by default.
If you like the idea - I can send a PR for it.

Sounds reasonable. I don't think I would consider the controllers as "bloat" since they are few and small, but it doesn't hurt anyone to make them optional either, so long as they remain default features.

This crate looks modular enough that you should be able to add a LookAngles or LookTransform component to your existing camera, and then have your own motion smoothed out. I'm having trouble figuring out how to do this though. Do you think you can add an example?

@maaku I believe you can take a look at one of the existing controller impls for an example.


Update CDX parser to generate GraphQL nodes

Update the cycloneDX parser to generate the new nodes based on the graphQL changes. Unit tests also have to be updated to match.

@dejanb has started work on this.

Completed with #603  and followup issue #632


Error: corrupted input

I have transferred xz file into JS using XMLHttpRequest. The content type of response is application/x-xz, and I checked response content which seems complete. Then I use LZMA.decomporess(request.response, ...) and got corrupted input error. What is wrong in my case? Thanks in advance.

LZMA-JS doesn’t support xz files, just the raw lzma format. Your best option is probably to provide the data in lzma format on the server side, if that’s possible.

Also fun fact in case anyone runs into this. Python converts to xz format by default when using lzma.compress. To fix this one must pass lzma.FORMAT_ALONE to the format argument to get data compressed in a way that LZMA-JS can use. Ran into this when compressing a gob of binary data in Python that needed to be decompressed by JavaScript for use in a plot.

I got "Corrupt input data" too.
Confirmation
Here is my confirmation code process.
First, I type the following command found at LZMA-JS Advanced Demo.
$ echo "Hello, world." | lzma -3 | hexdump -C | cut -c9-60
  5d 00 00 40 00 ff ff ff  ff ff ff ff ff 00 24 19
  49 86 e7 d6 3b 91 a7 78  42 d7 c2 fe e2 25 55 d4
  ff ff eb 7b 00 00

Then, I use the following Python script.
import lzma

# The output of `echo "Hello, world." | lzma -3 | hexdump -C | cut -c9-60`
s = b'\x5d\x00\x00\x40\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x24\x19\x49\x86\xe7\xd6\x3b\x91\xa7\x78\x42\xd7\xc2\xfe\xe2\x25\x55\xd4\xff\xff\xeb\x7b\x00\x00'

lzma.LZMADecompressor().decompress(s)
# => b'Hello, world.\n'

As you can see, I can decompress successfully.
However, I tried on the result is from LZMA-JS Advanced Demo
5d 00 00 01 00 0d 00 00 00 00 00 00 00 00 24 19
49 98 6f 16 02 8c e8 e6 5b b1 47 f3 de f4 89 ff
ff 30 44 00 00

import lzma

# The output of LZMA-JS Advanced Demo
s = b'\x5d\x00\x00\x01\x00\x0d\x00\x00\x00\x00\x00\x00\x00\x00\x24\x19\x49\x98\x6f\x16\x02\x8c\xe8\xe6\x5b\xb1\x47\xf3\xde\xf4\x89\xff\xff\x30\x44\x00\x00'

lzma.LZMADecompressor().decompress(s)
# Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
# _lzma.LZMAError: Corrupt input data

As you can see I got "Corrupt input data." Maybe my usage is wrong, but I think LZMA-JS probably has a problem.

It looks like there is an error in function $SetDecoderProperties, file lzma_worker.js:
for (i = 0; i < 4; ++i) {
   dictionarySize += (properties[1 + i] & 255) << i * 8;
}

Because of the <<i*8 the dictionarySize goes to enormous values.

Are there any known workarounds for @dongli's, and @nwtgck's corrupted data issue?

Are there any known workarounds for @dongli's, and @nwtgck's corrupted data issue?

Same problem. Testing with xz raw:
# compress into raw format
echo "hello" | xz -c --lzma1=lc=3,lp=0,pb=2 --format=raw > qr.xz

# test decode with xz - prints hello
cat qr.xz | xz -d --lzma1=lc=3,lp=0,pb=2 --format=raw

Try to decode with LZMA-JS
const fs = require("fs");
const LZMA = require("lzma");

const path = 'qr.xz';
const file = fs.readFileSync(path);
const byteArray = new Uint8Array(file);
const decomp = LZMA.decompress(byteArray);
console.log(decomp);

Fails with exception:
➜  RAMDisk node decomp.js
/Volumes/RAMDisk/node_modules/lzma/src/lzma_worker.js:313
            throw new Error("corrupted input");
            ^

Error: corrupted input
    at $init_0 (/Volumes/RAMDisk/node_modules/lzma/src/lzma_worker.js:313:19)
    at $LZMAByteArrayDecompressor (/Volumes/RAMDisk/node_modules/lzma/src/lzma_worker.js:344:9)
    at Object.decompress (/Volumes/RAMDisk/node_modules/lzma/src/lzma_worker.js:2554:29)
    at Object.<anonymous> (/Volumes/RAMDisk/decomp.js:7:21)

@mauron85 I'm having same issue, have you figured the problem?

@xseman For my needs, I switched to https://github.com/cscott/lzma-purejs. It generates proper raw lzma. Only it adds some padding (additional information) to the end of file, but I just trimmed that and can extract with comnand line xz.
Also forked lzma-purejs, but basically I made it es6 module and reorganised some components, so it can be tree shaken.|
https://github.com/mauron85/lzma-purejs


Seeing libc errors with sample

When I try to compile your first example, I see:
error[E0432]: unresolved import `libc::termios`. There is no `termios` in `libc`
 --> C:\Users\jont\.cargo\registry\src\github.com-1ecc6299db9ec823\termion-1.0.0\src\termios.rs:4:9
  |
4 | pub use libc::termios as Termios;
  |         ^^^^^^^^^^^^^^^^^^^^^^^^

error[E0432]: unresolved import `libc::ioctl`. There is no `ioctl` in `libc`
  --> C:\Users\jont\.cargo\registry\src\github.com-1ecc6299db9ec823\termion-1.0.0\src\size.rs:33:9
   |
33 |     use libc::ioctl;
   |         ^^^^^^^^^^^
...

Is there something I need to do to get libc also compiling?  I'm trying to build this on Windows, if that also has an impact.

Unfortunately, it doesn't work on Windows, due to the odd term API.

Ah, I missed the "Supports Redox, Mac OS X, BSD, and Linux (or, in general, ANSI terminals)."
Hopefully you'll be able to add support for Windows also.  I've been looking for a simple terminal crate.

Hopefully you'll be able to add support for Windows also. I've been looking for a simple terminal crate.

I have no Windows machine available for testing, but having Windows support would be very nice. If somebody makes a PR, I'll certainly merge it.

I may look into it.  There's already other term crates, too.  Would be great if they could merge support/feature set...


[Bug]: Gemini Pro is not giving titles to my conversations

What happened?
Hello I was wondering why gemini pro conversations didn't have titles
Steps to Reproduce
Start a conversation with gemini pro
What browsers are you seeing the problem on?
Chrome
Relevant log output
No response
Screenshots

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Neither is Anthropic Claude FWIW, not sure what's special about these two AIs. Is there a config we're missing?

Neither is Anthropic Claude FWIW, not sure what's special about these two AIs. Is there a config we're missing?

No it's not supported for either but I may add the functionality in the next day or two. Closing in favor of the older issue: https://github.com/danny-avila/LibreChat/issues/1422


Fixed pid type error

In generate_form_link function conditions compare pid argurment (type str) and its value (type int), that results in e.g. not providing comment (and not setting it as blocked) to the resulting form

Original code was structured around str, which I forgot to redocument as int :)
Still, this is a valid PR and other projects may already supply str per documentation.
Thanks for noticing!

Original code was structured around str, which I forgot to redocument as int :)
Still, this is a valid PR and other projects may already supply str per documentation.
Thanks for noticing!