feat: use PCM audio data to build a sound

Description
Would be nice to be able to send PCM data to the player and have the player be able to render it. This could be done by extending the SoLoud C++ AudioSource class to have a BufferStream class that can be used as a normal AudioSource sound.
The advantage of this is that the player can be used as a normal [AudioSource] and can play audio data that could be streamed from the web or created directly from Flutter. This new kind of [AudioSource] can then be able to use all the features of the flutter_soloud like filters, faders, oscillators and so on.
Think for example of some AI web services that let you compose voices like this.
Additional Context
Would also be nice to have a way to send PCM data directly to the output device without creating a new [AudioSource]: compose some audio PCM in Flutter and send to the player output. But this is maybe for another feature.

Experimenting with the BufferStream I came up with some good results, but opinions and suggestions are welcome.
What I have done so far is to simulate a WebSocket connection that streams PCM data using ffmpeg.
Here the source code. The only needs to use it, are to have ffmpeg and websocketd installed, and in the main.dart modify final audioPath to point to an audio file on your PC.

Here you choose in which format you want to send the PCM audio data.
In the websocket branch there is an example that demonstrates this feature in `lib/buffer_stream/websocket.dart' file.
https://github.com/user-attachments/assets/e7c443ef-7c06-410a-91a4-804b698a328f
lib/buffer_stream/generate.dart is an example of how to generate PCM audio data within an Isolate and send them to the AudioBuffer.
I have tested on Linux, Android, and Windows. It should work also on Mac and iOS, if you feel good please try on those.
cc: maybe @maks could be interested in this?

@alnitak thank so much for tagging me on this as yes I'm very much interested!
I've quickly glanced at this so far but it looks like exactly what I would like to be able to do: generate and playback PCM data in realtime. But I think maybe I'm missing something here: in the example you are pre-generating the data into a in-memory buffer (the for loop in generateWailingSirenSoundPCM()) as I don't see any callback that would be coming from the native audio system code that requests the audio data on the fly (as it requires it for pushing out to the OS's sound output API)? For me thats the main thing: I want to be able to make musical instruments so need to generate the audio data in realtime with low latency, ie. as the user plays notes via for example a MIDI controller input.


Support Reading Prometheus WAL directly instead of Remote Write as a Sidecar to reduce memory usage

Today using the remote_write to promscale works but it does require prometheus to use more memory and direct network calls. One option seen with other similar projects is to read the Prometheus TSDB WAL directly to then process and ship out to an external store (here would be timescaledb).
Do you think this is something that could be added to make deploying promscale as a sidecar easier and reduce prometheus resource usage?
remote_write memory usage: https://prometheus.io/docs/practices/remote_write/#memory-usage

Thank you for sharing your views @mitchellmaler. 25% increase in memory is significant. I will discuss this with the team.

@mitchellmaler we discussed about this. This method has possibilities of loosing data in case of a crash, which is something not desirable. Still, I think is something nice to have in future if more people request for it. Thank you very much for giving your suggestion.

I am curious on how this differs from today using remove_write, where if Prometheus crashed that promscale would not get the data. Reading the WAL would give you the most us to date data up to the point of Prometheus stopped and in that case it is up to the user to make sure they have it deployed HA so it can tolerate crashes.
Using https://github.com/prometheus/prometheus/tree/master/tsdb/wal you could read the TSDB and keep it in memory/disk to be sent to Timescaledb. This would reduce the memory usage of Prometheus substantially and if it crashed you would be able to read up to the last write to the TSDB (using shared volume) and another HA Prometheus could take over if the threshold is met.

I am curious on how this differs from today using remove_write, where if Prometheus crashed that promscale would not get the data. Reading the WAL would give you the most us to date data up to the point of Prometheus stopped and in that case it is up to the user to make sure they have it deployed HA so it can tolerate crashes.
Using https://github.com/prometheus/prometheus/tree/master/tsdb/wal you could read the TSDB and keep it in memory/disk to be sent to Timescaledb. This would reduce the memory usage of Prometheus substantially and if it crashed you would be able to read up to the last write to the TSDB (using shared volume) and another HA Prometheus could take over if the threshold is met.

Reading the WAL would give you the most us to date data up to the point of Prometheus stopped

WALs are flushed at a regular interval. If Prometheus comes down before that, then it will lose the data again. Remote-write works using wal reader, which means, as soon as a segment is written, it is streamed. This means that both have almost similar chances of loosing data and its amount.

keep it in memory/disk to be sent to Timescaledb

I agree, this has some advantages as well (low mem usage). Would be definitely good to have once we have the bandwidth.

closing this issue for now, as we haven't received multiple user requests. Sending data using remote_write has been the most widely used integration with Prometheus.
Would consider adding this to our backlog, if we receive more user requests/feedback to support reading from Prometheus WAL.
Thanks!


Error event streams not showing up on redis.client.on('error');

This is happening on mac osx, node_redis 2.6.0-2
var redis = require("redis"),
client = redis.createClient({
retry_strategy: function (options) {
console.log(options);
return new Error('The server refused the connection');
},
port: 80
});
client.on("error", function (err) {
console.log("Error " + err);
});
In the past, version(2.5.3), this would emit an error, however, now, after the upgrade, no error is emitted. Instead, just the "end" event is emitted. Is that supposed to be how it works...?

Hm, this was an intended change but I guess it's to much of a BC to keep it for the actual release. I tried to prevent any error being emitted as long as it is handled otherwise and in this case you already know that you end the connection, as the retry_strategy returns an error and therefore node_redis is always going to close the connection and stop retrying.

I think that may not be a good assumption right? What if you are already connected and then you get a hset error for some reason. Shouldn't that throw an error? I noticed that its not doing that if the retry_strategy is set too....

@ngoctranfire that should not throw an error. The command errors should be handled in the callback and not being emitted. This is only the case if you did not set a callback but a Redis error is returned to make sure the user is able to notice such an error. And this does not change with retry_strategy used or not. The only thing that changes in that case is that a retry is not going to emit an error anymore but return it in that very function.
Therefore there's a very explicit way how to handle those.

@BridgeAR That makes a lot of sense now. I was using the promise versions and then combining it with async/await in the ES7 babel version. The errors were just not thrown and I need to use try{}catch{} statements. That was my issue. Thanks for the clarifications!


add a func of send  msg to peer

this func will be used after fully compatible with abc

unit test passed.
re-run func test will ok


Add git node staging command

Add a new git node staging command that automates cherry-picking commits into staging branches.
It works by cherry-picking all commits that have no conflicts and skipping any commits that have conflicts while automating the sending of GitHub PR interactions to request backport for these commits, sending a message to the original PR and properly labelling it.
Usage:
Fetches a commit list using branch-diff and automatically cherry-picks / skips commits based on whether or not they land cleanly:
  git node staging

Sets a custom reporter to be used at the end of the automated operation:
  git node staging --reporter=markdown

Limits to 10 the number of commits to be cherry-picked:
  git node staging --pagination=10

Defines the release line (usually this can be inferred from the ncu.branch value as it should not be required most of the time):
  git node staging --releaseLine=22

Automates the backport request message, this won't run any of the branch-diff or cherry-pick routines. Useful for when you removed a faulty commit from the branch and want to signal to PR author and collaborators that commit now needs backporting, just use its PR#:
  git node staging --backport=12345

The skipGH option avoids gh hitting API requests. As such it enables for running a local-only / dry-run if the releaser wants to validate the final result of the staging branch without having to incur on all the automated messages spamming the Pull Requests:
git node staging --skipGH

Report
A report is generated at the end of operations listing information on the commits that were successfully cherry picked and commits that failed to be cherry-picked due to conflicts.

[!IMPORTANT]
Below is a short example of how these reports look like. The following report was generated for an operation that was limited using --pagination=10

Cherry-pick Report
6 successfully cherry-picked commits:

a21be0294d0 build,win: enable pch for clang-cl windows, build, v8 engine, tools, needs-ci, gyp
c7f82ec9780 doc: fix the return type of outgoingMessage.setHeaders() http, doc, author ready
d8c552a060b doc: edit onboarding guide to clarify when mailmap addition is needed doc, author ready, commit-queue-squash
a5590083cdd node-api: add napi_create_buffer_from_arraybuffer method buffer, c++, node-api, needs-ci, commit-queue-squash
6c836aa97ee meta: bump step-security/harden-runner from 2.9.1 to 2.10.1 meta, dependencies, github_actions
47b6c6748ba meta: bump github/codeql-action from 3.26.6 to 3.26.10 meta, dependencies, github_actions

4 commits that failed to cherry-pick:

71785889c89 lib: prefer logical assignment lib / src, author ready, needs-ci
4988bb549e7 tools: enforce ordering of error codes in errors.md tools, errors, author ready, dont-land-on-v18.x, dont-land-on-v20.x
9f9069d3130 fs: fix linter issue fs, fast-track, author ready, needs-ci, commit-queue-squash
90f56dbad9c module: throw ERR_NO_TYPESCRIPT when compiled without amaro util, module, errors, author ready, needs-ci, strip-types

More
The automate cherry-pick logic also includes local persistency of the ongoing commit list, in case a fatal error happens during the command execution, it's possible to resume after cleaning up the git repo state by running git node staging again.

I understand why you are doing it but I'm afraid of the notification spamming that this generates when there are many conflicts. It's also actually not a good idea to ask for backports on most PRs because landing backports in the wrong order increases the risk of conflicts.

I understand why you are doing it but I'm afraid of the notification spamming that this generates when there are many conflicts. It's also actually not a good idea to ask for backports on most PRs because landing backports in the wrong order increases the risk of conflicts.

I was afraid of it too. Maybe collapse all the messages into a single issue and add the backport-requested label to each one that might reduce the spamming?

I really like that! It would solve many problems:

No more spam
Single place to collaborate on backports
Encourage doing the backports in the correct order
More visibility, by having an open issues instead of comments on merged PRs

Right, I agree that having a single place to manage all the backports has many benefits, I think a simple way to tweak the implementation is to use thegh cli to open a new issue with that markdown report in which the commits that failed to cherry-pick section is the list of commits that need backporting in their expected order. Then the releasers and any other collaborator could work on that.
That said, I still believe that automatically placing the label is a fundamental step that needs to be part of the logic here but we can remove the automatic-commenting to avoid spamming.

@targos to reinforce on the idea here, my hope is that this is going to be a much more sane workflow for the releasers.
As anedoctal evidence, for the current release I'm working on I signed off for the day after running the script (which spammed a bunch of comments and added all the labels) and the next day when I woke up in the morning I noticed that @aduh95 jumped in and landed a dozen of the straightforward backports. This happened without any coordination but it does reinforces my believe that if we put a little bit more structure around the process we can have a workflow that is going to be way less demanding on a single releaser.
And of course this new workflow won't prevent the releaser themselves to go through the list of backport-requested commits on their own time and land the straighforward conflict resolutions.

I agree with everything you said.

FWIW I'm trying to implement some automation in https://github.com/aduh95/node/commit/38125cf8c8d9c87f6425be92df5045e22c226600, but it's failing atm

FWIW I'm trying to implement some automation in https://github.com/aduh95/node/commit/38125cf8c8d9c87f6425be92df5045e22c226600, but it's failing atm

that's awesome as the end goal is to have it running in a GitHub action!
do you believe you can tweak it easily to use git node staging instead?

What's missing to have this PR released? Can we land it and create an action to run this once per week to a specific release line? (I suggest v23.x)

As we discussed in the Release meeting, I think that by default, the tool should ask before taking any non-local action (add a label, add a comment), and there should be CLI flags to disable those prompts if e.g. you know you only want to add label and no comment. It should also give the opportunity for the backporter to fix the conflict.
This is not a blocking concern for landing this PR, though it would be a blocking concern to document git node staging as the de facto tool for backporting.

Please tell me when this is ready for a deep review.

Dropping a quick status update here since I just put in some time on this again over the weekend. It now includes many major changes we discussed here such as:

No longer automatically comments on PRs that were unable to land cleanly.
No longer automatically skips commits that were unable to land cleanly, by default the tool now works in an interactive mode that exits when finding a commit that is unable to land cleanly and allows you to manually fix it. In cases the releaser is able to fix a commit, running: git cherry-pick --continue & git node staging --continue will resume the cherry-pick to staging session. When the releaser is unable to fix it manually, running: git cherry-pick --skip followed by git node staging --skip will skip that & label that PR with the backport-requested label, optionally the command git node staging --backport <PR_ID> is available to automatically send a comment to that PR, letting the author know that the manual backport is necessary. (This new behavior is inspired by the feedback from @aduh95)
The behavior of automatically skipping commits that were unable to land cleanly is now an opt-in setting, it can be used with: git node staging --autoSkip
Reports can now be posted automatically to a GitHub issue at the end, the releaser can choose different destinations such as a file or stdout but by default a prompt will ask confirmation to open an issue on GitHub.

Please tell me when this is ready for a deep review.

@targos I just used it again to help me prepare https://github.com/nodejs/node/pull/56329 - It currently still lacks tests in order to land but it should be in a good enough state for review! As in, I don't plan to change anything drastically at this point.


Selected elements in compare view are covered by a white box

Issue and tracking information
When comparing models or merging them if the user clicks on a diff on either the left- or right-hand panes of the Content Merge Viewer, the element in the diff gets covered by a white rectangle hiding the element's name and icon.
Developer's time Estimated effort to fix (hours):
Developer's Actual time spent on fix (hours)
Issue reporter to provide a detailed description of the issue in the space below
Steps to reproduce

Add or clone the the git repo in the modelmergereproducer.zip from the cx-reproducers/bug/415 repository
Import the project
Open the "Git History" view
Checkout the "conflicting_branch"
Right-click on the "feature_branch" and select "Merge"
A dialog appears saying that there is a conflict.
Close the dialog and go to the "Git Staging" view
Right-clock on the BasicPubSub.uml file under "Unstaged changes" and select "Merge Tool"
After a little bit, model differences are computed and shown. The Content Viewer should show the following:

Click on the icon next to "<Package> defn" to unfold

Click on the diff for "<CXEnum> Color", you see the following:

Expected result
The selected element's name and icon should be visible.
Actual result
The selected element is covered by a white rectangle hiding its name and icon.
Environment
Ubuntu 20.04
CentOS 7

NGC has approved this issue to be investigated

An update on this issue.
The problem is in EMF Compare. I have a working solution, but I'm having some trouble integrating it. I'm trying to integrating through an extension point, but EMF Compare is not picking up my extension, and this lead me to discover that it is also not picking up a critical extension from Papyrus Compare. So it looks like we are loosing some Papyrus Compare functionality. I'll see if I can resolve this easily or if we need a separate issue. If I can't resolve it easily, I can still do a patch, but in general it's preferable to override behaviours via extensions rather than patches, as doing patches incurs in technical debt that we would have to pay if we need to upgrade EMF Compare.

Thank you for the update. I definitely agree that where possible, we should stick to extensions if possible. It is frustrating that EMF Compare is having this issue. Does it have the issue in the version of papyrus we are using in the v2 stream, or the v3 stream, or both?

Sorry, I forgot to answer. The issue happens on both streams. That's probably because they both use almost the same version of EMF Compare, where the error comes from. The old Papyrus comes with EMF Compare 3.3.11 and the newer one with 3.3.24, so I'm guessing they haven't gotten around to fix this.
As an update, as I mentioned earlier, I'm trying to override the EMF Compare code with an extension, and I mentioned that the Papyrus-Compare extension which I want to override wasn't being picked up. Well, I have more info on this. It turns out that it's not picked up when launching the comparison from the Git History view, but it is picking it up when launching the comparison from the Synchronize view (which only allows you to compare workspace items with some commit, and not comparisons between two commits). I'm still not sure how to override this extension yet, but at least now I know that there are two different scenarios, and it looks like the extension selection depends on the type of elements selected (two commits from the Git History View vs. a Papyrus model file in the workspace and a commit).
In hindsight it's not entirely surprising, as I remember that Philip, the author of Papyrus Compare, told me that support for EGit was limited.
So the matter seems to be about how extensions are registered and how so-called "ViewerDescriptors" (the thing that describes the Viewer to be overriden) are selected depending on the type of elements selected.

Depends on: Issue #501

I've marked this issue as depending on #501 because it turns out that issue is concerns how compare/merge viewers (and their extensions) are picked up to be shown when doing a comparison from the Git History view. Since the solution to this issue requires overriding a viewer (so that selected elements are displayed correctly, and Papyrus Compare functionality is active), we need to solve that one first.
I'll add more details on that issue.

I'm a bit overdue with an update on this issue.
As I mentioned earlier, in order to fix this one, I need to override some behaviour from EMF Compare. This lead me to discover Issue #501, at the heart of which is how certain extensions are picked by Eclipse Compare and EGit. In particular, in order to override the EMF Compare behaviour, I needed to implement an extension which was not being picked up by EGit and Eclipse Compare. This led me to a deep dive into the early stages of a compare operation, and in particular to try to understand the differences between a comparison launched from EGit and one from the Synchronize view. Both turn out to be different. Papyrus Compare works with the Synchronize view but not with EGit, so when using EGit, you get a plain EMF Compare operation. That might be enough for many purposes, in particular when comparing .uml files, but it will skip the Papyrus Compare features. But the bigger problem with respect to this bug, is that the extensions picked up in the two scenarios are different, and neither picks the extensions that would provide the fix. After a long time working on that, I think I have exhausted all alternatives to try to make Eclipse Compare, EGit and EMF compare, pick up the right extensions. It seems like the code that I need to override, cannot be overridden through an extension point. So at this point the only alternative left is to do a patch. That's what I will try to do now.

Thank you for the update, we appreciate your perseverance on this issue. Do we think there will be any chance that such a patch (if you are able to create one) would be accepted by the community eventually?

I'm not sure. The EMF Compare team seems rather inactive. Their last commit was 3 months ago, and all the commits from last year were simple releng work. No new features or bug fixes. There only seem to be three developers left. The activity in the forum is also very low. I asked a question last June and it's still the third newest. So the project seems to be, if not dead, in a kind of comma. Also, my experience with the Xtext team left me quite discouraged. When I proposed them to provide the patches I made replacing log4j1 with log4j2, their response was lukewarm at best. They asked for more details, which I provided extensively, and then they went quiet. Because of that, I desisted from trying to give patches to other projects, because if one of the projects rejects it, then log4j2 will still pop-up in Papyrus. It seems like the open-source community around Eclipse is dwindling, or is less enthusiastic. I know that a couple of projects (including Papyrus) have left the Eclipse Simultaneous Release train, due to its new high frequency release timeline (releases every 3 months). This causes a lot of problems for many projects which have only a very small number of active developers. Aside from the technical challenges of keeping up with the release cadence, I think there is a bit of an attrition factor. Just recently, the lead (and only developer) of the OCL project decided to quit. So whether the community accepts patches might depend on a few things. If the patch is simple and straightforward and doesn't make substantial changes, meaning it requires very little effort from the relevant project's core team, then there is a chance. But more substantial changes may not be so easily accepted. For this particular bug, I may be able to make it simple enough. But the patches required to fix Issue #501 go deeper, so I wouldn't count on those being accepted.

An update.
The fix works on streams/v2 after introducing a feature patch for org.eclipse.emf.compare.rcp.ui. However, on streams/v3 it seems to have opened a dependencies can of worms in the build. It doesn't seem like an easy solution is possible. I'm investigating possible solutions.

I think you can merge them. We won't have the ability to test them anytime soon.


MacRoman-CP437

I came across a file that went through an old Mac and an old PC, and is now on a Linux ext4 filesystem.
Filename string bytes as given by debugfs: journ\xc3\x87e.txt displayed as "journÇe.txt" in UTF-8. The correct filename should have been "journée.txt".
In [11]: bytes('journÇe','utf-8')
Out[11]: b'journ\xc3\x87e'

In [12]: bytes('journÇe','macroman')
Out[12]: b'journ\x82e'

In [13]: bytes('journée','cp437')
Out[13]: b'journ\x82e'

In [14]: bytes('journée','utf8')
Out[14]: b'journ\xc3\xa9e'

The filename is in french.
French 15th most frequent letter (> 2%) is diacritic e: é.
French bigrams containing the letter é are frequent too, with 4 in the top 100.
So there is a point to solve this problem.
Currently FTFY is not detecting/correcting the string:
In [68]: print(ftfy.fix_encoding(u'journ\xc3\x87e'))
journÇe

In [69]: print(ftfy.fix_text(u'journ\xc3\x87e'))
journÇe

We can use iconv to fix the filename:
➜  ls journÇe.txt
journÇe.txt
➜  ls journÇe.txt | iconv -f utf8 -t mac | iconv -f cp437 -t utf8
journée.txt

examples in the wild: google search for libertÇ

I'm going to consider this issue part of #18. That's not to say it's not important -- it's just hard. I think that #18 is the biggest functionality gap in ftfy, and I welcome any suggestions at all for how to address it.

Hi Rob, first of all my compliments for this awesome framework.
I'd like to share my thoughts on this matter. I ain't a programmer or a ML specialist but I do have a bit of technical knowledge of online software, data frameworks and applications. Nor do I exactly know how FTFY works on code level, I just think I understand the general concept.
The first question is do you want to setup more context to decide the possible solution? Looking at a single problem or a single string might give you an idea on a solution but then you presume all mojibake problems are equal, which as we can see is not always true.
So in the cases where the solutions might not be that easy you need to know more before you can be sure so fix the issue otherwise you might change something wrong.
sidestep: Now that is already a good question. Is the fix worst then the error if it is done. Lets say of all the issues FTFY can't fix that it would make the best guess, that 80% is guessed right. Then I as a user should have that option to take that risk. So a setting how secure/sure FTFY must be before fixing the mojibake. /sidestep
Now to get more context you can add a ton of options. Some hard, some more easy, most cost processing time etc etc. But I always like the easy solutions, something that is already there and might not be the magical solution but at least makes a significant dent in the problems.
So what about that the user can give context. For example in our case we know what language the string is and therefor/also we know the allowed characters. That information should give FTFY more information about the possible solution.
Second idea that I had is: with a bit more effort is to send more strings to see if in lets say 100 unfixable strings with 2 or 3 or 4 fixes can be solved. As this gives you a ton of more context on the problems in general, it must give you far more statistics on the probability if the fix will be correct.
Third idea is: Work together. And I mean FTFY with a human. And make a start in ML. So let FTFY suggest solutions and let the user decide. Based on these decisions suggest solutions in other equal mojibake problems and if more then x times this fix has been decided that it automatically applies this next time. Again with this you set context and "learn" FTFY a fix for your case.
I hope this shines some light on your question or at least gives you some new ideas.
Just my 2 cents.


New ts0601_garage quirk

I have created a new file for the t0601 garage quirks.
Just one for now.
Will require changes in HA to expose the button entity.
Reference:

https://github.com/zigpy/zha-device-handlers/issues/1260?notification_referrer_id=NT_kwDOAehH_bMyOTEzNTExMzgzOjMxOTk5OTk3#issuecomment-1315838957

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at ?%

Totals

Change from base Build<PHONE_NUMBER>:
0%

Covered Lines:

Relevant Lines:
0

💛  - Coveralls

Codecov Report
Base: 83.55% // Head: 83.58% // Increases project coverage by +0.03% :tada:

Coverage data is based on head (b6e5fe0) compared to base (1891e3f).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #2107      +/-   ##
==========================================
+ Coverage   83.55%   83.58%   +0.03%
==========================================
  Files         252      253       +1
  Lines        8128     8146      +18
==========================================
+ Hits         6791     6809      +18
  Misses       1337     1337

Impacted Files
Coverage Δ

zhaquirks/tuya/ts0601_garage.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Thanks David


proposal: library of common actions

The idea is to provide a python package (or packages), which would offer tools
for simple actions (not tasks) such as:

common file operations (copy, move/rename, mkdir, rmtree..)
extended file operations (create an archive, unarchive...)
great inspiration can be found in library of such tools for Gulp or Grunt.

The package would be external to doit to keep the core clean.
And it would strongly simplify (and shorten) amount of code to put into dodo.py.
Writing cross-platform dodo.py would be much easier.
Proposed resolution of this issue:

set up new repository (e.g. doit-action-library)
mark it as experimental one
play with this topice there, possibly discuss ideas and issues in given repository issues.

Well there is https://github.com/pydoit/doit-sys and https://github.com/pydoit/doit-web
I am not sure I ever released/documented this things.
If you look projects like Ansible/Grunt and others these kind of packages have thousands of
pull-requests.
In doit it is so easy to write your own that I am not sure it is worth to have a third package.
It would be acceptable only if:

it was completely configurable (so users do not need to patch).
self documented, otherwise the complexity to look the docs is bigger than writing a bit of glue code.

Also I was doing some experimentation when I wrote those.

There can be few levels of such libraries:

level 0 "snippets": collection of snippets providing typical constructs in dodo.py, preferably with pure python
level 1 "pure python lib": pure python (without calling shell commands). This allows for cross platform usage, especially when commands differ on platforms. This was the idea.
level 2 "adapter to other cli": deploying some other tool, e.g. called via command line. If the tool is easy to install and use and it has wide set of features reusable in our dodo.py files, we can gain a lot just by joining doit forces with that library.
level 3 "whatever comes": library of any sort of actions disregarding Linux/Windows portability.

I am aware of complexity of such task and it can be disaster to usability and create too much requirements to our support. For that reason:

if we do it, we shall experiment first without going to big scale
we shall not be responsible for maintaining those libraries (gulp and grunt also do not maintain the libs on their own, they just allow publishing, searching and using the tools).
we shall reuse existing infrastructure (as pypi.python.org etc) with minimal support to features we feel as required for that purpose.

I consider this as possible long term goal.


Upgrade to Ruby 2.2.2

Neither kgio 2.7.4 or json 1.8.0 would compile, so bumped both
versions. kgio is only bumped via unicorn, so upgraded that too.
Syck is no longer provided, so bump safe_yaml to accommodate this.
Trello: https://trello.com/c/sgAorLlb/58-upgrade-signon-to-ruby-2-2-medium

:fireworks:


Fix BigQuery docs build

The snippet references needed to be fixed up to include the new optional resource parameters.

No problem - but I won't merge this until #4554 is in, and I can add a second commit to make the equivalent changes.


[FEATURE REQUEST] Create var "rsyslog_server" to specify explicit server

Define a default var rsyslog_server for collections/logging/roles/rsyslog/
Context
We have an rsyslog_server outside of the bluebanquise iceberg
We would like to be able to point our rsyslog_clients to it explicitly rather than it being inferred from networks[j2_node_main_network] variables
Feature
In collections/logging/roles/rsyslog/defaults/main.yml
Add default variable that is set to null
rsyslog_server:

In collections/logging/roles/rsyslog/templates/client.conf.j2
Add accommodation to check use rsyslog_server if bluebanquise variables are not present (see additional elif clause below).
{% if networks[j2_node_main_network] is defined
      and networks[j2_node_main_network]['services'] is defined
      and (networks[j2_node_main_network]['services']['log4'] | default(networks[j2_node_main_network]['services']['log'], true)) is defined
      and (networks[j2_node_main_network]['services']['log4'] | default(networks[j2_node_main_network]['services']['log'], true)) is iterable %}
    {% for rsyslog_server in (networks[j2_node_main_network]['services']['log4'] | default(networks[j2_node_main_network]['services']['log'], true)) %}
        {% if (rsyslog_server.ip4 is defined and (rsyslog_server.ip4 | ansible.utils.ipaddr)) or (rsyslog_server.hostname is defined and rsyslog_server.hostname is not none) %}
# ### begin forwarding rule ###
# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$ActionQueueFileName fwdRule{{ loop.index }} # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
        {% if rsyslog_server.ip4 is defined and (rsyslog_server.ip4 | ansible.utils.ipaddr) %}
*.* @@{{ rsyslog_server.ip4 }}:{{ rsyslog_port }}
        {% elif rsyslog_server.hostname is defined and rsyslog_server.hostname is not none %}
*.* @@{{ rsyslog_server.hostname }}:{{ rsyslog_port }}
        {% endif %}
# ### end of the forwarding rule ###
        {% endif %}
    {% endfor %}
{% elif networks[j2_node_main_network]['services_ip'] is defined and (networks[j2_node_main_network]['services_ip'] | ansible.utils.ipaddr) %}
# ### begin forwarding rule ###
# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$ActionQueueFileName fwdRule # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
*.* @@{{ networks[j2_node_main_network]['services_ip'] }}:{{ rsyslog_port }}
# ### end of the forwarding rule ###
{% elif rsyslog_server is not none % }
# ### begin forwarding rule ###
# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$ActionQueueFileName fwdRule # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
*.* @@{{ rsyslog_server }}:{{ rsyslog_port }}
# ### end of the forwarding rule ###
{% endif %}

Dear @vedmonds
Apologies for the delay of my answer, I was totally off during 3 weeks.
Ok, I understand your need.
You could precedence things for this specific role, by specifying custom vars in the playbook for the role at call, like:
- role: rsyslog_server
  vars:
    networks:
     ...
    j2_node_main_network: net-admin

This would answer your need.
But I understand an optional direct way would be simpler.
Ok for me ! :blush:
I will do this asap !

Fixed in https://github.com/bluebanquise/bluebanquise/pull/949 :slightly_smiling_face:
Let me know if it doesnt work as expected.

Sorry for the delayed response.
This works! Thank you! :)


header-tabs don't work on mobile

https://ui.mantine.dev/component/header-tabs/
click burger icon not tabs appears

Hi, did you by any chance find the solution? I'm facing the same issue.

see https://github.com/gengjiawen/ui-playground

@rtivital @ajnart @leochoo Can you please help triage the issue?

The only header that works is the Mega Menu one. Everything else refuses to open, the animation on the hamburger triggers but nada.


feat: add date suffix in addition to UUID suffix for a well organized data lake (GCS Load Patten)

Following #72, the BigQueryGcsStagingSink Class now references {self.table_name}__{uuid.uuid4()} as the table in DEFAULT_BUCKET_PATH = (     "gs://{bucket}/target_bigquery/{dataset}/{table}/extracted_date={date}/{batch_id}.jsonl.gz"
https://github.com/z3z1ma/target-bigquery/blob/2d59eae0aa4a5468ed8ba5d04e8d239e05e373ee/target_bigquery/core.py#L310-L319
This means that when using the GCS load patten (other than the default append), a new entry is created with a different UUID. Since the 'directories' in a GCS path are not sortable by date and only by name, this commit removes any form of date sortability. Prior to #72, the table name was created via {self.table_name}__{int(time.time())} which gave a form of sortability on date.
After the change to UUID in the table name, this is now the GCS output:
tablename__UUID
    extracted_date=YYYY-MM-DD
        data.jsonl.gz
tablename__UUID1
    extracted_date=YYYY-MM-DD
        data.jsonl.gz

This would be the desired output that aligns with the description of the GCS Staging Load patten as 'a well organized data lake' in the README.md and also complying with the reason for #72.
tablenname__YYYYMMDDHHMMSS__UUID
    extracted_date=YYY-MM-DD
        data.jsonl.gz
tablenname__YYYYMMDDHHMMSS__UUID1
    extracted_date=YYY-MM-DD
        data.jsonl.gz

This would ensure a well organized data lake. Another option is to implement a ULID instead however the above approach has the benefits of having a more human readable date combined with the uniqueness of a tablename to comply with the reason for #72

For a ULID implementation instead, this will require adding the python-ulid package to pyproject.toml. The latest version of python-ulid = "2.2.0" is compatible with Python >=3.9 so it will not be satisfied with the current Python poetry dependency of python = "<3.12,>=3.8" unless this is bumped to ``python = "<3.12,>=3.9". The version of the python-ulid` package that is compatible with the current Python poetry dependency is 1.1.0.

This is sufficient. We can explore ULID in another issue. But this is a good callout and catch. I didnt think about it when I merged but that was probably the thought originally 😄


React 18.13.x compatibility

Currently the package json lists react-dom as dependency with ^18.12.0 while react is pinned to 18.12.0. Why is this so?
In my project I am using react and react-dom 18.13.1 and this causes the plugin to use react 18.12.0 and react-dom 18.13.1 so I am getting this warning:
└─┬ @etchteam/storybook-addon-status 4.2.4
  └─┬ react-dom 18.3.1
    └── ✕ unmet peer<EMAIL_ADDRESS>found 18.2.0 in @etchteam/storybook-addon-status

Hey @jbojcic1 we've fixed this in v5, thanks for the report!


[Bug-FE][UI Next][V1.0.0-Alpha]Project workflow import error report

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened
Project workflow import error report

What you expected to happen
Project workflow import is normal
How to reproduce
Project workflow import error report
Anything else
No response
Version
dev
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This seems to be no problem.


fix: patch set-output command

:tada: This PR is included in version 1.8.25 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Support use for background image or provide example code for it

I tried to use it for background image but it doesn't work. I'm new to react so I'm not really sure if I'm doing it correctly:
   render() {         return (
               <FullsizePicture                 sources = {[                     {                         srcSet: "https://images.unsplash.com/photo-1470619549108-b85c56fe5be8?dpr=2&auto=format&w=1024&h=1024",                     }                 ]}             >
Welcome!!!
                                </FullsizePicture>
         );     };
img is a void element tag and must neither have children nor use dangerouslySetInnerHTML.
in img (created by Img)
in Img (created by Picture)
in picture (created by Picture)
in Picture (created by FullSizePicture)
in div (created by PictureWrapper)
in PictureWrapper (created by FullSizePicture)
in div (created by Wrapper)
in Wrapper (created by FullSizePicture)

@mjamil0 this is now supported in v3, please see example at https://codesandbox.io/s/github/braposo/react-responsive-picture/


Update PHPCS

as requested in https://github.com/phpstan/phpstan-strict-rules/pull/206#issuecomment-1475835384

Failing here.

already on it, thanks

Thank you!


⚠ ClamAV has degraded performance

In 705a812, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 82 ms

Resolved: ClamAV performance has improved in 86ec75f.


Clarifying Implementation Steps of Chat 5.1

🗣 Description
Updated 5.1 steps for editing rules for clarity.

💭 Motivation and context
Closes #448
Based on Internal Discussion about clarifying steps for rule configuration polices

🧪 Testing
N/A no code changes.

✅ Pre-approval checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[x] If applicable, All future TODOs are captured in issues, which are referenced in the PR description.
[x] The relevant issues PR resolves are linked preferably via closing keywords.
[x] All relevant type-of-change labels have been added.
[x] I have read and agree to the CONTRIBUTING.md document.
[x] These code changes follow cisagov code standards.
[x] All relevant repo and/or project documentation has been updated to reflect the changes in this PR.
[ ] Tests have been added and/or modified to cover the changes in this PR.
[ ] All new and existing tests pass.

✅ Pre-merge Checklist

[ ] This PR has been smoke tested to ensure main is in a functional state when this PR is merged.
[ ] Squash all commits into one PR level commit using the Squash and merge button.

✅ Post-merge Checklist

[ ] Delete the branch to clean up.
[ ] Close issues resolved by this PR if the closing keywords did not activate.

Since we have decided to move DLP to common controls I will close this PR


String Index Out Of Range (bare) / 401 (with /oauth2/default) - when using example

I can not run the server with "our company" configuration and am unsure why. After creating an SPA Client in Okta with OpenID Connect I clone the project. I adjust the configuration as follows:

http://localhost:4200/implicit/callback set as Login redirect URI in Okta
Adjust my application yaml and app module typescript as shown below
Test if https://company.okta.com/.well-known/openid-configuration works (it does with the real company name)

application.yml:
    okta:
      oauth2:
        client-id: 20-digit-Client-ID-string
        issuer: https://company.okta.com

app.module.ts:
    const config = {
      issuer: 'https://company.okta.com',
      redirectUri: 'http://localhost:4200/implicit/callback',
      clientId: '20-digit-Client-ID-string'
    };

Java version:
java version "1.8.0_192"
Java(TM) SE Runtime Environment (build 1.8.0_192-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.192-b12, mixed mode)

Maven version:
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T20:41:47+02:00)
Maven home: /usr/local/Cellar/maven/3.6.0/libexec
Java version: 1.8.0_192, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/jre
Default locale: en_NL, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.6", arch: "x86_64", family: "mac"

JAVA_HOME:
/Users/m.ton/.jenv/versions/oracle64-<IP_ADDRESS>

If I then run ./mvnw spring-boot:run from the server folder I get the following error:
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.1.0.RELEASE:run (default-cli) on project demo: An exception occurred while running. null: InvocationTargetException: Error creating bean with name 'okta.client-com.okta.spring.config.OktaClientProperties': Could not bind properties to 'OktaClientProperties' : prefix=okta.client, ignoreInvalidFields=false, ignoreUnknownFields=true; nested exception is org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'okta.client.org-url' to java.lang.String: String index out of range: -1 -> [Help 1]

If I change my config to the below "default" from the clone my backend runs just fine.
application.yml:
    okta:
      oauth2:
        client-id: 0oagqzcu86BBOq2jF0h7
        issuer: https://dev-737523.oktapreview.com/oauth2/default

Obviously I can't onboard users that way haha, but it runs. If I use
https://company.okta.com/oauth2/default by the way (this isn't what is shown in the Issuer field in Okta, but just in case) I receive the following error:
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.1.0.RELEASE:run (default-cli) on project demo: An exception occurred while running. null: InvocationTargetException: Failed to process import candidates for configuration class [org.springframework.boot.autoconfigure.security.oauth2.resource.OAuth2ResourceServerConfiguration]; nested exception is java.lang.IllegalStateException: Error processing condition on org.springframework.boot.autoconfigure.security.oauth2.resource.ResourceServerTokenServicesConfiguration$JwkTokenStoreConfiguration: 401 Unauthorized -> [Help 1]

Could you tell me what I can try/provide to get it running on my machine? I'm at my wits end. Using Java11 + the Issuer as shown in Okta gives the same error, albeit formatted a little differently:
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:2.1.0.RELEASE:run (default-cli) on project demo: An exception occurred while running. null: InvocationTargetException: Error creating bean with name 'okta.client-com.okta.spring.config.OktaClientProperties': Could not bind properties to 'OktaClientProperties' : prefix=okta.client, ignoreInvalidFields=false, ignoreUnknownFields=true; nested exception is org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'okta.client.org-url' to java.lang.String: begin 0, end -1, length 25 -> [Help 1]

Hey @MarcelTon!
Okta has two different types of issuers one for the Okta Dashboard/Admin console this is typically formatted as https://company.okta.com the other is for API Access Management (any of your applications) and is typically seen as: https://company.okta.com/oauth2/default
Our Spring integration performs local validation of the access token by default, and this can ONLY be done for the API Access Management issuer (so this is likely why one worked and the other didn't)
You can work around this by validating the access token remotely by setting the property okta.oauth2.localTokenValidation=true (or equivalent yaml).
That said, Spring Security 5, does NOT yet support remote token validation. So this is a bit of a dead end for the short term (and because of this, this property will NOT work with our major next release)
The easiest path forward is to use the /oauth2/default style issuer.
Which leads us to the 401, I'm not following this part completely, you mentioned said it worked well, but in some cases you get a 401?  Can you clarify this part?

Hi @bdemers , thank you for your quick response.
My title was unclear, I will try to clarify on the 401 (1 and 2) and would like to ask a follow-up question regarding the API Access Management issuer (3).

When using the config issuer: https://company.okta.com for the server I receive the InvocationTargetException with the "String index out of range"
When using the config issuer: https://company.okta.com/oauth2/default for the server I receive the IllegalStateException with the "JwkTokenStoreConfiguration: 401 Unauthorized"

This is reproducable behaviour simply done by a fresh checkout, changing the two config parameters and running ./mvnw spring-boot:run from the server dir. The problem might very well lie with the 401, I just don't really know where to proceed in Okta to solve this access problem!

My colleague is owner of the Okta account, is that what you mean by the API Access Management issuer? And if so, I don't understand how this would explain the difference since we're both running the server app with the same config - his machine builds (and runs), while mine doesn't even build. The server build process doesn't know anything of the user building it correct?

I will play around with the LocalTokenValidation and see if it offers more insight, thank you again!

@MarcelTon
What comes back from a request to https://company.okta.com/oauth2/default/.well-known/openid-configuration (from your browser)?

When using the config issuer: https://company.okta.com for the server I receive the InvocationTargetException with the "String index out of range".

I think that you should set the org-url: okta.client.org-url= https://company.okta.com

For a https://company.okta.com server without Authorization server, you must validate the idToken, you need set security.oauth2.resource.jwk.key-set-uri=https://company.okta.com/oauth2/v1/keys security.oauth2.client.client-id={$client.id} in your application.properties and remove the okta oauth2.
And you need add @Override         public void configure(ResourceServerSecurityConfigurer resources) throws Exception {             resources.resourceId("0oa9utvmghGro8CW61t7");         }  in the ResourceServerConfigurerAdapter class.


failed to compile gen_key.c

Description

Type: Bug
Priority: Minor

Bug
OS
windows
mbed TLS build:
Version: 2.13.1
OS version: 17134.345
Configuration: default config
Compiler: Visual Studio Build Tools v141
Error
error C2664: 'const mbedtls_pk_info_t *mbedtls_pk_info_from_type(mbedtls_pk_type_t)': cannot convert argument 1 from 'int' to 'mbedtls_pk_type_t'
1>d:\gen_key.c(301): note: Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)
1>d:\gen_key.c(301): error C2660: 'mbedtls_pk_setup': function does not take 1 arguments
1>d:\mbedtls\mbedtls\pk.h(207): note: see declaration of 'mbedtls_pk_setup'
1>d:\gen_key.c(318): error C2664: 'int mbedtls_ecp_gen_key(mbedtls_ecp_group_id,mbedtls_ecp_keypair *,int (__cdecl *)(void *,unsigned char *,size_t),void *)': cannot convert argument 1 from 'int' to 'mbedtls_ecp_group_id'
1>d:\gen_key.cp319): note: Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)

Hi @phoenixmcallister,
thank you for your report! I agree that an explicit cast should be added to programs/pkey/gen_key.c on the two occasions you pointed out. We'll fix that.
Kind regards,
Hanno

Opened https://github.com/ARMmbed/mbedtls/pull/2175 to address this.

ARM Internal Ref: IOTSSL-2617


Support SPAR: add new roles

Describe the task
As required from SPAR team, they need 3 new roles:

Orchard Manager – internal (or ministry), no need forest client number
Orchard Manager – external (or non-ministry), this one requires forest client number
TSC_Admin, no need forest client number

Acceptance Criteria

[x] Write the flyway script to add three roles, add role purpose
[x] Add three roles to all env, SPAR_DEV, SPAR_TEST and SPAR_PROD
[x] Requesting team has provided role purpose

Additional context

Add any other context about the task here.
Or here

Meet with Sabina and Craig and Olga, they have a concern in the future if they want to pass the token from old spar to new spar, maybe it's better to keep using the old role name. Consider the possibility and that's the future plan, we'll still use the new role name for consistency in FAM, and they are fine with it, in the future if they still want to old role name, we can switch for them.

In the PR it has these 3 new roles and descriptions to be added:
TSC_ADMIN user role allows a ministry SPAR user to access SPAR administrative screens and perform administrative functions such as seedlot registration approval, or seedling request approval

SPAR_SPR_MINISTRY_ORCHARD user role allows ministry SPAR users to access the A class registration screens and to create, review, update and submit A class registration forms.
Ministry users may enter and update lots within their own Organizational Unit.

SPAR_SPR_NONMINISTRY_ORCHARD user role allows non ministry SPAR users to access the A class registration screens and to create, review, update and submit A class registration forms.
Non-Ministry Users may enter and update lots where their sign in client profile matches the Applicant Client.

SPAR team is okay to use the role name SPAR_TSC_ADMIN, SPAR_MINISTRY_ORCHARD and SPAR_NONMINISTRY_ORCHARD for consistency

And pr mentioned the descriptions are adjusted version to fit into max column size (200) in FAM.
Just want to confirm that: does the team confirm they are ok for the description we adjusted for them?
Thanks Catherine.
@MCatherine1994

Hi Ian, thanks for checking, we didn't change much about the description, it's almost the same.

verified in dev db:


Memory Validator fails on release APK

Watch Face does not contain a watchface.xml file
com.google.wear.watchface.dfx.memory.InvalidTestRunException: Watch Face does not contain a watchface.xml file
    at com.google.wear.watchface.dfx.memory.WatchFaceData.fromResourcesStream(WatchFaceData.java:113)
    at com.google.wear.watchface.dfx.memory.ResourceMemoryEvaluator.evaluateMemoryFootprint(ResourceMemoryEvaluator.java:118)
    at com.google.wear.watchface.dfx.memory.ResourceMemoryEvaluator.evaluateInHumanReadableMode(ResourceMemoryEvaluator.java:80)
    at com.google.wear.watchface.dfx.memory.ResourceMemoryEvaluator.main(ResourceMemoryEvaluator.java:52)

For one of the watchfaces at https://github.com/android/wear-os-samples/tree/main/WatchFaceFormat
build with
./gradlew watchface:assembleRelease

Contents of the release APK
jar tf watchface/build/outputs/apk/release/watchface-release.apk
META-INF/com/android/build/gradle/app-metadata.properties
META-INF/version-control-info.textproto
AndroidManifest.xml
res/Af.xml
res/Yd.png
res/li.xml
res/mf.png
resources.arsc

Confirming Debug build validates correctly.

Release APK from the above installs and works fine.

Support for resources.arsc files handles this issue: https://github.com/google/watchface/compare/main...arsc

This is now fixed


Add a custom parameter to disable action pruning in the UP grounder

Codecov Report
Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (da8ab70) 84.96% compared to head (b7bf8a8) 84.95%.
Report is 11 commits behind head on master.

Files
Patch %
Lines

unified_planning/engines/compilers/grounder.py
80.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #557      +/-   ##
==========================================
- Coverage   84.96%   84.95%   -0.01%
==========================================
  Files         200      200
  Lines       26433    26436       +3
==========================================
+ Hits        22458    22460       +2
- Misses       3975     3976       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

OK for me.
For future reference: we needed this to construct a grounding that remains valid even if the initial state is changed (this is not true if static fluents are used for simplification)


Serialize Work Order as Proto.

This is a single large PR that serialize all work orders as proto. It is a part of moving to the distributed database.
Foreman creates work order protos in WorkOrdersContainers, and then reconstructs work order from protos before dispatching to workers. Note that in the distributed Quickstep, the work order reconstruction part happens in Shiftboss.
@jianqiao Please take a look for my changes on operators, as it might affect your work on separating the  front-end and the back-end. Thanks!

@zuyu Can we add an abstraction around to short-circuit the creation of WO using protobufs? In a single-node case, we should bypass.
Also, in the distributed case, wouldn't the Foreman in the Director simply create the WOs rather than go through the protobuf way? protobufs are only needed when sending across processes (Director to the Worker). We should not add the protobuf overhead unnecessarily.

@pateljm Yes, I could use macros (e.g., QUICKSTEP_DISTRIBUTED) to differentiate the ways to generate work orders in the single-node and the distributed cases. But I don't have a better approach to do so, as both cases have only minor differences in some implementations in the function level (e.g., getAllWorkOrders for each operator).
In the distributed case, serializing work orders is required, as components used to create actual work orders (e.g., StorageManager and QueryContext in workers) are not accessible to Foreman in Director.
On the other hand, I could compare the overhead of w/ and w/o anwork order serializations. If it is minor, we may keep the serializations.
And as a follow-up, we should better to use Shiftboss in the distributed case only.
Btw, I am debugging a test failure related to serializing SortMergeWorkOrder. I did not encounter this issue when using protobuf 3 beta version.


ASCENDING PRIORITY QUEUE in C

🚀 Feature
Have you read the Contributing Guidelines on Pull Requests?
Yes !
AIM :: TO IMPLEMENT ASCENDING PRIORITY QUEUE in C.
Motivation
As i didn't found ASCENDING PRIORITY QUEUE in C , So i am willing to contribute it...
Pitch
Hi I am GSSoC'21 participant,
"I am looking to work on the
I will write the clean and optimized code , by keeping best case time and space complexity in mind and
with  brief commenting and algorithm.
I will use C language.
So i would to like @bhav09 , @HarshCasper sir to please review this issue , so that i can do PR :)
/assign

/assign


Declutter sidebar by supplying explict short titles.

Also, fix up 'resource_manager' docs to avoid propagating sub-menus.

@theacodes I expect that the short titles I picked might be open to debate (because "branding") but they have to be better than what we have now.


Support team deploys

This PR adds support to team-owned projects in Vercel. This is done through a new "Team slug" field in the "Create new" dialog, which we use to fetch the team's id.
The team's id is what allows us to fetch these projects from Vercel, as specified in their documentation. It's a pretty simple change and I didn't go the extra mile on validation and the likes, but at least we error out when we can't find the team's id.
Future improvements could include a team name's badge in deploy-item, links to the project in Vercel, loading states in the "Create new" dialog, etc.
Let me know if you need anything else to have this merged, @ndimatteo 🙌
PS: I also took the liberty to remove react-icons as a dependency as that wasn't being used and adds a large overhead to the install bundle of this plugin

@hdoro thank you for this! 🙏
I just went through it all, here's what I added/changed:

per your suggestion, added a "team name" badge to the deploy items
Added an error tooltip when deploy items get an Error state, displaying the error message returned from Vercel
Updated a number of items using the new @sanity/ui elements

New version is up on NPM as well 😎


Reference all children jss to tss syntax

Hello!,
In the migration process to mui v5. Previously used a lot of the below synax:
"& > *" : {
  ... reference all children css here
}

In migrating, I can't seem to figure out the syntax to accomplish the same thing with tss-react:
export const useStyles = makeStyles()((theme) => ({
  row: {
    height: "50px",
    cursor: "pointer",
    "& > *": {
      paddingTop: 0,
      paddingBottom: 0,
      paddingRight: theme.spacing(1),
      paddingLeft: theme.spacing(1),
    },
    "&.Mui-selected": {
      backgroundColor: theme.colors.TEAL["1000"],
    },
    "&.Mui-selected:hover": {
      backgroundColor: theme.colors.TEAL["900"],
    },
  },
})

Hi @KellyP5,
It works the same as in JSS.
the codz you are sharing should work.

Hi @KellyP5, It works the same as in JSS. the codz you are sharing should work.

Interesting, my guess is it is being overwritten by MUI theme then? Not sure why it is being ignored. I can adjust the setting in the primary theme or adjust the CSS on the components directly but haven't been able to get the above inheritance syntax to work

my guess is it is being overwritten by MUI theme then?

Pobably, inspect with the browser devloppement tools and see what's up...

Not sure why it is being ignored.

Matchall selectors are very low specificity. Any other rule will overwrite them.
You can abitratly increace specificity by adding extra & though.
The more you add, the more specific your selector will get, example:
export const useStyles = makeStyles()((theme) => ({
  row: {
    height: "50px",
    cursor: "pointer",
-    "& > *": {
+   "&&& > *": {
      paddingTop: 0,
      paddingBottom: 0,
      paddingRight: theme.spacing(1),
      paddingLeft: theme.spacing(1),
    },
    "&.Mui-selected": {
      backgroundColor: theme.colors.TEAL["1000"],
    },
    "&.Mui-selected:hover": {
      backgroundColor: theme.colors.TEAL["900"],
    },
  },
})

This did the trick, thanks a lot @garronej


Pong results do not match paper

Hi Danijar,
Thanks for sharing this amazing repo and creating a robust model-based RL algorithm! I've been playing with the replay buffer and trying to reproduce some of the results. I run the code on Pong with command python dreamerv3/main.py --logdir ./logdir/uniform_pong --configs atari --task atari_pong  --run.train_ratio 32 with the default configurations on a Ubuntu 22.04 LTS with a RTX 3090 GPU. Somehow, the agent does not work on the Pong task over 400K env steps (according to the first version of the paper). I'm not sure what went wrong. I've tried with the default uniform replay (cyan curve in figure), a mixed replay (gray curve) with ratio of (0.5, 0.3, 0.2) and uniform replay with compute_dtype: float16 (magenta curve) since I've seen some warnings from CUDA and XLA.

Here are the package versions that I installed:

python                            3.11.9
jax                                   0.4.28
jax-cuda12-pjrt               0.4.28
jax-cuda12-plugin          0.4.28
jaxlib                               0.4.28+cuda12.cudnn89
ale-py                              0.8.1
gymnasium                     0.29.1
tensorflow-cpu               2.16.1
tensorflow-probability    0.24.0

Please let me know if anything was not set up properly. Thank you!

As far as I know, the run.train_ratio should be 1024 for Atari100k.

Hey,
First of all i think the first comment on this is right, you should increase the train_ratio. That was confusing for me too at first but that should solve the issue. Quick off topic question tho: I see you are running pretty recent versions of tensorflow-cpu as well as jax. Did u run into any issues where the pip installation stated that jax requires mldtype >= 4.0 and tensorflow requires that library to be version 3.2?

As far as I know, the run.train_ratio should be 1024 for Atari100k.

Thanks @IcarusWizard and @NonsansWD  I'll try your suggestion!

Hey, First of all i think the first comment on this is right, you should increase the train_ratio. That was confusing for me too at first but that should solve the issue. Quick off topic question tho: I see you are running pretty recent versions of tensorflow-cpu as well as jax. Did u run into any issues where the pip installation stated that jax requires mldtype >= 4.0 and tensorflow requires that library to be version 3.2?

I have not run into that particular issue. I'd suggest that you install tensorflow-cpu first (maybe a less recent version) then install JAX.

Alright good to know. In the end i was able to fix my issue and everything works fine. The only problem im left with is i just realized that the resulting folder called "replay" does not contain raw frames but instead a lot of data like rewards and so on. Do you by any chance know a way of obtaining a video of the agents steps or something so i can watch it do its stuff without too much effort? I feel like im missing something cause i also dont know where to get these wonderful score plots or do i have to construct that plot myself with matplotlib? sorry for going off topic


chore: remove node ports

chore: remove node ports
Our kubectl-plugin no longer needs node ports to reach our services.
Signed-off-by: Tiago Castro<EMAIL_ADDRESS>
chore: temporarily pin io-engine to v0
This will be changed to v1 when the control-plane supports v1.
Signed-off-by: Tiago Castro<EMAIL_ADDRESS>
fix: run pre-commit hooks under the nix-shell
Signed-off-by: Tiago Castro<EMAIL_ADDRESS>

bors merge


NikitaK / Task - {update PA page design}

Changes:

update PA page design

Type of change

[ ] Bug fix
[ ] New feature
[x] Update feature
[ ] Refactor code
[ ] Translation to code
[x] Translation to crowdin
[ ] Script configuration
[ ] Improve performance
[ ] Style only
[ ] Dependency update
[ ] Documentation update
[ ] Release

There is one code smell that can be addressed as well 🙏

Please rebase this branch yeah @NikitaK-deriv

Your PR contains commits from many other PRs, you'll need to rebase this correctly. Please reach out to one of us if you need assistance.

@NikitaK-deriv can try and rebase this branch properly, it has so many changes that you didn't work on.
Maybe @sean-binary can help with that.


API Updates (breaking)

Removes FFI
Removes Aff functions
Adds general foldHashes
Removes hashChanged
Changes the way matches works (filters failures)

This adds some browser tests, but is only run manually.

Do you think that all the functions in Routing should be moved to Routing.Hash? They are hash specific, and I might write a pushState version since we have use for it.

@natefaubion not sure, I'm not that familiar with this lib. @garyb @cryogenian might have an opinion on that.

Having pushState support would be nice. It has been asked about a couple of times. Moving stuff to Hash makes sense to me!


Fix links to point offsite rather than relative

Good catch! There's one more in the title on line 1 😬 If I find time to merge/update this I'll fix that one as well if you don't feel like doing it ❤

This fix was included in release 1.0.1, thanks for your contribution! 🚀


[Inference Client] Factorize inference payload build

This PR  is a first attempt to factorize the payload build in multiple InferenceClient methods. In fact, there's some repetitive logic across several methods for handling inputs and parameters, so here we introduce a new (private) helper function to factorize this logic.
Key changes

Adding a helper function that:

handles both raw content (images or audio) and string/dict inputs uniformly.
base64 encodes raw content when at least one parameter is present.
filters out None values from the parameters.
.. and returns an _InferenceInputs object containing the json payload and raw data if any. (I don't have a strong opinion on this, we can also return a Tuple instead).

A unit test to verify the correct behavior of the refactored logic.

⚠ These changes only affect the internal/private functionality of the InferenceClient and AsyncInferenceClient.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

thanks @Wauplin for the review! I addressed your suggestions and I ended up adding an expect_binary flag as it's the simplest way to handle the special case of image and audio (path, URL and binary) inputs :)

thanks @Wauplin! I think we're good to merge this one


Unfair starting positions

Users aren't always equidistant from one another. This puts some players at an immediate advantage, since some players may be closer to weak opponents than others. This adds volatility to the rankings.
Here is one example. @djma and @erdman were placed right next to one another. They had to fight over the high production mass between them. @revoklaw got to enjoy an uncontested, high production mass because @tarriel was a relatively weak opponent.

Is this really a bug?  @erdman's relative position to @djma is same as
@revoklaw to @tarriel, each with high productivity area between them.  The
imbalance from this game is that the bot opposite @revoklaw did not fight
for the high productivity area.  I don't think there's much you can do --
you're always going to have a relatively weaker bot in any given match, and
someone else will benefit from that, and it does cause some volatility in
the rankings, but it seems natural and unavoidable.  Or maybe you're seeing
something else that I'm not?
On Sun, Nov 20, 2016 at 12:51 PM, Michael Truell<EMAIL_ADDRESS>wrote:

Users aren't always equidistant from one another. This adds volatility to
the rankings.
Here is an example
https://halite.io/game.php?replay=ar1479672367-299694187.hlt. @djma
https://github.com/djma and @erdman https://github.com/erdman were
placed right next to one another. They had to fight over the high
production mass between them. @revoklaw https://github.com/revoklaw got
to enjoy an uncontested, high production mass.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HaliteChallenge/Halite/issues/246, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAqgctBzJHKjNHTkwCZniR5wdhYpmFF8ks5rALK9gaJpZM4K3psq
.

When testing my bots locally, I've found that replaying with the same map with every possible permutation of players usually reveals biased starting positions (the bot in x position is more likely to win), even in 1v1 matches against itself.
At first I was concerned, but over enough games biased starting positions should average out among all competitors. To me, this isn't much different from sometimes playing as white or black in Chess. Any one match is insufficient to make a definitive statement of raking between contestants.

I think I found a map that could be considered "unfair".  I was surprised
to find it.  Just thought of this thread -- apologies if this should go
into a new or different thread.  Here is the map and game:
https://halite.io/game.php?replay=ar1480061447-2832866290.hlt
The starting positions are fine -- it's the symmetry that is at issue.
Like every other map, this one looks to be symmetrical at first glance
(click on 'p' to show the production areas).   However, look closely at the
high productivity area that @brianvanleeuwen and @erdman fight over in the
very beginning.  It looks like his side and my side are mirror-reflected
left-to-right, but when you start clicking around the different boxes, it's
easy to find corresponding pairs that "should" have the same strength and
production values, but they are, in fact, different.
For instance, his 7th box is a (53,6).  The corresponding box on my side is
a (52,6).  There are several other examples with larger differences in
magnitude in that higher production area surrounding where our initial
contact takes place.
On Tue, Nov 22, 2016 at 4:12 AM, Nick Malaguti<EMAIL_ADDRESS>wrote:

When testing my bots locally, I've found that replaying with the same map
with every possible permutation of players usually reveals biased starting
positions (the bot in x position is more likely to win), even in 1v1
matches against itself.
At first I was concerned, but over enough games biased starting positions
should average out among all competitors. To me, this isn't much different
from sometimes playing as white or black in Chess. Any one match is
insufficient to make a definitive statement of raking between contestants.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HaliteChallenge/Halite/issues/246#issuecomment-262225523,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAqgcvOULpIs7_bSjCXqDkHkZ3eNGdtFks5rAtw1gaJpZM4K3psq
.

Should be resolved; see other thread.


Safety Parser breaks unit tests

From time to time the unit test suite fails due to the safety parser relying on dynamically loaded data.
======================================================================
FAIL: test_multiple2 (dojo.unittests.tools.test_safety_parser.TestSafetyParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/dojo/unittests/tools/test_safety_parser.py", line 49, in test_multiple2
    self.assertIsNone(finding.cve)
AssertionError: 'CVE-2020-28493' is not None

======================================================================
FAIL: test_multiple_cves (dojo.unittests.tools.test_safety_parser.TestSafetyParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/app/dojo/unittests/tools/test_safety_parser.py", line 36, in test_multiple_cves
    self.assertIsNone(finding.cve)
AssertionError: 'CVE-2019-12385' is not None

Can we
a) change the unit test so they do not rely on external data being correct, or,
b) change the safety parser to just process a report. in general the idea is that scans are performed outside defect dojo and the results are imported.

@damiencarol

it's know pb. the DB of safety parser is maintained by the safety team. it's a real anomaly that our unit tests catch.
I'm just thinking that maybe we should change the way we load this DB.
Will work on it ASAP.

Or just remove the assert on the CVE, looks like it's not reliable provided by the safetyparser db.

When I download the json manually for safetydb the cve data is in there and looks the same as in the asserts.

ok, I get it. the DB just changed! I mean new vulnerability where discovered and appeared in the DB...
So, I guess we need to modify the unit tests to not rely on the DB directly. but maybe a custom DB for the unit tests.

I need to investigate the best solution.

Or just remove the assert on the CVE, looks like it's not reliable provided by the safetyparser db.

@valentijnscholten => I think it's the best solution for now as CVE are not part of the report itself. so => #4192
Pushed another pull request uptream to have CVE directly in the JSON reports => https://github.com/pyupio/safety/pull/340
it will reduce the code in this parser and make the download of the DB useless, so no more error on DB updates every month

Fixed for now by removing the asserts.


[core] Move codes of cross partition to paimon-core

Purpose

These codes should be reused by paimon-spark.

Tests

API and Format

Documentation

+1


Move plant checklist into MIxS as a checklist

Right now is floating around on its own. Needs to be incorporated into the main MIxS file as a checklist and have IDs assigned.
See issue #202.

aim for v7

See https://gensc.org/the-plant-specimen-contextual-data-consensus/

I'm trying to sort out if rhizosphere soil is plant-associated or soil ... I always assumed soil with some plant -associated extension slots because there's nothing in the plant-associated extension to indicate if the soil or the root was sequenced aside from a slot "plant_structure" but that's optional. With that... what is this plant vs plant_associated???

The URL https://gensc.org/the-plant-specimen-contextual-data-consensus/ doesn't resolve, and it doesn't appear to be in archive.org
This seems to be the consensus reference
https://academic.oup.com/gigascience/article/5/1/giw002/2756883?login=false
There is an attachment which is a word doc, I converted it here:
https://chat.openai.com/share/c780d6c3-0685-4fd9-8c25-5a1d6bb82b51


very good design ,but can give us more detail document to explain?

@kilgaloon
has any detail document for quick start ?,i think why so few person stars is that the leprechaun is not friendly to users to teach them how to build schedule system  with it.
your readme is far from  enough.
I expect see the document as soon as possible.because i like the leprechaun.

Hey @kilgaloon
I think the most important thing is to introduce in general what leprechaun is, what parts it is composed of, and what responsibilities each part bears.
The officially defined recipe type (schedule, hook, cron) should be used in any scenario. How to quickly connect with existing systems. I took a look at the overall code structure of leprechaun. It should be the biggest design ambition I have seen in the open source library for the timed task class. But I think there is still a lot to be done to reach your ultimate goal. But as users we only have a vague concept.
I hope that you can express some of your design ideas. So we can understand that there is a client, api? Are we using cli or http when we manage recipes? There are also recipe files and so on for not using the associated database. I have so much, or because I hope to integrate leprechaun into existing systems.
Thanks a lot

maybe，nsq is good example

@zppro i had idea about something similar document that can describe closely idea behind Leprechaun and some of the design decision i put in it. Currently i lack free time to create something like that, i will try my best. I have more ideas for Leprechaun regarding new functionalities and code changes and updates but like i said, currently i lack free time to do that.
Thanks for you contributions an ideas.

I worked a bit on this and here you can find some basic explanation of some general things: IDEA


Deep async

@jhnnsrs , can you check how this works with your real-world example? I do not think this is good code, but makes things pass, when I try the s3fs test in https://github.com/dask/s3fs/pull/374 with --count=10.

sorry for coming back so late.
Fix works great for datasets smaller than (1024,1024,50,2,1) but falls short on anything beyond, same
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 1844, in _run_once
    handle = self._ready.popleft()
IndexError: pop from an empty deque

Requirements:
aiobotocore==1.1.1
aiohttp==3.6.2
aioitertools==0.7.0
asciitree==0.3.3
async-timeout==3.0.1
attrs==20.2.0
botocore==1.17.44
chardet==3.0.4
dask==2.28.0
docutils==0.15.2
fasteners==0.15
fsspec @ file:///code/filesystem_spec/dist/fsspec-0.7.4%2B309.g77564f5-py3-none-any.whl
idna==2.10
jmespath==0.10.0
monotonic==1.5
multidict==4.7.6
numcodecs==0.7.2
numpy==1.19.2
pandas==1.1.2
python-dateutil==2.8.1
pytz==2020.1
PyYAML==5.3.1
s3fs==0.5.1
six==1.15.0
toolz==0.11.1
typing-extensions==<IP_ADDRESS>
urllib3==1.25.10
wrapt==1.12.1
xarray==0.16.1
yarl==1.6.0
zarr==2.4.0

Thanks!

I was running with a size (1024, 1024, 10, 9, 1) and repeats --count=10 and passing, but if I up the time even further, it breaks for me too :|. Increasing the sleep time only converts the exceptions into deadlocks.
I am puzzled that

you can get an IndexError inside a block explicitly protected by except IndexError
that a sleep makes any difference (this is the only thread handling events, other threads can only add events; but maybe it allows them to drop references and let garbage get collected?)
zarr should not be calling mkdirs on every single chunk write. Even for a local file-system, I find that it hurts performance for many-small-writes quite a lot; we know that all the chunks live in the same dir

--EDIT--
I have found the one-line fix (does not need change to fsspec) :). However, I am still not understanding exactly why the multiply-nested calls caused the race, and what to do about it. This will pobably turn up elsewhere.

Since the fix is in s3fs, this need not be merged (it is questionable code anyway, and you should see my local version); I'll leave this open here for now.


Feat(shop): Add new feature like notes skin

[ ] Design and implement a feature to allow users to change note skins (themes or styles).
[ ] Create multiple skin options for notes (e.g., different colors, textures, or layouts).
[ ] Add a UI element in the shop to let users select and purchase note skins.
[ ] Ensure note skins are applied consistently across all user devices.
[ ] Store selected note skins in user settings for persistence.
[ ] Test compatibility of skins with other features like drag-and-drop and sorting.
[ ] Update documentation to explain how the note skin feature works.

Hey, could you assign me to this issue? I'd love to start working on it. :)

Hey, could you assign me to this issue? I'd love to start working on it. :)

I've been updating the sticky memo app to use MUI from traditional CSS, and it’s coming along well! I'm aiming to handle skins like states, where each skin has an ID managed by recoil-persist, allowing users to switch between them as they like. If possible work like this so it'll be helpful. Let me know if you need anything else 🤝

can you assign me

can you assign me

Hi @dipak-kumawat assigned it to you, once completing the feature, Give me a PR.


Broken links in chapters

As per our shiny new broken link checker (see #24), we have a great overview of all the broken links in the guide. We should fix them :-)
See https://travis-ci.org/NLeSC/guide for an overview of all broken links.
Chapters with broken links atm:

[ ] Software/Communication
[ ] Software/Code Review
[ ] Software/Documentation
[ ] Languages/Java
[ ] Languages/Javascript
[ ] Languages/Python
[ ] Languages/R
[x] Contributing

Chapter owners, please fix :-)

I'm on it. I thought about fixing links in my part but when I'm doing it I can as well fix them for whole guide, no need for bothering all maintainers. @jspaaks @jvdzwaan @cwmeijer @vincentvanhees @LourensVeen

@nielsdrost any idea where this should point? .editorconfig?
See the collection of config files for automated coding style [here](https://github.com/NLeSC/knowledge/tree/master/software/config)

@LourensVeen from code review chapter:

Klocwork (http://www.klocwork.com/Code-Review-Tools)

Link dead, doesn't appear to offer any code review tools

I don't think it make sense to include in the guide something which seams to be dead.
The point of the guide is to give people useful readily applicable information.
Plus now that we check for dead links we should avoid having them on purpose.

@cwmeijer In java chapter:

SonarQube (http://www.sonarqube.org/) is an open platform to manage code quality.
On our Jenkins build server we do a SonarQube analysis for each Jenkins job. Out SonarQube > instance is hosted at https://sonar.esciencecenter.nl and is behind a login.
Sonar Qube can analyze Java, Python and Javascript.

It looks like there is no sonar hosted by us any more.

Just to be sure, let's ask @sverhoeven :-) Do we still have any SonarQube installation running that should be mentioned in the Java guide?

Just heard from @nielsdrost that we don't have SonarQube installed anymore. I'll update the guide.

That is correct we are not running our own SonarQube and Jenkins server anymore.

@nielsdrost any idea where this should point? .editorconfig?

The java code style is already referred to from the Java Chapter. The editorconfig is mentioned and linked to one paragraph above the broken link. So I vote we remove the whole sentence.

done

@jspaaks The link to http://localhost:8000 on the javascript page should be excluded from the checker by adding --exclude localhost:8000 to the blc command in .travis.yml file.


[Bug][Widget responsiveness] When we convert the List widget from fixed to auto layout (using algo), it dispays only the first list ITEM and rest data is not renderinf

List widget items are displayed. Working fine, moving to done.


#2279 Add Python 3.6 to README.

Description
Adds Python 3.6 to the list of supported versions in README.rst.
Motivation and Context
Closes #2279.
Have you tested this? If so, how?
I successfully built the documentation locally using the Writing documentation docs.

Thanks @mattayes !


Slots usage

consider using slot recipes

Thanks for the issue @anubra266  😄
Slot recipes are coming soon 👍

Just released v0.2.0 which replaces most components to slot recipes 👍


Performance issue with large number of ui-grids in one page

I have around 100 ui-grid tables in a single page encapsulated within a directive, grouped to show conditionally. As a result, even though at any point in time the page will show no more than 6 tables, it however takes an inordinate amount of time for the page to load, presumably due to some sort of initialization for all the 100 odd tables. The data for each of the ui-grid tables does not get loaded at page load. Once the page is loaded, the user would have to click on different nodes of a tree to trigger the data loading for a set of tables that correspond to each node. How would I go about fixing the initial page load issue? Is there something I need to do with regards to some sort of ui-grid options which will help mitigate the huge page loading time?

I was able to solve the problem by using ng-if instead of ng-show/ng-hide. Thanks for all the work on this control.


Tidy: deprecate + rename init/constructor

why
We have params being passed in to init that are also in the constructor. This is redundant and doesn't make sense. Cleaning this up. Also deprecating initFromPage in favor of browserbaseResumeSessionId. Lastly, change browserBase to browserbase.
Now you can just do
const stagehand = new Stagehand({ yourParams });
await stagehand.init()

what changed
param names + docs
test plan
evals CI

is there a migration guide somewhere for this change?

Deprecate initFromPage in favor of browserbaseResumeSessionID in constructor


blending of colors should take place in linearized RGB

https://github.com/3MFConsortium/spec_materials/blob/master/3MF Materials Extension.md#12-srgb-and-linear-color-values
Can most easily be handled in VectorField3DComposed-element

We might have a new attribute: "blendingspace", with options: "raw" (default) and "linear".
For volumetric color the blendingspace MUST be linear. What would be the outcome if not specified, or specified "raw". Option1: it is an error to either reject or warn. Option 2 is to accept it., however this would require supporting and testing two use cases.
If any other future 3D property needs to be linearized, that should be specified in the extension describing such property, together with the linearization formulae.
What about ScalarFieldComponent, do we have that attribute available, even not used yet, to help extendibility to any future property?


Added template.HTML to templateFuncs

Issue: https://github.com/matcornic/hermes/issues/38

Codecov Report

Merging #39 into master will decrease coverage by 0.32%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   98.45%   98.13%   -0.33%
==========================================
  Files           3        3
  Lines         907      910       +3
==========================================
  Hits          893      893
- Misses          8       11       +3
  Partials        6        6

Impacted Files
Coverage Δ

hermes.go
73.68% <0%> (-4.1%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 23ab47d...e0c2b27. Read the comment docs.


Would you consider support for more object storage providers?

Hi there,
I just discovered this project today, and it looks fantastic - thank you for making it!
Would you  consider supporting more providers than just AWS  if they are already boto/boto3 compatible ?
I see this code in the repo here which I think is responsible for uploading snapshots to an object storage service like AWS S3:
def get_s3_bucket():
    from boto.s3.connection import S3Connection

    conn = S3Connection(app_settings.S3_ACCESS_KEY,
                        app_settings.S3_SECRET_KEY)
    return conn.get_bucket(app_settings.S3_BUCKET)

def s3_upload(key, data):
    from boto.s3.key import Key
    bucket = get_s3_bucket()
    k = Key(bucket)
    k.key = key
    k.set_contents_from_file(data, rewind=True)
    k.set_acl('public-read')
    k.set_metadata('Content-Type', 'text/csv')
    return k.generate_url(expires_in=0, query_auth=False)

In a project I work on we use the boto library to talk to s3 compatible services from other providers, and I think to add this support to django-sql-explorer, it would mainly require a adding two new parameters to the code that connects to S3.
We have a comparable method on a project I work on where we fetch an S3-compatible bucket, which does a somewhat similar job to get_s3_bucket above.
def object_storage_bucket(bucket_name: str):
    """
    Return the bucket identified by `bucket_name` for uploading
    and downloading files.
    """
    session = boto3.Session(region_name=settings.OBJECT_STORAGE_REGION)

    object_storage = session.resource(
        "s3",
        endpoint_url=settings.OBJECT_STORAGE_ENDPOINT,
        aws_access_key_id=settings.OBJECT_STORAGE_ACCESS_KEY_ID,
        aws_secret_access_key=settings.OBJECT_STORAGE_SECRET_ACCESS_KEY,
    )
    return object_storage.Bucket(bucket_name)

While the example I have shared uses boto3, instead of boto, I think it would not be too complicated to add the extra params, the region_name, and endpoint_url to the code connecting to an object storage service.
Doing so would then make this library compatible with a range of S3-compatible providers (i.e. Cloudflare, Digital ocean, Scaleway, etc), as well as open source  S3-compatible object storage software like Minio, Garage and so on.
I'm currently adding django-sql-explorer to a project I'm working on but once I'm a bit more familiar with it, would you accept a PR to add this functionality, and assuming it meets contribution standards, giving some pointers on how to have this gracefully integrated into the project?

Hi @mrchrisadams - how did you find the project & happy to hear it's useful to you!
I certainly welcome any functionality that enhances the project and what you're proposing sounds great.
One thing to note, @lawson89 has put together a PR which updates the boto usage, see #501. I'm trying to find the time to review this - you may want to take a look as well to understand what's going on and offer any comments with the consideration for what you're thinking of doing.
In terms of contributing, I've had a pre-commit branch that I've been trying to get merged for a while - that might make it easier for everyone to meet a consistent code quality level. But on the whole, it sounds like this would need well commented code, with docs updates and tests. Nothing out of the ordinary.

@marksweb @mrchrisadams I think the above is a great idea. Looking at what I implemented I think we would just need to add region and endpoint (I should have added these both tbh in my PR). I also like making the setting names generic
My code:
def get_s3_bucket():
    s3 = boto3.resource('s3',
                        aws_access_key_id=app_settings.S3_ACCESS_KEY,
                        aws_secret_access_key=app_settings.S3_SECRET_KEY)
    return s3.Bucket(name=app_settings.S3_BUCKET)

Hi @lawson89 , yeah that looks about right.
I've annotated this some more.
def get_s3_bucket():

     # start up a session with non-AWS region. This might seem a bit
     # when we define the endpoint further down, but if you don't I think boto
     # complains that the endpoint and region do not match. For exmaple with
     # scaleway we use `nl-ams`
     session = boto3.Session(region_name=settings.OBJECT_STORAGE_REGION)

    # once you have instantiated your session with the alternative region specified
    # you still need to authenticate yourself. Here you list the endpoint to connect to.
    # In our case we use `https://s3.nl-ams.scw.cloud` to connect to the
    # Amsterdam service from scaleway
    s3 = boto3.resource('s3',
                        aws_access_key_id=app_settings.S3_ACCESS_KEY,
                        aws_secret_access_key=app_settings.S3_SECRET_KEY,
                        endpoint_url=settings.S3_ENDPOINT
     )

    # now we have our connection to the S3 style object storage service. We are
    # free return the relevant bucket to manipulate elsewhere in our code.
    return s3.Bucket(name=app_settings.S3_BUCKET)

If you tag me on this from the PR, I'm happy to test this in the coming week or so.

@marksweb  - I found this project via Simon Willison's blog below:
https://simonwillison.net/2021/May/10/django-sql-dashboard/
We use MariaDB where I work, and this project's use of Django's database connection code made it easy to understand and see how we could use this.
Thank you again for working on it 👍

@mrchrisadams - if you could kick the tires on this PR and let me know if you see any issues

You're referring to this PR, right?
https://github.com/groveco/django-sql-explorer/pull/505


container_network_interfaces should be a variable?

The documentation states this is a list of resources created by the server.  Should this be a variable instead of a parameter?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 97b00525-8267-2738-58f1-2214ea40d11b
Version Independent ID: 2c57d585-8b6d-3ed7-13a6-28c89fb7cf2c
Content: azure.mgmt.containerinstance.models.ContainerGroup class
Content Source: docs-ref-autogen/azure-mgmt-containerinstance/azure.mgmt.containerinstance.models.ContainerGroup.yml
GitHub Login: @lmazuel
Microsoft Alias: lmazuel

It looks like this got linked to the wrong page.  I will close this, as I opened another one for the correct page.

It looks like this got linked to the wrong page.  I will close this, as I opened another one for the correct page.


New version: KeeperSecurity.Commander version 16.11.21

Pull request has been created with WinGet Updater :rocket:
Microsoft Reviewers: Open in CodeFlow

##[error] Unsupported line end format KeeperSecurity.Commander.installer.yaml

The installer file appears to have CRLF (\r\n) style line endings. Can these be replaced with LF (\n) style line endings?


Fix deprecated use of Fixnum

Using fixnum in ruby 2.4 generate warnings. change shouldn't have any real world impact (unless someone has a replica set with 2^63 members)

Coverage remained the same at 99.671% when pulling 239bc19899ffdd01e0aabdbb35f7c8bac030fd5f on fcheung:patch-1 into ea98e05853a1dcbe0a5d615ba6ee498852b0ee38 on mongomapper:master.

Coverage remained the same at 99.671% when pulling 239bc19899ffdd01e0aabdbb35f7c8bac030fd5f on fcheung:patch-1 into ea98e05853a1dcbe0a5d615ba6ee498852b0ee38 on mongomapper:master.

Already merged w/ other commits.  Thanks again for your help @fcheung.
Closing this one out.


chore: improve package quality

Description

closes https://github.com/traefik/faency/issues/336

I decided to use a MIT license by default, but if you have reasons to prefer another one, feel free to tell me.
I noticed that https://github.com/traefik/traefik has an MIT license as well, hence I think it should be alright? Is there someone responsible who should be advised of this choice?
Misc:

adds license
adds keywords
adds exports

How to test

run npx @skypack/package-check, check everything is valid
check license is appropriate
review changes

Good PR checkboxes

[ ] Change has been tested
[ ] Added/Updated tests
[ ] Added/Updated stories
[x] PR follows conventions
[x] Labels are set
[x] Project is linked

Good Review checkboxes

 ℹ  Copy the snippet and paste in the review field to fill it
- [ ] I've tested the changes
- [ ] I've agreed on the unit tests (soon to come)
- [ ] I've checked the stories
- [ ] I've read the code and understood it
- [ ] I don't have any more questions
- [ ] I've described any optional improvements
- [ ] I checked PR follows [conventions](https://github.com/traefik/faency#how-to-contribute)

I would suggest checking first with the OSS team first as a license being set is immutable.

:tada: This PR is included in version 7.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Sidekob::Job renamed to Sidekiq::JobRecord

It appears as the latest Sidekiq release - 6.2.2 - renames an internal class Sidekiq::Job to Sidekiq::JobRecord.
This change breaks the Sidekiq patch here.
I'm not entirely sure how to fix this while still supporting previous releases. Below would be a very rough guess and I'm happy to PR if it's suitable.
if Gem::Version.new(Sidekiq::VERSION) >= Gem::Version.new('6.2.2')
  Sidekiq::Job.prepend ActiveJob::Uniqueness::SidekiqPatch::Job
else
  Sidekiq::JobRecord.prepend ActiveJob::Uniqueness::SidekiqPatch::Job
end

@dwightwatson Thank you for the report. :+1:
I adapted the fix a bit and added Sidekiq 6.2.2 to the test matrix. https://github.com/veeqo/activejob-uniqueness/pull/30/files
Release v0.2.1 is coming.

Awesome, thanks for the quick turnaround!