[FLINK-7443] [metrics] MetricFetcher store and deserializer fields now final

The MetricStore and MetricDumpDeserializer fields in the MetricFetcher should be final, as they are not meant to be overwritten and are even used for synchronization.

+1 but I'm only seeing an improvement and not a bug

true, will change the jira category. merging.


[AMORO-1744] Rename the name of classes with the arctic identifier in the mixed-format-trino module

Why are the changes needed?

Close #1744
Brief change log
Rename the name of classes with the arctic identifier in the mixed-format-trino module
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before making a pull request

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@xleoken Thanks for the contribution.
But I don't think renaming arctic to amoro in all places is a good solution here.
The new project name amoro means a lot more than just the mixed-format now.
So I tried to resolve this issue by #2835, can you help to review it?

@xleoken Thanks for the contribution.
But I don't think renaming arctic to amoro in all places is a good solution here. The new project name amoro means a lot more than just the mixed-format now. So I tried to resolve this issue by #2838, can you help to review it?

Got.


Drag and drop on windows 11

Description
Dragging and dropping files from files to another app doesnt work
Steps To Reproduce
Try to drag and drop a file/folder to another app and it wont work
Expected behavior
Should be able to drag and drop like normal
Files Version
Version: <IP_ADDRESS>
Windows Version
Windows 11 OS Version: 10.0.22000.282
Relevant Assets
debug.log
debug_fulltrust.log

I also tried to drag-and-drop to upload it to the browser (e.g. Gmail) and not working

Merging with #6941


Fixes a visual bug in the search bar when opening a view controller

@philium That's okay! It wasn't that much extra work since I was adding search anyway.

@pgruenler This is ready for re-review when you get a chance.


Cannot import wish history

"Cannot Find Te Wish History Uluru! Bake Thread and Open the Wish History First!" and the import URL is not displayed.

Delete the webCaches folder in GenshinImpact game\GenshinImpact_Data. Then open wish history again.

Thank you


Added filter by provider to chargeback report filter tab

@h-kataria @gtanzillo Please review
cc @simon3z @sergio-ocon

@miq-bot add_label darga/yes, providers/containers, ui, chargeback

@zeari as far as I understand it's impossible to create a report across all the providers (all providers all projects).
This changes the previous behavior where it was possible to create a report across multiple providers.

@zeari as far as I understand it's impossible to create a report across all the providers (all providers all projects).
This changes the previous behavior where it was possible to create a report across multiple providers.

That can be easily added but do we want to allow that? Feels like this could be a mess for the user.

@zeari some test failures that look like legit failures

@zeari Does this change only apply to chargeback for containers?

Temporarily assigning to @zeari as we're still reviewing/discussing this.
@miq-bot assign zeari

@zeari Does this change only apply to chargeback for containers?

Yes, this makes the change only for container projects.
Should VM's embrace a 'by entitity' chargeback report, it will be available for them as well.

👍 LGTM

@gtanzillo We should hold on with the merge until we figure out if we should enable cross provider chargeback reports. Marking WIP for now.
@miq-bot add_label wip

@zeari I had thought the same, that the requirements was for a single provider.  If there is a need to do so across multiple providers, a solution in the UI could be a drop down which has checkboxes next to each of the provider names.

@zeari I had thought the same, that the requirements was for a single provider. If there is a need to do so across multiple providers, a solution in the UI could be a drop down which has checkboxes next to each of the provider names.

Is there an example of this somewhere?

Since now we know we want this feature, lets get this merged and make UI adjustments later.

@miq-bot remove_label wip

@zeari IIUC the project uid comes from the real project uid in OpenShift/Kubernetes. On the other hand the provider uid is the guid generated and assigned by ManageIQ, right?
LGTM :+1:
Re-assigning to @gtanzillo for review/merge cc @blomquisg @chessbyte
@miq-bot assign gtanzillo

@zeari IIUC the project uid comes from the real project uid in OpenShift/Kubernetes. On the other hand the provider uid is the guid generated and assigned by ManageIQ, right?

@zeari actually this question was related to the other PR (adding columns), but it came up now.

@zeari IIUC the project uid comes from the real project uid in OpenShift/Kubernetes. On the other hand the provider uid is the guid generated and assigned by ManageIQ, right?

Now that I think of it, wouldnt it be nicer to to have the provider hostname instead?

Now that I think of it, wouldnt it be nicer to to have the provider hostname instead?

@zeari yes it's probably more friendly to the OpenShift operator.

@zeari yes it's probably more friendly to the OpenShift operator.

Ill address this in another PR. Meanwhile, this PR is ready for review\merge.

Ill address this in another PR. Meanwhile, this PR is ready for review\merge.

Ping @gtanzillo

@gtanzillo UI side of changes look good.


Parameterize ProgressEvent

@sandersn, how about this?

Is this useful anywhere else? I expected that lots of other event types would want to have custom targets?

Might be better to start rather than doing all in once. IMO this will at least prevent a future breaking change "Where is my FileReaderProgressEvent" as we already saw one.

If this is a start that will have some follow-up, I want to understand what the follow-up will be. Are there other subtypes of ProgressEvent that will provide different arguments to T?
If the only subtype of ProgressEvent is FileReader, then I think it's too expensive and complex to create a generic type for just one case.

Are there other subtypes of ProgressEvent

Current users of ProgressEvent in lib.d.ts include ApplicationCache, Document, GlobalEventHandlers, and XMLHttpRequest,

How would ApplicationCache customise ProgressEvent.target? Would T=ApplicationCache ?

Yep, and T=XMLHttpRequest.
This can be generalized for nearly all Event sub-interfaces, so we even could do some automation in the future rather than manual generic field.

OK, that makes sense. Can you make the change for ApplicationCache, Document, GlobalEventHandlers, and XMLHttpRequest in this PR?


8 6 1

All QA comment addressed 👍

Hi @abhsheksingh
Thanks for making the changes.
Just a couple more needed:

The README.md file needs updating (as it still says to copy and paste the tables into the excel file)

You've still got 2 separate scripts doing the update (update_8.6.1.R and function.R) and you now call them both in compile_tables.R - however, both of them output a dataframe called csv_output so the second script is overwriting things from the first script (I think...). I recommend combining the code from update_8.6.1.R and function.R into one script (called update_8.6.1.R) and only calling this script in compile_tables.R - it will make the code much simpler, and there doesn't seem to be a need for the separate scripts...

Otherwise, the 2 outputs are saved as expected and the html checks file looks much clearer now the dates have been turned 😄

Hi @abhsheksingh
Thanks for making the changes.
Just a couple more needed:

The README.md file needs updating (as it still says to copy and paste the tables into the excel file)
You've still got 2 separate scripts doing the update (update_8.6.1.R and function.R) and you now call them both in compile_tables.R - however, both of them output a dataframe called csv_output so the second script is overwriting things from the first script (I think...). I recommend combining the code from update_8.6.1.R and function.R into one script (called update_8.6.1.R) and only calling this script in compile_tables.R - it will make the code much simpler, and there doesn't seem to be a need for the separate scripts...

Otherwise, the 2 outputs are saved as expected and the html checks file looks much clearer now the dates have been turned 😄

Thanks Katie for the review & comments, I made the changes.

Sure Thanks Katie,  I will change the Readme file and then merge it to main.


After commit eeb309c97cdbbf1aa9921f108bcb7dc2b79efb52 (Bring sdata back into use) Harvey does not boot properly in QEMU

Hi,
This is what I have just observed. Going back and forth over the commit log resulted in the isolation of the commit mentioned in the title. I use the following shell script to boot Harvey in QEMU:
https://gist.github.com/dmgolubovsky/2895c66c1b93906c03840cc01fc0aa7a
This script is slightly modified from the original script to run CPU server, just to run a terminal with large resolution (using qxl for video).
Maybe I am doing something wrong, maybe sdata has some unexpected behavior.
Expected behavior
Harvey boots with terminal service starting, rio screen appears
Actual behavior
The boot process is stuck after starting dns.
Steps to reproduce

git checkout f5bfe81
./bootstrap.sh && time ARCH=amd64 util/build
GO9PRUNT # the script referred to in the first gist

then it boots, starts rio

git checkout eeb309c
./bootstrap.sh && time ARCH=amd64 util/build
GO9PRUNT # the script referred to in the first gist

then gets stuck on starting dns, and rio screen does not appear.
QEMU output
When it boots (before the commit)
https://gist.github.com/dmgolubovsky/c421a9e312a41e88e124046c3c916bfd
When it does not boot (after the commit)
https://gist.github.com/dmgolubovsky/b4fec7eb56db955bc46027cd6af60b15
Thanks.

But if I change the machine flag to q35,accel=kvm I can boot into rio even with the current master.
QEMU output:
https://gist.github.com/dmgolubovsky/d3656c131f50d9dffb4ddbe9f77837c2
pci(8) output:
https://gist.github.com/dmgolubovsky/b7acdbe4c5bb6f9965387df12c3cc123
What can be the difference making sdata so sensitive?
Thanks.

This is great work, do you mind pasting a diff here of that file, so we have a complete record.

This is diff between https://gist.github.com/dmgolubovsky/2895c66c1b93906c03840cc01fc0aa7a and the current script. The only meaningful change is replace pc - > q35 in the machine flags. Will be easier to apply manually IMHO.
This is not exactly the GO9PRUN script from the repo, but should work same way in the script from the repo.
--- usr/harvey/tmp/G.pc 2016-06-12 16:35:28.903411636 -0400
+++ /home/golubovsky/bin/GO9PRUNT       2016-06-12 15:12:16.807311129 -0400
@@ -5,6 +5,7 @@
trap : 2
export HARVEY=$HOME/src/harvey
+export KVM=$HOME/src/kvm-exp/external/qemu-2.6.0/x86_64-softmmu/qemu-system-x86_64
export KVM=/usr/bin/kvm
$HARVEY/util/ufs -root=$HARVEY &
@@ -13,7 +14,7 @@
export machineflag=pc
if [ "$(uname)" = "Linux" ] && [ -e /dev/kvm ]; then
export kvmflag='-enable-kvm'

  export machineflag='pc,accel=kvm'

  export machineflag='q35,accel=kvm'
  if [ ! -w /dev/kvm ]; then
          # we don't have access as a regular user
          export kvmdo=sudo

There is a presentation http://www.linux-kvm.org/images/0/06/2012-forum-Q35.pdf (by someone from the KVM team) explaining differences from the traditional pc (440fx). I am OK just with this workaround so I can continue my work.
Thanks.

Well, sdata needs a second round. I440FX needs isa too, I'm preparint devpnp for that. It comes until ICH7 and some nforce/via similar age. Qemu q35 is a ICH9 model which is ahci. Avoiding pc option ensures no error/hang on booting for now. But the problem is that sdata must be fixed at all.

Try:
kvm -s -m 2048 -monitor stdio -serial /dev/pts/2 -machine q35 -net nic,model=rtl8139 -net user,hostfwd=tcp::1522-:1522 -redir tcp:17010::17010 -redir tcp:17013::17013 -device ahci,id=ahci -drive id=disk,file=$PATH_TO_YOUR/harvey.qcow2,index=0,cache=writeback,if=none -device ide-drive,drive=disk,bus=ahci.0 -append "service=cpu nobootprompt=tcp fs=<IP_ADDRESS> auth=<IP_ADDRESS> nvram=/boot/nvram nvrlen=512 nvroff=0 mouseport=ps2 vgasize=1024x768x8 monitor=vesa" -kernel sys/src/9/amd64/harvey.32bit -vga vmware

This will boot with ahci driver.

sdata has been dropped. I brought sdata to be compatible with ASUS P5KPL/1600 which is where I have Harvey running in real iron. Sadly, that ICH7 series hasn't AHCI. sdata is old hardware, 32 bit related even if some core duo series were using i440FX model at early 2008. Unless someone has a good sdata driver replacing for legacy IDE/SATA/PATA chipset, Harvey won't support anymore. Source file for now is still in kernel tree, for researching purposes.


N5 file can't be opened from private S3 bucket with shared queue

Opening an n5 file in a private bucket like so works:
final ImageJ imageJ = new ImageJ();
        imageJ.ui().showUI();
        try {
            S3Utils.setS3AccessAndSecretKey( new String[]{"acess","key"} );
            SpimData spimData = (SpimData) new SpimDataOpener().openSpimData("path/to/bucket/image.xml", ImageDataFormat.BdvN5S3);
            BdvFunctions.show(spimData);
        } catch ( SpimDataException e) {
            e.printStackTrace();
        }

but with a shared queue doesn't. Same as above but with:
SpimData spimData = (SpimData) new SpimDataOpener().openSpimData("path/to/bucket/image.xml", ImageDataFormat.BdvN5S3, new SharedQueue( N_FETCHER_THREADS ));

Errors with:
mpicbg.spim.data.SpimDataException: Error while trying to read spimDataServer returned HTTP response code: 403 for URL: https://s3.embl.de/meechan-user/test/images/bdv-n5-s3/mri-stack.xml
	at org.embl.mobie.io.SpimDataOpener.openBdvN5S3(SpimDataOpener.java:110)
	at org.embl.mobie.io.SpimDataOpener.openSpimData(SpimDataOpener.java:69)
	at run.Temp.main(Temp.java:70)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.intellij.rt.execution.CommandLineWrapper.main(CommandLineWrapper.java:64)

Related to: https://github.com/mobie/mobie-viewer-fiji/issues/621

@KateMoreva - would you have time to look into this?

@KateMoreva - would you have time to look into this?

Yes, I'll look into this today or/and tomorrow.

This is fixed now :)


Fixed typo

Fixed typo - changed 'light' to 'lite'

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@CarolynRowe  changes look correct. Could you please confirm and merge? Thanks!

@CarolynRowe Changes look correct. Could you please review and merge? Thanks!

Let's discuss these changes.  I'm OK with the first change, but have questions about the other changes.


Error - unknown variant default, expected one of any, boolean, number, string at line 1 column 25634

I am attempting to convert a Postman Collection v2.0 and v2.1 with the online tool, and I receive the following errors
unknown variant `default`, expected one of `any`, `boolean`, `number`, `string` at line 1 column 25634

I cannot share the Postman Collection until I've scrubbed it, but I have not been successful at getting it to convert my stuff.

Turns out either Postman or the person who originally created the collection put in erroneous data types. Using a Find & Replace to change type": "default" to type": "any" made the error go away, but now I don't have any type definitions, but that's on me to fix and default is just as good as any.


Bug with dotnut chart

Creating a 'dotnut' chart, i get error "Uncaught (in promise) TypeError: this.xyRatios is null".
And when i create a 'pie' chart whith same options, all is correct.
I use ApexCharts v3.27.3 and my code is :
let options1 = { labels: ["cat 1","cat 2","cat 3"], series: [44, 55, 41], legend: {position: 'bottom'},
chart: {type: 'dotnut', width: '100%', height:'350px'}, plotOptions: {pie: {dotnut: { size: '30%'}}} };
let chart1 = new ApexCharts(document.getElementById("fortest"), options1);
chart1.render();
Changing to 'pie' chart, all is correct :
let options1 = { labels:  ["cat 1","cat 2","cat 3"], series: [44, 55, 41], legend: {position: 'bottom'},
chart: {type:'pie',width:'100%',height:'350px'} };

Chart type should not  bedotnut, it has to be donut


Latency issue possibly caused by avformat_find_stream_info

Live streaming through Nginx to avconv to bmdplay to Quad SDI card.
Was getting 2x a latency value (5-6s). Once through avconv and once through bmdplay. I was able to almost eliminate the latency through avconv with explicitly setting framerate (since I know it) and also setting the max time that avformat_find_stream_info looks for stream information.
-r 30 -probesize 750000 -analyzeduration 750000
full command:
avconv -r 30 -max_delay 1000 -probesize 750000 -analyzeduration 750000 -fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut -strict experimental -c:v copy -c:a pcm_s16le - | /home/vidcontrol/Builds/'Blackmagic DeckLink SDK 10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 10 -C 1 -f pipe:0
I am looking for something similar in bmdplay. Even better, since my format is fixed (I control the encoding and the RTMP streaming), I would like to be able to provide all of these hints up front so that I can reduce start up latency through bmdplay.
In fact, there is a note in the libav code about doing just this
from libav/libavformat/avformat.h @ ln 1445

@todo Let the user decide somehow what information is needed so that

  we do not waste time getting stuff the user does not need.

*/
int avformat_find_stream_info(AVFormatContext *ic, AVDictionary **options);
avconv using avformat looks for these supplied by user
nb_audio_sample_rate
nb_audio_channels
nb_frame_rates
nb_frame_sizes
nb_frame_pix_fmts
then picks a codec (decoder) which will have it's own set of parameters and then uses avformat_find_stream_info to fill in the blanks.
In bmdplay I think it would be possible to set these after here:
378    avformat_find_stream_info(ic, NULL);
ic->max_delay = 1000;
ic->framerate = 30;
ic->flags |= AVFMT_FLAG_NOBUFFER;
ic->probesize = 750000;
ic->max_analyze_duration = 750000;
ic->fps_probe_size = ?????;
ic->strict_std_compliance = true;
Of course the right way to do this is through the command line args.
Am I on the right track?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/22974241-latency-issue-possibly-caused-by-avformat_find_stream_info?utm_campaign=plugin&utm_content=tracker%2F256764&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F256764&utm_medium=issues&utm_source=github).

I mistakenly wrote above that this code should be after but it should be before.
Using the following test code I was able to reduce the latency in bmdplay by ~30%:
ic->max_delay = 1000;
ic->flags &= AVFMT_FLAG_NOBUFFER;
ic->probesize = 1000000;
ic->max_analyze_duration = 1000000;
ic->fps_probe_size = 30;
for (int i = 0; i < ic->nb_streams; i++) {
    AVStream *st = ic->streams[i];
    st->avg_frame_rate.num = 30;
    st->avg_frame_rate.den = 1;
}

Thanks for reminding me, I'll add soon a nolatency option.

I'll add a no-latency option to skip that part completely soon, thank for
reminding me!

Luca,
Thanks. Not pushing but our event is in three weeks. Which part would you skip? The function call I noted or somewhere else?
Thanks again.
Glenn
—
Sent from Mailbox
On Thu, Jun 25, 2015 at 12:28 AM, Luca Barbato<EMAIL_ADDRESS>wrote:

I'll add a no-latency option to skip that part completely soon, thank for
reminding me!
Reply to this email directly or view it on GitHub:
https://github.com/lu-zero/bmdtools/issues/58#issuecomment-115109107

In theory all the information should be already provided by NUT (that's why I'm using it), in theory the find_stream_info call should be a no-op.
If it is not currently, dropping it shouldn't impact the rest as long NUT is used, possibly the -syncpoints none variant.

Got it. I'll give it a try
—
Sent from Mailbox
On Thu, Jun 25, 2015 at 12:54 AM, Luca Barbato<EMAIL_ADDRESS>wrote:

In theory all the information should be already provided by NUT (that's why I'm using it), in theory the find_stream_info call should be a no-op.
If it is not currently, dropping it shouldn't impact the rest as long NUT is used, possibly the -syncpoints none variant.
Reply to this email directly or view it on GitHub:
https://github.com/lu-zero/bmdtools/issues/58#issuecomment-115114771

Looks like there is an audio parameter missing, so it isn't just dropping that call.
You are doing -c:v copy, decode it with -c:v rawvideo or -c:v v210 if you want 10bits output.

Luca,
A question, the video is h.264 encoded by Teradek Cube, 720p@30fps. I am trying to output through bmdplay and Quadlink SDI card at 1080i@59fps. Having a very difficult time getting scaling to work. Is this related to using v:copy?
avconv -r 30 -max_delay 1000 -probesize 500000 -analyzeduration 3000000 -fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut -strict experimental -c:v copy -c:a pcm_s16le - | /home/vidcontrol/Builds/'Blackmagic DeckLink SDK 10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 12 -C 1 -f pipe:0
I get a 720p region in the upper left corner. Thoughts? Suggestions?
I am not too familiar with Avconv and this may be better posted over there.
Glenn

You are not doing any scaling. bmplay does not scale for you. You do it
with avconv. Add a -s 1920x1080 or use -vf scale
On 11 July 2015 at 08:39, glennolsen<EMAIL_ADDRESS>wrote:

Luca,
A question, the video is h.264 encoded by Teradek Cube, 720p@30fps. I am
trying to output through bmdplay and Quadlink SDI card at 1080i@59fps.
Having a very difficult time getting scaling to work. Is this related to
using v:copy?
avconv -r 30 -max_delay 1000 -probesize 500000 -analyzeduration 3000000
-fflags nobuffer -i rtmp://localhost:1935/AV2015Local/stream2 -f nut
-strict experimental -c:v copy -c:a pcm_s16le - |
/home/vidcontrol/Builds/'Blackmagic DeckLink SDK
10.4.1'/Linux/Samples/bmdtools-master/bmdplay -m 12 -C 1 -f pipe:0
I get a 720p region in the upper left corner. Thoughts? Suggestions?
I am not too familiar with Avconv and this may be better posted over there.
Glenn
—
Reply to this email directly or view it on GitHub
https://github.com/lu-zero/bmdtools/issues/58#issuecomment-120587313.

--
Håkon

Thanks, I pasted the wrong command I have tried both. Turns out that it is my error. I am streaming 720p and the Blackmagic card is scaling the SDI output to 1080i because that is how I configured it in Desktop Video Utility. Looks good for now. On to the next issue that I will post in another thread.
Thanks again.
Glenn


Desktop App: Add a download progress bar at the bottom of the screen

There is currently no indication to the user that after they have clicked on a file to download, and picked the download location that the file is actually downloading.
Proposed design:
https://projects.invisionapp.com/share/YDMY3VHWKTA#/screens/308612367

The download bar can be reopened by going to “View > Downloads” if there are downloads to show.

If you're interested please comment here and come join our "Contributors" community channel on our daily build server, where you can discuss questions with community members and the Mattermost core team. For technical advice or questions, please  join our "Developers" community channel.
New contributors please see our Developer's Guide.
JIRA: https://mattermost.atlassian.net/browse/MM-22239

Let me take a stab at this one

Hi, I am interested to take this up! Can I take a look?

@laxmy, apologies for the delay. Absolutely you can take this! Let us know if you're still interested and I'll assign this to you and remove the "Up for Grabs" label

@esethna Thanks. I would like to give  it a go. Please assign it to me.

@laxmy you got it! Let me know if you have any questions. You can also ask our desktop devs if you have any questions at https://community-release.mattermost.com/core/channels/desktop-app

@Willyfrog can help answer any technical questions. Once you've had a chance to dive in we're also happy to review your proposed approach before you get too deep into the coding

Sure thing. Will get back to you or @Willyfrog once I wrap my head around it

@laxmy are you still interested in taking this on?

@laxmy are you still interested in taking this on?

@esethna This design is a fair bit older. I'm definitely up for doing this feature, but maybe the design should be looked at a second time?
cc @matthewbirtch

Good question, I'll leave that up to @matthewbirtch if we want to revisit the design.

@esethna This design is a fair bit older. I'm definitely up for doing this feature, but maybe the design should be looked at a second time?
cc @matthewbirtch

@devinbinnie the design did get revisited actually, but this github ticket didn't get the update - not sure how that gets synced. Here's the jira ticket with the updated design: https://mattermost.atlassian.net/browse/MM-22239
However, I should probably update the screens again since the desktop app has changed significantly since then.
Should we close this ticket and re-open a new HW ticket once I've taken another look at it?

Yeah that works for me. Closing this.


Failing builds due to codacy-coverage dependency joi engine >=8.9.0 requirements!

I've got some legacy node v6 builds failing due to an engine requirement from one of your deps:
https://travis-ci.org/mgcrea/redux-rest-resource/jobs/393570711
yarn install v1.3.2
[1/4] Resolving packages...
[2/4] Fetching packages...
info<EMAIL_ADDRESS>The platform "linux" is incompatible with this module.
info<EMAIL_ADDRESS>is an optional dependency and failed compatibility check. Excluding it from installation.
error<EMAIL_ADDRESS>The engine "node" is incompatible with this module. Expected version ">=8.9.0".
error Found incompatible module
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command "eval yarn " failed. Retrying, 2 of 3.

$ yarn why joi
yarn why v1.7.0
[1/4] 🤔  Why do we have the module "joi"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found<EMAIL_ADDRESS>info Reasons this module exists
   - "codacy-coverage" depends on it
   - Hoisted from "codacy-coverage#joi"
info Disk size without dependencies: "156MB"
info Disk size with unique dependencies: "256MB"
info Disk size with transitive dependencies: "272MB"
info Number of shared dependencies: 4
✨  Done in 0.65s.

Since your package is supposed to be used for testing having it stuck only on node >=8.9.0 is a bit counter-productive.

@mgcrea we had some issues with old versions and due to some security one we decided to update this.
At the moment the requirement is "node": ">= 6.0.0" and our tests are passing https://github.com/codacy/node-codacy-coverage/blob/master/circle.yml#L3
Not sure why they did not fail initially. Do you have any suggestion for an alternative?

Closing for inactivity.


Centos 7 vuild fails to compile due to missing headers

Centos 7 builds failed to compile due to missing header after Socket refactoring. The fix puts the needed headers.

Test PASSed.


[PROPOSAL] Add guidelines for tests to maintainers.md file

What kind of business use case are you trying to solve? What are your requirements?
We need to define standards and guidelines around the tests for the contributions to improve the quality of the code.
What is the problem? What is preventing you from meeting the requirements?
Currently, we dont have any guidelines around tests hence code contributions and merges are very subjective at this point in time
What are you proposing? What do you suggest we do to solve the problem or improve the existing situation?
Suggest adding below section to https://github.com/opensearch-project/.github/blob/main/MAINTAINERS.md
Test the Changes
Check whether tests already exist in the project to test the contribution. If so, make sure to execute it every time before merging the changes.  If the test doesn’t exist, understand the type of testing requirement ( Integration? Unit? Acceptance? ) for the contribution and see if we can re-use the framework by setting up a new testing harness for the project.
If adequate testing for a contribution does not exist, it is our responsibility to make sure the tests are added either by requesting the contributor to add one or more tests OR by adding the tests ourselves. If we are merging a fix, the added tests should fail without the fix. If we are merging an enhancement, we should make sure that the tests thoroughly exercise the new functionality.
What are your assumptions or prerequisites?
Describe any assumptions you may be making that would limit the scope of this proposal.
What are remaining open questions?
List questions that may need to be answered before proceeding with an implementation.

I like the spirit of this. I would rephrase it slightly and be shorter and more prescriptive. Particularly "we" and "us" are misused in the proposal and if it's directed at maintainers then it should say what the maintainer responsibilities are. Could be something like this:
Require Automated Tests
Provide and maintain continuous integration, test harnesses and code coverage for the project. Ensure tests are executed for every contribution before code is merged. Accept only pull requests with sufficient test coverage, including but not limited to unit, integration and acceptance tests. Encourage improvements that increase the amount of automated testing, call out and fill gaps in testing by opening issues and asking the community for help increasing quality of the software through automated testing.


how to get an image with marker and zoom level?

There are two methods to get the image , but how to get an image with marker and zoom level?
Uri getStaticUri(Location center, int zoomLevel, {int width, int height}) {
return _buildUrl(null, center, zoomLevel ?? defaultZoomLevel,
width ?? defaultWidth, height ?? defaultHeight);
}
Uri getStaticUriWithMarkers(List markers, {int width, int height}) {
return _buildUrl(
markers, null, null, width ?? defaultWidth, height ?? defaultHeight);
}

By default, the static maps API will adjust the zoom to fit around your markers so we didn't have a need for that. I'm not sure what the static maps behavior would be if you specified markers and a zoom level but you should be able to play with this part of the plugin to try it out.


Update to compliance-trestle ~= 3.3

Updates compliance-trestle dependency to enable oscal 1.1.2 document use.

@rahearn Thank you so much for updating Trestle and applying the corresponding code fixes!
Could I request to include one additional update on samples_public/heterogeneous/result_to_compliance.py (https://github.com/rahearn/compliance-to-policy/pull/1)?


[feature] teku pretty print subcommand

Description
In (under development) new beacon-fuzz tooling, we are using client tools like zcli, lcli, ncli, ... and we compared the result/return value between all tools.
I would to have a teku subcommand to preform pretty print on most common type, like those ones:

Attestation
BeaconBlock
BlockHeader
Deposit
ProposerSlashing
SignedBeaconBlock
SignedBlockHeader
SignedVoluntaryExit
BeaconState
VoluntaryExit

That will allow us to have an easy way to test SSZ decoding and parsing between teku and other clients. We consider zcli as the reference spectest tool.

Is the aim here to actually have the pretty output that you're then parsing to determine if the parsed result matches or just to see if we consider the input valid or not?

Both ;) but main goal is to compared parsing result since output will be a JSON.


My cursor bounces all over the place.

Is there anything that can fix this cursor problem from bouncing all over the place?

This is already a known bug and if you would've red my comment then you would've seen that this issue has already adressed in issue #7847
Please use chatrooms if you want tips how to tackle this problem. You can also view our full list of official chatrooms.

So it was a year ago and it obviously won't get fixed.
Sent from my Verizon 4G LTE smartphone
-------- Original message --------
From: Lauri<EMAIL_ADDRESS>Date: 4/15/17 3:28 PM (GMT-08:00)
To: freeCodeCamp/freeCodeCamp<EMAIL_ADDRESS>Cc: Fatbacker<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [freeCodeCamp/freeCodeCamp] My cursor bounces all over the place. (#14423)
This is already a known bug and if you would've red my comment then you would've seen that this issue has already adressed in issue #7847https://github.com/freeCodeCamp/freeCodeCamp/issues/7847
Please use chatrooms if you want tips how to tackle this problem. You can also view our full list of official chatrooms.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/freeCodeCamp/freeCodeCamp/issues/14423#issuecomment-294321349, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AaQvECSo_Nm0FpNH68MCfrZv7DrnOroPks5rwUR1gaJpZM4M-ehH.

@fatbacker feel free to make a Pull-request for it with a fix..


Is it possible to show all dependencies at once?

I wanto see all depdendencies in the time interval I select.
Image that slices and its depedencies formed a graph.
Now an dependency arrow is presented only when one of the slice in the graph the dependency involved in is selected.
Is there any way to show al the depedencies in the interval I select?

If I understand correctly, you want to see all flows in and out of all slices within an area selection.
This can be achieved by clicking on the "Flow Events" tab and selecting the "Show" checkbox as in the screenshot below.


Minute 09 missing from market creation wizard

As said in title, there is no 09 option in the dropdown.

Thanks @danuker!
This has been fixed now. Closing.


x/text: iterator does not handle zero width joiner

What version of Go are you using (go version)?

$ go version 1.20

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
I'm using the go playground
What did you do?

package main

import (
	"fmt"

	"golang.org/x/text/unicode/norm"
)

func main() {
	var ia norm.Iter
	s := "👩🏾🦰👱🏾🧑🏾⚖"

	ia.InitString(norm.NFD, s)
	rev := ""

	for !ia.Done() {
		n := ia.Next()
		rev = string(n) + rev
	}

	fmt.Println(rev)
}

Playground:  https://go.dev/play/p/Jq9CnUd2zA8
What did you expect to see?
It should print 🧑🏾⚖👱🏾👩🏾🦰
I think it has to do with the library not handling zero width joiners.  If I use this string:  "😀😂"  it works as expected.
What did you see instead?
⚖🏾🧑🏾👱🦰🏾👩

CC @mpvl via https://dev.golang.org/owners

The underlying issue is that x/text doesn't handle multi-rune unicode grapheme clusters.
In the meantime, you could use this third-party module uniseg to iterate through grapheme clusters.


Abort Mission report format can become broken

Description
Some inputs can break the Abort-Mission report, causing it to confuse the number of col/row spans needed in case of certain tests
Example files attached
abort-mission.zip

My investigation found cases, where the same thread appeared to be executing multiple things at a time according to the json test data report. This was most likely due to incorrect (too aggressive) concurrency configuration as the Jupiter tests were using concurrent both on the test and class levels using per_method lifecycle. This meant that sometimes when an instance was initialized, the threads were reported incorrectly.
As the number of occurrences was low, and simply reducing the concurrency to run the methods of the same class on the same thread and use per_class lifecycle fixed the issue, there will be no code change to attempt a fix on Abort-Mission side for now.


Update US-GA-Forsyth

Closes: https://github.com/openaddresses/openaddresses/issues/2221

# us-ga-forsyth-1544624945921.geojson.gz
ok - 99389/99667 (99.7%) valid addresses
ok - 0 outside of boundary (dropped)
ok - 207 have an unsupported number format
ok - 71 are missing street names
ok - 0 could not be parsed
ok - 0 suffered misc failures


In init new collection scaffold a playbook and role

We should include some "ansible" content, even if they are just short simple files.

Related JIRA issue: https://issues.redhat.com/browse/AAP-19822


Undocumented options

Looks like there's some undocumented options that can be set with HighsOption:
https://github.com/ERGO-Code/HiGHS/blob/master/src/lp_data/HighsOptions.cpp#L762
Should do two things:

document and support in wrapper
add documentation for upstream

Initial documentation of advanced options done, simply note that they exist, their types, and that they are undocumented.


ch_detected_temperature displayed 0

Hi.
After updating to version 2.0 I no longer see the temperature detected by the internal thermostat (ch_detected_temperature), it always displays 0, before this update it was always displayed correctly. I have a Genus One boiler. Some idea? Thank you.

After 11th of June implemented functionality stopped working due to dropped support of requests and thus 80-90% of integration were rewritten to work with new requests.
I have checked in my boiler that sensor.ariston_ch_detected_temperature displays correct value.
So it hints that your boiler provides 0 for specific parameter that is being parsed. It means that requests for your boiler must be checked on sent/received data in case detected temperature is stored in different structure in your case or it does not report such data at all.
I use Fiddler and configure proxy on it and connect Android Wifi to that proxy and run Ariston application (open pages, which trigger requests).

Just to be clear: data needs to be collected for your boiler specifically as I have no idea what can be an issue.

I have a same problem! You have find solution


How to add a recursive list of paths in the PYTHONPATH manager

Description
What steps will reproduce the problem?
I want to know if Spyder provides the ability to add a recursive list of folders/directories in the PYTHONPATH manager.
1.
2.
3.
What is the expected output? What do you see instead?
Please provide any additional information below
Version and main components

Spyder Version: 2.3.8
Python Version: 2.7.6
Qt Versions:  4.8.6, PyQt4 (API v2) 4.10.4 on Linux

Optional dependencies
pyflakes >=0.5.0   :  None (NOK)
pep8 >=0.6         :  None (NOK)
pygments >=1.6     :  1.6 (OK)
IPython >=1.0      :  1.2.1 (OK)
zmq >=2.1.11       :  14.0.1 (OK)
pandas >=0.13.1    :  0.17.1 (OK)
sphinx >=0.6.6     :  1.2.2 (OK)
psutil >=0.3       :  2.2.1 (OK)
rope >=0.9.2       :  None (NOK)
jedi >=0.8.1;<0.9.0:  None (NOK)
matplotlib >=1.0   :  1.5.1 (OK)
pylint >=0.25      :  1.4.3 (OK)

I want to know if Spyder provides the ability to add a recursive list of folders/directories in the PYTHONPATH manager.

No, it doesn't, and there is no need for that. When you add your topmost directory to PYTHOPATH, you can import what's beneath it by using the following syntax:
from topdir.subdir.subsubdir import foo

For this to happen, the only thing you need to do is adding empty files called __init__.py (notice the two underscores to both sides) to topdir, subdir and subsubdir. That's all :-)

In python3 you don't need empty __init.py__ anymore. I don't remember which version exactly.


Update readme file

Thank you for taking the time to work on a Pull Request for this project!
To ensure your PR is dealt with swiftly please check the following:

[x] Your submissions are formatted according to the guidelines in the contributing guide
[x] Your additions are ordered alphabetically
[x] Your submission has a useful description
[x] The description does not end with punctuation
[x] Each table column should be padded with one space on either side
[x] You have searched the repository for any relevant issues or pull requests
[x] Any category you are creating has the minimum requirement of 3 items

couple things:

remove extra whitespace at end of lines
remove extra newlines at end of sections (should be 1, not 2)
your fork is not up-to-date with the current mater. Please rebase so that your diff does not include regressions on entries you are not updating (labstack, for example).

@davemachado Have updated, as commented above.

Bump


Your library is unaware of the recently introduced 64 bit transaction id's

Meaning: int's are not sufficient to store Event.TranId anymore, resulting in failures.
http://developer.oanda.com/rest-live/64-bit-transition/

Thank you for reporting this.
You should be fine as long as you're on a 64bit OS but I'll update the code shortly so that it will also work 32bit platforms.

Well I'm currently running it on OS X (el capitan) 64 bit.
Forced a 64 bit build with: GOOS=darwin GOARCH=amd64 go run *.go
I have patched oanda/events.go and replaced all TranId int's with uint64..
Still no luck :-) Could it be the json decoding or so?

I'm sure I ran the tests successfully even after the 64bit transition (but I'm on 64bit Linux).
Please feel free to share your patch as a PR but I'll take a closer some time later this week regardless.

I opened a work in progress PR. So, the interesting part is that:
Event{TranId:<PHONE_NUMBER>1, AccountId: 3952343 Type: MARKET_ORDER_CREATE}
All good for that order type.
But changing your account's margin (that was what I tried to do several times):
TranId is<PHONE_NUMBER>, resulting in ApiError{Code: 13, Message: Transaction not found, Moreinfo: http://developer.oanda.com/docs/v1/transactions}

Emailed OANDA api support, because the raw message for margin type already contains a 32 bit max int value. Asked them to verify if this is actually going wrong on their side.

I don't think this issue is caused by the lack of 64-bit support.
I tried building the package using GOARCH=386 and it failed to build so you must have been using a 64-bit build already (where an int is already 64 bits afaik).  I want to support 32 bit architectures too so I'll fix any issues there.  Currently my WIP is on branch "support-64-bit-ids" if you're interested to take a look or contribute.
I received a similar error as yours (on a 64-bit build) so I'd be interested to hear the response from OANDA api support.

Hi @kciredor - I have merged my changes for 64 bit Id support on 32 bit architectures into master.
If you still have the same problem can you please provide an example program that I can use to debug the problem?
Thanks.

Very nice @santegoeds much appreciated.
You can reproduce easily by forcing a SET_MARGIN_RATE (simply change your account margin manually and listen for the event). If you fmt.Println(msg) on events.go:544, you'll see a 32 bit int already. Fairly confident it's not your (awesome) library issue now.
I'm still waiting for OANDA support, the response I received so far besides the auto reply was "we're investigating". Will definitely keep you posted!

Hi @kciredor I just fixed a bug where SET_MARGIN_RATE events/transactions were not correctly unmarshalled.
This would have resulted in a panic if you tried to access get the Rate() from the event.  If this was the case then please retry.
Ps I renamed Rate() to MarginRate().

Cool! Did not call Rate() yet (got stuck with the int32 TranId issue). Still waiting for OANDA...
Meanwhile I opened a PR fixing the TradeCreateEvent call for you.

Great, just merged the PR.  Many thanks!

Received response from OANDA:
"Overflown transactions will be only returned for SET_MARGIN_RATE and FUND related transactions on fxPractice only as it is set by a different system. Live fxTrade platform transactions are all intact."
Closing this issue. Thanks for your time @santegoeds !


docs: mention our bundle file clearly.

Note that we are using the bundle file already in our docs since 787441d.
Preview: https://deploy-preview-31300--twbs-bootstrap.netlify.app/

One copy tweak, and then okay to open for review?

Pushed some changes and rebased.

Homepage now only shows the JS bundle option.  Let's streamline there, and expand on inner docs pages.
Intro page has been rearranged to clarify the JS quick start options. New headings, rearranged paragraphs, etc.
Improved labels and commented out the second JS option in the starter template.

Should be good to go after another review from @XhmikosR.

Perfect, thanks!

BTW backporting might be tricky, not sure when I'll get to do it.

@mdo this is a little tricky to backport, if you could take a stab at it, I'd appreciate it 🙂


Road to v1.0 stable?

I want to ask if you have any roadmap in mind for first stable version? It can help other repositories using highway-env, by not having to change their code frequently because of changes in the highway-env.

Hello, @mhtb32 , is that possible to add the DDPG et al. for the continuous space in Highway-env? I have tried to use DDPG+HER to realize this, but failed. Thanks in advance.

@narutoten520 highway-env package is a collection of gym environments itself and has nothing to do with the RL models you want to train. The DDPG model needs continuous action environments, and any highway-env environment with continuous actions can be used for training a DDPG agent, like parking-v0. There is a python notebook that trains exactly the DDPG + HER on the parking-v0 environment using the stable-baselines package, which you can check here.
You can check the docs for further information about highway-env, and I suggest you comment on related issues(or make a new one) in the future to ask about highway-env functionalities.
Thanks.

@mhtb32 Thanks for your response and suggestion. I will use the new comment for my next issue, Thanks.

Hi @mhtb32, thank you for answering narutoten.
Regarding your original question, I do not have anything planned in particular, so now is as good a time as any for a first stable version. I may just finish the few missing sections in the documentation before settling.

v1.0 has been released (before merging the multi-agent feature).


improve scripted column

the help and editor needs some 'attention'

xcell provides an idea how scripted cells similar to a spreadsheet can work. Maybe we can adapt the LineUp implementation and provide a better usability providing suggestions while typing.

In LineUp v4.0.0-alpha.8 the whole help text is gone.

Though I opened a new issue to improve the scripted column -> https://github.com/datavisyn/lineupjs/issues/116


Pseudo functions NOT and HAS are not efficient

the css expression is parsed for each node matched.

Very neat @benbenw.
Thank you very much for contributions - you earned the top contributor place for this month. Or even for a year :))


Modal content has bottom border-radius when no header is present, but a footer is.

This LOC adds a border radius if there is no header, but does not take in to account the presence of a footer, resulting in some funkiness: https://github.com/salesforce/design-system-react/blob/master/components/modal/index.jsx#L207

This is reproducible with the first example on the demo page https://react.lightningdesignsystem.com/components/modals/ by removing the title prop.
Ideally this would add a border-radius class (using $border-radius-medium design token?) to the modal instead of an inline style, so that we could redefine it in trailhead (our theme's border-radius is 0.5rem for modal windows).

It seems like we should fix it for this instance and provide a good default of no radius on the bottom, as to custom styles, is contentStyle with a SLDS token sufficient?

Oh, I see what you are saying. If SLDS doesn't have a border radius class (and I don't think it does) https://www.lightningdesignsystem.com/utilities/borders, you will need to file a feature request with them for a class.
For another possible solution, I'm open to using JSON from the design tokens library for inline styles, too. There's not a good framework for it though in this library at this time.

I published a pull request that solve the problem. #1976


GCP config file not found - " no such file or directory"

When i'm trying to run the script as follows:
docker run -e TZ=America/Chicago -e<EMAIL_ADDRESS>-e PASSWORD=1231231123 -e GCP_CONFIG_NAME=weeklyepicgames-xxxxxxxxxxx.json -e RUN_ON_STARTUP=true -v /mnt/user/appdata/epicgames-freegames/:/usr/app/config:rw charlocharlie/epicgames-freegames:latest

I receive the error even though I've checked "/usr/app/config/" directory and the "fweeklyepicgames-xxxxxxxxxxx.json" is clearly there.
><EMAIL_ADDRESS>start /usr/app
> node dist/index.js

[2020-04-25 02:16:10.405 -0500] ERROR (24 on 0da6d9c67ee3): ENOENT: no such file or directory, open '/usr/app/config/fweeklyepicgames-xxxxxxxxxxx.json'
    Error: ENOENT: no such file or directory, open '/usr/app/config/weeklyepicgames-xxxxxxxxxxx.json'
(node:24) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'body' of undefined
    at main (/usr/app/dist/index.js:309:24)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:24) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:24) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Are you mixing up the host and container directory? /usr/app/config is the container directory. The host directory is the first of the pair, so /mnt/user/appdata/epicgames-freegames/ in the example.

Yes, I mixed up directories. Moved my "fweeklyepicgames-xxxxxxxxxxx.json" to "/mnt/user/appdata/epicgames-freegames/" and it worked.


[Question] Modified Labeled Heatmap

Hi team,
I saw your showcase about Labeled Heatmap, it's so good.
I intend to make the correlation matrix base on this example, and the result should be like this example:
http://saurfang.github.io/d3-corrplot/doc/
However, I wonder can I only draw only haft of the square (it means the triangle), and custom the size of datapoint as its value..
Thanks

Hey @thinhly197
Couple questions:

are you asking to only draw the upper half of the the labeled heatmap, eg above diagonal?
are you asking for tooltips on that show the value of the cell?

gotcha! For question 1, if you dont want those cells to be there you don't have to pass them in, for instance here is a modification to the data set up from my Labeled Heatmap example
const data = alphabet.reduce((acc, letter1, idx) => {
  return acc.concat(
    alphabet.map((letter2, jdx) => ({
      x: `${letter1}1`,
      y: `${letter2}2`,
      color: (idx + jdx) % Math.floor(jdx / idx) || idx
    }).filter((letter2, jdx) => jdx >= idx))
  );
}, []);

(note the filter!) You could also do it by setting the colorType on XYPlot to be "literal" and then specifying the color on the cell to be something like color: 'rgba(255, 255, 255, 0)'.
Can you say more about this line from your original question?

custom the size of datapoint as its value

Thanks for your remind.
It likes the attached image

The value of each cell is the long float number. I tried to style it, but it cannot affect.
<LabelSeries           data={data}           labelAnchorX="middle"           labelAnchorY="baseline"           getLabel={d => ${d.value}}           style={{             width: '60%',             overflow: 'hidden',             whiteSpace: 'nowrap',             textOverflow: 'ellipsis',           }}         />

i think there might be a bug in the way labelSeries does some things. You might changing your style prop to
style={{style: { width: '60%', overflow: 'hidden', whiteSpace: 'nowrap', textOverflow: 'ellipsis', }}}

it still lost the style prop. Anyway, thanks for your support!


Git for Windows prefers HOMEDRIVE+HOMEPATH to expand ~

See https://github.com/git-for-windows/git/issues/4625 detailing my findings.
Because of this behavior, the current means of patching USERPROFILE is inadequate on Windows.

In the upstream issue, Git for Windows is not planning to address the deficiency, as it's long established and was intentionally introduced.
They did give a good suggestion, however, to set HOME even on Windows, and Git will honor that. Let's do that.


OpsCenter: How to handle CT SNS Subscriptions

CT creates SNS subscriptions to the AUDIT account email for each region by default.
We need to decide how to handle them, e.g. whether they should be forwarded to the OpsCenter.

Questions we'd ask ourselves:

is anyone interested in this information? (it's the guardrails, which can be critical)
if so, shall we pass them into the OpsCenter (which is our dedicated communication strategy)
if not, we discard them


Isolated Variable object for each Query tab

Describe your feature request
There is only one textbox for variables, and it its shared amongst all query tabs. It would be useful if each Query tab would have an isolated Variables textbox.
Are you willing to contribute this feature?
No
Related pull request (if applicable)
No

This is now the new behaviour of variables in Surrealist 2.0


Update Northstar to v1.7.1

ghcr.io/cpdt/northstar-dedicated-dev:dev.20220515.git304835f is verified to start and load.

Thanks!


Consider auto-updating package.json contents

Following #566: I'm particularly fond of Greenkeeper for CI and npm-check-updates for local development. Staying up-to-date on packages is generally a good idea:

You get the latest bug fixes and improvements, particularly security patches
You don't fall out of date with API needs (e.g. #566!)
Users have fewer duplicate npm dependency versions resolved

Considering the bump to 16.4.1 totally broke this library, I agree with having more tests before adding green keeper :)

@Daniel15 looks like I don't have permissions to enable Greenkeeper on this repo, would you mind taking a look?


Fix riscv machine timer for long running platform w/ software reset

Signed-off-by: Jaron Kelleher<EMAIL_ADDRESS>

Background info:
Symptom: The CLINT is constantly interrupting the system when the platform has been long-running and the MCU only is reset. This will not work for long running systems that have their cores soft reset.
Cause: The last_count variable is initialized to 0 when the kernel starts. When updating the mtimecmp register, this last_count value is used as the base count and then some incremental number is added to it to ensure the CLINT interrupts at a consistent frequency. The problem arises when the platform has been running for an extended period of time, and the mtime register is not reset with the MCU. Even more specifically, this problem arises when mtime grows above 32 bits. In this scenario, the first update of mtimecmp register will set the upper 32 bits of the mtimecmp register to 0 or 1 (likely 0). This will immediately trigger an interrupt since this will be much smaller than the mtime register (who has an upper 32 bit value greater than 0). This pattern continues forever.
Solution: To fix this, the last_count variable needs to be set to the value in the mtime register at initialization.
Alternate solution: Let the dticks value be a u64_t, rather than casting it to a u32_t.


Allow user to skip Send to Playground warning

A request came in to allow a user to skip the warning when sending to playground that you will overwrite the existing programs.

I also would like an option “do not show this message again” with a checkbox at the bottom here. It would increase workflow for some.

This is reasonable and will result in one less button press for savvy users.

This has been added, closing.


Search Not Working for Recent Episodes

A user reports that "search.nashownotes.com no longer lets you search up to the current show. The max is episode 1135"

Thanks for reporting. Seems that http://feed.nashownotes.com/ changed quite substantially which broke the periodic updater rake task.
Should be fixed now
https://github.com/mcclymont/nasearch-ruby/commit/eb2f46d88698d01d5c7662271efcfb10ddea4a53


Actions - Traceability Schema

User scenario
Persona: DRI
DRI got paged that one of their AKS cluster is facing an issue. DRI wants to find out what changed on the cluster from last known good state. Was there any code or config change that happened ? If yes what exactly changed and how can this be reverted to mitigate the LSI.
Guiding principles
What is a deployment ?
A deployment is something which changed the state of a given resource. Traceability means that an user can find the answers to the following three questions

Where it got deployed ?
How it got deployed ?
What got deployed ?

The resource  schema has been structured into three sections to provided answer for these questions.

Target Resource (the WHERE)
The target resource identifies the cloud resource on which the GitHub workflow deployed changes to. It consists of the properties of a resource which were impacted by the deployment.

WorkflowRun (the HOW)
The workflow run provides details around the workflow run (runUrl, timestamps, refs, workflow details etc. ). Workflow details is essentially how changes can be pushed.

Artifacts (the WHAT)
The artifact details provides a list of data which was associated with a deployment. For kubernetes for example a container image is what gets deployed. Artifacts can also contains code-coverage data, test results, blob urls etc. This is a list of everything that is associated with the deployment

Scope
The scope in V1 is to limit to GitHub Workflow generated Deployment through Actions and Targeting any Cloud type resource
Deployment Report
Deployment report provides a summary of the deployment in a generic way. It is an open cloud neutral schema which can be pushed to any data platform and later queried from to provide insights into state changes for a given resource. Deployment Report will enable us in establishing traceability for the actions.
Schema
{
    "id": "<unique identifier for the current report>",
    "timestamp" : "<creation time for this report>",
    "schema-version": "alpha-1",
    "targetResource": {
        "provider": "<cloud provider>",
        "properties": {
            "id" : "<unique identifier for the target resource",
            "type": "<type of the target resource>",
            "properties": {
                <set of properties of the target resource which were impacted by the deployment>
                ...
                }
            }
        },
        "workflowRun": {
            "provider": "GitHub",
            "properties" : {
                "workflow": {
                    "repository": "<identifier of the repository>",
                    "path": "<path of the workflow file>",
                    "name": "<name of the workflow>",
                    "ref": "<ref for the workflow>",
                },
                "runUri": "<url for the workflowRun",
                "createdBy": "<handle of the user who created the run>",
                "jobName": "<name of the job doing the current deployment>",
                "creationTimestamp": "<workflow creation time>",
                "modifiedTimestamp": "<workflow last modified time>",
                "status": "<status of the workflow>",
                "commit": "<commit SHA which was deployed in this workflow>"
            }
        },
        "artifacts" : [
            <List of metadata associated with the deployment>
            {

            }
        ]
    }
}

Azure Kubernetes deployment report example
{
    "id": "cdfed638-6530-40cb-8d26-ce503cddb207",
    "type": "/Microsoft.DevOps/deployment",
    "timestamp" : "2020-06-04T07:59:40Z",
    "schema-version": "alpha-1",
    "properties": {
        "targetResource": {
            "provider": "Azure"
            "id" : "/subscriptions/c00d16c7-6c1f-4c03-9be1-6934a4c49682/resourceGroups/k8sRg/providers/Microsoft.ContainerService/managedClusters/contoso-k8s-cluster",
            "type": "Microsoft.ContainerService/ManagedClusters",
            "properties": {
                "namespace": "prod-wcus",
                "kubernetesObjects": [
                    {
                        "kind": "services",
                        "name": "contoso-service",
                        "version": "v1"
                    },
                    {
                        "kind": "deployments",
                        "name": "contoso-deployment",
                        "version": "v1"
                    }
                ]
            }
        },
        "workflowRun": {
            "provider": "GitHub",
            "properties" : {
                "workflow": {
                    "repository": "contoso/contoso-app",
                    "path": "./github/workflow/cd.yml",
                    "name": "Contoso Deployment",
                    "ref": "refs/heads/release"
                },
                "runUri": "https://github.com/contoso/contoso-app/actions/runs/123",
                "createdBy": "contosoUser",
                "jobName": "build-and-deploy",
                "creationTimestamp": "2020-06-04T07:59:40Z",
                "modifiedTimestamp": "2020-06-04T08:59:40Z",
                "status": "success",
                "commit": "870d8ae5d7558661d59c1d805d41fd53e43a9a6c"
            }
        },
        "artifacts" : [
            {
                "type" : "container",
                "image" : "",
                "dockerFileUrl" : ""https://github.com/contoso/contoso-app/blob/bc191dd28f7066e6e35a78d238af4f15de11ecee/Dockerfile"",
                "helmchartUrls" : [
                    "https://github.com/contoso/contoso-app/blob/bc191dd28f7066e6e35a78d238af4f15de11ecee/helmchart"
                ],
                "manifestUrls" : [
                    "https://github.com/contoso/contoso-app/blob/bc191dd28f7066e6e35a78d238af4f15de11ecee/manifest"
                ]
            }
        ]
    }
}

Some salient points of deployment report

Deployment report can be generated by any cloud publishing job.
It is basically a JSON file generated in temp folder on the job agent.
Users can choose to publish the deployment report to any data platform to generate traceabiliy information.

Following are some of the properties which will be part of the deployment report

Property
Type
Data Type
Purpose

id
Mandatory
uuid
Uniquely identifies a deployment report. Required for aggregating and querying data

timestamp
Mandatory
string
Creation timestamp of the deployment report in UTC required for ordering

schema-version
Mandatory
string
Version of schema used required for deciphering and validating

targetResource
Mandatory
object
Defines the cloud resource on which the deployment was targeted

targetResource/provider
Mandatory
enum
Cloud provider of the target resource (AWS/Azure/GCS/OnPrem) required to identify the resource

targetResource/id
Mandatory
string
Required to uniquely identify the resource

targetResource/type
Mandatory
string
Required to group resources and properly parse the properties

targetResource/properties
Optional
string
Metadata of the target resource required to provide granular details

workflowRun
Mandatory
object
Provides workflow run information for the current deployment

workflowRun/workflow
Mandatory
object
Provides workflow information for the current workflow run

workflowRun/workflow/repository
Mandatory
string
Complete repo path in org/repo-name format used to get workflow run information

workflowRun/workflow/path
Mandatory
string
Relative path of the workflow yml file. This is used to uniquely identify the workflow

workflowRun/workflow/ref
Mandatory
string
Required to identify branch details for the run

workflowRun/runUri
Mandatory
URL
Complete URL of the run. Use to identify the run

workflowRun/createdBy
Mandatory
string
User handle who triggered the run, required to establish traceability

workflowRun/jobName
Mandatory
string
The name of the job which is creating the deployment report

workflowRun/creationTimestamp
Mandatory
string
Creation timestamp of the workflow run in UTC. Required to establish time-line for a given deployment

workflowRun/modifiedTimestamp
Mandatory
string
Last modified timestamp of the workflow run in UTC. Required to establish time-line for a given deployment

workflowRun/status
Mandatory
string
Status of the job (success/failed). Required to provide information to diagnose workflow failure issues.

workflowRun/commit
Mandatory
string
The commit SHA which was deployed in the workflow run, required to find commit delta between runs.

artifacts
Optional
object
Metadata associated with the deployment, can be container images, test results, blob urls etc.

Discussion Notes:

Schema needs a version, so that we can handle back compatibility as we rev this.
Is Provider a Mandatory or Optional? Would you have predefined providers (Azure, AWS, GCP, OnPrem) listed or will any 'string' be ok?
Would Target resource type be a URI or String? Is it String because we need the schema to scale to On-premise targets?
I am assuming we are supporting only GitHub provider for Workflow. This seem like a good scoping call.
I am assuming that schema validation will look for "String : String" in Property bag for TargetResource, but will have rich strongly type schema validation for GitHub property bag in Workflow properties.
Remove the bootstrapconfiguration property from Workflow properties
Can you rephrase the "Purpose" column to explain the real need of those properties, instead of just explanation on their names.
Do we need to take traceability down all the way till Actions. Example: Multiple Action with same job can target same resource with different deployment status. How do we uniquely trace that?
Should Artifacts also have Provider, type and Property bag instead of flat properties?
Can we add one example for IaaS (VM Deployment) and one example for PaaS (AWS Beanstalk) with different artifact sources (Build drop and NPM or Nuget Package)

Discussion Notes:

Schema needs a version, so that we can handle back compatibility as we rev this.
Is Provider a Mandatory or Optional? Would you have predefined providers (Azure, AWS, GCP, OnPrem) listed or will any 'string' be ok?
Would Target resource type be a URI or String? Is it String because we need the schema to scale to On-premise targets?
I am assuming we are supporting only GitHub provider for Workflow. This seem like a good scoping call.
I am assuming that schema validation will look for "String : String" in Property bag for TargetResource, but will have rich strongly type schema validation for GitHub property bag in Workflow properties.
Remove the bootstrapconfiguration property from Workflow properties
Can you rephrase the "Purpose" column to explain the real need of those properties, instead of just explanation on their names.
Do we need to take traceability down all the way till Actions. Example: Multiple Action with same job can target same resource with different deployment status. How do we uniquely trace that?
Should Artifacts also have Provider, type and Property bag instead of flat properties?
Can we add one example for IaaS (VM Deployment) and one example for PaaS (AWS Beanstalk) with different artifact sources (Build drop and NPM or Nuget Package)

Here are few more open questions that came up in review with @smalpani-gh

DockerFileUrl demands that Docker image be downloaded so that we can inspect the tag. This will be extra step during CD. Should this download be done just for traceability?
Clarify where the k8s Deployment object traceability belongs, is it in targetResource properties or within the artifact properties?
Why do we need Manifest Url? Who is the consumer?
What is modified timestamp for a workflow run?
Does jobName belong to workflowRun or workfFlow? Clarify.
Why is workflow properties not a Uri but instead name/path etc?
Why does dockerFile, Helm and Manifest Url all under artifact and not in Workflow inputs?

cc: @rachitb777 , @ammohant , @azooinmyluggage , @pulkitaggarwl , @ds-ms

Here are few more open questions that came up in review with @smalpani-gh

DockerFileUrl demands that Docker image be downloaded so that we can inspect the tag. This will be extra step during CD. Should this download be done just for traceability?
Clarify where the k8s Deployment object traceability belongs, is it in targetResource properties or within the artifact properties?
Why do we need Manifest Url? Who is the consumer?
What is modified timestamp for a workflow run?
Does jobName belong to workflowRun or workfFlow? Clarify.
Why is workflow properties not a Uri but instead name/path etc?
Why does dockerFile, Helm and Manifest Url all under artifact and not in Workflow inputs?

cc: @rachitb777 , @ammohant , @azooinmyluggage , @pulkitaggarwl , @ds-ms

Yes,  traceability would need this step to surface the Docker file against the deployment resource. The Docker file path is stored as a label in the docker image.
K8s Deployment is the run time kubernetes object create in the cluster. It belongs to the Target resource.
The AKS deployment centre spec demanded surfacing the manifest in the UI.  In general if there is an intent to see if the desired and the current state differ, manifest can help in deriving that information along with the cluster state.
Inherited the definition from the proposed workflow resource. Modified timestamp can be done away with
JobName belongs to workflow
Artifact we have URIs as the source of the same can vary and hence URI, workflow path is the workflow identifier
Open item for discussion , can consider moving it to action context within workflow section or target resource k8s context
Details of the entry point to our experience can be found here

Yes,  traceability would need this step to surface the Docker file against the deployment resource. The Docker file path is stored as a label in the docker image.
K8s Deployment is the run time kubernetes object create in the cluster. It belongs to the Target resource.
The AKS deployment centre spec demanded surfacing the manifest in the UI.  In general if there is an intent to see if the desired and the current state differ, manifest can help in deriving that information along with the cluster state.
Inherited the definition from the proposed workflow resource. Modified timestamp can be done away with
JobName belongs to workflow
Artifact we have URIs as the source of the same can vary and hence URI, workflow path is the workflow identifier
Open item for discussion , can consider moving it to action context within workflow section or target resource k8s context
Details of the entry point to our experience can be found here

What is the right naming : Is it Microsoft/GitHub https://github.com/opencontainers/image-spec/blob/master/annotations.md. http://label-schema.org/rc1/

What is the right naming : Is it Microsoft/GitHub https://github.com/opencontainers/image-spec/blob/master/annotations.md. http://label-schema.org/rc1/

Artifact: What other types will be there and why everything for K8s is of type container artifcat.

Artifact: What other types will be there and why everything for K8s is of type container artifcat.

Design question : How can we work around the need to download image and running docker inspect

Design question : How can we work around the need to download image and running docker inspect

Design question : How can we work around the need to download image and running docker inspect
Two ways that we have discussed earlier. One is store it somewhere just like ADO but we understood from GH is building some sort of a store that we can leverage.  Second is  to have an environment variable store the Docker file path , this is not a full proof way.

Design question : How can we work around the need to download image and running docker inspect
Two ways that we have discussed earlier. One is store it somewhere just like ADO but we understood from GH is building some sort of a store that we can leverage.  Second is  to have an environment variable store the Docker file path , this is not a full proof way.

@smalpani-gh - This the final schema for the traceability we are closing on. Pls let us know of comments if any.
@rachitb777 @rohit-batra   We can commit the final version of the schema once Sanjay gives the go ahead.

@rachitb777 Can I close this issue since we have deferred this work? We can open a new issue when we plan to pick this up.


[SPARK-29348][SQL] Add observable Metrics for Streaming queries

What changes were proposed in this pull request?
Observable metrics are named arbitrary aggregate functions that can be defined on a query (Dataframe). As soon as the execution of a Dataframe reaches a completion point (e.g. finishes batch query or reaches streaming epoch) a named event is emitted that contains the metrics for the data processed since the last completion point.
A user can observe these metrics by attaching a listener to spark session, it depends on the execution mode which listener to attach:

Batch: QueryExecutionListener. This will be called when the query completes. A user can access the metrics by using the QueryExecution.observedMetrics map.
(Micro-batch) Streaming: StreamingQueryListener. This will be called when the streaming query completes an epoch. A user can access the metrics by using the StreamingQueryProgress.observedMetrics map. Please note that we currently do not support continuous execution streaming.

Why are the changes needed?
This enabled observable metrics.
Does this PR introduce any user-facing change?
Yes. It adds the observe method to Dataset.
How was this patch tested?

Added unit tests for the CollectMetrics logical node to the AnalysisSuite.
Added unit tests for StreamingProgress JSON serialization to the StreamingQueryStatusAndProgressSuite.
Added integration tests for streaming to the StreamingQueryListenerSuite.
Added integration tests for batch to the DataFrameCallbackSuite.

Test build #112110 has started for PR 26127 at commit fb36fa6.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17106/
Test PASSed.

Test build #112110 has finished for PR 26127 at commit fb36fa6.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class CollectMetrics(
class DataTypeJsonSerializer extends JsonSerializer[DataType]
class DataTypeJsonDeserializer extends JsonDeserializer[DataType]
case class CollectMetricsExec(

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112110/
Test PASSed.

Merged build finished. Test PASSed.

I like an idea in general. Just a sake of understanding, is this meant to be an alternative of #21721 or we still have #21721 unresolved even after this?
And this approach would have same issue pointed out in #21721 - it won't work for continuous processing. I would like to see some discussion/decision that whether we can ignore the continuous processing when designing some API for streaming queries.

Test build #112158 has started for PR 26127 at commit 6937a9a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17149/
Test PASSed.

This seems nothing to do with DS v2, but a new API in Dataset that allows users to collect metrics.
From the PR description, this should support both micro-batch and continuous streaming, as epoch is a general concept. I haven't looked into the code yet, but do we support continuous streaming in this PR?

Test build #112158 has finished for PR 26127 at commit 6937a9a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112158/
Test PASSed.

StreamingQueryListener doesn't work with continuous mode. Please refer #24537 which proposes to support StreamingQueryListener in continuous mode. (I haven't had a look in details though... I'd spend my time on continuous mode once we are clear it's still making progress.)

In continuous mode, the task is never completed - the patch seems to do something when task is completed which will never happen.

      // Only publish the value of the accumulator when the task has completed. This is done by
      // updating a task local accumulator ('updater') which will be merged with the actual
      // accumulator as soon as the task completes. This avoids the following problems during the
      // heartbeat:
      // - Correctness issues due to partially completed/visible updates.
      // - Performance issues due to excessive serialization.

Test build #112224 has started for PR 26127 at commit a58c04e.

Test build #112224 has finished for PR 26127 at commit a58c04e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112224/
Test PASSed.

If it's justified to have ProgressReporter supports micro-batch only, I think it's also justified to add more features to ProgressReporter.

Alright, let me update this one.

Test build #114590 has started for PR 26127 at commit cdc390a.

@cloud-fan I have updated, PTAL

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19418/
Test PASSed.

Test build #114590 has finished for PR 26127 at commit cdc390a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114590/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19455/
Test PASSed.

Test build #114636 has started for PR 26127 at commit cdc390a.

Test build #114636 has finished for PR 26127 at commit cdc390a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114636/
Test PASSed.

Test build #114738 has started for PR 26127 at commit cb69e55.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19562/
Test PASSed.

Test build #114739 has started for PR 26127 at commit a07474c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19563/
Test PASSed.

Test build #114738 has finished for PR 26127 at commit cb69e55.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114738/
Test FAILed.

Test build #114739 has finished for PR 26127 at commit a07474c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Subquery(child: LogicalPlan, correlated: Boolean) extends OrderPreservingUnaryNode
case class InsertAdaptiveSparkPlan(session: SparkSession) extends Rule[SparkPlan]
class HiveThriftServer2AppStatusStore(
class HiveThriftServer2HistoryServerPlugin extends AppHistoryServerPlugin

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114739/
Test FAILed.

retest this please

Test build #114758 has started for PR 26127 at commit a07474c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/19581/
Test PASSed.

Test build #114758 has finished for PR 26127 at commit a07474c.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class Subquery(child: LogicalPlan, correlated: Boolean) extends OrderPreservingUnaryNode
case class InsertAdaptiveSparkPlan(session: SparkSession) extends Rule[SparkPlan]
class HiveThriftServer2AppStatusStore(
class HiveThriftServer2HistoryServerPlugin extends AppHistoryServerPlugin

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/114758/
Test PASSed.

Merging this. Thanks for the reviews!


Provide NamedInterfaces and shared modules lookup strategies

While trying to setup spring-modulith on an existing project that has already multiple modules, each one designed to used hexagonal architecture, I realized that spring-modulith does not support hexagonal architecture.
I think that by allowing spring-modulith modules detection to be more customizable, by providing additional extension points like strategies for identifying NamedInterfaces from a module base package, it should be possible to support this architecture, and others.
I made a rough experiment here:

spring-modulith changes
the hexagonal-project

Note that it would require to open API to create NamedInterfaces and NamedInterface.
Additionally, it would probably be useful to provide a lookup strategy to identify shared modules, when it's possible to automatically detect them, and avoid manual declarations through @Modulithic attribute.

Thanks for opening up this ticket. I think before jumping to conclusions (Spring Modulith not supporting hexagonal architecture) and suggesting changes to the implementation, it would be helpful if you laid out what you're trying to achieve, what means you're using to implement those goals. We can then elaborate whether those are the appropriate ones or whether there are alternative approaches.
Fundamentally, any of the separation of concerns architectures (Hexagonal, Onion) are an implementation detail to a module. With jMolecules, a library exists that allows declaring the architectural stereotypes language in the codebase. In combination with Spring Modulith's named interface concept, we know about several applications using Hexagonal Architecture and Spring Modulith in combination.
Any chance you provide a minimal reproducer to demonstrate what you're looking for? I cannot really tell that from the sample project you linked to. Ideally without any references to third-party frameworks that might already come with precompile opinions? The last time I interacted with JHipster, the team was pretty opposed to a domain-oriented code arrangement approach, insisted on a rather technical first-level decomposition strategy, which is at odds with decomposion for encapsulation.

Closing due to the lack of feedback. Feel free to reopen with further information


nixos/sysctl: dynamic inotify watches or higher defaults

Nowadays most applications require a good amount of inotify watches,
so raise our default to what other distros do. If kernel supports it
enable dynamic setting.

ping: this is supposed to be ready :shrug:

@GrahamcOfBorg test bittorrent
@GrahamcOfBorg test boot.biosCdrom
@GrahamcOfBorg test boot.biosNetboot
@GrahamcOfBorg test boot.biosUsb
@GrahamcOfBorg test boot.uefiCdrom
@GrahamcOfBorg test boot.uefiNetboot
@GrahamcOfBorg test boot.uefiUsb
@GrahamcOfBorg test installer.bcache
@GrahamcOfBorg test installer.btrfsSimple
@GrahamcOfBorg test installer.btrfsSubvolDefault
@GrahamcOfBorg test installer.btrfsSubvols
@GrahamcOfBorg test installer.encryptedFSWithKeyfile
@GrahamcOfBorg test installer.grub1
@GrahamcOfBorg test installer.luksroot
@GrahamcOfBorg test installer.lvm
@GrahamcOfBorg test installer.separateBoot
@GrahamcOfBorg test installer.simple
@GrahamcOfBorg test installer.swraid
@GrahamcOfBorg test installer.fsroot

@alyssais Can we merge this? All feedback seems to be duly attended.

This change LGTM.

I don‘t understand the max_user_instances default.
I guess it makes sense for max_user_watches. The commit explains that the default value changes with 5.11 so we set that same value for everything below 5.11. Seems reasonable.
For max_user_instances we only change the default for everything above 5.12 if I understand kernelAtLeast correctly. Why is that?
It might be just right, but it looks like different cases. So describing them both with "Dynamic since X" confuses me. Can you maybe elaborate that?

Can you maybe elaborate that?

The upstream commit formulation is so intricate that I don't dare to give an approximate interpretation here inline.
https://github.com/torvalds/linux/commit/ac7b79fd190b02e7151bc7d2b9da692f537657f3

Okay, do you know what harm it would do do raise that limit unconditionally?
And related: Back porting a new default is one thing, but are we sure it’s a good idea to override an existing default for current and future kernels?

Thanks for the continued discussion!

Okay, do you know what harm it would do do raise that limit unconditionally?

I don't for sure, but I assume that it's an int value, so it cannot receive more than MAX_INT.

And related: Back porting a new default is one thing, but are we sure it’s a good idea to override an existing default for current and future kernels?

This was inspired by how other distributions do it and was argued in the related issue: https://github.com/NixOS/nixpkgs/issues/36214

Okay, do you know what harm it would do do raise that limit unconditionally?

I don't for sure, but I assume that it's an int value, so it cannot receive more than MAX_INT.

That’s not what I meant. I was wondering why we only do the override from 5.12 and newer.
But since then I think I understood it. I suggest adding a comment of the form:
From version 5.12 inotify watches count against a RAM resource limit, so it’s safe to increase this limit to the maximum
(If this is factually correct.)

And related: Back porting a new default is one thing, but are we sure it’s a good idea to override an existing default for current and future kernels?

This was inspired by how it was argued in the related issue: #36214, seems to be a requirement for docker or rather: in general name-spacing. I innocently assume because of those namespaces actually count towards this maximum.
Since this is a maximum boundary and implemented as a default, it is in any case as close to a pareto-improvement as we can get (doesn't do harm to anybody, but good to many). The maximum is only hit, if you actually use it.

Well the resulting harm would be if the user creates a resource exhaustion they don‘t want. The kernel has this limits for a reason. But if my phrasing above is correct, it’s fine to raise it.

I think I am 100% in favor of the instances change. It does basically what the kernel devs recommend.
I am a little bit uncertain about the watches change. The kernel wants to establish a new, system dependent default here and this PR would raise the limit for older kernels higher then what new kernels would pick on most systems. I don‘t think this is likely to cause any problems, but maybe someone more confident can merge this.

Cleaning up my follow-up list. Please anyone feel free to pick this up.


adding support for optional keys in templates

Hello Martin,
this pullrequest is to support optional keys
'{pro}/[{episode}/]{seq}/{shot}'
like this in square brackets.
I dont know whether you favour this at all, but I like the them a lot.
As one can reduce the amount of templates a lot.
let me know what you think.
Cheers
johannes

Thanks for this - will take a look over the weekend.

Hi Martin,
if you dislike the idea or the way I implemented it. Please let me know.
I first thought of writing one regex that builds the optional structure.
To be honest after testing it I was a bit afraid and thought the list approach is, well not sexy, but a bit more easier to read and easier for me to implement.
I guess that is a design issue. If you dislike the lists of regexes then I can give it a shot of putting it into a single regex... though I am not sure if that is doable at all...
cheers
johannes

Hey,
Didn't get as much chance to play with this as I wanted over the weekend. However, here are my current thoughts:
As stated in #29, I still feel that putting this behaviour in the template then conflicts with the idea of having multiple templates and that it also makes the logic more complicated (not only in implementation, but also usage). It also leads to the question of when to use multiple templates vs optional keys.
I think what might be better is to add this behaviour as a helper / preprocessor instead for those that want it. So given some strings in this format the helper generates the relevant template instances and returns them from which best match can then be returned. Of course this line of thinking does mean that #18 needs doing!
Maybe I am missing something though - what is the driving force behind wanting to reduce the amount of templates considering they can be generated programmatically?

Hi Martin,
one point is I am used of having optional keys in templates :-)
For me personally it is easier to be honest.
In our past feature production we had well over 120 ish templates (with optional keys).
Though the system we used was limited. With proper optional keys we now could reduve the amount drastically.
for example
we have templates for image sequences, that can be used for quicktimes, one has to lose the frame token and even sometimes the stereo token.
For me it is easier to write one template with the stereo key and frame key optional instead of having four... then if sth beforehand is optional it gets another multiplier...
so we already 8 templates that could be written with one...
Maybe the best match approach is sth I have not givin much thought yet....
What I tend todo with our old system and lucidity already is usually this:
-get a template from a path (in the pipeline this tends to be an opened workfile)
-retreive the tokens modify/add/delete them and format them again with the same template or another one I happen to know by name.
If I am having optional keys in there I can use the same template for stereo image sequences, image sequences, movies and stereo movies in projects with an episode or without.
So i can leave out an entire naming based matching logic or hardcoding of template names.
Or how would you approach sth like this ?
getting a template only by tokens/placeholders frightens me as we would have to guarantee that there is no template that consists of the exact same set of tokens...
Cheers and thanks for looking into this.

Sorry for the delay - I totally missed that you had replied!
I understand where you are coming from - I'm just not sure it fits into Lucidity at the Template level. One of the reasons that I avoided supporting reading from config natively in Lucidity is that I felt it more powerful and flexible to be able to manipulate them programatically. I also like how this means that the rest of the logic is very clear and it is simple to work out (and prioritise) which Template is used for which situation later.
However, as mentioned, I'm open to adding some helper / preprocessor stuff to Lucidity which would give what you want I think, but just at a higher level than the templates - perhaps linked to reading from config files.
Can you send through some concrete examples of your template combinations so I can get more of a feel for what is involved and perhaps give an example of how I might handle it now?

Hey Martin,
sorry for my delay now and thanks for the answer, maybe it would be good to have some other connection to talk about this...
I guess your approach regarding the design of a pipeline and its origins around filenames is different to my understanding.
I dont quite understand (designwise) how I should create the templates programatically.
Probably this is the root we should clarify before going further :-)
Cheers
Johannes

Sure! - I've pinged you via email.

Any news on getting this merged?

Hey,
we agreed to better move this into a sort of preprocessor/generator.
So it is more flexible.
The only thing I have not thought about yet is how to propagate the differentiation between optional missing keys and normal missing keys then...
So basically I would want to rework this a bit... though I dont know when ....

Cool :smile: Look forward to this!


skip maturity- & fecundity-at-length calculations when values at age are input directly?

@latreesedenson-NOAA and @Matthew-Smith-NOAA reported an r4ss plotting issue https://github.com/r4ss/r4ss/issues/709 with their red snapper model which I tracked down as related to "Inf" values in the BIOLOGY table.
I replicated the issue in the attached modification of "Simple".
When maturity option 4=read age-fecundity is combined with placeholder 999 values for the maturity and fecundity parameters, fecundity-at-length is calculated as 999 * weight ^ 999 though these values have are not used in the model.
I can make a fix to r4ss to skip the plots of length-based maturity and fecundity when maturity option 4 is used, but could we also report "NA" values for the "Mat_len", "Spawn", and "Fecundity" columns in BIOLOGY for models that use age-based maturity options or empirical weight-at-age? NA could be reported even if the fecundity is still being calculated, but I supposed skipping the calculation could also be nice (though presumably not very time-intensive if it only happens once).
An additional option would be to also skip the input of the parameters lines, saving users from adding placeholder values, but I assume that would take a lot more work.
Simple_maturity4.zip

I've been wanting to bypass a bunch of calculations like this when doing empirical wt-at-age.  Add this one to the list.  Let's prioritize it.

My preference is to do the code modification where these quantities are first calculated in ss_biofxn.tpl's get_mat_fec() function.
I think setting the vector to all 0.5 values would be better than all 0.0 or 1.0, but open to suggestions there.  Cannot set to NA because they are numeric.
The code is already bypassing the mat fec vector calculations when WTage_rd == 1.  So no action needed there, but perhaps the length-based quantities could also be set to 0.5 when this option is used.

I think the revised headers are a good idea. What we can do in r4ss is just rename the old names, if present, to the new names and base all plotting code on the new names. That's how we've been dealing with changes like this in recent years and it works well, especially when there's a 1-to-1 match of old to new.

@iantaylor-NOAA Thanks for asking these questions!

shouldn't the pull request https://github.com/nmfs-stock-synthesis/stock-synthesis/pull/349 have automatically created an r4ss issue?
The issues are created when the pull request is merged in rather than when it is created, which is why no issue was created.
what's the best way to test the new SS3 branch with the new r4ss branch, which might also require a new branch in the workflow repo?

I think everything you brought up makes sense  and how I would do it! I'm not sure there is a good way besides thinking carefully about how to coordinate changes. As a general philosophy, I would test in branches and merge in when you have high confidence that it will work.