Benchmark for Gravity Waves

Added simple benchmarks for running the 'WaveNet' neural net.

Running this as a quick test has the following timings:

Direct: ~0.265 s / iteration
Forpy: ~0.200 s / iteration

This is roughly consistent with what we saw in the full MiMA runs, so we probably now want to probe this in more detail to understand where we are losing time.

@jatkinson1000 I decided to gather a larger sample size as I think there is some variance in the runtime.
I have pushed a new commit (1dce3da) to this branch which improves the statistics output and adds the variance.
I then run the code (on my local machine 13th Gen Intel(R) Core(TM) i7-13800H 14 cores @ 5200 MHz, 32 Gb ram) using the gfortran compiler (GNU Fortran (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0) and the following commands:
$ ./benchmarker_cgdrag_torch ../pytorch saved_model.pth 1000 10
$ ./benchmarker_cgdrag_forpy ../pytorch run_emulator_davenet 1000 10

Here is my output for 1000 runs of the model compute function:
$ tail *out
==> cgdrag_forpy.out <==
 check iteration      995 (     0.970 s)
 check iteration      996 (     1.027 s)
 check iteration      997 (     0.827 s)
 check iteration      998 (     0.876 s)
 check iteration      999 (     1.031 s)
 check iteration     1000 (     0.969 s)
min    time taken (s):     0.5562
max    time taken (s):     3.1618
mean   time taken (s):     0.8627
stddev time taken (s):     0.2038

==> cgdrag_torch.out <==
 check iteration      995 (     0.716 s)
 check iteration      996 (     0.766 s)
 check iteration      997 (     0.713 s)
 check iteration      998 (     0.668 s)
 check iteration      999 (     0.668 s)
 check iteration     1000 (     0.668 s)
min    time taken (s):     0.5571
max    time taken (s):     3.0239
mean   time taken (s):     0.7521
stddev time taken (s):     0.1627

This actually implies that the direct coupled method (cgdrag_torch) is actually faster.
I also ran this for intel compiler (ifort (IFORT) 2021.10.0 20230609) and it was the same:
==> cgdrag_forpy.out <==
 check iteration      995 (     0.667 s)
 check iteration      996 (     0.823 s)
 check iteration      997 (     0.722 s)
 check iteration      998 (     0.774 s)
 check iteration      999 (     0.948 s)
 check iteration     1000 (     1.211 s)
min    time taken (s):     0.5524
max    time taken (s):     6.3702
mean   time taken (s):     0.8259
stddev time taken (s):     0.4090

==> cgdrag_torch.out <==
 check iteration      995 (     0.656 s)
 check iteration      996 (     0.866 s)
 check iteration      997 (     0.811 s)
 check iteration      998 (     0.766 s)
 check iteration      999 (     0.711 s)
 check iteration     1000 (     0.714 s)
min    time taken (s):     0.5487
max    time taken (s):     3.1084
mean   time taken (s):     0.7754
stddev time taken (s):     0.1916

Just run this on CSD3 as an sbatch job with the following results:
 ====== DIRECT COUPLED ======
min    time taken (s):     0.1342
max    time taken (s):     0.2251
mean   time taken (s):     0.1847
stddev time taken (s):     0.0169

 ====== FORPY ======
min    time taken (s):     0.1189
max    time taken (s):     0.1990
mean   time taken (s):     0.1641
stddev time taken (s):     0.0111

so 12% slower direct coupled on HPC for intel build.

Great work. Remember you can return to "before" you set OMP_NUM_THREADS by running command:
unset OMP_NUM_THREADS


cluster: add support for disabling circuit breakers

Description:
This PR adds the ability to completely disable circuit breakers via configuration.
Signed-off-by: Venil Noronha<EMAIL_ADDRESS>Risk Level: Med
Testing: Existing tests + a new test
Docs Changes: N/A
Release Notes: Added a note about this feature
Fixes #4337
/cc @snowp

I agree that a breaking change is unacceptable, at least if it happens quickly and/or without warning.
However, I think the current default is very arbitrary, and caught be off-guard. I think a default of UNLIMITED makes more sense. But I don't know how to gracefully change the default.

For setting individual fields to unlimited, what about using zero to mean this, instead of making it a one_of?

@ggreenway I like the idea of using a zero notation for unlimited resources. That would prevent this from being a breaking change, and that'd make this feature an opt-in.

@venilnoronha This might be contrived, but IMO this is still a breaking change because we are eliminating the possibility of an operator setting a limit to 0. There might not be a real need for this, but at the same time we are at a stage with the project that we can't make assumptions about what people might be relying on.
So I am now falling closer to @mattklein123's opinion that while the current situation is not optimal, it preserves all current functionality and still allows to effectively disable the limits.

So I am now falling closer to @mattklein123's opinion that while the current situation is not optimal, it preserves all current functionality and still allows to effectively disable the limits.

I'm personally probably OK with making 0 unlimited, but I still question whether this change is really needed. Can we just make a doc change and/or FAQ to more clearly indicate how to set the limits to max value if people want to "disable" them?

Sure, closing this PR as the solution doesn't favor the current situation.


Abort function for io.Copy

Hello @hajimehoshi !
I was moved by the method of playing music by combining go-mp3 and oto library with io.Copy. here
I would like to create a mechanism like jukebox using this technique and incorporate it into my game using ebiten.
So I searched for a way to interrupt music play by interrupting io.Copy and implemented.
Here is an example of an interruption sample written based on your sample.
Would you please give me comments or advice?

Hi Kemo-kemo-san!
As far as I know, there is no feasible way to suspend copying of io.Copy.
Instead, how about making a 'abortable' copy function? The (pseudo) code would be:
func AbortableCopy(w io.Writer, r io.Reader, abortCh <-chan struct{}) (int, error) {
  type readResult struct {
    buf []uint8
    err error
  }
  readCh := make(chan, readResult)
  go func() {
    buf := make([]uint8, 4096)
    for {
      n, err := r.Read(buf)
      r := ReadResult{buf[:n], nil}
      if err != nil && err != io.EOF {
        r.err = err
      }
      readCh <- r
      if err != nil {
        break
      }
    }
    close(readCh)
  }()
  written := 0
loop:
  for {
    select {
    case buf, ok := <-readCh:
      if !ok {
        readCh = nil
        break loop
      }
      n, err := w.Write(buf)
      if err != nil {
        return err
      }
      written += n
    case <-abortCh:
      return written, nil
    }
  }
  return written, nil
}

Note that I've not tested that code and there might be something wrong in that.

Hmm, that code has a bug: writtenBytes is not accurate when aborting. I don't think it is not hard to fix though.

I would like to create a mechanism like jukebox using this technique and incorporate it into my game using ebiten.

Sorry I missed this comment. If you are using Ebiten, you can use Player's Pause. http://godoc.org/github.com/hajimehoshi/ebiten/audio#Player.Pause Doesn't this work?

Hi hajime-hoshi-san!
I understood that the Copy function should be able to Abort. Thank you so much for the code of AbortableCopy! And now, I am convinced that ebiten's Player is the best choice for me.
I'm sorry to have disturbed you....
I'll try to implement the jukebox in my ebiten game!

No problem. Thank you for using Oto and Ebiten!


Detect weak Diffie-Hellman parameters

https://freedom-to-tinker.com/2015/10/14/how-is-nsa-breaking-so-much-crypto/

@Yamakaky This test is now implemented in my v2.0.0 release: https://github.com/jtesta/ssh-audit/releases/tag/v2.0.0

Nice!


Add support for zio-http

Given the recent announcement of zio-http, adding zio-http servers and clients to guardrail would remove the need for zio-interop-cats + http4s when generating for a zio codebase.

Hi any chance to getting this done soon @blast-hardcheese ? :)

I've not started implementation at this time, but I'm happy to give guidance for those interested in pursuing this.
I opened this issue mainly as an indicator of intent after reading the blog posts with performance benchmark numbers, it was particularly amazing to see.
As I do not have any active ZIO projects to speak of after changing companies, I don't think it would be a service to the ecosystem if I were to not reflect popular idioms in whatever implementation I'd come up with.
If there's time and capacity, I'm happy to chat here, in gitter, or via a Zoom chat or equivalent (timezones permitting).
Thanks for pinging me though, good to know there's active interest around this feature!

Hi guys! I've started working on this, but cannot promise it would be fast

Hi guys! I've started working on this, but cannot promise it would be fast

Thank you for giving this a shot! Please let me know if there's a way I can assist, I'm also usually pretty responsive over Matrix (room linked from README)

Would it make sense to extract it into a separate repository using the SPI approach to keep the core repository more focused?

Either way is fine for now, the module system is stable but there are no examples of doing this yet (especially when it comes to the test suites, unfortunately, I'll open an issue to fix that). Whatever helps make this easier!

Hi any progress here? ZIO-http is gaining user base and this would be honestly quite huge for this project. @zeal18 has done quite some work there, how much remains? Would be possible to finish it by someone else or directly by @blast-hardcheese ? Thank you

@Peppi-Ressler The biggest barrier to entry is knowing what the end state for generated code would look like.
If someone can supply an example repo of a high quality zio-http application, including stuff like file uploads, json encoding and decoding, and service composition.
Thank you for the continued interest, and I'm very excited to see that zio-http is gaining traction!

Right now, zio-http introduced Endpoint based API, kind of similar to tapir and it looks quite good. It uses zio.Schema so it supports out of the box json, protobuf, bson and more will eventually come. Also they have added a lot of examples https://github.com/zio/zio-http/tree/main/zio-http-example/src/main/scala/example + some docs here https://github.com/zio/zio-http/tree/main/docs. So I think it should provide most of the info you would need. Using endpoint way it would give us "for free" encoding/decoding to other protocols than just json.
If you would be interested in example repo using endpoint API, ZIO 2 with new layers and service pattern I could come up with something

Oh and one more thing. If you have Endpoint specification, it creates client basically for free as well. So hardest part would be to come up with endpoint definition template and that would be enough for end users to either implement server or use it in client

I tried to model a zio-http handler's hypothetical output based on the structure that guardrail generates for other modules, maybe it could be used as a starting point for someone who decides to work on it.
https://github.com/stanislav-chetvertkov/configuration-repo/blob/main/example/src/main/scala/api/StoreHandler.scala
Even thought https://github.com/zio/zio-http/tree/main/zio-http-gen is now available it is still incomplete and lacks many features, so a dedicated guardrail module for zio-http is still worth the effort

Thank you Stanislav. I wish I could put more effort towards this currently, if anyone wants to cut their teeth on this I'm happy to give my time to help a new contributor get up to speed.
Alternately, if anyone is able to fund this work that would be another way to contribute. Feel free to reach out at<EMAIL_ADDRESS>or chat with me in person at LambdaConf in a few weeks

I decided to give it another try, will be working on it  https://github.com/guardrail-dev/guardrail/pull/1979
using Endpoints api should indeed simplify a lot of things

@987Nabil has implemented a big portion (in not all!) of this task in https://github.com/zio/zio-http/pull/2562
Maybe you @stanislav-chetvertkov can use it/build on it/integrate it into guardrail.


It's not working on Mac platform

The editor seems freeze and it can't play when I add the oit render feature, and there are small black blocks in the editor windows as shown below.

What pipeline are you using? Does this happen with all pipelines or only a specific one? This looks like a custom scene, have you tried any of the provided samples? Which Unity version are you using?
I don't have a Mac and I am not able to test on these devices. So without any help I am not really able to debug or test for this platform.

I'm using urp pipeline and I have a try on Windows platform, the urp pipeline is ok


Extend make verify to fully check charts/kueue/templates

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
Extend make verify to fully check charts/kueue/templates
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?

NONE

/cc @tenzen-y
/cc @alculquicondor

/lgtm
/approve


The width difference between design and display.

Describe the bug
What's the bug?
I prepared a structure as shown in the figure. However, the designed w1 and w2 should be 0.425 and 0.375, respectively. It is clear that the w1 in the fabricated figure by the factory is 1nm smaller, while w2 is 1nm larger. What could be the reason for this? Thank you!
To Reproduce
What's the code to reproduce the behavior? What commands or code did you write to get the error?
You can add screenshots to help explain your problem.
Make sure you include the code for others to reproduce your issue.
import gdsfactory as gf
from import_other_gds_to_newlayer import extract_polygons_to_new_layer

@gf.cell
def bragg_cell(w1:str=0.4, w2:str=0.6, l1:str=0.2, l2:str=0.4,
               straight=gf.components.straight):
    print(w1,l1,w2,l2)
    c = gf.Component()
    r1 = c << straight(length=l1, width=w1,layer=(1,0),)
    r2 = c << straight(length=l2, width=w2,layer=(1,0),)
    r2.connect(port="o1", destination=r1.ports["o2"],allow_width_mismatch=True)
    c.add_port("o1", port=r1.ports["o1"])
    c.add_port("o2", port=r2.ports["o2"])
    return c

total_mz=gf.Component()
total_mz<<bragg_cell(w1=0.425,w2=0.375)
total_mz.show(show_ports=True)

Expected behavior
What would you like to happen?
Suggested fix
How could we fix the bug?

The problem arises from the ports. I assume you are using a 1nm database unit as a base.
If you want to use straights, they need the width in a multiple of 2 dbu. This is in order for the port to land on grid.
You could use allangle components or shapes directly.
I would suggest to go for just increase or decrease both widths by 1nm. That will fix it and the actual produced device will not be really different as your process will not be accurate enough for 1nm resolution anyway.


Add ability to add title to viz.draw

Is your feature request related to a problem? Please describe.
I want to visualize several graphs and distinguish them with a title.
Describe the solution you'd like
An optional argument for an image title.
Describe alternatives you've considered
Adding title outside of image.  A bit clumsy.
Additional context
I'm trying to compare graphs.

One can add a keyword argument to draw() inside pywhy-graphs/viz

Hi @robertness , @adam2392. I am looking to contribute to the library and this looks like a good first issue. Do you mind if I take this?

Sure!


[stdlib] remove preconditions from compatibility entry point

The old implementation of UnsafePointer.withMemoryRebound had no preconditions.
When implementing SE-0333, we forwarded the old entry point to the new implementation, which has preconditions.
This change removes the precondition from the compatibility entry point, reverting its previous behaviour.

Resolves rdar://90462471

@swift-ci please smoke test


Create directories in shared paths if they do not exist?

I believe capistrano does it automatically, but I may be mistaken.

I don't think this is a good idea, even if Capistrano did it.
If the directories are magically created but there is nothing there, your release will fail anyway. IMHO, creating these folders and filling them should be part of the machine provisioning, not the deployment scripts

Yeah, you're right.


pollInterval None should cancel polling?

Hi folks
I have been using react state to gate pollInterval on some queries, as some tasks progress on the back end.  It took me a while to figure that None does not cancel a set interval.  Should it? It seems like it should?
Seems likely a one liner.  I can take a look if youre interested in a PR.
Thanks
Alex

Gonna close this as I think the suggested solution will work.


barrier: fix 'uint8_t' is not a member of 'std'

Fixes a build failure on https://github.com/NixOS/nixpkgs/pull/275620

@ofborg build barrier


PostgreSQL Support

Hey guys, this is awesome!
We are looking for an alternative to Makara - are there any plans to support PostgreSQL?

This is very much required.
Please add support for postgres as well.

@stingrayzboy The guys over at Nasdaq started working on a proxy adapter for PostgreSQL: https://github.com/Nasdaq/active_record_proxy_adapters


updating caption

this pr closes/resolves/fixes issue #2
##done

created a branch
made a commit
##todo
[]merge the branch
[]delte the branch

Perfect!


[TrieLookupTable] added destructor for TrieLookupTable and added a test case

Additional Description:
In the past TrieLookupTable didn't have a customized destructor. The default destructor will use call-stack to recursively find all unique_ptr and reset them. If the string added to Trie is very long, for example, with 40000 characters, the Trie will have a depth of 40000 and call-stack will encounter stack-overflow.
Added a destructor for TrieLookupTable and added a unit test case with a long string. In addition, added size = "small", to BUILD to silence a warining:
 WARNING: //test/common/common:utility_test: Test execution time (14.9s excluding execution overhead) outside of range for MODERATE tests. Consider setting timeout="short" or size="small".
No memory leak was detected by asan.
Risk Level: low
Testing: Passed all the unit test cases. Passed a crashed test case reported by network_readfilter_fuzz_test.
[Optional Fixes #Issue] https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24558&sort=-opened&can=1&q=proj%3Aenvoy status%3DNew
/cc @asraa
/cc @samkerner

Thanks, could you please add the fuzz test as regression test, and also include some information on where a user may add a long strings to a Trie? (eg in this case the prefix lookup table).
@mattklein123 Can you take a quick pass at this?

Should we add fuzz test into the existingtest/common/common/utility_fuzz_test.cc, or create a new dedicated fuzzer for TrieLoopupTable? (And by the way, should it be a seperate PR, or just adding it here is ok?)

Should we add fuzz test into the existingtest/common/common/utility_fuzz_test.cc, or create a new dedicated fuzzer for TrieLoopupTable? (And by the way, should it be a seperate PR, or just adding it here is ok?)

Good point. I meant the network fuzz test case that caused the crash from the issue to at least server as a regression test in this PR.
In terms of fuzz testing the Trie, a separate PR would be good (so we can get this PR as a fix out). I think the only place outside of the inline headers (which are fixed at compile time, and isn't configured by users or traffic) is this usage in redis filter. Perhaps adding a small utility fuzzer is the way to go, if the usage of this is expected to increase.

@mattklein123 can you take a quick look at this destructor and if it ever came up when building the header map? (probably not, since inline headers are fixed)

@mattklein123 can you take a quick look at this destructor and if it ever came up when building the header map? (probably not, since inline headers are fixed)

It didn't. Would it be better to just release assert / some type of error handling the trie max size and then check that in the fuzz test? Is there any user input that actually hits this? Mostly just trying to understand if we need this extra complexity in the prod code or not.
/wait-any

It didn't. Would it be better to just release assert / some type of error handling the trie max size and then check that in the fuzz test? Is there any user input that actually hits this?

The only places that this would happen is from user configured prefix lookup tables, which comes down to user configuration for a very long string that may cause this crash. An API validation would work here, but it would go against API policy, but the other possibility is to just throw an error on initialization of the redis filter. @jianwen612 WDYT?

Yeah I would be fine with just enforcing a max size, but up to you all.

It didn't. Would it be better to just release assert / some type of error handling the trie max size and then check that in the fuzz test? Is there any user input that actually hits this?

The only places that this would happen is from user configured prefix lookup tables, which comes down to user configuration for a very long string that may cause this crash. An API validation would work here, but it would go against API policy, but the other possibility is to just throw an error on initialization of the redis filter. @jianwen612 WDYT?

I think both ways are ok. But even though limiting the length in redis_filter fixes this issue, we still need to warn future developers that this utility has its limitation on the depth(and this depth is not fixed. It depends on the data type and current stack).
I agree that adding extra complexity into production code is risky. So if you think a safer solution is better, I would close this PR and make a new one.

I agree that adding extra complexity into production code is risky. So if you think a safer solution is better, I would close this PR and make a new one.

This would be my preference if you are OK with it. Thank you!

I agree that adding extra complexity into production code is risky. So if you think a safer solution is better, I would close this PR and make a new one.

This would be my preference if you are OK with it. Thank you!

Do you think limiting the length to 500 chars is ok?


When release 1.0.0-beta.6 ?

Many fixes are already merged into master branch, but these are not released. A firebase-tools requires cloud-functions-emulator, and it always show info logs because #287 is not released...
CHANGELOG.md will be changed as following:
##### 1.0.0-beta.6 - XX XXX 2019

###### Bug fixes
- [#287](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/pull/287) - Update engines requirement
- [#286](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/pull/286) - Add support for Firebase Remote Config events
- [#210](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/issues/210) - "logs" location is not writable if node.js is installed via a package manager
- [#207](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/issues/207) - Hot reloading does not work for subdirectories in Linux
- [#204](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/pull/204) - Inspect fails when the supervisor is running on an unexpected port.
- [#199](https://github.com/GoogleCloudPlatform/cloud-functions-emulator/pull/199) - Handle missing logfiles correctly

###### Other
- Updated dependencies
- Fix documents and help messages

Thanks.

+1

Also: 1.0.0-beta.5 requires lodash 4.17.5 (exactly that version --- no leading ^), which has a known vulnerability, https://nvd.nist.gov/vuln/detail/CVE-2018-16487, that is showing up in GitHub's security alerts.
https://github.com/GoogleCloudPlatform/cloud-functions-emulator/blob/c592053f8b299fa0c51997022088161b6f13aa3b/package.json#L87
It has already been fixed on master with https://github.com/GoogleCloudPlatform/cloud-functions-emulator/pull/246 but not yet released.

Look at this project and this issue. Look at GoogleCloud youtube channel and email list.
Expectation versus reality.
The issue was opened Dec 30, 2018.

Working on this. I'll track publishing here.
npm version prerelease --preid="beta.6"


Add Configuration Options

TODO

[ ] src/elements/control-issue.ts:5: TODO #60 move to config file link
[ ] src/elements/control-issue.ts:86: TODO #60 get this issue title and label settings from config from input? link
[ ] src/elements/control-issue.ts:87: TODO #60 label is not created with right config (color + description) link
[ ] src/util/find-todo-stream.ts:7: TODO #60 move to config link

Tracked Branch: refs/heads/main | Tracked commit: 6a32e4634580ee71ea6f22ba0307010ebe8a8d18


Fix for PR #1504, orthographic matrix

The code in PR #1504 computed the height of the viewport incorrectly so it was always negative, thus causing problems when using the ortho camera. This PR fixes that oversight.

@JeffM2501 Oh! Good catch!

@JeffM2501 Oh! Good catch!


Add 3D translations and scale

Hi, are you interested in adding translation and scale for 3D layers?
I have implemented both in my fork: https://github.com/jnk22/napari-manual-transforms/tree/feat_translation-scale-3d
If so, I would be happy to open a pull request and provide some more context if needed.

hey @jnk22, I'm not personally doing too much with napari these days, but would be more than happy to review and merge your PR if you'd like it to be part of the plugin.  I took  quick look at your branch and it seems well designed.

Thanks for your quick reply!
I have now opened #2.


Remove TestFlightRO Config pack.  RealismOverhaul itself now has these and will satisfy the dependency.

Can you resubmit with just the changes to TestFlight.netkan? The RO folks have frozen the config netkan so it won't waste resources any more, but still maintains the Identifier.

I'm sorry but I don't entirely know what you mean?  The CKAN side has always been confusing to me.
Resubmitting this with just the changes to TestFlight.netkan doesn't do me any good, as the driving force here is to remove the TestFlightConfigRO mod, since it no longer exists as of the next version of Test Flight.  The RO Configs are now in RO itself, and @NathanKell has or will be modifying RO so that it satisfies the TestFlightConfig dependency.

@jwvanderbeck  I believe what @politas is saying is that I already did do this stuff. :) Because I did.

You took care of removing my netkan as well?  If so, then you sneaky bastard you :p

What he did was rename TestFlightConfigRO.netkan to TestFlightConfigRO.netkan.disabled, which will stop the Netkan bot from trying to find any new versions, while maintaining the existence of the TestFlightConfigRO identifier in case someone wants to try using that name in the future. (Which is admittedly extremely unlikely, but maybe someone will create a "Random Operations" mod in the future and want to have a special configuration for TestFlight to support it.
Additionally, I've put in a change to the current release of TestFlightConfigRO (the .ckan file), which changes the abstract so that it says it is deprecated as of Realism Overhaul v10.9.4. (Well, actually, it turns out I did that on the last-but-one version of TestFlightConfigRO, so I'll just add that change to the latest one as well.)


Include hyperbars title bar in window borders

It looks weird to me that the hyperbar titlebars are not included in window borders. Can we have an option to make title bar be inside the border and not on top of it?

Possibly also add gradients and active/inactive colors so the title bar can be more integrated with the borders.


Create a route to microservice to be accessed from outside the cluster.

As a [role]
I need [function]
So that [benefit]
Details and Assumptions

[document what you know]

Acceptance Criteria
Given [some context]
When [action taken]
Then [outcome observed]

Route is - https://inventory-hd2585-dev.apps.sandbox-m2.ll9k.p1.openshiftapps.com/


mobile: Remove toBridge from Envoy::Http::Client

Envoy::Http::Client no longer uses C wrapper types, so the toBridge name no longer makes sense.
Risk Level: low (rename only)
Testing: CI
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

/assign @abeyad

/retest


cannot send tx

Hi, I had run the relayer2, and I tried to fine tune the config.
When I change the indexer.keepFiles to false to save my ssd storage,but I found I cannot send tx through relayer2. Was keep refiner json file necessary?
In addition, when I keep the indexer.keepFiles to true, the relayer2 can send tx normally, but it also rarely cannot send out my tx about send 20 txs 1 will miss.

Hi @hollejacklin
the indexer.keepFiles config is not related to tx sending, we nevertheless verified that it is not reproducible in our tests setups. For further investigation, can you please share the followings;

have you used standalone-rpc for installation of relayer2 and refiner or have you installed them from scratch
can you please share the config file you are using and logs while having the issue

thanks!

Hi @akeyder
I installed refiner(borealis-engine-lib v0.18.0) and relayer2(v2.0.0) separately.
I just try it again using indexer.keepFiles=false, and it send tx normally. I will test it for long time for more information.
if I found it didn't send out the tx, the log report the near tx hash and aurora tx hash, but I cannot found it on aurora explorer and near explorer. it looks like the tx was made but the tx was on chain. However I send the same tx to my relayer-V1 at same time was normally. (I run relayer V1 and V2 at the same time for backup line)

Hi @hollejacklin
the txns you send either via relayer v1 or v2 are using the same path to reach the chain and communicating with same contract so I would not expect any problem there. But this part of your comment caught my attention
I send the same tx to my relayer-V1 at same time was normally. (I run relayer V1 and V2 at the same time for backup line)

what exactly do you mean by sending same txn at the same time from both relayers (v1 and v2)?
I am not sure how such request is processed but it may cause some kind of race condition hence undefined behavior.
please explain your setup (topology) in more details and also would be great to have logs if it is reproduced again.
thanks

Hi @akeyder,
The following was my relayers and app structure.
app1(javascript VM1)->relayer-V1(VM1)->nearcore(VM1)->WAN_IP1
app1(javascript VM2)->relayer-V2(VM2)->refiner_nearcore(VM2)->WAN_IP2
app1 in VM1 and VM2 used same aurora account and same function. If app1 send out a tx from VM1 and VM2, the relayer-V1 and relayer-V2 will feed back same tx hash. However, sometimes only the app1(VM1) can get receipt, the app1(VM2) get nothing.

Hi @hollejacklin
unless you have already changed it in config,  relayer2 sends txns in async fashion by default, while relayer1 sends them in sync. I suspect this may cause timing related issues in your app1 logic so;

can you please change the relayer2 config endpoint.engine.asyncSendRawTxs to false and restart relayer2 and perform the same test again.
if this still not helps, I would recommend running the same tests with only relayer2 if possible

It looks for setting endpoint.engine.asyncSendRawTxs to false.
Thanks.


Get DaskExecutor working locally.

I was able to easily run the ETL using the LocalExecutor. docker-compose up creates two dask workers and runs the ETL using the DaskExecutor. This ran but got hung up on various parts of the ETL and produced some dask warnings about tasks holding onto process locks for too long.

Running etl_fast.yml using the DaskExecutor produced a couple of
 FileNotFoundError: [Errno 2] No such file or directory: '/pudl/outputs/cache/2021-09-14-2222-a1c3c13c-fd40-4aa3-8814-c25d0fcf88e6/dataframes/0cdfb9fa15ac11eca3f90242ac120003/boiler_fuel_eia923'

errors for different tables. I checked the prefect flow chart and the tasks seem to be executed in the correct order which makes me think this is an issue when caching with multiple processes.
Luckily running the fast etl using the LocalDaskExecutor works! This executor works on a single node and should provide a speed up so it is good enough for this first iteration of cloudification. We will need to use the DaskExecutor if we want multi-node execution.


Eval of parent does not run after raise in multi-child setting

I have a parent component P with more than one child (A and B). There is a list of A's, indexed by some unique id (which is also the slot type). P has a button to add new A's to the list, and every A can raise a delete command (eg, when a delete button is pressed), which should instruct P to delete it from the list.
This works fine except in the following situation:

There is an instance of B present
I add a new A to the list
I want to delete that same A

I can always delete the A's that are initially in the list. And once the whole P is re-rendered, I can delete every A again. Also, if no instance of B is present, everything works.
With some trace statements I can see that in the case where deleting does not work the callback in the last argument of slot' is called, but then the delete query of P is never evaluated.
Let me know if you have an idea what could cause this, otherwise I'll try and post an example that reproduces this.

Ok, weirdly this issue has disappeared after I got rid of all the unnecessary re-renders due to get (see #400). Maybe you shouldn't investigate too much before it creeps up again or I build a reliably failing example. :slightly_smiling_face:

I'll close this for now since I've never seen it again.


it can't apply themes from server

when i input this command

hugo server --theme=hyde --buildDrafts

ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 template: theme/partials/head.html:14:39: executing "theme/partials/head.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/head.html
ERROR: 2015/08/18 template: theme/partials/sidebar.html:4:23: executing "theme/partials/sidebar.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo in theme/partials/sidebar.html
ERROR: 2015/08/18 Error while rendering 404 page: template: theme/404.html:8:123: executing "theme/404.html" at <.Site.BaseURL>: BaseURL is not a field of struct type *hugolib.SiteInfo

getting the error like that.
how can i solve this problem?

Duplicates http://discuss.gohugo.io/t/i-cant-apply-hugo-themes-from-the-server/1588

i wrote that huh

.Site.BaseUrl got renamed to .Site.BaseURL in hugo 0.15 (AFAIK). I had the opposite problem your having (upgrading from 0.14 to 0.15-DEV) and replaced all instances of the former with the latter, which fixed my issue.
Try updating your version of hugo to 0.15, or replacing all instances of .Site.BaseURL with .Site.BaseUrl


Strips multipart messages in BigQuery before comparing.

Fixes #1434. H/T to @cyluss
/cc @jonparrott

LGTM

Awesome.


WIP: Fix issues with multiple task links

Test Task
Test Task

Issue reproduced on this run: https://github.com/Teamwork/github-sync/runs/3281027832?check_suite_focus=true
And fixed in the latest.


FERN.Lab logo is not shown correctly in README

There is just a white space, no logo is shown:

It may also be nicer to put the FernLab_Logo.svg to the docs directory instead of having it in the package root directory.

@danschef if you use the black mode you can see it. Should we place the image for white mode since it is the default one?
@DaRa42 what do you think?

@romulogoncalves yes I would go for white mode as default and adapt the image for it

@DaRa42 is this solved by #5 ?

I will test it

No it seems not

It should be fixed by:
https://github.com/FernLab/cookiecutter-py-package/commit/2d77b69cd1965146e7379a4b67cc4a11f264f2a8


ZeroDivisionError: division by zero

run
for SCENE_NAME in brandenburg_gate; do
scripts/data_generation.sh /nas/datasets/Heritage-Recon/${SCENE_NAME}
done
python tools/prepare_data/prepare_data_cache.py --root_dir /nas/datasets/Heritage-Recon/brandenburg_gate --dataset_name phototourism --cache_dir cache_sgs --img_downscale 1 --semantic_map_path semantic_maps --split_to_chunks 64
[WARNING] batch has 0 intersections!!
[WARNING] batch has 0 intersections!!
[WARNING] batch has 0 intersections!!
Traceback (most recent call last):
File "/nas/users/NeuralRecon-W/tools/prepare_data/prepare_data_cache.py", line 176, in
dataset = dataset_dictargs.dataset_name
File "/nas/users/NeuralRecon-W/./datasets/phototourism.py", line 123, in init
self.read_meta()
File "/nas/users/NeuralRecon-W/./datasets/phototourism.py", line 667, in read_meta
curent_percent = valid_num / current_len
ZeroDivisionError: division by zero

/nas/users/NeuralRecon-W/datasets/phototourism.py(667)read_meta()
666                         current_len = rays.size()[0]
--> 667                         curent_percent = valid_num / current_len
668                         padding_length = int(np.ceil((self.depth_percent * current_len - valid_num) / (1 - self.depth_percent)))

ipdb> current_len
0
for id_ in tqdm(img_ids_split):
ipdb> id_
15

I also meet this problem.

have you solved this yet?


[docs] Update the postgres container initialisation example

The container needs to be marked as a compatible substitute for the official postgres image.
Otherwise, with recent versions of testcontainers, the postgres container will fail to initialise with:
Failed to verify that image 'debezium/postgres:11' is a compatible substitute for 'postgres'.

@jakzal Thanks for the PR.
@obabec Could you please try to upgrade TestContainers in our project and check if the source code change is needed? If yes we need to create a new Jira and make a fix to make sure this requirement is not lost.

@jpechane no problem. Same PR or a new one?

@jakzal Thanks for checking, applied! I just linked the version update commit into its own Jira - https://issues.redhat.com/browse/DBZ-5990


Implement Promise.allSettled()

Issue: https://github.com/then/promise/issues/169
Promise.allSettled() :

Specification: https://tc39.es/ecma262/#sec-promise.allsettled
Docs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
TS: https://github.com/microsoft/TypeScript/blob/main/src/lib/es2020.promise.d.ts

I created PR https://github.com/then/promise/pull/171 to fix it but not sure that this project is alive and maintainers can review it quickly

cc @ForbesLindesay

@dankreiger big shout out

@dankreiger big shout out

I'm not a maintainer so I can't merge it. I wonder where the maintainers are.

@ForbesLindesay could you please review this pr


ROX-24337: helpful error message to wrong bundle format

Description
Added Helpful error messages in the case of wrong bundle format being passed through, included cases such as both or no bundles as well.
Checklist

[ ] Investigated and inspected CI test results
[x] Unit test and regression tests added
~- [ ] Evaluated and added CHANGELOG entry if required~
~- [ ] Determined and documented upgrade steps~
~- [ ] Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)~

If any of these don't apply, please comment below.
Testing Performed
Manual Testing: I tested for

[x]  operator bundle

[x] helm bundle

[x] both bundles

[x] no bundle.

The following block checks for the validity of the provided init bundle. (helm install ... -f <init-bundle.yaml>)

Throw an error if no init bundle was provided
Throw an error if the wrong init-bundle format was provided (user wrongly passed an operator-formatted init-bundle to helm). We detect the presence of an operator init bundle by checking for the Kubernetes secret annotation "init-bundle.stackrox.io/name", that suggests a user has wrongly used helm install ... -f <operator-init-bundle.yaml>.
Warn if the user specified both init bundles, e.g. helm install ... -f <helm-init-bundle.yaml> -f <operator-init-bundle.yaml>

Compared the expected to the actual result and all 4 checks were correct.
My input for the command line:
helm install --dry-run -n stackrox \    --create-namespace \    -f /path/to/operator/bundle/file \    -f /path/to/helm/bundle/file \    --set central.persistence.none=true \    --set imagePullSecrets.useExisting=<name> \    --set clusterName=<name> \    stackrox-secured-cluster-services \    ./stackrox-secured-cluster-services-chart``
Depending on the test case, the file paths were removed.
My test case script is: init-bundle.test.yaml where these 4 test cases were checked again.
Reminder for reviewers
In addition to reviewing code here, reviewers must also review testing and request further testing in case the
performed one does not seem sufficient. As a reviewer, you must not approve the change until you understand the
performed testing and you are satisfied with it.

/retest

/retest

/retest


updated Regex to look for more file extensions

I'm not a regex master but I believe that this should change the script from looking exclusively for .JPG files, to also include .JPEG files and .PNG files.

Tested with /r/Wallpapers and verfied that it works as expected for JPEG and PNG files.
Thanks for the submission!


Style InstantSearch's components instead of using connect-[component]

As much as possible, we should try to style directly the InstantSearch's component, intead of using the connect functions.
Example :
const Highlight = connectHighlight(
  ({ highlight, attribute, hit, indexContextValue, ...props }) => {
    const parsedHit = highlight({
      highlightProperty: '_highlightResult',
      attribute,
      hit,
    })

    return (
        ...
    )
  }
)

This could result in the dashboard being a little faster than it currently is.

We are currently not adding new features to the mini-dashboard as per the notice on the readme. I'm closing this for now. feel free to re-open it if the situation changes.


Add FSharp.Data Tests and Documentation to work with JSonValue

Would be very useful to have unit tests (i see the project is empty) and/or documentation to work with FSharp.Data.JsonValue lenses

The tests that execute are the same as can be found for other implementations, there are some ifdefs in the source:
Note the references in fsproj here:
https://github.com/fsprojects/Fleece/blob/master/test/Tests.FSharpData/Tests.FSharpData.fsproj#L9-L10
The ifdefs are here:
https://github.com/fsprojects/Fleece/blob/master/test/Tests/Lenses.fs#L10-L15
Having additional tests and documentation would be neat though 😃


Tools lose updates when drawing quickly due to async graph

Drawing quickly with, e.g., the Pen tool causes some points to disappear when drawn.

No longer an issue due to the pen and path tool not relying on the previous graph evaluation.


Unauthorized to create EFS storage class

When creating infrastructure with GPU nodepool, creation of the EFS storage class fails with "Unauthorized".

Currently there is no solution for this error, but a workaround exists. The storage class has to be created manually with the already created EFS file system id.

Just as a remark: Today I used g5.4xlarge nodes instead of p3.2xlarge and the error did not occur.
The problem might be related to the kind of GPU nodes.

The reason for this is the expiration time of 15m for the default token that is created/queried on EKS cluster creation.
If some operation takes too long, e.g. the initial scale up of a GPU node instance to avoid scale from 0 problems, the token expires before the EFS storage classes is created.
This can be solved in the definition of the kubernetes provider by not specifying token = data.aws_eks_cluster_auth.cluster.token argument. Instead, the following configuration causes a new token to be get automatically:
exec {
  api_version = "client.authentication.k8s.io/v1beta1"
  args = ["eks", "get-token", "--profile", "change_me", "--cluster-name", data.aws_eks_cluster.cluster.name]
  command = "aws"
}

See https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#exec-plugins for details.


Add code coverage reporting

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@414fec3). Click here to learn what that means.
Patch has no changes to coverable lines.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the GitHub App Integration for your organization. Read more.
Additional details and impacted files
@@           Coverage Diff           @@
##             main       #2   +/-   ##
=======================================
  Coverage        ?   65.08%
=======================================
  Files           ?       24
  Lines           ?      507
  Branches        ?      224
=======================================
  Hits            ?      330
  Misses          ?       19
  Partials        ?      158

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Do train animatediff on Flux.1 Dev asap

Is this project even alive???
i want animatediff for 64+ frames and with 1024x1024 resolution. Youtubers told this repo is the place.
Can you do it asap??
Make sure you open source the weights!
I can even pay like $10-20 as a tip if you do it.

I think if you train animatediff on flux.1 dev it can beat SORA? train it for the whole night, ( not just 2-3 hours). i have heard training for longer time can make it better quality


Add oldaskconfig the default conf program

Closes #133
This is the default option, for kconfig-conf.
Just a small modification to the "oldconfig":
@@ -82,9 +81,7 @@
     if isinstance(node.item, Symbol):
         sym = node.item

-        # Skip symbols that already have a user value
-        if sym.user_value is not None:
-            return
+        new = sym.user_value is None

         # Skip symbols that can only have a single value, due to selects
         if len(sym.assignable) == 1:
@@ -98,9 +95,9 @@
         # Loop until the user enters a valid value or enters a blank string
         # (for the default value)
         while True:
-            val = input("{} ({}) [{}] ".format(
+            val = input("{} ({}) [{}]{} ".format(
                 node.prompt[0], _name_and_loc_str(sym),
-                _default_value_str(sym)))
+                _default_value_str(sym), _new_value_str(new)))

             if val == "?":
                 _print_help(node)
@@ -242,5 +239,11 @@
     return sym.str_value

+def _new_value_str(new):
+    if new:
+        return " (NEW)"
+    return ""
+
+
 if __name__ == "__main__":
     _main()

Rather unfortunately, it appears the original author of Kconfiglib disappeared.  The Zephyr project has been granted ownership of the PyPI package via PEP-541 request, and I'm going to do a release based on a new repository in the zephyrproject-rtos organization here: https://github.com/zephyrproject-rtos/Kconfiglib
:arrow_right: Action required: @afbjorklund   Are you able to re-open your pull request on the new repository?

Can't make a PR, since you didn't fork the same project. Would have to make a new clone...

Can't make a PR, since you didn't fork the same project. Would have to make a new clone...

You can just close this PR and open a new one agains the new repo. We did not make it a fork because we want it to become the new canonical upstream.


Please, add new options for my family

Please, Add new options to protect my kids from objectionable activity.If you add website block feature and instantly  Clear cookies and delete logins.It will be great help for my nice and heavenly family.  Pardon me ,I do not know how to use github and where to write a request.But I need this useful rat to monitor my kids. Thanks for your philanthropic work.

Hello @Abraham313
I will definitely consider your suggestions

Thanks for your kind response.I will wait for your next version.I hope I can monitor my kids properly without hurting their unadulterated soft mind .


Table 26 - B0 is double for MCMC compared to MLE

MLE looks correct (comparing to last year) and in param_est_table() has:
        bo = (mddq %>% filter(Label == "SSB_Initial") %>% pull(Value) / 2e3) %>% f(0),

(so divide by 2000 to get females, and make the units tons), whereas MCMC part has:
        bo = f(median(.x$mcmc$`SSB_Initial`) / 1e3, 0),

so just divide by 1000. Presume the latter should be changed to 2e3, as I remember some talk about SSB definition changing this year in SS (!). I can fix but just wanted to check.

As I have it open I will change it in a single commit that we can fix if necessary.


Keep optimization-dropped fields as a data-less component of Array

meant to eventually fix #180
We want to be able to use the necessary columns optimization with something like this:
import dask_awkward as dak
import numpy as np
import uproot

x = uproot.dask({"https://github.com/CoffeaTeam/coffea/blob/master/tests/samples/nano_dy.root?raw=true": "Events"})
y = dak.zip({"pt": x.Muon_pt, "eta": x.Muon_eta})

# add another field lazily (we will never actually use it)
z = dak.with_field(y, x.Muon_phi, "phi")

# function which uses pt and eta (but not phi)
def some_func(pt, eta):
    return pt * np.cosh(eta)

w = some_func(z.pt, z.eta)

# currently fails because there is a node in the graph that expects
# `Muon_phi` (but it gets dropped by the optimization as unused
# (correctly!))
w.compute()

where Muon_phi is indeed kept as a field, but contains no data in memory (because it is unnecessary for the final computed result)

@jpivarski @agoose77 - made some decent progress here. Something is partially working but could use some input.
Current state (feel free to look through the whole diff of the PR but I'll try to write it out with some explanation):

After determining the necessary columns we now check them against the original set of columns to figure out what is "removed" (really just "not read from disk")
Now also pass this list of removed columns to the project_columns interface. This is so the callable class that is going to do the actual ak.from_xyz call at the dask node knows about the other fields that will not be read from disk.
When the column-projectable callable is actually called at the Dask node, we tack on the removed columns after the main (existing) read of ak.from_xyz:

def __call__(self, source):
    a = ak.from_parquet(source, columns=self.columns)
    # before this PR we'd return a now

    # this PR adds this logic mock_dataless are the columns we want to mock w/o data:
    for entry in self.mock_dataless or []:
        # split on "." so we get lists of strings for fields of fields
        entries = entry.split(".")
        if len(entries) == 1:
            a = ak.with_field(
                a,
                np.broadcast_to(np.int64(0), (len(a),)),
                entries[0],
            )
        else:
            ## This will error if the top level field is not already available
            a = ak.with_field(
                a,
                np.broadcast_to(np.int64(0), (len(a),)),
                entries,
            )

        return a

(You can wee for this PoC I'm just using a int64 0).
Two issues right now.

We're looping over all of the "dataless" fields and creating a new ak.Array each time. This appears to be N^2 with the number of columns; so it looks like we'd benefit from some kind of bulk augmentation function.
The way it's written above fails in the else block if we have some subfield structure (like foo.bar.baz getting turned into ["foo", "bar", "baz"]) and the first field is not already present in the array.

Happy to hear your thoughts on this!

@douglasdavis it's late my end, so this is just a superficial response.
This looks interesting! One note that I can add is that we would like for the order of the columns to be the same whether all fields are read, or only a subset. It might be nice for a different interface to specify missing columns, e.g. a dict (which are now ordered by default) that indicates if the field should be read:
columns = {
	'x': True,
	'y': True,
	'z': False
}

Will look over properly tomorrow! :)

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@605682f). Click here to learn what that means.
The diff coverage is 93.75%.

:exclamation: Current head f73a2d4 differs from pull request most recent head c1d4687. Consider uploading reports for the commit c1d4687 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##             main     #184   +/-   ##
=======================================
  Coverage        ?   94.49%
=======================================
  Files           ?       19
  Lines           ?     2033
  Branches        ?        0
=======================================
  Hits            ?     1921
  Misses          ?      112
  Partials        ?        0

Impacted Files
Coverage Δ

src/dask_awkward/lib/structure.py
95.42% <ø> (ø)

src/dask_awkward/lib/io/parquet.py
90.39% <90.90%> (ø)

src/dask_awkward/layers/layers.py
98.27% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

I've rebased onto main, made some fixes for changes in awkward, and handled a form-layout deviation case (unmasked arrays for non-unmasked forms)

The unhandled case that's failing the CI is described in this Gist. I believe it's a duplicate of https://github.com/apache/arrow/issues/30043, meaning it's likely not going to be fixed any time soon. We can drop trivial nodes like UnmaskedArray, so that's probably what needs to be done in this case.

6483a08 handles (option, no-option) for a (layout, form) pair, which fixes this bug. I think that's the only place that it's needed; we only see this option being introduced if one reads a subset of columns. If one reads a subset of columns, then re-hydration always happens.
I suppose that a user could only read a subset of columns, which would trigger this bug (in plain Awkward, too, I suppose).

@agoose77 I've been giving this PR a scan and also taking it for a spin locally with a Parquet dataset. I've not run into any problems
I've also started working on adding an API (starting with a decorator) that will allow libraries (at this point that just means uproot 😛) to wrap the concrete callable such that unproject_layout will be called on the returned array with the original form. As things currently stand, in the builtin parquet IO support in dask-awkward, we can write in an unproject_layout call ourselves, but we definitely want to make it usable by others.

@douglasdavis fab! Locally, I am experimenting with modifying unprojection to create fake arrays instead of content / index objects. I'll see if that bears much fruit!

Should I give this a spin with nanoevents / uproot soon?

@lgray Yea I think so, I'm typing up a little update on an API for rehydrating an array in the method that instantiates it

I made a new branch to experiment with the API I mentioned in my comment above. The commit which adds two different possible interfaces: https://github.com/douglasdavis/dask-awkward/commit/d91ddae5d04eae5702ce1a8cab8fa84e881b97e0
The first via a decorator interface and the second via a mixin interface.
In both cases the functor that does the reading from disk just needs a self.original_form attribute. You can see in that commit above how I've implemented it for the builtin from_parquet reading functor _FromParquetFileWiseFn; naming and things can be changed but so far this works for me locally. (I've done a little task graph surgery testing to see that we indeed get an array with the complete form but a bunch if 0 buffers on the fields that are optimized away

This PR now depends upon https://github.com/scikit-hep/awkward/pull/2479 which introduces support for a placeholder array.

@douglasdavis I'm thinking more about https://github.com/dask-contrib/dask-awkward/pull/184#discussion_r1202611928
This PR addresses the problem of keeping the array metadata the same (e.g. array.fields) by including columns that are never ultimately touched.
e.g.
array = ak.Array(
    ak.contents.ListOffsetArray(
        ak.index.Index64(...),
        ak.contents.RecordArray(
            [...],
            fields=["x", "y"]
        )
    )
)

result = ak.sum(array.x)

In the above example, we always need to touch the offsets given in the ListOffsetArray, but we don't need to load the content corresponding to field y.
The example given in the body of this PR is actually a slightly different problem:
x_rec = ak.Array(
    ak.contents.ListOffsetArray(
        ak.index.Index64(...),
        ak.contents.RecordArray(
            [...],
            fields=["x"]
        )
    )
)
y = ak.Array(
    ak.contents.ListOffsetArray(
        ak.index.Index64(...),
		...
    )
)
array = ak.with_field(x_rec, y, "y")
result = ak.sum(array.x)

Here, we would normally touch both x_rec.layout.content.offsets and y.layout.offsets in order to perform the broadcasting required by with_field. That "fix" is discussed above in https://github.com/dask-contrib/dask-awkward/pull/184#discussion_r1202611928 to ensure that this column is read. In other words, this PR right now reads y, even though it doesn't survive to the final result.
This is technically correct. For example, in this concrete example,
import awkward as ak
x_rec = ak.to_regular([
    [{'x': 1}],
    [{'x': 2}]
])
y = ak.Array([
    [1, 2, 3],
    [4, 5, 6, 7]
])

array = ak.with_field(x_rec, y, "y")
result = ak.sum(array.x)

the result is 11, despite x_rec.x summing to 3; the offsets of y lead to repetition of x. We cannot further optimise this because the result does depend upon both sets of offsets.
We could conceivably update Awkward to implement asymmetric broadcasting, such that ak.with_field(x_rec, y, "y") only broadcasts y against x; we've been considering that in other contexts, so it's natural to raise it here. With this change, the value of the result would not depend upon the offsets of y. But, crucially, y could easily have offsets that are not asymmetrically broadcastable with x_rec that would raise an exception at runtime. If we implemented an optimisation such that we just ignore offsets during broadcasting if they don't affect the result value, then the above would succeed. But, if y ended up in the task graph, this earlier result would then fail because the broadcast would be properly evaluated.
In other words, creating records via ak.with_field cannot be considered (isn't) metadata-only, even though it's lightweight[^perf]. Transient fields will have offsets touched, so they must come from the same list if we're to ensure that there is no IO penalty to them. That's why the uproot.dask form rewriting solves this particular case; it can both retain the important context that both sets of offsets have the same form key, and it also can build records without broadcasting.
@jpivarski I think this should be on your radar, though we can discuss it later today.
[^perf]: broadcasting lists requires comparing offsets, although we could make it even faster if we add an this.offsets is that.offsets optimisation.

Thanks for that detailed writeup, @agoose77! Please correct me if I'm misunderstanding here, but this is the general takeaway I'm getting:
The need for form rehydration has popped up a few times and this PR addresses that need. However, the example I give at the top of the PR is not the best example to give as the motivation for form rehydration. And in the PR as it exists you've ensured that the original need is met (form rehydration with the unproject_layout function. And still in this PR you ensure that necessary columns for things that involve broadcasting (like the zip example that I originally wrote out) are covered in the wildcard pruning logic of the optimize.py file. Am I following this correctly?

@douglasdavis succinctly put, that is exactly it!

thanks @jpivarski and @agoose77 for helping to get this feature in!


Demographic results should be handled like other modules

I'm writing a custom demographics module based on Pregnancy. A custom intervention averts pregnancy-related deaths. However, there's no way to record true results (after demographic and intervention updates). Most other modules update_results near the end of sim.step. It's not as easy as calling update results because demographic modules have customized update_results, these will need to conform.

This would be an good topic to discuss during a Starsim meeting. In seeking to align Births with Pregnancies/Deaths, I want to store data on who the mothers are and when they give birth. I was considering (1) creating a property sim.people.mother and sim.people.ti_mother or (2) using sim.results.

Closed by #504


reset relay publish status screen if it is closed for 5 seconds

Currently if you click outside the modal it will disappear, but once you try to write a new note it will show up again with the relay statuses of your previous publish.

Fixed


Resolve Sass deprecation warnings

Move vanilla declarations above Sass @includes
Wrap division in a calc()

Shouldn't break anything anyway, but have tested on review app and it's no more broken than it already is.


fix: typo

Signed-off-by: Gaius<EMAIL_ADDRESS>
Description

Rename status to state

Related Issue

Motivation and Context

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation Update (if none of the other choices apply)

Checklist:

[x] Code compiles correctly.
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Codecov Report

Merging #809 (edd440e) into main (807ce94) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #809      +/-   ##
==========================================
+ Coverage   41.00%   41.02%   +0.02%
==========================================
  Files          98       98
  Lines        7904     7904
==========================================
+ Hits         3241     3243       +2
+ Misses       4256     4253       -3
- Partials      407      408       +1

Flag
Coverage Δ

Object-compatibility-e2etests
?

e2etests
?

unittests
41.02% <ø> (+0.02%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

client/daemon/peer/peertask_stream.go
55.96% <0.00%> (+0.61%)
:arrow_up:


Feature: Update merchant to be empty and with validation

Details

This PR is about to add an error message when merchant is empty on expense reports in request money and split bill
Fixed Issues

$ https://github.com/Expensify/App/issues/32222
Tests

Login to the app
Click on "Request money" or "Split bill"
Select an amount
For IOU request:
Select user, not a workspace
Merchant should be empty and no validation errors
You are able to create a request with empty merchant
Create request
Go to edit page
No (empty) string visible for the merchant
Try to edit with empty string the merchant
You were able to edit with empty string
For Expense request:
Select a workspace
Look that merchant is empty, but validation error is appear that you need to provide a merchant
Create request
Go to edit page
try to edit merchant as empty field
validation error will appear

[x] Verify that no errors appear in the JS console

Offline tests

N/A automated tests
QA Steps

Same as tests

[X] Verify that no errors appear in the JS console

PR Author Checklist

[X] I linked the correct issue in the ### Fixed Issues section above
[X] I wrote clear testing steps that cover the changes made in this PR

[X] I added steps for local testing in the Tests section
[X] I added steps for the expected offline behavior in the Offline steps section
[X] I added steps for Staging and/or Production testing in the QA steps section
[X] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[X] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[X] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[X] I included screenshots or videos for tests on all platforms
[X] I ran the tests on all platforms & verified they passed on:

[X] Android / native
[X] Android / Chrome
[X] iOS / native
[x] iOS / Safari
[X] MacOS / Chrome / Safari
[X] MacOS / Desktop

[X] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[X] I followed proper code patterns (see Reviewing the code)

[X] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[X] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[X] I verified that comments were added to code that is not self explanatory
[X] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[X] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[X] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[X] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[X] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[X] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[X] I verified the JSDocs style guidelines (in STYLE.md) were followed

[X] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[X] I followed the guidelines as stated in the Review Guidelines
[X] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[X] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[X] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[X] I verified that if a function's arguments changed that all usages have also been updated correctly
[X] If a new component is created I verified that:

[X] A similar component doesn't exist in the codebase
[X] All props are defined accurately and each prop has a /** comment above it */
[X] The file is named correctly
[X] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[X] The only data being stored in the state is data necessary for rendering and nothing else
[X] If we are not using the full Onyx data that we loaded, I've added the proper selector in order to ensure the component only re-renders when the data it is using changes
[X] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[X] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[X] All JSX used for rendering exists in the render method
[X] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[X] If any new file was added I verified that:

[X] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[X] If a new CSS style is added I verified that:

[X] A similar style doesn't already exist
[X] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[X] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[X] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[X] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[X] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[X] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[X] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

@mountiny Okay, pushed changes, removed required for isScanRequest expense report, now it works as before my changes.
If I understand you correctly only this required in smart scan was an issue, right?

I ma making a test build for easier testing

:test_tube::test_tube: Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! :test_tube::test_tube:

Android :robot:
iOS :apple:

https://ad-hoc-expensify-cash.s3.amazonaws.com/android/32486/index.html
https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/32486/index.html

Desktop :computer:
Web :spider_web:

https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/32486/NewExpensify.dmg
https://32486.pr-testing.expensify.com

@mountiny Damn, thats super weird, this happens after I merged main into this branch..

I cant even see console.logs in MoneyRequestConfirmationList.js component..

Okay, I found, there was some refactor and all my code is unused now.. :D

Discussing in slack

@mountiny updated new code(hopefully everything), tested and works fine

Some weird TS issue unrelated to my code

@mountiny merged main again, lint TS issues was resolved, but some test which is unrelated is failing
And for some reason tests are running for so long, 10 mins so far :D

:test_tube::test_tube: Use the links below to test this adhoc build on Android, iOS, Desktop, and Web. Happy testing! :test_tube::test_tube:

Android :robot:
iOS :apple:

https://ad-hoc-expensify-cash.s3.amazonaws.com/android/32486/index.html
https://ad-hoc-expensify-cash.s3.amazonaws.com/ios/32486/index.html

Desktop :computer:
Web :spider_web:

https://ad-hoc-expensify-cash.s3.amazonaws.com/desktop/32486/NewExpensify.dmg
https://32486.pr-testing.expensify.com

@waterim the rundown of the tests with @trjExpensify has been successful only updated was required to the design of the required field which should he been addressed already I believe.
Once the conflicts are resolved we can proceed with the PR

@mountiny conflicts resolved

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

@koko57 @waterim
Been testing around and noticed two issues:

the merchant validation should only happen on submission according to our guidelines. When you are creating the request, the merchant fields should not show as required. the error should pop up only once the form has been submitted and the required merchant field is empty
On the split bill screen or in Scan request the merchant field shows up as required, but then it disappears as correctly it should not be there when scanning in workspace as we are running smartscan

https://github.com/Expensify/App/assets/36083550/710b532b-b69f-4e7c-8357-c4f8fb62efcb

@mountiny I think I fixed the first issue, will push the changes in a minute. I just wanted to ask - if we fill in the merchant field it should disappear (I mean move to the hidden fields under Show more button)?
https://github.com/Expensify/App/assets/28020445/67a1c96f-5729-4e0f-a5ea-c3ab0ae1f732
It's a current behaviour, I changed it in my commit and now it's visible for workspace request. Let me know if I should restore hiding of this field.
For the second issue - I don't really understand, could you explain what is the expected behavior here? And is it on recording? In which second it starts?

the issue if that the merchant should not even be visible for receipt scan creation. We run smartscan which should fill in the merchant so its not useful to have it in there.
In the video you could see that the merchant was showing in the scan flow which is incorrect

if we fill in the merchant field it should disappear (I mean move to the hidden fields under Show more button)?

It should not disappear, for manual requests for workspaces, the merchant is required so it should be visible at all times
The bug there is that the field is showing as required when you navigate to the page immediately, thats not how we handle required fields, it should start showing as required only once the user click the button to proceed and the merchant is missing/ validation fails

It should not disappear, for manual requests for workspaces, the merchant is required so it should be visible at all times

so in my last commit, I applied this change for this field to be always visible

The bug there is that the field is showing as required when you navigate to the page immediately, thats not how we handle required fields, it should start showing as required only once the user click the button to proceed and the merchant is missing/ validation fails

yes, now it works that way.
Thanks for explaining the issue with the Scan request, I'll take care of it 🙂

@mountiny for scans - it should be fixed now

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/mountiny in version: 1.4.20-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
cancelled 🔪

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/marcaaron in version: 1.4.20-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

Coming from https://github.com/Expensify/App/issues/33786:
Manual request with scan receipt case was not considered.


initialize the library with a frequency plan

Initialize the library to TTN_EU868. #32

[ ] Include US915
[ ] One method for frequency plan that sets power index, data rate (move from reset()), frequencies, channels etc

@johanstokking ok I didn't see the enableFsbChannels()... Do I also move the dr switch/case from reset() to configure_channels() ?

@Nicolasdejean let's keep reset() for resetting ABP and OTAA settings, and configure everything related to the frequency plans in init(): data rates, channels, duty cycles, etc.

@johanstokking I enable the channels in a loop for EU in configure_EU868 or by calling enableFsbChannels() in configure() for US in configure. Is there an enable loop I forgot ?

@FokkeZB @Nicolasdejean why is this PR so big? Should it be rebased?

I guess, this has a lot of commits already in a long time.. 48 commits?! It should be rebased or maybe just re-done one a new branch of off master.


Allow setting priority on listeners/subscribers

I am not sure since it is hard to find any documentation on this. But it seems like both the Listeners and the Subscribers don't allow to set a priority. This would be a very useful addition.
If it is somehow possible to do this it would be good to add an example to the documentation.

It is not supported by our event system. If you need to support this, wrap your event handlers in a "priority-aware" event handler.
If you need this sort of functionality, consider providing a more extensive feature request in https://github.com/doctrine/common/issues

OK thanks!


SessionManager: Allow backend storage to be configurable

This allows using the built-in session feature without having to use the CacheMap, which is limited to the current process and not suitable in larger deployments.
In my use case of drogon, I am running multiple pods behind a single load-balancer, so the built-in session handler won't work without sticky load balancing, which isn't feasible and is more of a bandaid fix. This allows the user to optionally specify their own implementation of backing storage, for example using redis. By default, CacheMap is used and current usages should require no changes, it is opt-in only.

@DarkLordZach Thanks so much for the patch. this is very helpful for multiple pods scenarios.
I have a quastion, I used synchronous methods in the old session manager because data is stored in memory, the read/write on sessions dosen't block the current thread. but for a custom session storage (e.g. a redis implementation) the calling on synchronous interfaces may block current thread, this will cause a large throughput loss in high concurrency scenarios. So is it necessary to change the synchronous interface to the asynchronous interface?

@an-tao , Yes, using an asynchronous interface with std::future does seem much better, I didn't do that because I was trying to change as little code as possible. I can fix that today.
My other question was if it was possible to switch from std::any to std::variant<...>, because std::variant is so much easier to handle for serialization/deserialization. I was thinking it could include common types like bool, std::string, size_t, and perhaps even Json::Value. I do realize that this is probably a breaking change though.
Finally, for the sessionPtr stuff you mentioned, yes, I do think it needs to be virtualized, but it should require minimal effort. All that needs to happen is marking the methods as virtual. Since the ptr is returned from SessionStorageProvider, the user already has control over which implementation is used. As for how sessionPtr will connect to redis, I was planning on using RAII, where the session maintains a reference/shared-ptr to the redis client and gets the value in the constructor and then sets it in the destructor.

@DarkLordZach Thanks for your reponse.
If you use std::future, the futureobj.get() is a new blocking point to the current thread, I think maybe we should use callbacks to get session data.
As you said, the changes you mentioned will break the current APIs, so we need more discussion about them. You can start to perfect this PR after we have fully discussed.
@rbugajewski @vedranmiletic @ihmc3jn09hk

@DarkLordZach If we use std::variant instead of std::any, does that mean we can only store data of some predefined types to sessions? This seems to increase the constraints on the type of data stored in sessions.

Yes, but the problem with std::any is that the only way to get objects out of it is to use std::any_cast, which only works if you use the explicit type. Thus, if the backend cannot directly store the std::any (for example, on redis), it either has to explicitly check for every type the user might store or some other type has to be used.

Yes, but the problem with std::any is that the only way to get objects out of it is to use std::any_cast, which only works if you use the explicit type. Thus, if the backend cannot directly store the std::any (for example, on redis), it either has to explicitly check for every type the user might store or some other type has to be used.

You are right, the current API only suitable for memory storage.

At this point of development API breaking changes should be prioritized lower in my opinion, so that it would be OK to change the API. But I think we should find a better solution and strive towards getting the best one for a plain memory storage and key-value stores like Redis at the same time, not find a compromise that will hinder the design and usage of one or the other. The current API is beneficial for plain memory storage as it is as flexible as possible. Changing the API to std::variant won’t be a big deal API wise, but it will make the plain memory storage less versatile. On the other hand a configurable key-value store in the backend is a rather common use case and it would be great to have it as a core part of the framework.
I wonder how often you’ve a case that it is necessary to provide a “drop-in” replacement for plain memory storage? From my experience it is rather clearly defined what gets stored in a session and what in a key-value store. In this case we could maybe decouple both APIs, keep std::any for plain memory storage, and use std::variant for Redis? Do you know how the situation looks like in Amazon’s DynamoDB or Memcached?


Add 'Author' field for texts

It would be nice if we could attribute the Author of the text we're adding (if known). Above / below 'Source' might be a good place.

Blocked by #250 - This should be optional on the book listings, need show/hide for that to work.

#250 done, not blocked now.


build: version 3.7.10 broke build rolling back to 3.7.9

Summary
Latest release of release please is failing, not sure why. Rolling back to a working version to unblock releases for now.
https://github.com/trussworks/react-uswds/actions/runs/5548198197/jobs/10130859807
https://github.com/USSF-ORBIT/ussf-portal-client/actions/runs/5512846343/jobs/10050194817
https://github.com/USSF-ORBIT/ussf-portal-client/actions/runs/5535147685/jobs/10101056417

Looks like it might not be release-please but a change in the github action runner that caused the issue. Looking at options.


KAFKA-13718: kafka-topics describe topic with default config will show segment.bytes overridden config

More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.
Effectively, commenting out the log.segment.bytes value to treat it as a default value instead of STATIC_BROKER_VALUE. Should have no effect given that the value is hardcoded in the code. Upon running integration tests, test which pass consistently passed.
Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@showuon Thanks! Happy to help contribute to the project!


A few improvements to MolStandardize::Normalizer

This PR contains a few improvements to MolStandardize::Normalizer:

use ROMOL_SPTR consistently (cosmetic change)
allow using multiple alternative normalization patterns that match the same functional groups. I have added a test case where multiple custom patterns all match malformed azides. With the current RDKit master the Normalizer keeps adding the custom pattern again and again until it reaches the maximum number of normalization restarts (200 by default), and then applies no other normalization SMARTS. With the fix included in this PR this does not happen anymore.
add missing parameter to NormalizerFromData factory method (Python wrappers): apparently one of the parameters had been forgotten in the Python signature
add NormalizerFromParams factory method (Python wrappers). This is analogue to the NormalizerFromData method, minus the string. I know that one could use the constructor that takes a file and a number of restarts in alternative, but given there is a CleanupParameters class why not using it?

@ptosco : the windows build failures are real.  I pointed out a couple of the problems via single comments

I’ll do them as fast as I can, but between vacation and the UGM I am super slow


Added Ctrl+Shift+S for Save All

LGTM. Thanks @Zerrick !

Released in https://github.com/Microsoft/vscode-notepadplusplus-keybindings/releases/tag/v1.0.2


VB-2198 update govuk frontend version

New version of GOV.UK Frontend available: Release GOV.UK Frontend v4.6.0 · alphagov/govuk-frontend

Updated library version.
Book a visit: accordion internally improved, new property rememberExpanded

Looks good. I wonder if we now no longer need the accordionId variable generating and passing to the view? This would mean we could remove:

https://github.com/ministryofjustice/book-a-prison-visit-staff-ui/blob/62a79a66867fa852773f752053bbe1c49018f377/server/routes/visitJourney/dateAndTime.ts#L80
https://github.com/ministryofjustice/book-a-prison-visit-staff-ui/blob/62a79a66867fa852773f752053bbe1c49018f377/server/views/pages/bookAVisit/dateAndTime.test.ts#L33
https://github.com/ministryofjustice/book-a-prison-visit-staff-ui/blob/62a79a66867fa852773f752053bbe1c49018f377/server/views/pages/bookAVisit/dateAndTime.test.ts#L205

(I haven't actually tried this; just a suggestion and may not work 🙂!)


Append sequence if one is already loaded

Modify Sequencer.parse() to take a file_object,  Sequencer.parse(file_object). Moved self._sequences to SequenceFileHandler init.
Modify SequencerWidget.load_sequence to append sequences if self.data is not None.

@mcdo0486, I have pushed in the branch new updates based on your feedback and in particular:

renamed the class SequenceFileHandler to reflect the fact that is not linked to a particular file
added the option to append or replace when loading a new file
added persistent editor mode to show combobox
few other changes.

Do you mind having a look and check if you still see some points open ?


Is  9f411b3 still necessary

In 9f411b3 , patch 0.1.4,

Added a system of postinstall to patch selenium-webdriver http client behavior (temporary)

The patch http.js present in the project is now 6 months old. I don't really get why it exist but maybe selenium-webdriver got an actual patch ?
https://github.com/simonradier/node-red-contrib-selenium-wd2/blob/7b8567d345afbf5498983c82544675b231ca9424/package.json#L39
This fix doesn't feel really good to use, especially since cp doesn't exist in windows, so you basicaly can't use the package in windows if you don't know what you're doing and don't use git bash

Hi @MyGg29,
This fix is needed because Selenium node.js version 4.0.X has a weird implementation of the webdriver management which prevent webdriver to be modified by node-red. The good news, is that I'm working on a 0.2.x which get rid of selenium webdriver implementation. No more dirty cp and it works on Windows / Linux / MacOS.
I'll publish a beta soon, so you will be able to test it.
I hope it answers your expectations.
Best regards,

It does ! Thank you !


Fix typo - an -> a

Description
Describe your pull request and problem statement here.
Completed Tasks

[x] I have read the Contributing Guide.
[x] The pull request is complete (implemented / written).
[x] Git commits have been cleaned up (squash WIP / revert commits).
[x] I wrote tests and ran bundle exec rake locally (if code is attached to PR).

Coverage decreased (-0.01%) to 93.494% when pulling d6c163b4754665cbaf09f2654c85c9fe574abb12 on matrinox:master into d816482a0d4850506c3bcccc9434550c536c28c6 on lsegal:master.

Thanks!


Closes ciena-frost/ember-frost-list#121

This project uses semver, please check the scope of this pr:

[ ] #none# - documentation fixes and/or test additions
[ ] #patch# - backwards-compatible bug fix
[x] #minor# - adding functionality in a backwards-compatible manner
[ ] #major# - incompatible API change

CHANGELOG

#121 - Support loading state in frost-list

Changes Unknown when pulling 920201e4af3628c60322f14940b95376cf07ea48 on notmessenger:is-loading-indicator into ** on ciena-frost:master**.

Changes Unknown when pulling 854db7bac23275eaf3a616048d87e854a634c56c on notmessenger:is-loading-indicator into ** on ciena-frost:master**.

@sglanzer Regarding the exploration of the explicit need for isLoading my thinking is that we would need to change the items property definition in the frost-list component - https://github.com/ciena-frost/ember-frost-list/blob/master/addon/components/frost-list.js#L28-L31 - to only accept a Promise rather than an Object or Ember.Object.  This way a loading (resolution) state could be monitored.

@notmessenger ok, let's avoid that for now and stick with the isLoading solution.

@sglanzer Or we could have it also accept a Promise and if one is passed then the loading state can be inferred.  The only way you get this loading indicator functionality is to use a Promise.

Sure, but an explicit isLoading still won't hurt in that case, so I'm fine with this, just need the adjustment to the way the loading indicator is specified and I'm good to merge


Error with Zelig when using negbinom

Hi,
We keep getting this error message when running our model:
Error in zclist$hook(new.res, new.call, match.call(), ..., data = d.f) :
robust contains elements that are not supported.
In addition: Warning message:
In value[3L] : There was an error fitting this statistical model.
Does anyone know what it means?
This is the code:
z.out <- zelig(ViolentCrimeRate ~ FoundationsDensity100k + NetFlowRate + TurnoutPercentage + PropwoHauptschulabschluss + YouthRate + MaleRate + UnemployedPercentage + MarriageRate, model="negbinom", DistrictData)
We already tried to code all variables numeric and/or integer but it still keeps saying the same error.
Thanks for your help.

Do you have the data so that we can reproduce the error?