Use Chrome 59 for PNG cheatsheet rendering

Zero dependencies to render a PNG now.
PDF rendering should work also however Chrome 59 doesn't capture the text. Seems like a bug.

will it make more sense to rewrite this in bash?

if we use bash then i'd prefer a one liner and dropping all the conditional checking. fine with that?

a one liner and dropping all the conditional checking
Yes, I like one liner! I think you could still do some checks just so when anyone runs it, it can give meaningful errors.

also @sid-github or @tirodkar , WDYT?

Ok, added bash version.


Title bar hidden

Is there a way to hide the title bar and still have the possibility to drag the window around? Only works when the window is not focused... Thanks!

Duplicate of #28


Allow enabling metrics and output roles

Hello 👋
I opened a couple of PRs a few months ago, wondering if they're okay to get merged in or if there's outstanding work required for them to be accepted
Could be my bad for not opening a related issue.
The PRs are:
https://github.com/terraform-community-modules/tf_aws_ecs/pull/39
https://github.com/terraform-community-modules/tf_aws_ecs/pull/38
Thanks,
Chris

Hi! I think they just fell off my inbox. 😄  I'll take a look at em!

Merged! I'll cut a new release shortly!

Awesome thanks, got one more too that I'm just rebasing 😬 ... https://github.com/terraform-community-modules/tf_aws_ecs/pull/45

It's good to go now 👍


Support for natural sort order

Sorting currently does not use a natural sort order and will sort layers like the following:
1 skin
10 sky
11 light
2 background
The desire is that natural sort order is supported. This would sort like the following:
1 skin
2 background
10 sky
11 light

Added natural sort.


Remove lib comparison

I do not believe this gives a good/honest overview of the 3d space in React Native. We could think about improving this section but it might just be easier to remove it.

What is the current status with:

https://github.com/wcandillon/react-native-webgpu

? Is it already ready to be used? I think this should be added to the comparison here, sure

RN WebGPU is a lower-level cross platform GPU API so the positioning with RN Filament.
But now we run threejs/three-fiber out of the box (model loading, leveraging existing ecosystem, etc) and we also support worklet thread and non mobile platforms (macOS, vision pro).
But maybe let's not do the comparison stuff, I dunno


Make GitHub mirror of wlroots on Freedesktop

Given the instability/downtime of the freedesktop gitlab server, a mirror should be made on GitHub and used as the submodule.

make it then

open an issue there :troll:

WAIT I think I got it one sec

will it autoupdate tho

It should? Idk too much about mirrors

ok then if you want you can make a pr pointing to the repo

lollll


Pre-alpha ARM Storage package

This is the first cut of the ARM Storage work. At present, some calls fail because the Swagger used to generate the code has errors. The client.go and models.go files are generated (as noted in their front-matter). common.go and token.go are helper files.

Hi @brendandixon, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

@brendandixon this is still experimental / incomplete. Can't merge to here, also I can't do the preliminary review here as the repo have far too many watchers and they will be getting emails about all those. Please send the PR to your own repository and cc us there.


Not recording audio

I'm trying to record video and audio using puppeteer stream, but it's getting written only the video to the output file:
async function videoRecorder() {
  require("puppeteer-stream");
  const puppeteer = require("puppeteer");
  const fs = require("fs");

  const filename = `./recordings/test.mp4`;

  const file = fs.createWriteStream(filename);

  const browser = await puppeteer.launch({
    executablePath:
      "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
    headless: true,
    defaultViewport: null,
    devtools: false,
    args: [
      "--window-size=1920,1080",
      "--window-position=1921,0",
      "--autoplay-policy=no-user-gesture-required",
    ],
    ignoreDefaultArgs: ["--mute-audio"],
  });

  const page = await browser.newPage();

  await page.goto(pathToPageWithVideo, {
    waitUntil: "load",
  });

  const stream = await page.getStream({
    audio: true,
    video: true,
  });

  stream.pipe(file);

  setTimeout(async () => {
    await stream.destroy();
    file.close();
    console.log("finished");
  }, 10000);
}

i'll have a look tomorrow

i'll have a look tomorrow

You are writing to a .mp4file however the default format is webm.
Also can you send the pathToPageWithVideo value

You are writing to a .mp4file however the default format is webm.
Also can you send the pathToPageWithVideo value

@Flam3rboy I've tried with webm and din't work either.
The pathToPageWithVideo  I was testing with was https://www.rtp.pt/play/p8157/e518677/telejornal

@Flam3rboy I've tried with webm and din't work either.
The pathToPageWithVideo  I was testing with was https://www.rtp.pt/play/p8157/e518677/telejornal

also note that chrome doesn't support video/mp4.
Have a look at the HTML5 MediaRecorder Options

also note that chrome doesn't support video/mp4.
Have a look at the HTML5 MediaRecorder Options

weirdly enough everything works without any mimeType options and saving it as an .webm file

weirdly enough everything works without any mimeType options and saving it as an .webm file

so you got it to save a file with audio and video by setting the mimeType to video/webm and saving as webm?

so you got it to save a file with audio and video by setting the mimeType to video/webm and saving as webm?

yes everything the same

yes everything the same

@Flam3rboy yes I got it just opened it with another player, thanks.
One more thing, how can I save it as mp3 audio only?

@Flam3rboy yes I got it just opened it with another player, thanks.
One more thing, how can I save it as mp3 audio only?

only if the browser supports mp3 just specify the mimeType

only if the browser supports mp3 just specify the mimeType

Is there anyway to stream it instead of downloading the video? I want to get a Media Stream of the Browser Tab


wrong deathmessage  + other wrong message that have the flair

http://gyazo.com/b0bbbcb0114bab1bb6821ad9b4085645

This was already fixed in 4592b5699ee6b7f0ad2cf25f617b26df85a7308c


Swift 3 Compile Error

Swift 3 Compile Error

[x] I have looked at the Documentation
[x] I have read the F.A.Q.

Issue Info

Info
Value

Platform
iOS

Platform Version
10.0

SnapKit Version
0.15.0

Integration Method
cocoapods

Issue Description

As you can see, SnapKit comes up compiling error.
Hope it can be solved quickly :)

@sunshineclt you must use 3.0.0 for Swift 3.0 you indicated you are currently on 0.15.0.

@robertjpayne Thank you for your assistance! But the document in http://snapkit.io/docs/ is out-dated.


[PIP 141] Pulsar BOM

Motivation
When designing NAR modules loaded by Nifi in the broker such as protocol handlers, proxy extensions, Pulsar IO connectors, etc..., it's important that the dependencies that are common to the module and the broker are as close as possible to prevent incompatible library exceptions (NoSuchClassError, NoSuchMethodError, IncompatibleClassChangeError, etc ...) at runtime. If a class is both in the NAR and in the broker, the broker one will be loaded.
Goal
This proposal is to define a BOM (Bill Of Materials) for Pulsar.
A BOM is a special kind of POM that contains all the dependency versions that are used by the project and can be imported in another project. Currently there is a dependencyManagement section in Pulsar's parent POM but it's not always possible to derive from this parent POM as it imports a lot more things than the dependency versions and external projects usually prefer to have their own parent POM.
External projects can import this BOM and use the same library versions as Pulsar at compile/test time.
API Changes
No API changes
Implementation
The dependencyManagement section of Pulsar's parent POM and related properties will be extracted in a POM and put in a pulsar-bom directory.
The pulsar-bom artifact shall be built and released independently from the rest of Pulsar project (not a maven module).
The Pulsar's parent POM dependencyManagement section is replaced by:
  <dependencyManagement>
    <dependencies>

      <dependency>
        <groupId>org.apache.pulsar</groupId>
        <artifactId>pulsar-bom</artifactId>
        <version>2.10.0-SNAPSHOT</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

The CI will have to build pulsar-bom before building Pulsar.
Reject Alternatives
The BOM could be part of a distinct Git project. This would be harder to handle for contributions that modify both the BOM and Pulsar.

It is possible to use the Pulsar parent POM as a BOM. See for instance how it's done in Pulsar adapters https://github.com/apache/pulsar-adapters/pull/35 . The Parent POM dependencies are imported with
    <dependency>
        <groupId>org.apache.pulsar</groupId>
        <artifactId>pulsar</artifactId>
        <version>${pulsar.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>

It probably doesn't bring much value to separate it in a distinct POM.
So closing for now. Don't hesitate to reopen if needed.

vote for reopening, since a SBOM (Software Bill of Materials) in the Software Package Data Exchange (SPDX) format
(an international open standard (ISO) for communicating SBOM information)
is very helpful and already becomes mandatory for some application/in some organizations to manage security, compliance, and compatibility.
Other major projects provides it e.g. kubernetes see  https://sbom.k8s.io/v1.21.3/source
here is a great blog post, how this can help to map it to Vulnerabilities databases
e.g. directly to Open Source Vulnerabilities (OSV) database
which aggregates information across multiple ecosystems (e.g., Python, Golang, Rust) and databases (e.g., Github Advisory Database (GHSA), Global Security Database (GSD)).
https://security.googleblog.com/2022/06/sbom-in-action-finding-vulnerabilities.html
There is also a tool from kubernetes which helps generating the bom https://github.com/kubernetes-sigs/bom

+1 to reopening. Using the parent POM isn't really an option since it declares a number of dependencies that are not part of Apache Pulsar. A true BOM should only publish the modules that are part of the project and not any third-party dependencies.

I just had a nasty nasty stack overflow error in my IDE that was masking a Maven verison conflict error once I added client-original and local-runner to my POM. After much digging, I was able to resolve it using GRPC's BOM. Can we please have a Pulsar BOM that ensures the entire ecosystem is consistent?
We can't use a parent POM, we have our own. You need a BOM that you can import "sideways" to manage the transitive versions.


Update tvtv.us_us-local.channels.xml

Added KCNCDT1.us, KDVRDT1.us, WCNYDT1.us and Syracuse, NY, USA, locals from #1080
Resolves #1080

Thank you for your help!


In message received from PvPool.power_status target_power is None until someone set value to the microgrid

What happened?
I would like to receive pv power on every update of pv power or its bounds.
I receive power status from pv_pool.power_status.new_receiver() but until I send pv_pool.propose_power, the target_power is None.
Also on every update pv status is send 2 times. I expect 1 message per update.
async def main():
    await initialize_microgrid()

    pv_pool = microgrid.new_pv_pool(priority=1)
    pv_power = pv_pool.power.new_receiver()
    pv_status = pv_pool.power_status.new_receiver()

    counter = 0
    async for selected in select(pv_power, pv_status):
        if selected_from(selected, pv_power):
            print(f"PV power: {selected.message.value}")
        elif selected_from(selected, pv_status):
            print(f"PV status: {selected.message}")

        counter += 1
        if counter % 10 == 0:
            await pv_pool.propose_power(Power.from_watts(-counter))

Logs:
PV status: _Report(target_power=None, _inclusion_bounds=Bounds(lower=Power(value=-30000.0, exponent=0), upper=Power(value=0.0, exponent=0)), _exclusion_bounds=Bounds(lower=Power(value=0.0, exponent=0), upper=Power(value=0.0, exponent=0)))
PV status: _Report(target_power=None, _inclusion_bounds=Bounds(lower=Power(value=-60000.0, exponent=0), upper=Power(value=0.0, exponent=0)), _exclusion_bounds=Bounds(lower=Power(value=0.0, exponent=0), upper=Power(value=0.0, exponent=0)))
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV power: -39 kW
PV status: _Report(target_power=Power(value=-10.0, exponent=0), _inclusion_bounds=Bounds(lower=Power(value=-60000.0, exponent=0), upper=Power(value=0.0, exponent=0)), _exclusion_bounds=Bounds(lower=Power(value=0.0, exponent=0), upper=Power(value=0.0, exponent=0)))
PV status: _Report(target_power=Power(value=-10.0, exponent=0), _inclusion_bounds=Bounds(lower=Power(value=-60000.0, exponent=0), upper=Power(value=0.0, exponent=0)), _exclusion_bounds=Bounds(lower=Power(value=0.0, exponent=0), upper=Power(value=0.0, exponent=0)))

What did you expect instead?

target_power has correct pv power since startup. (not None)
I expect 1 message per update.

Affected version(s)
v1.0.0-rc1200, v1.0.0-rc1300,
Affected part(s)
Microgrid (API, component graph, etc.) (part:microgrid)
Extra information
No response

target_power is the power we want to set, not the measured power.  So, I think the behaviour is correct.  But maybe we can make it easier to understand.

Also on every update pv status is send 2 times. I expect 1 message per update.

This is also a known issue, just hasn't been prioritized: https://github.com/frequenz-floss/frequenz-sdk-python/issues/818#issuecomment-2203358082

Should we convert this issue to a docs improvement issue?

The power_status docs are definitely confusing as they talk about batteries, and I guess we could highlight there that is only for receiving the status of the power manager itself and not the readings from the components.

The PVPoolReport.target_power docs are not super verbose but the attribute itself seems clear that is a target and not the actual read value, no? Maybe we can be more verbose in the docs anyway.

Should we convert this issue to a docs improvement issue?

Yes, please! :)


docs: added 'Getting a File Tree' section

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

Hey there - thanks for your interest in contributing. Could you explain the need for this addition?

Thank you for opening this pull request.
This is a standard message notifying you that we've reviewed your pull request and have decided not to merge it. We would welcome future pull requests from you.
Thank you and happy coding.


Consolidate RAT settings in parent pom

Create a single config for RAT in parent.
Update the RAT exclusion list to filter out hidden directories

Signed-off-by: Gera Shegalov<EMAIL_ADDRESS>

build


Stuck in infinite internal loop, using mixins and recursive generic types

I'm running into a problem, when using mixins and recursive generic types. The weird behavior is that it works fine with 3 generic types, but enters an infinite loop with 4 generic types.
Dart doesn't run.

Why does it get stuck with 4 generic types but not with 3?
Is there any work around?

It's difficult to explain, so it's easier to just show the examples.
How to reproduce
Simply import a file with the following code, or try to run it using an empty main function.
✅ Runs fine

3 Classes & Mixins
3 Generic types extending

Click to view example
/* -= Classes =- */

class A<
  A1 extends A<A1, B1, C1>,
  B1 extends B<A1, B1, C1>,
  C1 extends C<A1, B1, C1>
> {}

class B<
  A1 extends A<A1, B1, C1>,
  B1 extends B<A1, B1, C1>,
  C1 extends C<A1, B1, C1>
> {}

class C<
  A1 extends A<A1, B1, C1>,
  B1 extends B<A1, B1, C1>,
  C1 extends C<A1, B1, C1>
> {}

/* -= Mixins =- */

mixin MixinA<
  A1 extends MixinA<A1, B1, C1>,
  B1 extends MixinB<A1, B1, C1>,
  C1 extends MixinC<A1, B1, C1>
> on A<
  MixinA<A1, B1, C1>,
  MixinB<A1, B1, C1>,
  MixinC<A1, B1, C1>
> {}

mixin MixinB<
  A1 extends MixinA<A1, B1, C1>,
  B1 extends MixinB<A1, B1, C1>,
  C1 extends MixinC<A1, B1, C1>
> on B<
  MixinA<A1, B1, C1>,
  MixinB<A1, B1, C1>,
  MixinC<A1, B1, C1>
> {}

mixin MixinC<
  A1 extends MixinA<A1, B1, C1>,
  B1 extends MixinB<A1, B1, C1>,
  C1 extends MixinC<A1, B1, C1>
> on C<
  MixinA<A1, B1, C1>,
  MixinB<A1, B1, C1>,
  MixinC<A1, B1, C1>
> {}

❌ Gets stuck

4 Classes & Mixins
4 Generic types extending

Click to view example
/* -= Classes =- */

class A<
  A1 extends A<A1, B1, C1, D1>,
  B1 extends B<A1, B1, C1, D1>,
  C1 extends C<A1, B1, C1, D1>,
  D1 extends D<A1, B1, C1, D1>
> {}

class B<
  A1 extends A<A1, B1, C1, D1>,
  B1 extends B<A1, B1, C1, D1>,
  C1 extends C<A1, B1, C1, D1>,
  D1 extends D<A1, B1, C1, D1>
> {}

class C<
  A1 extends A<A1, B1, C1, D1>,
  B1 extends B<A1, B1, C1, D1>,
  C1 extends C<A1, B1, C1, D1>,
  D1 extends D<A1, B1, C1, D1>
> {}

class D<
  A1 extends A<A1, B1, C1, D1>,
  B1 extends B<A1, B1, C1, D1>,
  C1 extends C<A1, B1, C1, D1>,
  D1 extends D<A1, B1, C1, D1>
> {}

/* -= Mixins =- */

mixin MixinA<
  A1 extends MixinA<A1, B1, C1, D1>,
  B1 extends MixinB<A1, B1, C1, D1>,
  C1 extends MixinC<A1, B1, C1, D1>,
  D1 extends MixinD<A1, B1, C1, D1>
> on A<
  MixinA<A1, B1, C1, D1>,
  MixinB<A1, B1, C1, D1>,
  MixinC<A1, B1, C1, D1>,
  MixinD<A1, B1, C1, D1>
> {}

mixin MixinB<
  A1 extends MixinA<A1, B1, C1, D1>,
  B1 extends MixinB<A1, B1, C1, D1>,
  C1 extends MixinC<A1, B1, C1, D1>,
  D1 extends MixinD<A1, B1, C1, D1>
> on B<
  MixinA<A1, B1, C1, D1>,
  MixinB<A1, B1, C1, D1>,
  MixinC<A1, B1, C1, D1>,
  MixinD<A1, B1, C1, D1>
> {}

mixin MixinC<
  A1 extends MixinA<A1, B1, C1, D1>,
  B1 extends MixinB<A1, B1, C1, D1>,
  C1 extends MixinC<A1, B1, C1, D1>,
  D1 extends MixinD<A1, B1, C1, D1>
> on C<
  MixinA<A1, B1, C1, D1>,
  MixinB<A1, B1, C1, D1>,
  MixinC<A1, B1, C1, D1>,
  MixinD<A1, B1, C1, D1>
> {}

mixin MixinD<
  A1 extends MixinA<A1, B1, C1, D1>,
  B1 extends MixinB<A1, B1, C1, D1>,
  C1 extends MixinC<A1, B1, C1, D1>,
  D1 extends MixinD<A1, B1, C1, D1>
> on D<
  MixinA<A1, B1, C1, D1>,
  MixinB<A1, B1, C1, D1>,
  MixinC<A1, B1, C1, D1>,
  MixinD<A1, B1, C1, D1>
> {}

Environment

Dart SDK version: 2.19.2 & 3.0.0-edge
Operating system: Windows 11

We seem to get into an infinite loop in dart::TypeArguments::InstantiateFrom
#0  0x000055d86482ab69 in dart::VMHandles::IsZoneHandle (handle=94388435951120) at ../../out/DebugX64/../../runtime/vm/handles.cc:55
#1  0x000055d8649329e3 in dart::Object::IsZoneHandle (this=0x55d885321e10) at ../../out/DebugX64/../../runtime/vm/object.cc:2911
#2  dart::Object::IsNotTemporaryScopedHandle (this=0x55d885321e10) at ../../out/DebugX64/../../runtime/vm/object.cc:2919
#3  dart::AbstractType::OnlyBuddyInTrail (this=this@entry=0x55d885c72458, trail=trail@entry=0x7f73d6263228) at ../../out/DebugX64/../../runtime/vm/object.cc:20945
#4  0x000055d864939bec in dart::TypeRef::InstantiateFrom (this=0x55d885c72458, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:22651
#5  0x000055d8648cbd6e in dart::TypeArguments::InstantiateFrom (this=this@entry=0x55d885c71db0, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=num_free_fun_type_params@entry=0, space=space@entry=dart::Heap::kOld, trail=trail@entry=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:7360
#6  0x000055d864934fa7 in dart::Type::InstantiateFrom (this=0x55d885c71d68, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:21743
#7  0x000055d864939cc2 in dart::TypeRef::InstantiateFrom (this=<optimized out>, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:22661
#8  0x000055d8648cbd6e in dart::TypeArguments::InstantiateFrom (this=this@entry=0x55d885c71c00, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=num_free_fun_type_params@entry=0, space=space@entry=dart::Heap::kOld, trail=trail@entry=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:7360
#9  0x000055d864934fa7 in dart::Type::InstantiateFrom (this=0x55d885c71bb8, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:21743
#10 0x000055d864939cc2 in dart::TypeRef::InstantiateFrom (this=<optimized out>, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:22661
#11 0x000055d8648cbd6e in dart::TypeArguments::InstantiateFrom (this=this@entry=0x55d885c71ae0, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=num_free_fun_type_params@entry=0, space=space@entry=dart::Heap::kOld, trail=trail@entry=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:7360
#12 0x000055d864934fa7 in dart::Type::InstantiateFrom (this=0x55d885c71a98, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:21743
#13 0x000055d864939cc2 in dart::TypeRef::InstantiateFrom (this=<optimized out>, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:22661
#14 0x000055d8648cbd6e in dart::TypeArguments::InstantiateFrom (this=this@entry=0x55d885c71960, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=num_free_fun_type_params@entry=0, space=space@entry=dart::Heap::kOld, trail=trail@entry=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:7360
#15 0x000055d864934fa7 in dart::Type::InstantiateFrom (this=0x55d885c71918, instantiator_type_arguments=..., function_type_arguments=..., num_free_fun_type_params=0, space=dart::Heap::kOld, trail=0x7f73d6263228, num_parent_type_args_adjustment=0) at ../../out/DebugX64/../../runtime/vm/object.cc:21743

Hey, I find out that the code with four generic types enters an infinite loop is due to a limitation of Dart's type inference algorithm. When you have more than three generic type parameters that depend on each other in a circular way, the type inference algorithm may fail to deduce the correct types and get stuck in an infinite loop.
You could use a separate interface to represent the circular dependency, and have the classes and mixins extend/implement that interface instead of each other directly.
See the example I provided of how you could use a separate interface to represent the circular dependency:
`interface CircularDependency<
A1 extends A<A1, B1, C1>,
B1 extends B<A1, B1, C1>,
C1 extends C<A1, B1, C1>

{}

class A<
A1 extends A<A1, B1, C1>,
B1 extends B<A1, B1, C1>,
C1 extends C<A1, B1, C1>

implements CircularDependency<A1, B1, C1> {}

class B<
A1 extends A<A1, B1, C1>,
B1 extends B<A1, B1, C1>,
C1 extends C<A1, B1, C1>

implements CircularDependency<A1, B1, C1> {}

class C<
A1 extends A<A1, B1, C1>,
B1 extends B<A1, B1, C1>,
C1 extends C<A1, B1, C1>

implements CircularDependency<A1, B1, C1> {}

/* -= Mixins =- */
mixin MixinA<
A1 extends CircularDependency<A1, B1, C1>,
B1 extends CircularDependency<A1, B1, C1>,
C1 extends CircularDependency<A1, B1, C1>

on A<A1, B1, C1> {}

mixin MixinB<
A1 extends CircularDependency<A1, B1, C1>,
B1 extends CircularDependency<A1, B1, C1>,
C1 extends CircularDependency<A1, B1, C1>

on B<A1, B1, C1> {}

mixin MixinC<
A1 extends CircularDependency<A1, B1, C1>,
B1 extends CircularDependency<A1, B1, C1>,
C1 extends CircularDependency<A1, B1, C1>

on C<A1, B1, C1> {}`

Well here, the CircularDependency interface is used to represent the circular dependency between the three generic type parameters, and the classes and mixins extend/implement that interface instead of each other directly. This avoids the type inference issues and allows the code to compile without entering an infinite loop.
I hope this helps :)

This example doesn't stuck in the infinite loop, it finishes in about 15m on my machine. However, it hits an edge case in the handling of recursive types in the VM, resulting in a huge number of TypeRef objects created and added/queried in a TrailPtr (> 10000), which quickly becomes a bottleneck.
I'm going to try to get rid of TypeRefs and simplify handling of recursive type, as outlined in https://github.com/dart-lang/sdk/issues/52022.


Bad IUPAC sequence in glycowork/glycan_data/v9_df_species.csv

I found the following bad IUPAC sequence in glycowork/glycan_data/v9_df_species.csv:
{Gal(b1-?)GlcNAc(b1-?)}{Neu5Ac(a2-?)}Gal(b1-4)GlcNAc(b1-2)Man(a1-3)[Gal(b1-4)GlcNAc(b1-2)Man(a1-3)]Man(b1-4)GlcNAc(b1-4)GlcNAc
It has two Man(a1-3) linkages connected to the same Man(b1-4) residue.

n

Thanks for picking that up! We'll fix it in the root dataset; though that will only roll out with 1.1 as changing the dataset usually causes some downstream changes etc.

Commit 517f3ae has solved this issue and we have put methods in place to prevent that kind of error to enter the database again. The file is already available on the dev branch and will be merged into main soon (hopefully end of the week)


Allow completely dismissing the admin map

Allow the map to be completely hidden if desired (instead of just collapsed really small)

Closing in favor of #674


ci: Unit test coverage as part of the CI/CD pipeline

Purpose

This PR completes #2447
It adds a job to run unit tests and collect coverage
Adds code coverage as a PR comment
Uploads the summary in build job

Pull Request Type
What kind of change does this Pull Request introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[x] Other... Please describe: Changes to our CI configuration files and scripts

Superseded by #2515 - @HenryvanderVegte - the comments have been resolved there


functions implementation

Jitterbit function API implementation:

[x] #12
[x] call expressions

[x] optional/required arguments
[x] evaluation where possible

Literal and operator implementations are prereq for this:

[x] #2
[x] #3

Closed by b60a645


Improve background processing

Attachment processor: transfer the files in separate jobs
Process rows in the background to improve concurrency
Group the rows by common changes (ex. for products by product slug)

Originally part of the roadmap in #5

Process rows in the background to improve concurrency

This would be extremely useful given that, with long files, the background processing library (eg Sidekiq) might crash in the middle and leave unfinished rows. Since the status of the SolidusImporter::Import is left as processing, these never are processed. This has happened a few times on a client project 🥲
Not only that, the way we import Orders right now is super fragile - we import all of them in an after_import call, many times breaking without a visible error, since there's only a context[:success] = false there which provides no useful error messages whatsoever
For files with thousands of lines, we're also building a huge context hash since we need it all in memory for the after_import call.
Since we already have the rows as raw Hash data, I don't understand why we need all the rows to have the context of the import from the beginning. We only need the context from related rows.
I sketched a still early draft solution of a possible solution for both problems: https://gist.github.com/cesartalves/df919b261012499ed30dfd11a6a1a672

Move processing of related Rows to a Job
Move the after_import call of the importer to such a job (would probably need to rename this slightly, maybe to after_group_import, but since the Order importer is the only one that actually implements this, I don't see why not!)

I may write a Draft PR soon!


change(state): Write non-finalized blocks to the state in a separate thread, to avoid network and RPC hangs

Motivation
This PR sends queued non-finalized blocks down a channel to the writer task, which makes state block writes concurrent
Part of https://github.com/ZcashFoundation/zebra/issues/4937.
Designs

https://github.com/ZcashFoundation/zebra/issues/4937
https://docs.google.com/drawings/d/1FXpAUlenDAjl8nkftrypdAPsj0jr-Ut9gZlSP57nuyc

Solution
Use block write task to:

Commit non-finalized blocks.
Propagate errors from queued parent blocks to queued children and skip validate_and_commit calls.
Finalize blocks on best chain past MAX_REORG_HEIGHT.

Related cleanups:

Removes unused chain_tip_sender and non_finalized_state_sender fields of StateService.

Review
This PR is part of regular scheduled work.
Reviewer Checklist

[x] Will the PR name make sense to users?

[ ] Does it need extra CHANGELOG info? (new features, breaking changes, large changes)

[x] Are the PR labels correct?
[ ] Does the code do what the ticket and PR says?
[ ] How do you know it works? Does it have tests?

Follow Up Work

Avoid removing and re-adding known_utxos by nesting the SentHashes and QueuedBlocks inside another struct that holds that tracks the known_utxos
Avoid now-redundant checks against the chains in the Request::AwaitUtxo call by skipping to db.utxo(&outpoint).map(|utxo| utxo.utxo) instead of routing it to the ReadRequest after checking the queue and sent hashes on StateService.
Organize block write task into an object.

I'm still looking into why value_pool_is_updated fails and chain_tip_sender_is_updated hangs

Looks like there's a timing issue, seemingly related to the CommitFinalizedBlock call.
components::mempool::tests::vector::mempool_cancel_mined failed with:

The application panicked (crashed).
Message:  assertion failed: (left == right)
left: 1,
right: 0
Location: zebrad/src/components/mempool/tests/vector.rs:475

https://github.com/ZcashFoundation/zebra/actions/runs/3153382100/jobs/5129789960#step:10:3615
Same failure happened on main in the most recent commit 75a6797
https://github.com/ZcashFoundation/zebra/actions/runs/3153584855/jobs/5130226183#step:10:3674

There was a panic on the testnet sync
The application panicked (crashed).
Message:  only called while blocks is populated
Location: zebra-state/src/service/non_finalized_state/chain.rs:561
Metadata:
version: 1.0.0-beta.15+121.gcbd8cdb
Zcash network: Testnet
state version: 25
branch: non-finalized-block-commit-channel
git commit: cbd8cdb
commit timestamp: 2022-09-28T23:55:45Z
target triple: x86_64-unknown-linux-gnu
build profile: debug
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
0: zebra_state::service::write::write_blocks_from_channels with finalized_state=FinalizedState {
network: Testnet,
debug_stop_at_height: None,
db: ZebraDb {
db: DiskDb {
ephemeral: false, db: RocksDB {
path: "/home/ar/.cache/zebra/state/v25/testnet"
}
}
}
}
non_finalized_state=NonFinalizedState {
chain_set: {},
network: Testnet
}
non_finalized_state_sender=Sender {
shared: Shared {
value: RwLock(
PhantomData,
RwLock {
data: NonFinalizedState {
chain_set: {},
network: Testnet
}
}
),
state: AtomicState(0),
ref_count_rx: 2,
notify_rx: Notify {
state: 0,
waiters: Mutex(
PhantomData,
Mutex {
data: LinkedList {
head: None,
tail: None
}
}
)
},
notify_tx: Notify {
state: 0,
waiters: Mutex(
PhantomData,
Mutex {
data: LinkedList {
head: None,
tail: None
}
}
)
}
}
}
at zebra-state/src/service/write.rs:76

Looks like there's a timing issue, seemingly related to the CommitFinalizedBlock call.
components::mempool::tests::vector::mempool_cancel_mined

This is bug #5301, I'll increase its priority.

Adding the non-finalized state receiver to read_service didn't seem to hurt performance much, so I'm not too concerned about this, but I think if the watch channel is holding a reference to the chains, the Arc::make_mut calls in NonFinalizedState.finalize() may always find a strong count >1 and fall back on cloning all of the chain data.
That may be the case with the Arc::try_unwrap call on the parent_chain that falls back to a clone in NonFinalizedState.validate_and_update_parallel

Adding the non-finalized state receiver to read_service didn't seem to hurt performance, so I'm not too concerned about this, but I think if the watch channel is holding a reference to the chains, the Arc::make_mut calls in NonFinalizedState.finalize() may always find a strong count >1 and fall back on cloning all of the chain data in the non-finalized state.
That may be the case with the Arc::try_unwrap call on the parent_chain that falls back to a clone in NonFinalizedState.validate_and_update_parallel as well.

I'm not too concerned about it either, most of the large data in Chain is wrapped in Arc, inside their vectors or maps. If performance becomes a problem, we should try wrapping Utxo in Arc, it's the only potentially large data that isn't wrapped. (But it's usually small, so we'll need to benchmark if we make that change.)

I started a full and checkpoint sync by going to 'Actions/CI Docker', and running it manually:
https://github.com/ZcashFoundation/zebra/actions/workflows/continous-integration-docker.yml

Here is the running workflow:
https://github.com/ZcashFoundation/zebra/actions/runs/3178119035
Let's check back in about a day?

I just restarted my synced zebrad with this branch, and I got a block timeout:
2022-10-03T23:18:26.272444Z  INFO {zebrad="ef3e9f8" net="Main"}:init{config=Config { checkpoint_sync: true, debug_skip_parameter_preload: false } network=Mainnet debug_skip_parameter_preload=false}: zebra_consensus::chain: initializing chain verifier tip=Some((Height(1830536), block::Hash("000000000134d7c8df43537fd7faac1373d117e3e9c1faebb6d36a96d9610674"))) max_checkpoint_height=Height(1805214)
2022-10-03T23:18:26.272461Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::commands::start: initializing syncer
2022-10-03T23:18:26.272472Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::commands::start: initializing mempool
2022-10-03T23:18:26.272488Z  INFO {zebrad="ef3e9f8" net="Main"}: zebra_rpc::server: Trying to open RPC endpoint at <IP_ADDRESS>:28232...
2022-10-03T23:18:26.272533Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::commands::start: spawned initial Zebra tasks
2022-10-03T23:18:26.272534Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::gossip: initializing block gossip task
2022-10-03T23:18:26.272541Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool::crawler: initializing mempool crawler task
2022-10-03T23:18:26.272540Z  INFO {zebrad="ef3e9f8" net="Main"}: zebra_state::config: finished old database version cleanup task
2022-10-03T23:18:26.272551Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool::gossip: initializing transaction gossip task
2022-10-03T23:18:26.272550Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool::queue_checker: initializing mempool queue checker task
2022-10-03T23:18:26.272632Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: starting sync, obtaining new tips state_tip=Some(Height(1830536))
2022-10-03T23:18:26.275303Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.995% current_height=Height(1830536) network_upgrade=Nu5 remaining_sync_blocks=95 time_since_last_state_block=0s
2022-10-03T23:18:26.305211Z  INFO {zebrad="ef3e9f8" net="Main"}: zebra_rpc::server: Opened RPC endpoint at <IP_ADDRESS>:28232
2022-10-03T23:18:27.179475Z  INFO {zebrad="ef3e9f8" net="Main"}:init{config=Config { checkpoint_sync: true, debug_skip_parameter_preload: false } network=Mainnet debug_skip_parameter_preload=false}: zebra_consensus::chain: Groth16 pre-download and check task finished
2022-10-03T23:19:26.276929Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.997% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=62 time_since_last_state_block=0s
2022-10-03T23:20:26.278827Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.997% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=63 time_since_last_state_block=1m
2022-10-03T23:21:26.280399Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.997% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=64 time_since_last_state_block=2m
2022-10-03T23:22:26.281659Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.996% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=65 time_since_last_state_block=3m
2022-10-03T23:23:26.282543Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.996% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=65 time_since_last_state_block=4m
2022-10-03T23:24:26.283297Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.996% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=66 time_since_last_state_block=5m
2022-10-03T23:24:46.090050Z  WARN {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: error downloading and verifying block e=Invalid { error: Block(Transaction(InternalDowncastError("downcast to known transaction error type failed, original error: Elapsed(())"))), height: Height(1830583), hash: block::Hash("0000000000334454ba3daa548bf77b54735ece8e28ee9b73e480d4bb3b500a6a") }
2022-10-03T23:24:46.144306Z  INFO {zebrad="ef3e9f8" net="Main"}:sync: zebrad::components::sync: waiting to restart sync timeout=67s state_tip=Some(Height(1830578))
2022-10-03T23:25:26.284409Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.996% current_height=Height(1830578) network_upgrade=Nu5 remaining_sync_blocks=67 time_since_last_state_block=6m
2022-10-03T23:25:41.323297Z  INFO {zebrad="ef3e9f8" net="Main"}:crawl_and_dial{new_peer_interval=61s}: zebra_network::peer_set::candidate_set: timeout waiting for peer service readiness or peer responses
2022-10-03T23:25:53.145591Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: starting sync, obtaining new tips state_tip=Some(Height(1830578))
2022-10-03T23:26:00.914635Z  WARN {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: error downloading and verifying block e=Invalid { error: Block(Commit("block already sent to be committed to the state")), height: Height(1830588), hash: block::Hash("0000000000a82784cc38eb5155047eb4ef7218bba441361d3509cc54cb4ec1a7") }
2022-10-03T23:26:00.917005Z  INFO {zebrad="ef3e9f8" net="Main"}:sync: zebrad::components::sync: waiting to restart sync timeout=67s state_tip=Some(Height(1830584))
2022-10-03T23:26:26.285182Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.997% current_height=Height(1830594) network_upgrade=Nu5 remaining_sync_blocks=53 time_since_last_state_block=0s
2022-10-03T23:27:07.918808Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: starting sync, obtaining new tips state_tip=Some(Height(1830594))
2022-10-03T23:27:11.133649Z  WARN {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: error downloading and verifying block e=Invalid { error: Block(Commit("block already sent to be committed to the state")), height: Height(1830616), hash: block::Hash("00000000004da411d03906a2c60752d9b1ee23f7028b327de83083a48f565063") }
2022-10-03T23:27:11.133851Z  INFO {zebrad="ef3e9f8" net="Main"}:sync: zebrad::components::sync: waiting to restart sync timeout=67s state_tip=Some(Height(1830597))
2022-10-03T23:27:26.286034Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.998% current_height=Height(1830616) network_upgrade=Nu5 remaining_sync_blocks=37 time_since_last_state_block=0s
2022-10-03T23:28:18.135809Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: starting sync, obtaining new tips state_tip=Some(Height(1830616))
2022-10-03T23:28:26.287605Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: estimated progress to chain tip sync_percent=99.998% current_height=Height(1830616) network_upgrade=Nu5 remaining_sync_blocks=37 time_since_last_state_block=1m
2022-10-03T23:28:31.037042Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: extending tips tips.len=1 in_flight=19 extra_hashes=0 lookahead_limit=20 state_tip=Some(Height(1830623))
2022-10-03T23:28:31.340978Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: exhausted prospective tip set
2022-10-03T23:28:31.340990Z  INFO {zebrad="ef3e9f8" net="Main"}:sync: zebrad::components::sync: waiting to restart sync timeout=67s state_tip=Some(Height(1830623))
2022-10-03T23:28:31.412646Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool: activating mempool: Zebra is close to the tip tip_height=Some(Height(1830623))
2022-10-03T23:28:32.956587Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool: resetting mempool: switched best chain, skipped blocks, or activated network upgrade tip_height=Height(1830626)
2022-10-03T23:28:32.956614Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool: activating mempool: Zebra is close to the tip tip_height=Some(Height(1830626))
2022-10-03T23:28:36.413824Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool: resetting mempool: switched best chain, skipped blocks, or activated network upgrade tip_height=Height(1830636)
2022-10-03T23:28:36.413861Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool: activating mempool: Zebra is close to the tip tip_height=Some(Height(1830636))
2022-10-03T23:28:41.415867Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::mempool::gossip: sending mempool transaction broadcast request=AdvertiseTransactionIds({Witnessed(WtxId { id: transaction::Hash("d9c3da9f62a707d681b61a73cd8f24e2bddf21aca0c0e7ce203dd38d33cf14f6"), auth_digest: AuthDigest("ad3fe950acdf3fdcb7d7896e3c1ce4e72911267b779545281288f43ccaf675c6") }), Witnessed(WtxId { id: transaction::Hash("36a37d8562010d0cdedcc2c51d34510397c6a21b7190fadab1e891e235ff62f6"), auth_digest: AuthDigest("089841736a6084e2cf79fc075860c203b2460db79f7a60344c1fe03ee6e2676e") }), Witnessed(WtxId { id: transaction::Hash("578a8e17a2341861d24c2d8c8f1c1e631a80d6187697cdaa740de0faf8e60d3d"), auth_digest: AuthDigest("13f645d89d1299913e839fa21c5f2f85ad9fdb3a785ceeadaec151887f675090") })})
2022-10-03T23:29:26.288749Z  INFO {zebrad="ef3e9f8" net="Main"}: zebrad::components::sync::progress: finished initial sync to chain tip, using gossiped blocks sync_percent=99.999% current_height=Height(1830636) network_upgrade=Nu5 remaining_sync_blocks=12 time_since_last_state_block=0s
2022-10-03T23:29:38.342253Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: starting sync, obtaining new tips state_tip=Some(Height(1830636))
2022-10-03T23:29:38.664123Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: extending tips tips.len=1 in_flight=6 extra_hashes=0 lookahead_limit=20 state_tip=Some(Height(1830636))
2022-10-03T23:29:44.666403Z  INFO {zebrad="ef3e9f8" net="Main"}:sync:try_to_sync: zebrad::components::sync: exhausted prospective tip set
2022-10-03T23:29:44.666421Z  INFO {zebrad="ef3e9f8" net="Main"}:sync: zebrad::components::sync: waiting to restart sync timeout=67s state_tip=Some(Height(1830636))

It seems like this PR is not sending all the blocks that need to be committed, or they aren't being sent in the right order.

It seems like this PR is not sending all the blocks that need to be committed, or they aren't being sent in the right order.

I'm looking at these lines with some suspicion, though it does panic in the testnet sync without that check.

It seems like this PR is not sending all the blocks that need to be committed, or they aren't being sent in the right order.

I think I was wrong.
I added some metrics, it seems like we might have a race condition finding UTXOs.
Here is where we can find UTXOs:

in the block queues (finalized or non-finalized)
in the committed blocks (finalized or non-finalized)

Here is where we can't find UTXOs, but we need to be able to find them:

in the block channel (non-finalized only)

If we get a UTXO request after a block has been removed from the queue, but before it has been committed to the non-finalized state, those UTXOs time out, because they are never checked again. (We only check for UTXOs when a new block or UTXO request arrives in the state.)
There are two ways to fix this:

re-check pending UTXOs every time a block is committed to the state
store UTXOs of sent blocks until they are committed

Storing more UTXOs might be the better option, because we are currently CPU-bound. And we can avoid most of the extra memory usage by wrapping the UTXO set for each block in an Arc.

@mergifyio update

(Let's update this PR to the latest main branch, just in case that fixes anything.)

Storing more UTXOs might be the better option, because we are currently CPU-bound. And we can avoid most of the extra memory usage by wrapping the UTXO set for each block in an Arc.

I wonder if we just want to use two instances of QueuedNonFinalizedBlocks, rather than implementing and testing the same queue/UTXO functionality twice. (I don't know if that would work, just a random idea.)

Coverage failed in mempool_cancel_mined, time for another main branch update.

@mergifyio update

@teor2345  Thank you kindly for finding out why the timeout was happening!
I started a new Mainnet sync here: https://github.com/ZcashFoundation/zebra/actions/runs/3184782188

The full Testnet sync succeeded locally.
Sharing the known_utxos field seems less important now that I'm looking at the end-product (https://github.com/ZcashFoundation/zebra/pull/5339), we can potentially close that PR. I replaced the panic on duplicate hashes in QueuedBlocks with an early return here too and am running another full Testnet sync locally.

Sharing the known_utxos field seems less important now that I'm looking at the end-product (#5339), we can potentially close that PR

I agree it's a low priority (and it's not a release blocker). But we might still want to do it to simplify the state service implementation. That would also reduce the amount of code we have to test.
I have some more ideas for simplifications on top of PR #5339, I'll push another PR when I get time.

Updated workflow Now that https://github.com/ZcashFoundation/zebra/pull/5330 has merged: https://github.com/ZcashFoundation/zebra/actions/runs/3189995562

@Mergifyio update

I just restarted my synced zebrad with this branch, and I got a block timeout:

I re-did this local test, and it worked well.

@mergifyio update

Testnet sync succeeded locally again.
Updated workflow for the Mainnet sync now that #5330 has merged: https://github.com/ZcashFoundation/zebra/actions/runs/3190092327

Zebra tip / Log full-sync-to-tip test (end) timed out.
I'm running a full Mainnet sync locally.

Testnet sync succeeded locally again.
Updated workflow for the Mainnet sync now that #5330 has merged: https://github.com/ZcashFoundation/zebra/actions/runs/3190092327

Zebra tip / Log full-sync-to-tip test (end) timed out.

And we can't find out why, because PR #5330 stopped showing test output when running ssh commands.
I opened PR #5358 to see if we can fix that.

Zebra tip / Log full-sync-to-tip test (end) timed out.

There are two possible reasons for this:

bug #5125, fixed in my commits 29a8ada and 56cfa72 in this PR
our checkpoints are out of date and need to be updated (I'll open another PR)

I also merged PR #5358 into this PR, so I can run a full mainnet sync.

I made a few syncer tweaks, hopefully this full sync will pass:
https://github.com/ZcashFoundation/zebra/actions/runs/3201818199

I think we might need to check a few more blocks for UTXOs, I just pushed commit 6551a74.

Let's try a full sync again, with the latest UTXO, syncer, and GitHub actions changes: https://github.com/ZcashFoundation/zebra/actions/runs/3203613563

There were SSH failures, I started another here: https://github.com/ZcashFoundation/zebra/actions/runs/3206336794

We're still seeing a hang after verifying the final checkpoint, we need to fix that bug before we merge this PR.

2022-10-08T18:10:12.430218Z  INFO {net="Main"}:sync:try_to_sync:extend_tips:checkpoint: zebra_consensus::checkpoint: verified final checkpoint: starting full validation final_checkpoint_height=Height(1805214)
...
2022-10-08T18:39:52.199233Z  WARN {net="Main"}: zebrad::components::sync::progress: chain updates have stalled, state height has not increased for 13 minutes. Hint: check your network connection, and your computer clock and time zone sync_percent=98.307% current_height=Height(1805214) network_upgrade=Nu5 time_since_last_state_block=13m target_block_spacing=PT75S max_block_spacing=None is_syncer_stopped=false
2022-10-08T18:40:04.782992Z  WARN {net="Main"}:sync:try_to_sync: zebrad::components::sync: error downloading and verifying block e=ValidationRequestError { error: Elapsed(()), height: Height(1805215), hash: block::Hash("00000000008b4c288a9bbbd87ed95e7dd3e5d65d445deea4b5eb4caa70ae2134") }

https://github.com/ZcashFoundation/zebra/actions/runs/3206336794/jobs/5250967839#step:6:2115
In this case, we needed to find out what block 1805215 was waiting for before it verified.
I suspect it was some UTXOs:
https://github.com/ZcashFoundation/zebra/blob/ee0edef8571d3ed82848303b142c7ae72e7135fc/zebra-consensus/src/transaction.rs#L485
But it could have been something in the state non-finalized block commit request:
https://github.com/ZcashFoundation/zebra/blob/ee0edef8571d3ed82848303b142c7ae72e7135fc/zebra-consensus/src/block.rs#L276
It's very unlikely to be this Depth call:
https://github.com/ZcashFoundation/zebra/blob/ee0edef8571d3ed82848303b142c7ae72e7135fc/zebra-consensus/src/block.rs#L140
I'll modify my metrics branch to show what is happening here.

I'll modify my metrics branch to show what is happening here.

I didn't get to this today, feel free to have a go at it!

I compared the full sync before my sync tweaks:
https://github.com/ZcashFoundation/zebra/actions/runs/3190092327
With the full sync after the tweaks:
https://github.com/ZcashFoundation/zebra/actions/runs/3190092327
The sync after is about 25 minutes faster, and the jobs for recent blocks are faster than they used to be. So I think I'll keep most of the changes. (They should also reduce RAM usage.)

I added a workaround for #5125 in commit 74b29be.
That should be it for this PR, we just need to merge the checkpoint PR and get the full sync working.

I'm running a full sync with the new checkpoints here:
https://github.com/ZcashFoundation/zebra/actions/runs/3223410431
If any of the sync jobs time out, we can fix the workflows, but that doesn't need to block this PR merging.
(The timeouts could be from PR #5360 or this PR, but we want to wait to fix them until we have the final times from both.)

Failed to build zebra-test in the merge queue: https://github.com/ZcashFoundation/zebra/actions/runs/3228683588/jobs/5285115640

@Mergifyio update


Fixed flakes in mpl and plotly backend

This PR fixes the remaining flakes issues raised with:
flake8 --ignore=E,W,F999  .

Looks fine, will merge when tests passes.

I might as well do the tests in this PR as well. So don't merge just yet...

@philippjfr  Read to merge once the tests pass.

And don't forget to do a squash merge this time!

Looks good and the PR build passed, merging.


gsq-roles.ttl has 2 owner concepts

Hi @KellyVance
This vocab https://github.com/geological-survey-of-queensland/vocabularies/blob/master/vocabularies/gsq-roles.ttl
has 2 owner concepts.
Can you please fix?
Thanks, DC

done.
originator prefLabel fixed. From owner to originator


ContextValueFilter is not run on 1.2.22

ContextValueFilter is work on 1.2.11,but it not work on on 1.2.22

能否提供重现问题的testcase？


Fix handling of old-style classes

Pyjaco should either correctly support old-style classes (and we should have unit tests to make sure we have the semantics in place), or (favorably), we should give an error if the user attempts to compile an old-style class.

I suggest a warning instead of an error.

Yes, could be. But some kind of notice for sure. I'm not fully sure how different old- and new-style classes are, but I'd like to avoid subtle errors, where possible. So, if we warn the user but we know the code is going to break subtly, that should be an error, I think. But first of all, I have to figure out why it doesn't do anything at the moment.


question : why do you try to amend the SMILES of the Diastereomers generated by rdkit ?

Hello,
I have a question : inside the class rd_isomer in the isomer_engine.py you go to great length to amend the Diastereomers's  SMILES using the amend_configuration_w function. Is rdkit has trouble generating all possible diastereomer ?
Best,
Etienne Reboul

Hi,
Thanks for the question! RDKit is mostly fine for generating all possible diastereomers, but it could miss some configurations for stereo centers in a ring (example N=C1OC(CN2CC(C)OC(C)C2)CN1).
The total number of stereoisomers should be equal to 2**num_unspecified_steric_centers. If RDKit didn't give all configurations, the missing configurations were amended by finding enantiomers of existing SMILES (treating SMILES as text and inverting stereo centers).
Please let me know if you have further questions.
Best,
Zhen (Jack) Liu

Hello,
Thanks for your fast reply. So rdkit does miss some stereocenters for some reason but it seems like a minor problem. I do have an another question :  In the utils.py script inside the amend_configuration function , you are not using the unspecified_steric_centers , it is computed but never used :

As it is commented out of the script , you  use the number of stereo-centers instead. Is this an oversight or am I missing something ?
Best,
Etienne Reboul

Hi Etienne,
This is a great question! I did swing over using num_centers and num_unspecified_centers, but decided to use the total number of stereo-centers, num_centers here. This won't change the following condition because num is changed by the power of 2.
https://github.com/isayevlab/Auto3D_pkg/blob/ee93d7beae190f7b6c454d0488be3f41f5a5eb62/src/Auto3D/utils.py#L652
Please be reminded that the function in our discussion will only be triggered when the user sets enumerate_isomer = True. There are two scenarios, one is that the SMILES does not contain any stereo information at all. If RDKit lost some configuration, we could safely recover all by finding enantiomers. This is also the case where we found that RDKit missed some configuration in a ring; the other situation is that the SMILES specifies the part of the stereo information. There is a risk that we get extra configurations by finding enantiomers (chance is rare).
This is like a trade-off between the two scenarios, and I thought giving some extra enantiomers is better than missing some configurations. It will be much appreciated if you got an idea and want to contribute to this project :)
Best,
Zhen


fix(masthead): fixed search bar for dir='rtl'

Related Ticket(s)
#3984
Description
The search button remained in its original position on the right even when dir='rtl' was active. This issue remained even when the search bar was opened, glitching through the header menus.
Before:

After:

Changelog
New

Added new .scss classes to account for rtl style changes.

@IgnacioBecerra Thank you for jumping in! Did you set REACT_STORYBOOK_USE_RTL=true environment variable before you start up the Storybook? I basically expect no change to our JavaScript code. For advanced RTL styling, you can use directives in our CSS code (example).

Closing this PR since the issue doesn't occur when REACT_STORYBOOK_USE_RTL=true is active. Configuration on the adopter side must've been incorrect. #4157 provides the correct documentation.


CollectionView within XIB file?

Hi, I tried to use this library with my UICollectionView that is set up in an .XIB file but it doesn't seem to work.
Is the animation not supported for .XIB files?

I have'nt tested the library for .XIB
If needed i will add support for .XIB files in the next release.


state.js errors in ie8

Trying state.js out on a project that (urrgh) supports ie8. looks like it's the console.error statements causing the issue. Will post more info when I have it.

opening the example page gives us some more useful errors.
The transpiled version of the else transition seems to be throwing the error
https://github.com/steelbreeze/state.js/blob/master/lib/state.js#L657

yes, so it's the use of the else keyword in the function prototype that does the damage :disappointed:

Hi Coramc,
Thanks for the report... I think you've uncovered two issues here in IE8 (something that I must confess I've not tested).
Yes, the IE8 JS engine gets confused with else as a method, but there's also something a little bigger going on with the code that browserify emits that IE8 is not happy with.
I will take a look and see if there's a possible fix that's not too disruptive to others.

Looking further into this there's a bunch of other issues beyond those mentioned above; e.g. there looks to be no implementation of Array.prototype.reduce available in IE8.
I've created an IE8 branch and will continue to investigate there.

Hi Cormac,
I've added the required polyfills for the various Array methods I rely on that IE8 does not support. The example still does not work, but I think this is now more of an issue in the example code.

This issue has now been fixed on the IE8 branch; the following changes are only applicable to this branch.

The Transition.else method is renamed to Transition.Else.
The target attribute in the HTML script element when pulling state.js into your page is not supported; instead, the API is always bound to the global variable fsm.
Polyfills have been added to cater for the lesser implementation of ```Array````.

Hey David, thanks for addressing this so quickly


Silverlight version

is there plan to run this moudle in silverlight?
i use library and convert as silverlight version but there is some errors and diffrent on someobject that must be cast
any hope for silverlight 5?

Hello mahdiit, currently I have no plans to create Silverlight version of Pkcs11Interop as there seems to be a lack of demand for such version. You are actually the first person asking for it. Could you please share more info about the conversion process you are using?

hi
we have a web application that use security module like smart cart or token for login, as solution for client side for web application we are using applet and java,
silverlight work inside browser with no dependency to .net framework just a plugin and can read token and do other works that we need
if your library can have silverlight version we can develop some app that work without buggy applet that need java runtime also has more security again java applet.
but i am change your library and build my own silverlight class library, googling problems and fix them
some Marshal method needs change and some win32 exception must change
but if there was a official release it can be better
i see some commercial product like NCryptoki (very similar to your product and some class is same as your lib) that they have silverlight version  but open source solution for security module always is best
Mahdi Yousefi
Web developer
Sent from my iPad

On ۸ ژانویهٔ ۲۰۱۵, at ۲۳:۳۳, Jaroslav Imrich<EMAIL_ADDRESS>wrote:
Hello mahdiit, currently I have no plans to create Silverlight version of Pkcs11Interop as there seems to be a lack of demand for such version. You are actually the first person asking for it. Could you please share more info about the conversion process you are using?
—
Reply to this email directly or view it on GitHub.

Thanks for the detailed info Mahdi. I have managed to get Pkcs11Interop partialy working inside silverlight application but it requires elevated trust otherwise it is unable to access unmanaged PKCS#11 library. I am currently failing to understand why would anyone consider silverlight application to be a better choice than ActiveX component which can utilize full power of .NET and use already available versions of Pkcs11Interop. Have you considered the creation of a custom ActiveX component?

yes, we think
at first of all solutions we have activex but activex run only inside internet explorer
it sounds that Microsoft decide to retire IE in next version of windows and new browsers does not like activex (code name spartan)
Microsoft also has an activex component with name capicom.dll that works well as our needs can read certificates from windows cert store and sign pcks messages.
but we need multi platform, browsers solutions like silverlight and java applet
i use trusted silverlight
it has two simple step:
enable a registry key
sign silverlight app with valid signature
better than activex with .net framework installer or applet with JRE
Sent from my iPad

On ۹ ژانویهٔ ۲۰۱۵, at ۲۰:۵۱, Jaroslav Imrich<EMAIL_ADDRESS>wrote:
Thanks for the detailed info Mahdi. I have managed to get Pkcs11Interop partialy working inside silverlight application but it requires elevated trust otherwise it is unable to access unmanaged PKCS#11 library. I am currently failing to understand why would anyone consider silverlight application to be a better choice than ActiveX component which can utilize full power of .NET and use already available versions of Pkcs11Interop. Have you considered the creation of a custom ActiveX component?
—
Reply to this email directly or view it on GitHub.

Dear, I'm also interested in Silverlight 5 Version. My use case is the same as mahdiit. I'm according full-trust is a must.

I'll try to start experimental branch or new repository for Silverlight5 version next week. I will post more details here when available.

thanks from your works
i can share my changes in your library to work with silverlight with you if you need
i build and use silverlight version but can not test all of it
Sent from my iPad

On ۱۷ ژانویهٔ ۲۰۱۵, at ۱۹:۰۵, Jaroslav Imrich<EMAIL_ADDRESS>wrote:
I'll try to start experimental branch or new repository for Silverlight5 version next week. I will post more details here when available.
—
Reply to this email directly or view it on GitHub.

Thanks Mahdi, but there is no need to share your changes. I would like to step through the problems by myself alone so I can learn more about Silverlight specifics and also test the implementation thoroughly.

Hey guys, I would like to know your opinion on these topics:

Have you ever managed to run Silverlight app in-browser with elevated trust on OS X?
I can't get it working and according to MSDN forum post by Pete Brown trusted in-browser execution is a Windows-only feature. If it is true then Silverlight version of Pkcs11Interop would be also Windows-only.

Are you really performing all steps required to enable in-browser execution with elevated trust on Windows for your production applications?
It is working for me flawlessly but from my experience setting registry keys and importing code signing certificate into the trusted publishers store is a show stopper for regular users.

1- yes, windows only working for in browser
2- need a nsis setup script only about 60-70kb to run on user computer to enable registry key and add selfsign cerfiticate to windows store or use a valid certificate

Correct.

If you like I can contribute with Setup (Registry Key and Certificate)

Thank you guys for your prompt answers. They were really helpful.
@AntonioGa thank you for your offer but I plan to provide just the assembly optimized for Silverlight 5. It is up to each individual developer/integrator to link the assembly to his application and to sort runtime prerequisites in an optimal way for his deployment scenario.
BTW I have managed to get the full test suite running today in Silverlight test application. It has identified a few minor problems that still need to be solved but I expect to publish the code (Windows-only for now) in no more than a week.

Silverlight out-browser works on all os

Yes, trusted Silverlight application works for me Out-of-Browser on OS X Yosemite but I am getting System.MethodAccessException when I want to pinvoke unmanaged function such as dlopen() which dynamically loads PKCS#11 library (dlopen is unix equivalent of LoadLibrary function). I have found MSDN article "How to: Call Unmanaged Code from Trusted Applications" which states that:

Platform invoke in Silverlight can only be used on computers running the Windows operating system. There is no Mac support.

If this is true then there is nothing I can do to make Silverlight version work on OS X.

In my application, I don't have to support Mac and Linux users.

Hey guys, I have just pushed the changes required for Silverlight to the master branch. Source code can be retrieved via git or downloaded as a ZIP archive. Please let me know if it works for you.

thanks, i am testing and inform you about any problem, if exsist

i test for my work reading slots, keys, sign
no problem in silver light
thanks

Great. Thanks for letting me know.


Add LZ as webapp in the collab

LZ as image annotation tool with export of point coordinates compatible with MeshView

When saving the points in LZ, "save" generates a LZ files that can be opened in MeshView, but "save as" generates a zip file that is not recognized by MeshView. Can the redundant "save as" be removed ?

Export overlays button is removed from LocaliZoom now, that's the one which can generate .zip file, which is meant for Nutil, and contains no point clouds. Now it appears in WebWarp only.


sink: Implement DSG Sink

What problem does this PR solve?
What is changed and how it works?
Check List
Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation
Need to update key monitor metrics in both TiCDC document and official document

Release note
Please add a release note.
If you don't think this PR needs a release note then fill it with `None`.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.chend seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

PR currently has a large number of conflicts and I will close it for now. Please feel free to reopen it if you want to continue working on it subsequently. Thank you for your contribution!


UsbManager DeviceList is Empty

Hello,
I am trying to use this library to connect my Xamarin.Forms (mainly Android) app to my PC via USB. When trying just the example, everything compiles correctly on my tablet (Acer Iconia 7 B1-780), however no devices are ever found when connected to my PC. I have tried using both regular and OTG cables, neither have given me any success.
I have had success in connecting to an Arduino via the OTG cable, so I know the library works. Its just a matter of getting it to work with a PC connection that seems to escape me.  The laptop is a Dell Lattitute with an Intel USB controller chip, but don't know the vendor id or its product id. Is the problem just with the driver?
Any help on this would be greatly appreciated.
Cheers!

I think the problem is that your Android device and your PC are both acting as USB Hosts.  You can only have one USB host, which can talk to many devices.  This library is probably not the right tool for your task.  You may want to look at the jMTP library, as referenced in this Stack Overflow thread.

@CatimusPrime what was your final solution? did you get something working?


Takes a long time to build runes which have a large number of labels

Building the landmark_na rune on a macbook air with rune build Runefile.yml took 2h 21m 15s. The build process spent most of it's time on the following process:
[2021-05-26T19:28:34.923Z DEBUG rune_codegen::environment] Executing "cargo" "+nightly-2021-05-09" "build" "--target=wasm32-unknown-unknown" "--quiet" "--release"

This issue persists on a windows laptop as well.
Rune details:

Rune has 99424 labels which is by far the largest amount (next largest runes we have are mushroom and food with 4128 and 2024 labels respectively).
Rune size is 55.8 MB, however it is unlikely that the size of the rune is what's causing the issue (Building mushroom rune takes 1m 2s - Its size is 29 MB)

Additional Information:
Created landmark_na rune causes runic_mobile app to crash (on both iOS and Android). This has been outlined in issue #23 in the runic_mobile repo.
Discussion Points:

An idea that @akshr had was to move the labelling processing block out of the landmark_na rune into a second rune and chain them together.
@kthakore proposed creating a labeling only service where the rune points to the label file in the host device to use for labelling (avoids loading the labels within the rune entirely).

Copying across my response on slack:

I have a feeling I know what the issue will be... Basically, with 100,000 labels we'll be creating a Rust array literal with 100,000 elements. At that scale, the generated lib.rs will probably be megabytes in size, take forever to parse and the internal data structures managed by the Rust compiler will be massive. Hence why it takes forever to build, or doesn't complete at all.
The way we get around this is by moving it to a file, using include_str!() to embed the file into the binary as a string literal, then doing a bit of processing to turn it back into an array when running the _manifest() function.
The Runefile itself is probably massive, so we may want to figure out a way to move the labels out and into a separate file. A while ago, @kartik and I were throwing around the idea of saying "read this argument from a file". Maybe we should revisit that.

This seems quite similar to several issues I had when writing the include_dir!() crate:

Pathological compile times (https://github.com/Michael-F-Bryan/include_dir/issues/22)
Extremely large memory overhead (https://github.com/Michael-F-Bryan/include_dir/issues/33)
Unbounded memory growth (https://github.com/rust-lang/rust/issues/56900)

As mentioned in #320,

This should be resolved by #310 because you'll be passing a single (albeit long) string literal to the label proc block.


What is the framework for POET?

Dear Authors,
I am amazed that the performance of POET outperforms POFO so much. Since PyTorch does not have a traced backward graph, I am curious about which framework you are using.

Hi super-dainiu, thank you for your interest.
POET uses Gurobi for solving the MILP formulation. Gurobi can be replaced by cvxpy/coin-or or any open-source ILP solvers. We generate the graph and the corresponding dependencies ourselves. The network is defined using Pytorch-compatible methods.
To further highlight the benefit, POFO supports chain (linear) model graphs while POET is more generic and supports any arbitrary model graphs such as ResNets/BERTs etc.


Added Material Editor

Added a material editor that allows you to create custom materials (Stored in FedFigs/Materials as .vmt files) and apply them to your chams.
This seems to work pretty well and didn't cause any crashes or performance hits while using.
I've also added an option to disable the "Anti MOTD" feature. You can access it under Visuals->Misc->Removals

Good stuff, thanks LNX


migrating MoneyRequestAmountPage to functional component

Details>
Refactoring MoneyRequestAmountPage to a functional component
Fixed Issues

$ https://github.com/Expensify/App/issues/16278
PROPOSAL: https://github.com/Expensify/App/issues/16278#issuecomment-1593334438
Tests

Open a MoneyRequestAmountPage it's opened when we create a Split Bill or Request Money or Send Money
To test thoroughly change the currency, amount multiple times

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android / native
[x] Android / Chrome
[x] iOS / native
[x] iOS / Safari
[x] MacOS / Chrome / Safari
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

Screenshots/Videos

Web
https://github.com/Expensify/App/assets/27822551/802a61e7-7f13-49d6-904a-6911c2eb0923

Mobile Web - Chrome

https://github.com/Expensify/App/assets/27822551/ee20d88d-6657-4c31-9371-c4d2b8348658

Mobile Web - Safari
https://github.com/Expensify/App/assets/27822551/21db783e-6d0c-41e5-93c0-750bcdab7508

Desktop

https://github.com/Expensify/App/assets/27822551/033fe6d7-9d43-4470-81be-412f7596de47

iOS

https://github.com/Expensify/App/assets/27822551/c629b4ef-4c0f-476b-aa95-f6ad0be3e859

Android
https://github.com/Expensify/App/assets/27822551/747b36aa-0d87-40fd-8156-923e5008594d

@thesahindia , Updated the checklist

Thank you for pointing out @thesahindia . That occurred while resolving merge conflicts

Taking this one as C+ (reviewer)

@BhuvaneshPatil Let me know when this is ready for re-review

Hi @s77rt , I am working on the useEffect refactoring, will let you know once done

@s77rt I have pushed the changes, I think there are still more improvements need to be done for useEffect implementations. Please provide the feedback on the same.

@BhuvaneshPatil @s77rt Can we please prioritize getting this one over the finish line?

@BhuvaneshPatil I see the commits now. Seems to cover only two concerns: the useCallback and useRoute. Can you please address the rest of the comments (and the bug)? If anything is needed let's work on this on Slack

Bug that you have mentioned is resolved

about the comments for using previous state, I have added few comments in the conversation about the approach that I am using, please have a look.

I don't see any comments, can you please double check?
We have this comments unresolved:

discussion_r1246460263
discussion_r1245995332

Feel free to use the Slack thread I have tagged you in to collaborate faster if need be. Thanks!

@BhuvaneshPatil I see bunch of comments marked as resolved but they are not (and do not have a comment explaining why they were ignored). This makes the review process harder (having a deja-vu :sweat_smile:)

https://github.com/Expensify/App/pull/21005#discussion_r1240075766
https://github.com/Expensify/App/pull/21005#discussion_r1254317502
https://github.com/Expensify/App/pull/21005#discussion_r1245932267
https://github.com/Expensify/App/pull/21005#discussion_r1245934194
https://github.com/Expensify/App/pull/21005#discussion_r1245965065

@BhuvaneshPatil The last commit didn't resolve all the comments. I have also added some more requests. Can you please tag me once this is ready for review?

Hey @BhuvaneshPatil I see the comments marked as resolved, but I don't see any new commits yet 😕

@BhuvaneshPatil Thanks! The code is looking better, hopefully we will get to the testing phase today. I just see one left unresolved comment, please check:

https://github.com/Expensify/App/pull/21005#discussion_r1245965065

@BhuvaneshPatil The unresolved comment is about the title.

@s77rt , removed the useMemo implementation.

@s77rt , updated the code

@BhuvaneshPatil Thanks! Can you please resolve the conflicts?

Yes. Working on them. There we have new method - dismissModalIfNecessary. It's being called in useEffect, shall I add eslint disable for exhaustive hooks?

I don't see the need of useCallback here. Just wrap the function in it's own useEffect

I don't see the need of useCallback here. Just wrap the function in it's own useEffect

@s77rt Sorry, but I am afraid that I don't understand what you are implying here

@BhuvaneshPatil Yes, that should do it; but reportID is a ref so I think you meant to pass reportID.current.

@s77rt I have updated the code for dismiss modal

@BhuvaneshPatil Awesome! Code looking good. While testing on iOS I noticed some cursor jump behaviour, I think this bug exists in main but the jump behaviour seems to happen more often. Can you please confirm from your side if this is a regression or not?

Reviewer Checklist

[X] I have verified the author checklist is complete (all boxes are checked off).
[X] I verified the correct issue is linked in the ### Fixed Issues section above
[X] I verified testing steps are clear and they cover the changes made in this PR

[X] I verified the steps for local testing are in the Tests section
[X] I verified the steps for Staging and/or Production testing are in the QA steps section
[X] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[X] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[X] I checked that screenshots or videos are included for tests on all platforms
[X] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[X] Android / native
[X] Android / Chrome
[ ] iOS / native
[X] iOS / Safari
[X] MacOS / Chrome / Safari
[X] MacOS / Desktop

[X] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[X] I verified proper code patterns were followed (see Reviewing the code)

[X] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[X] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[X] I verified that comments were added to code that is not self explanatory
[X] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[X] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[X] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[X] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[X] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[X] I verified the JSDocs style guidelines (in STYLE.md) were followed

[X] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[X] I verified that this PR follows the guidelines as stated in the Review Guidelines
[X] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[X] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[X] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[X] If a new component is created I verified that:

[X] A similar component doesn't exist in the codebase
[X] All props are defined accurately and each prop has a /** comment above it */
[X] The file is named correctly
[X] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[X] The only data being stored in the state is data necessary for rendering and nothing else
[X] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[X] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[X] All JSX used for rendering exists in the render method
[X] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[X] If any new file was added I verified that:

[X] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[X] If a new CSS style is added I verified that:

[X] A similar style doesn't already exist
[X] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[X] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[X] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[X] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[X] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[X] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[X] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Web

Mobile Web - Chrome

Mobile Web - Safari

Desktop

iOS

Android

@mountiny We are probably good to merge, just need a confirmation from @BhuvaneshPatil about the iOS behaviour, it's seems to have a buggy cursor but I'm getting this on main too. So I guess we can move on with that

@s77rt , I checked on master and new implementation. The jumpy behavior is at both places

@BhuvaneshPatil Thanks for the confirmation.

Just waiting for a lint PR fix to avoid errors once merged.

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/Julesssss in version: 1.3.39-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/francoisl in version: 1.3.39-11 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


[Bug]:  Inconsistent Document Count Displayed in Knowledge Base

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Branch name
main
Commit ID
main
Other environment information
- Operating System: Windows 10
- Browser: Chrome
- ragflow version: latest

Actual behavior
The document count displayed on the main page is inconsistent with the actual number of files in the knowledge base, with fewer numbers being shown.
Expected behavior
The main page should reflect the actual number of documents present in the knowledge base.
Steps to reproduce
1. Log in to the ragflow project.
2. Navigate to the knowledge base section.
3. Observe the document count displayed on the main page.

Additional information

We are gona fix it.

Upgrade to the latest dev version of the docker image. Problem solved for new knowledgebases.
But the former data is still there.


Saturn trace sampler

Add a sampler that rely on Saturn config to set sample ratio.
@mlefebvre @KevGoDev
Depends on #241

Rebase, ready for review

cool lgtm


Contractor journal

My biggest PR in a while.
Recently it became clear that the contractor was not scaling well when storing lots of data.  Our tests showed that this amounted to about 35MB of data being encoded and written to disk (with fsync).
To address this, I wrote a json-based transaction journal. The journal records updates to the contracts instead of rewriting the whole contract. These updates are applied to the original object at regular intervals. The primary goal is to reduce latency when uploading/downloading, so for now, only those actions take advantage of the journal.
Things got a little ugly because the contractor persist object changed: it now uses string-keyed maps instead of slices in a few places.
Left to do:

fix bugs in journal (see TODOs in journal.go)
add metadata to journal format
test extensively on antfarm before even attempting with real contracts

Hooray. Eventually the hope is that the journal will turn into a full write ahead log for managing all sorts of complex persist updates that hopefully iteratively boost scalability and speed as we add them. I think this is a pretty good start, and while it may not iterate for a while, I'm looking forward to testing it, using it, and expanding it.

would be great if there was some way to test the cached revisions from the api package.
oh hmm. We could if build.Testing && randInt(10) { dropConn } in key places within the host. I'm not sure how gracefully our existing tests would handle that, but actually I think reasonably gracefully.

Maybe too much at the moment, but now that we're no longer building a general solution to work with any json object, we can reasonably drop the string parsing for setters for each type of update. And right now I think that's only the fc operations, as everything else just does a full flush (fine as it's not happening several times per second)

I would like to keep the updates arbitrarily extensible though. Like right now it's super easy to add another field to an update. Not sure the best way to achieve this with setters.

Specific update types seem like the right way to go. I will explore what that looks like.

Using specific updates, each revision results in about 2KB synced to disk. This is a 15,000-fold improvement over the current state of things, assuming a 30MB contractor.json :P
There is certainly room for improvement in that figure as well. By my estimates, we could get it as low as 300 bytes. But that's easy enough to add later, when we are confident in the safety of this scheme.
Benchmarking shows that update is still quite slow, and that the slowdown is entirely due to syncing. With syncing, each update takes about 11ms. Without syncing, that drops to <100us, and the JSON encoding becomes the bottleneck.
Left to do:

Make updates idempotent
Add a header to the journal with version metadata
Add checksums to each update set

With syncing, each update takes about 11ms.

That's an SSD number, HDDs will be slower.
When we split out the log, I'm pretty sure we'll be able to do staggered syncs, meaning 10 pieces all trying to sync at the same time would be able to do it in 11-30ms instead of in 110ms.
11ms is still a huge improvement over where it is right now. I'm currently able to do about 1 piece per second. With this improvement, I'm hoping it'll jump to 5 or 10.

It's not super clear to me how the cached revisions are working, nor that they are safe.
Can you put some comments somewhere explaining the process, why it's safe, and then add some DEBUG code to the save() / checkpoint() functions to make sure that the cached revisions are in fact being saved correctly?

Now that the sanity checks are in place, if you get this working on the antfarm I'll feel comfortable throwing it on the server. Then we can see what sort of speedups we're getting.

ok, working in the antfarm now, including upload+download after a full renew cycle.
The only thing left to test here is that it can successfully load the old contractor.json.

I will try it out on the server as soon as the renew cycle completes :)

I ran it on the server, and... VROOOM. Went from about 12mbps sustained upload to doing about 170mbps sustained upload. Those numbers are not exaggerated, they are the production values that I've been measuring.
So, basically this seems good to go. And also, a major, major improvement. Very excited to have this make the v1.1.1 release.


[Feature request - CTT-Windows-Util] Not asking for wifi and shuting down without any info why, when device running it is not connected to Internet !!!

-Problem-
I have CTT-Windows Utilities Pin on my Startup shortcut (dosnt matter for the feature request)
for the purpous of openning it without going trough file explorer or even by typing it in search for fast and efficient way to open directly the utility tool. But As im Im used to disconnect my Windows Os to the Internet EVERY time I dont need it. I found out That the software/script ... Start and shutdown Imediatly when its not communicating with Internet !!!!
This lack of log, warning or information of why the Script isnt starting, for non computer people, can be really confusing and frustrating
as it also is for ME(HackanHacker xD) for 50% of the time it does sart and shutdown(Im starting it)
even thought I did figured out what cause it (wifi off) after a few time.

-Solution-
I would like the script somewho to inform the user that its not communicating with internet which is needed for the script to initiate and start its automatic fonctions (Like downloading winget each use). at minimum!
The best tough would be it, as is run as/with administrator permission, to do it automaticly
Which is MY main "problem" speed and efficiency...
Well I dont think I need to say anything else here :P It is a simple problem to solve ! ;)

-No add. Context.-

Tkx for your considerations and for this great community project/product This is one of the greatest effort of making a user friendl;y tool to use !!! It as to continue that way, thats How Computing should be!!!

Oh i see So it would be better to make that solution on my ends. Absolutly fine for me thanks for the heads up !!!!


Removed gemspec

Revert of #6.
Mock collector becomes independent so gemspec is not needed.

[ ] depends on: https://github.com/ManageIQ/topological_inventory-collector-mock/pull/4

@agrare