Default metadata configurability

[x] I have searched the issues of this repository and believe that this is not a duplicate.

Summary
Concentratord does add metadata to the stats report. The concentratord should provide a mecanism to disable those metadata.
What is the use-case?
These metadata may not be needed and users may want to remove them to save bandwidth (e.g. mobile data plan).
Implementation description
I suggest adding a [metadata] section to the configuration file with a concentratord property that would default to concentrator_temp,concentratord_version,config_version,hal_version,model (array of string of enabled metadata).
[metadata]

concentratord="concentrator_temp,concentratord_version,config_version,hal_version,model"

Redefining this property to empty would allow the user to remove those metadata. The default behavior is retro-compatible.
Can you implement this by yourself and make a pull request?
Possibly.
Workaround
I could not find a workaround to remove the metadata. Any idea is welcome.

Have you evaluated what would be the saving in MBs on a monthly basis? In case of a cellular back-haul, I would first suggest the reduce the stats interval, e.g. from 30 seconds to 5 minutes.
Note that in case of ChirpStack, it would also cause the NS to malfunction as it needs the config version to validate that the Concentratord channel-plan is correctly configured. If not it will send a reconfiguration command (and it will keep sending this until the Concentratord is in sync).


Become compatible with semantic versioning specification

See semantic Versioning 2.0.0 specification.

Explicitly document that Bucket4j obeys Semantic Versioning specification.
Rename 2.2 branch to 3.0 branch, because of incompatible changes in the API.
Automate incompatible API changes detection via revapi-maven-plugin

This feature will be included in the scope of #70


Cannot use textobjects after merging nvim-treesitter/nvim-treesitter#2272

PR: https://github.com/nvim-treesitter/nvim-treesitter/pull/2272
|| Error detected while processing FileType Autocommands for "*":
|| E5108: Error executing lua ...r-textobjects/lua/nvim-treesitter/textobjects/select.lua:59: attempt to concatenate local 'query' (a table value)
|| stack traceback:
|| 	...r-textobjects/lua/nvim-treesitter/textobjects/select.lua:59: in function 'attach'
|| 	...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:458: in function 'attach_module'
|| 	...er/start/nvim-treesitter/lua/nvim-treesitter/configs.lua:481: in function 'reattach_module'
|| 	[string ":lua"]:1: in main chunk
|| No localrc found

Did you make sure to reinstall the parser (restarting Neovim) and upgrade to the latest version of both nvim-treesitter and nvim-treesitter-textobjects?

Note that the capture names have changed, so you may have to adapt the mappings in your setup.

Removing this snippet fixed it for me
      -- Or you can define your own textobjects like this
      ["iF"] = {
        python = "(function_definition) @function",
        cpp = "(function_definition) @function",
        c = "(function_definition) @function",
      },

Which probably should've been marked as a breaking change in https://github.com/nvim-treesitter/nvim-treesitter-textobjects/commit/2623fc7e82c876c394769e4ab9dff8bb2525e58f

@MunifTanjim can you add a "Rosetta stone" for your textobject queries (old -> new)?

This is not related to the linked PR; see https://github.com/nvim-treesitter/nvim-treesitter-textobjects/commit/2623fc7e82c876c394769e4ab9dff8bb2525e58f

The ability to specify custom directly in your mappings was removed (because it never really worked), so you need to adapt your config accordingly.
(This should have been marked as breaking, but we hadn't properly set up workflows around this.)

@clason I cannot believe this. I use customized mappings all the time :((.

@clason, thanks for clarifying the issue for me.

Note: customized mappings  still work, it's just that you can't specify custom captures  in those mappings directly; you first have to define them in a local query. The PR description and the updated README explain what you have to do now to get the same mappings back.


GA4GH converters only

Depends on this fork of ga4gh-schema that updates protobuf i POM to
<proto.version>3.2.0</proto.version>

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/386/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/387/
Test FAILed.

Added a test that demonstrates converting a list of GA4GH protobuf alignment objects to JSON using code
val adamRead1 = makeRead(10L, "10M", "10", 10).build()
val adamRead2 = makeRead(11L, "10M", "10", 10).build()
val gaReads: Seq[Reads.ReadAlignment] = List(AlignmentRecordConverterGA4GH.toGAReadAlignmentPB(adamRead1), AlignmentRecordConverterGA4GH.toGAReadAlignmentPB(adamRead2))
val result: ga4gh.ReadServiceOuterClass.SearchReadsResponse = ga4gh.ReadServiceOuterClass.SearchReadsResponse.newBuilder().addAllAlignments(gaReads.toList.asJava).build()
val resultJSON: String = com.google.protobuf.util.JsonFormat.printer().print(result)

produces GA4GH JSON String
 {
    "alignments": [{
      "readGroupId": "rg",
      "fragmentName": "read0",
      "improperPlacement": true,
      "numberReads": 1,
      "alignment": {
        "position": {
        "referenceName": "myCtg",
        "position": "10",
        "strand": "POS_STRAND"
      },
        "mappingQuality": 60,
        "cigar": [{
        "operation": "ALIGNMENT_MATCH",
        "operationLength": "10"
      }]
      },
      "alignedSequence": "AAAAAAAAAA",
      "alignedQuality": [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
    }, {
      "readGroupId": "rg",
      "fragmentName": "read0",
      "improperPlacement": true,
      "numberReads": 1,
      "alignment": {
        "position": {
        "referenceName": "myCtg",
        "position": "11",
        "strand": "POS_STRAND"
      },
        "mappingQuality": 60,
        "cigar": [{
        "operation": "ALIGNMENT_MATCH",
        "operationLength": "10"
      }]
      },
      "alignedSequence": "AAAAAAAAAA",
      "alignedQuality": [9, 9, 9, 9, 9, 9, 9, 9, 9, 9]
    }]
  }

This JSON conversion should probably be added as a utility function inside AlingmentRecordConverterGA4GH
@akmorrow13  - I think you said that producing  GA4GH JSON like this, starting with an ADAM AlingmentRecord, was the functionality you need for the notebook application, let me know if this does the job.

Added
AlignmentRecordConverterGA4GH.listGApbToJSON()
VariantContextConverterGA4GH.listGApbToJSON()

GA4GH protobuf to JSON converter functions

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/388/
Test FAILed.

@jpdna we should also remove the current GA4GHAlignmentRecordConverter and make sure it works with your new one.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/393/
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/394/
Test FAILed.

We will need to access these converters in python. To do so, they will have to be accessible through a java API.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/396/
Test FAILed.

We will need to access these converters in python. To do so, they will have to be accessible through a java API.

Let's talk about what that entails and where to get started.
Where is the java API?
are we talking about ADAM's
org.bdgenomics.adam.api.java
and if not where would such code go?

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/mango-prb/398/
Test FAILed.

Where is the java API?

Perhaps now is a good time to interject with another pointer to
https://github.com/bigdatagenomics/bdg-convert
Having implemented a bunch of these, I think there is maybe a bit too much boilerplate, but that is java for you.  I haven't been following this PR or mango very closely though, so I don't know all what is required here.  With an idea of what the source and target classes are, I could whip up the maven module with // todo: markers for the interesting bits.

Cool @heuermh ! The source is the mango-core/src/main/scala/org/bdgenomics/mango/converters scala classes, and the target is java ga4gh converter classes. Would this be a new module in mango (ie mango-converters?)

If we want to go from GA4GH Variant java class to/from bdg-formats Variant java class, then that should probably be hosted outside of mango.

Having reviewed this PR a bit, I see that is the case for the most part.  I'll code it up and use Feature as a demonstration.

Keep an eye on https://github.com/heuermh/bdg-convert-ga4gh.  We can figure out where it needs to live later I suppose.

Keep an eye on https://github.com/heuermh/bdg-convert-ga4gh.

ok, I've added basic support for converting bdg-formats Feature to GA4GH Feature.  The nice thing about the bdg-convert API is that you can ignore almost all of it.  Have Guice inject the converter you want into your constructor.  All of its dependencies and construction and horrible class names are hidden from you.
final class MyClass {
  private final Converter<org.bdgenomics.formats.avro.Feature, ga4gh.SequenceAnnotations.Feature> featureConverter;
  private final ConversionStringency stringency = ConversionStringency.STRICT;
  private static final logger = LoggerFactory.getLogger(MyClass.class);

  @Inject
  MyClass(final Converter<org.bdgenomics.formats.avro.Feature, ga4gh.SequenceAnnotations.Feature> featureConverter) {
    this.featureConverter = featureConverter;
  }

  void doIt() {
    org.bdgenomics.formats.avro.Feature source = ...;
    ga4gh.SequenceAnnotations.Feature target = featureConverter.convert(source, stringency, logger);
  }
}

OOC, what is the advantage of using dependency injection here?

OOC, what is the advantage of using dependency injection here?

Generally, check out the Best Practices linked from
https://github.com/google/guice/wiki/Motivation
Specifically:

All Converter<S,T> implementations are final, stateless, threadsafe, @Immutable
All implementation constructors can be package private
All implementation classes can be package private
Clients only @Inject directly the converter they need
Client never has to call new, or even know what the unwieldy class name might be
Converters are narrow in scope and easily testable
Converters are composable and potentially deeply nested, client doesn't have to know or care

I guess I just don't see how this is an improvement from how we are currently doing things? Our current converters may not be stateless, but they all are immutable. Additionally, I'm not sure I see how DI materially changes the scope of our converters or makes them easier to test. I feel like our approach to writing converters is already pretty narrowly scoped.
Otherwise, I didn't really get the pro WRT composability/nesting. I don't see a use case for deeply nested converters in what we're doing.
WRT:

All implementation constructors can be package private
All implementation classes can be package private
Clients only @Inject directly the converter they need
Client never has to call new, or even know what the unwieldy class name might be

All that's happening is that Guice creates an instance of the converter via reflection, no? Don't you need to know the class name to bind to the implementation?


Securing the communication (Ratelimits, Auth, etc)

https://medium.freecodecamp.org/how-to-secure-your-websocket-connections-d0be0996c556

Ratelimit
Authentication
Logging
Blocking
...

Consider these remarks about SANIC and make the service more robust
https://www.reddit.com/r/Python/comments/876msl/sanic_python_web_server_thats_written_to_die_fast/


Two start tags in sequence move inner end tag to the end of the outer one after round trip

If we have
The quick brown fox jumped over the lazy dog.

after fromXhtml( ) and back toXhtml( ) result will be
The quick brown fox jumped over the lazy dog.

It is expected result to be same as initial input.

Yes, I can reproduce this problem. I will work on fixing it -- thanks!

Fixed in 0.5.1. Thanks!


⚠ GitLab Server has degraded performance

In d227b55, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 6719 ms

Resolved: GitLab Server performance has improved in e025f5f.


salesforce: revenue and loss grouped by opportunity type query

This query sums the total amount from opportunities grouped by type. A total is displayed at the bottom

Looks good. Please keep tabs consistent in SELECT and WHERE clauses and capitalize the final ORDER BY.

Thank you for checking. I pushed some updates. Please review again. Thank you!

@trevorfox done with the requests here.


tests, Fix networkpolicy tests and support them upstream

Networkpolicy tests were skipped because when those tests were created, k8s didnt support them.
Now K8s does support them (2 years passed since).
Fixing Flakiness by using waiting for the nettwork policy to finish its async creation (using eventually on the ping).
Use PingFromVMConsole robust method.
Skip Kind providers, since its not supported on kubevirtci kind lanes out of the box.
Wait for Network Policy deletion, in order not to mix policies between tests.
Refactor tests.
Signed-off-by: Or Shoval<EMAIL_ADDRESS>None

how should we test it ?
it needs ocp + ovs

@oshoval Quique is now working on conversion of these tests to be runnable on Calico. After that, we can move forward with this one.

@oshoval @qinqon is now working on conversion of these tests to be runnable on Calico. After that, we can move forward with this
one.

ok thanks
adding hold meanwhile i guess
/hold

@phoracek
@EdDev
i think meanwhile we can do code review on how it should look, even if we dont merge until its tested?

/test pull-kubevirt-check-tests-for-flakes

/hold cancel
Skip was safe to remove, thanks Quique

Ready to review, PTAL

Thanks Quique for the eventually solution

/test pull-kubevirt-e2e-k8s-cnao-1.17

I'm concerned about the ping eventually approach.
Why was that not required until now ?... This has the potential to bloat - by a lot - the execution times on failed builds.

Those tests are were on skip this is the reason..
it takes a bit time for the policy to be applied, it uses Edy's mechanism that already saves time even when ping failed, because it wont wait for the timeout.
There is one bug anyway that i do need to fix beside the comments.

I'm concerned about the ping eventually approach.
Why was that not required until now ?... This has the potential to bloat - by a lot - the execution times on failed builds.

Those tests are were on skip this is the reason..
it takes a bit time for the policy to be applied, it uses Edy's mechanism that already saves time even when ping failed, because it wont wait for the timeout, but even stop on retcode != 0, instead of timeout.

is it possiable that the  Eventuallys will not be on the ping but on the appliance of the policy, because the way I see it, those are two different checks. I think that Eventuallys scope should be limited as possible because the tend to cover for bugs and here maybe(not sure) we should limit it to the policy appliance. is there a way to check that the policy was applied without pinging, and to use the Eventually just for the policy appliance check? like try to GET it, or something else, or ping is the right way?

There is one bug anyway that i do need to fix beside the comments.

I'm concerned about the ping eventually approach.
Why was that not required until now ?... This has the potential to bloat - by a lot - the execution times on failed builds.

Those tests are were on skip this is the reason..
it takes a bit time for the policy to be applied, it uses Edy's mechanism that already saves time even when ping failed, because it wont wait for the timeout, but even stop on retcode != 0, instead of timeout.

Is it possible that the Eventuallys will not be on the ping but on the appliance of the policy, because the way I see it, those are two different checks. I think that Eventuallys scope should be limited as possible because the tend to cover for bugs and here maybe(not sure) we should limit it to the policy appliance. is there a way to check that the policy was applied without pinging, and to use the Eventually just for the policy appliance check? like try to GET it, or something else, or ping is the right way?

There is one bug anyway that i do need to fix beside the comments.

Its possible, i want please first to fix the little bug that there is, and the comments,
and if possible please to consider it in a follow PR, because in case the policy wont be applied, we would catch that the ping wont success.

I'm concerned about the ping eventually approach.
Why was that not required until now ?... This has the potential to bloat - by a lot - the execution times on failed builds.

Those tests are were on skip this is the reason..
it takes a bit time for the policy to be applied, it uses Edy's mechanism that already saves time even when ping failed, because it wont wait for the timeout, but even stop on retcode != 0, instead of timeout.

Is it possible that the Eventuallys will not be on the ping but on the appliance of the policy, because the way I see it, those are two different checks. I think that Eventuallys scope should be limited as possible because the tend to cover for bugs and here maybe(not sure) we should limit it to the policy appliance. is there a way to check that the policy was applied without pinging, and to use the Eventually just for the policy appliance check? like try to GET it, or something else, or ping is the right way?

+1; I think you should get the VM "ready" (you an have your eventually there), then do a single ping. That's the pattern we have been using, and expresses better what we want to achieve.

There is one bug anyway that i do need to fix beside the comments.

just rebased upon Quique request

just rebased upon Quique request

just rebased upon Quique request

Did part of the comments, still need to do some, marked the done ones inline their thread.

Done all, will rearrange commits once we i get your blessing.
Thanks

/test pull-kubevirt-check-tests-for-flakes

/test pull-kubevirt-check-tests-for-flakes
Passed when there is both eventually on the ping and on the GET of the policy,
retesting to see if it passes couple times

/test pull-kubevirt-check-tests-for-flakes
(passed twice, want to see more times that its stable)

All the tests that check ping fail are failing on Kind IPv6 (1.18 locally passes),
Can we add a skip for kind ?
I see this on the docs of Ingress field
	// List of ingress rules to be applied to the selected pods. Traffic is allowed to
	// a pod if there are no NetworkPolicies selecting the pod
	// (and cluster policy otherwise allows the traffic), OR if the traffic source is
	// the pod's local node, OR if the traffic matches at least one ingress rule
	// across all of the NetworkPolicy objects whose podSelector matches the pod. If
	// this field is empty then this NetworkPolicy does not allow any traffic (and serves
	// solely to ensure that the pods it selects are isolated by default)
	// +optional

Maybe the "OR if the traffic source is the pod's local node" is the reason.
Will try kind locally to see if this is the same behavior all the time.

The pattern of adding review changes as individual commits at the tail is not something I can work with when reviewing. I am passing one commit at a time in order to limit my focus to a specific change and not to everything as a whole, I cannot practice that with this approach.
If this will change in this PR, let me know and I'll have a look.

The pattern of adding review changes as individual commits at the tail is not something I can work with when reviewing. I am passing one commit at a time in order to limit my focus to a specific change and not to everything as a whole, I cannot practice that with this approach.
If this will change in this PR, let me know and I'll have a look.

From what i understood from our chat now and back then, middle rearrangements wasn't the preferred method by couple Authors, and depends on the reviewer.
I will just rearrange it now as it should be in the end, since it would be easier for me, and needs to be done anyhow (thought just to do it in the end, but upon your request will do it before).

Rearranged the commits, PTAL
Removed the GET, as we spoke in the meeting, it doesn't fail, and just added delay to the Ping
so extended the ping eventually timeout.
Added skip on kind.
Will run several times flake finder on the PR to see its stable as did before.

/test pull-kubevirt-check-tests-for-flakes
passed 1/1 times so far

/test pull-kubevirt-check-tests-for-flakes
passed 2/2

/retest

Fixed according Quique comments, (waitForNetworkPolicyDeletion func).
Rebase.
PTAL

/lgtm

/retest

Did all comments of Ram and Edy,
removed the last commit (dual stack + refactor for it) and the WaitUntilVMIReady commit, will create a new PR for those.
Testing locally, will squash the new commit once there is a blessing.
PTAL

Did all comments of Ram and Edy,
removed the last commit (dual stack + refactor for it) and the WaitUntilVMIReady commit, will create a new PR for those.
Testing locally, will squash the new commit once there is a blessing.
But
I have a mistake, i need to add here the pingEventually stuff, dont review yet please

Added last commit that fix flakes, PTAL
one commit need to be squashed to the one below it (there i answered the latest requests)

Hopefully answered all Edy latest comments.
Arranged commits.
Rebased.
PTAL.

/lgtm

Updated according Quique and Ram comments.

/lgtm

/lgtm

/lgtm

/lgtm

Is there a plan to make this test dual stack?

Is there a plan to make this test dual stack?

I guess you mean this one: https://github.com/kubevirt/kubevirt/pull/4099

failures are not due to this PR
/retest

/approve

/retest

/retest

/retest


Update Dockerfile

Update python image tag
Current python:3.9 and python:3.9-slim docker images uses Debian bullseye , unfortunately this Debian version is missing some packages like libssl3, libtiff6 and libwebp7 so the original Dockerfile will not build.
I updated the tags with *-bookworm that is a more recent version with the required packages.

Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations

Docs

[ ] Link to documentation:

Pull Request Checklist

[ ] Privileged queries and mutations are either absent or guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migrations are either absent or optimized for zero downtime
[ ] The changes are covered by test cases
[ ] All new fields/inputs/mutations have proper labels added (ADDED_IN_X, PREVIEW_FEATURE, etc.)
[ ] All migrations have proper dependencies
[ ] All indexes are added concurrently in migrations
[ ] All RunSql and RunPython migrations have revert option defined

# docker build .
[+] Building 4.0s (11/24)                                                                                docker:default
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 87B                                                                                0.0s
 => [internal] load build definition from Dockerfile                                                                                0.0s
 => => transferring dockerfile: 2.23kB                                                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.9-slim                                                                                0.0s
 => [internal] load metadata for docker.io/library/python:3.9                                                                                0.0s
 => CACHED [build-python 1/7] FROM docker.io/library/python:3.9                                                                                0.0s
 => [internal] load build context                                                                                0.2s
 => => transferring context: 296.83kB                                                                                0.2s
 => CACHED [internal] settings cache mount permissions                                                                                0.0s
 => CANCELED [build-python 2/7] RUN apt-get -y update   && apt-get install -y gettext   && apt-get clean   && rm -rf /var/lib/apt/lists/*                                                                                4.0s
 => [stage-1  1/11] FROM docker.io/library/python:3.9-slim                                                                                0.0s
 => CACHED [stage-1  2/11] RUN groupadd -r saleor && useradd -r -g saleor saleor                                                                                0.0s
 => ERROR [stage-1  3/11] RUN apt-get update   && apt-get install -y   libcairo2   libgdk-pixbuf2.0-0   liblcms2-2   libopenjp2-7   libpango-1.0-0   libpangocairo-1.0-0   libssl3   libtiff6   libwebp7   libxml2   libpq5   shared-mime-info   mime-support   && apt-get clean   && rm   3.9s
------
 > [stage-1  3/11] RUN apt-get update   && apt-get install -y   libcairo2   libgdk-pixbuf2.0-0   liblcms2-2   libopenjp2-7   libpango-1.0-0   libpangocairo-1.0-0   libssl3   libtiff6   libwebp7   libxml2   libpq5   shared-mime-info   mime-support   && apt-get clean   && rm -rf /var/lib/apt/lists/*:
0.484 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
0.506 Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]
0.507 Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
0.744 Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8062 kB]
0.895 Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64 Packages [264 kB]
1.043 Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [18.8 kB]
2.175 Fetched 8554 kB in 2s (5015 kB/s)
2.175 Reading package lists...
2.892 Reading package lists...
3.545 Building dependency tree...
3.703 Reading state information...
3.831 E: Unable to locate package libssl3
3.831 E: Unable to locate package libtiff6
3.831 E: Unable to locate package libwebp7
------
Dockerfile:22
--------------------
  21 |
  22 | >>> RUN apt-get update \
  23 | >>>   && apt-get install -y \
  24 | >>>   libcairo2 \
  25 | >>>   libgdk-pixbuf2.0-0 \
  26 | >>>   liblcms2-2 \
  27 | >>>   libopenjp2-7 \
  28 | >>>   libpango-1.0-0 \
  29 | >>>   libpangocairo-1.0-0 \
  30 | >>>   libssl3 \
  31 | >>>   libtiff6 \
  32 | >>>   libwebp7 \
  33 | >>>   libxml2 \
  34 | >>>   libpq5 \
  35 | >>>   shared-mime-info \
  36 | >>>   mime-support \
  37 | >>>   && apt-get clean \
  38 | >>>   && rm -rf /var/lib/apt/lists/*
  39 |
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get update   && apt-get install -y   libcairo2   libgdk-pixbuf2.0-0   liblcms2-2   libopenjp2-7   libpango-1.0-0   libpangocairo-1.0-0   libssl3   libtiff6   libwebp7   libxml2   libpq5   shared-mime-info   mime-support   && apt-get clean   && rm -rf /var/lib/apt/lists/*" did not complete successfully: exit code: 100

I realised later that I had the outdated python:3.9 image, but I think this update might be useful to those who are trying to update an old installation.

We build these images daily, so they definitely build in their current shape.

As @patrys the images we have in the repo are used regularly and we haven't encountered any problems with them. We also regularly switch between versions when porting changes to older releases. I'll go ahead and close this PR. Feel free to reopen when you find a particular problem.
Additionally to that Stack Overflow mentions did not complete successfully: exit code: 100 should be fixed by pruning docker images.


[Issue] Don't include the bridge if we are not in Storyblok (check value _storyblok)

I create this issue to raise an internal solution or to document the use of the bridge differently to avoid projects loading the bridge in production without necessity and to standardize it between technologies (SDKs).
PR of a community member with an approx for Nuxt 3 -> https://github.com/storyblok/storyblok-nuxt/pull/227
Reference -> https://github.com/storyblok/storyblok-js/blob/5aa2bdb4de93cb2e07959e20c0359cc4f875bb74/lib/index.ts#L19

@thiagosaife, any comments or help on this matter?

I actually went ahead and made a suggested way to fix this. The only thing I am unsure about is the usage of URLSearchParams, but it should be fine to use it.

URLSearchParams is not supported by IE11, not sure if this browser is still supported by storyblok client api.
https://caniuse.com/urlsearchparams

@reinoldus given that ie11 isnt supported by anything pretty much I don't think that should be an issue, but as you said I'm not sure what is supported right now by storyblok-js.
Hopefully, Thaigo or someone can clear up this! I can switch to looking at the full string if needed, but I personally thing making an object makes more sense for readabilty.

@thiagosaife any update on this matter?
I would like to understand the difficulty of the issue, to me it seems like it's an easy fix, given that there are PRs already.
Couldn't we enable this fix with a configurable option, if you are concerned about compatibility issues?

Hey @reinoldus we fixed the thing about the console error that was likely polluting your Sentry in #239.
Keep in mind you can dynamically load the bridge at init time per environment, like this:
storyblokInit({
  bridge: process.env.NODE_ENV !== "production"
})

However, we'll keep this issue open as it might be a nice feature request :) thanks!

@alexjoverm
Thank you, I don't think that is the issue with my sentry but I'll check it out.
But I have to disagree: It's not a "nice to have thing", this library is loading unnecessary JavaScript on to clients which won't be used by anybody if they are not in the storyblok editor.
It's not a huge file, but it is still absolutely unnecessary and fixing it seems very easy to do.
Disabling the bridge is not an option for me, the USP of storyblok for me was that I can edit my live website with ease.

@alexjoverm I dont understand your suggestion for a fix being to disable the bridge in production. How is that an acceptable fix when I need to be able to run the editor in a built environment for my editors?
It could be that I misunderstood something!

@alexjoverm
additional question: How do I test the new version, if I am on storyblok-nuxt2, there hasn't been a release as far as I can tell.
And another thing that I don't understand: The storyblok-nuxt2 module installs as dependencies this module and the vue-module, but those don't seem to be used, because everything seems to be bundled into one dist file.
Deleting those modules from my node_modules dir doesn't seem to break anything it looks like those libs are installed for no reason as well.
This build behavior makes it also hard to just fork the lib and just use @SebbeJohansson fix in a fork.
If you don't wanna fix it, can you explain how the build process works, so we can at least fork?

@reinoldus there needs to be an update to the vue2 package because the vue2 package is limited to "@storyblok/js": "^1.8.5". The nuxt2 package should get the update right away after vue2 is updated.

@Dawntraoz Any news on this?

@SebbeJohansson @reinoldus we're actually discussing if would make sense to automatically enable/disable the bridge depending on if the frontend is loaded on the visual editor.
The reason we made the bridge conditional is so you can disable it in production. A usual case to use Storyblok is different environments is:

Preview/Stating: here you use the draft token and you want to load the bridge, so you want the visual editor to be interactive to change content.
Production: you use public token and don't want the bridge, as actually with the public token the interactivity is disabled.

@alexjoverm I think that it would make sense. I have a feeling that the most common way to do it is to only run bridge in preview mode despite if it is in a production or dev environment.
How to handle version and the bridge is the most common issue that people have. Mainly that they set version to "published" if it is in production, which means that bridge doesn't function.

@SebbeJohansson I will create a PR today with the new conditional 🎉, and we can adequately add some docs to clarify "published" and Public Token, so people understand that for previewing, they need the "draft" version and the Preview Token, not the production ones.

@Dawntraoz AMAZING! 🥳
Check #236 and close it if nothing should be kept going from that :)


Add org parser support

Org is a file format used by an Emacs major mode called org-mode for document editing and other features.
There exists an org tree-sitter parser: https://github.com/milisims/tree-sitter-org

I am trying to add the parser, but I found there are no suitable atoms and delimiters to be added?
The result with empty atom and delimiter lists:

It seems good! However I would like to ask how do atoms be generated if I left atoms and delimiters empty?

I also would love org support.
What happens if you move stuff around, e.g., list items or whole headings (refiling)? This is the main pain I have with org diffs.


Sample dashboards / grafana

Hi, thanks a tonne for this project, it's quite awesome!
I've used your sample influx script to dump the data to influxdb, and have Grafana plugged in over the top and been able to create a few graphs such as power usage, production, temp monitors, and wondering if you have anything built or what you use to visualize the data.
The few metrics I have are great, but its a bit funky trying to match some of the data to mirror the solaredge cloud, ie total kWh per day, or what sort of voltages I should be looking at.

hi @Bonn93,
Seeing that you are running, I've a question for you xD.
I've an error running the example_influxdb.py, maybe I'm running wrong.
I use the terminal with command: pythoon3 example_influxdb.py [and the other commands with database, user, password, host...]
Is correct? or I need to run with a python program or something? (I use a raspberry pi)
Can you explain me how do you run? (if you want)
Thanks
Nico

@iNaiks so I originally tested with a python/terminal and wrapped into a docker image that a built, so I could set a few things to work for me, I had an earlier version where it tried a new modbus library which didn't work well, and had to tweak the requirements.
But, sharing is caring! Here's my docker run script ( it was in k8s, but had some issues with influx storage and had to move to plain old docker whilst rebuilt things ).
I also use the latest 1.8.X release on InfluxDB, not 2.X as it had a bunch of weird issues.
#!/bin/bash
docker stop modbus
docker rm mobus
docker run -d \
--name=modbus \
--restart=always \
-e INVERTER_IP=<IP_ADDRESS> \
-e INVERTER_PORT=1502 \
-e INFLUX_HOST=<IP_ADDRESS> \
-e INFLUX_PORT=8086 \
-e INFLUX_DB=solaredge \
-e INFLUX_USER=solaredge\
-e INFLUX_PASSWORD=solaredge\
mbern/modbus-solaredge:latest

The dockerfile is more interesting and shows how I run it.
FROM python:3.7.9

RUN mkdir /modbus
WORKDIR /modbus
ADD ./python/solaredge_modbus /modbus
ADD ./requirements.txt /modbus
ADD ./example_influxdb.py /modbus

RUN pip install influxdb
RUN pip install -r requirements.txt
#RUN python setup.py install

ENV PYTHONUNBUFFERED=1

WORKDIR /modbus
CMD ["/bin/bash", "-c", "python3 example_influxdb.py ${INVERTER_IP} ${INVERTER_PORT} --influx_host ${INFLUX_HOST} --influx_port ${INFLUX_PORT}  --influx_db ${INFLUX_DB} --influx_user ${INFLUX_USER} --influx_pass ${INFLUX_PASSWORD}"]

It's not perfect, but does the job. Ideally, I need to raise a PR and would like to use python to read the OS.envs to build the config, and it does need more logging that works differently when run direct via python in your own shell.

Hi, thanks a tonne for this project, it's quite awesome!

Thanks!

I've used your sample influx script to dump the data to influxdb, and have Grafana plugged in over the top and been able to create a few graphs such as power usage, production, temp monitors, and wondering if you have anything built or what you use to visualize the data.

My setup is similar, solaredge_modbus, combined with solaredge_setapp (for module level reporting) and sdm_modbus (which is my kWh meter measuring import+export at the grid), all reporting to influxdb, with grafana on top.
On a PV panel I plot voltage_dc, power_ac, and temperature to get a general picture of the state of things. I then have individual graphs of module voltages and current grouped by orientation (east facing modules, and west facing modules). In dashboard style I show the latest power_ac, ac voltage, ac frequency, dc voltage, production today, this month, and all time (from solaredge_setapp).
On a more general panel which also show temperature readings from around the house, along with central heating stats, I show a large graph of the inverter's power_ac, voltage dc, and the consumption as calculated from the actual import or export power measurement by my kWh meter. This graph shows whether the system is producing anything, when the production and consumption spikes were, and gives a general picture of production and consumption throughout the day. I find this graph alone enough for a quick glance at how things are going. Add a couple dashboard style gauges for current power_ac, voltages and temperature, and you should be fine.

@Bonn93 @nmakel Thanks for the info,
But my first problem is build the python into a docker... (I know networks things, not programming, sorry)
I've a nas (Synology) with docker (for read and calculate the information) and a raspberry with a screen to see Grafana graphics.
What docker image I need to install or build with python?
Thanks
Nico

What docker image I need to install or build with python?

You don't need to run the script in a docker container. What error are you getting when you run the script currently?

@Bonn93 @nmakel Thanks for the info,
But my first problem is build the python into a docker... (I know networks things, not programming, sorry)
I've a nas (Synology) with docker (for read and calculate the information) and a raspberry with a screen to see Grafana graphics.
What docker image I need to install or build with python?
Thanks
Nico

So the dockerfile i linked above shows how you can build it, mines a little different because I use this repo as a submodule, that way I get all the version control and updates from this repo within my own repo.
But, if you read the dockerfile carefully, you'll understand how it works.
FROM python:3.7.9 is using the python docker image as the base, and i use python 3.7.9 specifically.
RUN mkdir /modbus
WORKDIR /modbus

These lines I create a directory in the container image, and set my working directory to this one.
ADD ./python/solaredge_modbus /modbus
ADD ./requirements.txt /modbus
ADD ./example_influxdb.py /modbus

These lines I add basically the entire repo to my workdir, but then I also add the requirements.txt file ( which is consumed by pip ) I then add the exact script into the container being example.influx.py to my workdir.
RUN pip install influxdb
RUN pip install -r requirements.txt

I then run pip ( which is like a package manger for python, included in the docker image you may need to install this locally to play/test ( apt install python3-pip ) for example.
Pip then installs all the libraries/packages defined in requirements.txt
CMD ["/bin/bash", "-c", "python3 example_influxdb.py ${INVERTER_IP} ${INVERTER_PORT} --influx_host ${INFLUX_HOST} --influx_port ${INFLUX_PORT}  --influx_db ${INFLUX_DB} --influx_user ${INFLUX_USER} --influx_pass ${INFLUX_PASSWORD}"]

Finally, I run the script via bash, it executes python, in a normal shell, this would be something like this:
python3 example_influx.py <IP_ADDRESS> 1502 --influx-host=<IP_ADDRESS>

Ultimately, ignore all the docker stuff, get it working with python locally first. Install python 3.7.9, install pip, run the pip install, run the script at let us know how you go with the basics, in a shell, it'll spit out stuff, or get the example.py working first.

Hi, thanks a tonne for this project, it's quite awesome!

Thanks!

I've used your sample influx script to dump the data to influxdb, and have Grafana plugged in over the top and been able to create a few graphs such as power usage, production, temp monitors, and wondering if you have anything built or what you use to visualize the data.

My setup is similar, solaredge_modbus, combined with solaredge_setapp (for module level reporting) and sdm_modbus (which is my kWh meter measuring import+export at the grid), all reporting to influxdb, with grafana on top.
On a PV panel I plot voltage_dc, power_ac, and temperature to get a general picture of the state of things. I then have individual graphs of module voltages and current grouped by orientation (east facing modules, and west facing modules). In dashboard style I show the latest power_ac, ac voltage, ac frequency, dc voltage, production today, this month, and all time (from solaredge_setapp).
On a more general panel which also show temperature readings from around the house, along with central heating stats, I show a large graph of the inverter's power_ac, voltage dc, and the consumption as calculated from the actual import or export power measurement by my kWh meter. This graph shows whether the system is producing anything, when the production and consumption spikes were, and gives a general picture of production and consumption throughout the day. I find this graph alone enough for a quick glance at how things are going. Add a couple dashboard style gauges for current power_ac, voltages and temperature, and you should be fine.
Edit: I should add that I calculate daily usage, import and export, by looking at values reported by my kWh meter only. It is possible to do basic difference calculations on values such as energy_total with queries such as:
SELECT difference(last("energy_total")) *1000 FROM "pv" WHERE $timeFilter GROUP BY time(1d)

I didn't even see your panel level stuff, I wonder if my SE10000H is supported, I did play with setapp stuff to get modbus going...
Do you think you could share the grafana dashboards/template you have? It would be pretty cool if that could be a grafana template, I think many others would like it, I know a few folks with solaredge inverters all trying for the same type of monitoring.

What docker image I need to install or build with python?

You don't need to run the script in a docker container. What error are you getting when you run the script currently?

Ok, but I prefer to run all the things on the nas with docker. And use the raspberry only for watch the graphics in Grafana.
How can I create a docker to run the python scrip. I've the influxDB server and grafana server, installed on other dockers.
Very Thanks
Nico

@iNaiks not trying to be rude here, but everything is in the above, you might need to go and understand the basics on containers and building images before you look at this.

@iNaiks not trying to be rude here, but everything is in the above, you might need to go and understand the basics on containers and building images before you look at this.

Yes, I have deleted everything and I have started again from the beginning and now everything works XD.
The only problem is that the NAS just broke ... But before that, it was perfect. Thanks a lot

The only think is sometimes the scrip stops and I need to open the terminal in docker container, and run again.
I've a docker container in a Synology. I access via web to the UI of docker and I have a terminal (bash) open in a specified directory (/home/solaredge_modbus-master), and then I run the command (python3 example_influxdb.py <IP_ADDRESS> 1502 --influx_host XXXX --influx_port 8086 --influx_db XXXX --influx_user XXXX --influx_pass XXXX --timeout 2)
A few hours later, the command stops (I can't see the error because when I go in the terminal via web on Synology, there isn't any error and is like a terminal console waiting for an input command)
There is anything that I can do for run again the scrips when stops or stop the scrip every hour and restart it again automatically?
Thanks Nico

Thanks @Bonn93 - Appreciate the feedback!
Here is my modified version of Niels' "example_influxdb.py" utility which includes the additional "graph" MEASUREMENT that I've used in Grafana. I wasn't sure whether it was appropriate to upload this directly to the project's Code repository without Niels' blessing, so leaving it here for anyone who wants it :
example_influxdb_with_graph_measurement_TuxFan777.zip
I'll also spend some time cleaning up a json export of the Grafana dashboard for you (minus db credentials, etc.).
Will post that as soon as it's ready.


add details

https://github.com/MicrosoftDocs/windows-powershell-docs/issues/1824

Docs Build status updates of commit c272515:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/international/Get-WinUILanguageOverride.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@VLG17 Please change the target branch of the PR from live to master to continue with the review.

@dariomws @get-itips @tseward
Peer review needed for this PR. Thanks.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

I like @dariomws suggestion for a simpler description and I would maintain the bold format for the cmdlet names that @VLG17 used.

Docs Build status updates of commit d32eecc:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/international/Get-WinUILanguageOverride.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


[java] Use openjdk icon for java

simple icons removed its logo probably because of Copyright things thus it looks like error on our page.
We will use openjdk ( thank you @Evernow for this idea )

Maybe the openjdk icon can be used instead?

Maybe the openjdk icon can be used instead?

thank you, very good idea 💯 🥇

LGTM


Upgrade process for OS X installer 0.16.0

Hi @aronasorman, @cpauya and @djallado .
This fixes #282
Upgrade Checklist :

[x] Remove /Library/LaunchAgents/org.learningequality.kalite.plist.
[x] Remove existing KA-Lite-Monitor.app in /Applications folder
[x] Retain .kalite

@arceduardvincent I tested your branch in my virtual machine. I installed 0.15.0 and download some videos, then logout and quit from the app. When I upgrade to 0.16.0, I observed that:

The /Library/LaunchAgents/org.learningequality.kalite.plist from root was deleted.
The KA-Lite-Monitor.app in /Applications folder was removed.
The .kalite is still in the home directory, and my downloaded videos is still in the content folder of .kalite.

So this is good to go then @djallado?  Coz we intend to use this during pre-installation for the 0.16.x installer.

@cpauya That's good to go. Here's the screenshot of my updated app.

The video I downloaded in 0.15.0 is still in the content folder of .kalite when I upgrade to 0.16.0.

Looks good, merging.


Replace vector with native arrays

Slower than std::vector but needed for future cuda version

closed by c7e3f061928da26cc90cc35d33bdb05f6ed0a420


Allowing other .dll files to be paired with a mod dll

If this is supported, great, I haven't been able to figure it out
if it isn't, then it would be nice to have it implemented

just put other dlls into the bin folder

its trying to load that DLL as a mod, and refuses to continue loading my mod


storage: Allow segment to be merged close to base size

Signed-off-by: Wish<EMAIL_ADDRESS>What problem does this PR solve?
Issue Number: ref #6414
Problem Summary:
What is changed and how it works?
In https://github.com/pingcap/tiflash/pull/5863 we will only merge segments when the merged size is <= 1/3 * base size. It improves performance but lead to more fragmented IO and higher memory usage.
This PR change the merge policy so that merged size could be <= 1 * base size.
Check List
Tests

[ ] Unit test
[ ] Integration test
[x] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/run-all-tests

/merge

/merge


[ROOTv6.28] Illegal RooListProxy in CascadeMinimizerGlobalConfigs

When trying to run combine with ROOT master, I ran into a RooFit error.
RooFit now protects you from creating a RooListProxy without owner, because there were some bugs in usercode where people forgot to register the owner and thus ended up with broken RooFit computation graphs.
Fortunately there were no faulty computation graphs in combine, but there are some RooListProxies without owners in CascadeMinimizerGlobalConfigs that trigger this error now:
https://github.com/cms-analysis/HiggsAnalysis-CombinedLimit/blob/102x/interface/CascadeMinimizer.h#L126
All RooListProxy members of CascadeMinimizerGlobalConfigs need to be changed to simply RooArgList:
	  RooArgList pdfCategories;
	  RooArgList nuisanceParameters;
	  RooArgList allFloatingParameters;
	  RooArgList parametersOfInterest;
	  RooArgList allRooMultiPdfParams;
	  RooArgList allRooMultiPdfs;

If you know me which branches I should apply this fix on, I can also do it.

Hi Jonas, 112x is a better target, since it is already compliant with ROOT v6.22.
We will figure out if we want to branch further, but in case we will rebase to that.

Hi Andrea, okay good to know then I'll base my branches on 112x.
I'll not make a PR just for this issue, I think there are more changes required to make combine work with ROOT master.
Right now I'm working with a CMS colleague, Massimiliano Galli, on trying combine with ROOT master for current Higgs fits. When we reached a first milestone I'll make a PR.

Closed by #772.


Come up with forward and backward compatibility policy for node-client-api

@kkanthet Is this done?

This can be closed , the basic rule is to support

Last client release of previous version with
Latest version of the server  with latest client release
EX: 1.0.6 with server 8.0-7
2.0.3 with 9.0-3
2.1.1 with 9.0-4

Upgrade from

Last release of previous version of the client to latest version of the client(same upgrade path for server)
previous version of the client to latest version of the client (same with server)
Ex: 1.0.6 to 2.1.1


[FLINK-3618] [gelly] Rename abstract UDF classes in Scatter-Gather implementation

Rename MessageFunction to ScatterFunction and VertexUpdateFunction to GatherFunction.
Change the parameter order in Graph.runScatterGatherIteration(VertexUpdateFunction, MessagingFunction) to Graph.runScatterGatherIteration(ScatterFunction, GatherFunction)

Thanks a lot @greghogan. Everything looks good. Can you please update the docs too?

Thanks @vasia, docs were an oversight but I think are covered now.

Thanks for the update @greghogan! +1 to merge.


IPv4 RFC791 fragmentation

:warning: This needs to be plugged in the proper place and tested before merging. :warning:
I believe the code is correct, but it has not been properly tested and this is my first patch which does some packet wrangling, so I am posting it early to gather feedback.

Updated with the following changes:

Fixed extracttion of the  IP Header Length field (it was incorrect).
Named constants for bitmasks of flags/fields, instead of magic numbers.
Added missing division by 8 for the fragment offset of the last packet.
Rebased against current ipv6_encap upstream branch.

I had to post a new PR to be able to base it in the new lwaftr_nutmeg branch (unfortunately, GitHub does not allow changing the base branch for a PR :disappointed:). Let's close this one and continue the review there.


i want to snipe givaway

ok it worked now see I don't want to snipe nitro I just want it to join the giveaway and the bot they use is different and the emoji so I edited the main.py and added the bots id and also put the emoji and it doesn't join the giveaway
edit the id bots id is<PHONE_NUMBER>94587432 and the emoji that they use is 👌 /774274450513985557

this is what i edited

if 'GIVEAWAY' in message.content:
if giveaway_sniper:
if message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432:
try:
if not edelay:
await message.add_reaction("👌")
except discord.errors.Forbidden:
print(""
f"\n{Fore.RED}{time} - Couldnt React to Giveaway" + Fore.RESET)
GiveawayInfo()
if edelay:
print(""
f"\n{Fore.GREEN}{time} - Giveaway Found!" + Fore.RESET)
GiveawayDelayInfo()
else:
print(""
f"\n{Fore.GREEN}{time} - Giveaway Sniped" + Fore.RESET)
GiveawayInfo()
if notification:
if edelay:
toaster.show_toast("Sniper",
f"Sniping Giveaway in {delay}s. Look into console",
icon_path="./drop.ico",
duration=7)
else:
toaster.show_toast("Sniper",
"Giveaway Sniped! Look into console",
icon_path="./drop.ico",
duration=7)
try:
if edelay:
sleep(delay)
await message.add_reaction("👌")
print("")
print(f"{Fore.GREEN}Giveaway Sniped with delay {delay} seconds!")
except discord.errors.Forbidden:
print(""
f"\n{Fore.RED}{time} - Couldnt React to Giveaway" + Fore.RESET)
GiveawayInfo()
else:
return
if f'Congratulations<EMAIL_ADDRESS>in message.content or<EMAIL_ADDRESS>won' in message.content:
if giveaway_sniper:
if message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432 or message.author.id ==<PHONE_NUMBER>94587432:
print(""
f"\n{Fore.GREEN}{time} - Giveaway Won" + Fore.RESET)
GiveawayInfo()
if notification:
toaster.show_toast("Sniper",
"Giveaway Won! Look into console",
icon_path="./drop.ico",
duration=7)


Create F2_14.c

The application you will be writing for is intended to run on a small, inexpensive embedded device that does not have a floating point unit. This means that the processor can only do integer arithmetic. However, the device does have a display and needs to display the result of floating point arithmetic to the user.  Because this is a small inexpensive processor there is no support for strings or any mathematical library functions. You must write all the code yourselves performing all the floating point math with integers only and no strings! Character arrays, or C strings, can't tell you their size but they do end with a '\0' character.
The requirements for the code are to write one of the functions presented below. All the functions return a boolean value indicating the success or failure of the operation. Improper data is the most likely cause for a function to fail. You may add your own additional helper functions (I had at least a half dozen) but you must include them for the review. Do not change the interface to these functions.
These two functions take as parameters two sets of mantissa and characteristic and a char array to hold the result of the arithmetic operations. The result of the add or subtract should be converted into char's and placed in the result array. The array must end with a '\0'. The 'len' parameter tells how many characters can be placed on the result array. For these functions to return true you must at a minimum store the characteristic of the result. If the result is a non-integer place as many of the digits as will fit in the result after a decimal point.
bool add(int c1, int n1, int d1, int c2, int n2, int d2, char result[], int len);
bool subtract(int c1, int n1, int d1, int c2, int n2, int d2, char result[], int len);
...
...
char answer[100];
int c1, n1, d1;
int c2, n2, d2;

c1 = 1;
n1 = 1;
d1 = 2;

c2 = 2;
n2 = 2;
d2 = 3;

//if the C string could hold at least the characteristic if(add(c1, n1, d1, c2, n2, d2, answer, 100))
{
    //display string with answer 4.166666666...
}
else
{
    //display error message
}
'''

I am working on this pull request.

I am working on this pull request num F2_14.

POTENTIAL IMPROVEMENTS:

I can see we don't have comments for any line in the code, so can we add comments for the code for better understanding, readability, and explaining the purpose of that code blocks?.
I can see the len variable represents the maximum length of the result array, but it is not currently used in the code.This can lead to potential buffer overflow issues if the result exceeds the allocated space in the array. To Improve the code , we need to add block of code to ensure that the result fits within the provided length.


Add dein#post_sync() for third-party upgrader

I'm creating an ddu-ui to update dein plugins.
It would be useful to create an function that third-party dein update ui can call after update.

OK.


add required dependencies on a RP3

added missing dependencies before running the setup:
sudo apt-get install python3-setuptools python3-dev libjpeg-dev

Thank you for PR-ing this!


[2.0.x] Update reload4j to 1.2.22

Please update reload4j to 1.2.22. From https://reload4j.qos.ch/news.html
2022-07-21 - Release of reload4j 1.2.22
• Fixed a newly discovered XXE vector vulnerability reported against Chainsaw. This issue was reported by PJ Fanning.

Thanks for report!

Thanks for the fast response @grgrzybek !


Error after Updating

Error with payments after Updating to latest Version (4.4.1)
Also Reset of module does not work.
After Reset Payment Methods can not be activated (Error while deleting old payment methods).
Error is caused by missing Tables. (See Screenshot)
After inserting Tables from "src/Install/DatabaseTableInstaller.php" manually via phpmyadmin module starts working again.

Environment

PrestaShop version: <IP_ADDRESS>
Mollie module version: 4.4.1
PHP version: 7.3.31

Hey @waschier-design ,
Thanks for your message!
Maybe you could clarify from with version did you made the update to 4.4.1 please?
Looking forward to your reply!
--
Best regards,
Invertus Support team

We were upgrading 1 shop from Version 4.3.0
Other PrestaShop (same PS Version) was on Mollie Version 3.5.2
Both were upgraded versions from Mollie Module Version 2.0.x
and had problems with upgrading before.

Hey @waschier-design ,
Since the issue isn't active for some time, I will mark it as resolved.
Let me know if you still encounter any issues, we can re-open the ticket then!
--
Best regards,
Invertus Support team


Add automated benchmarks to track tool performance

In order to be able to optimize things (e.g. #105 ), we need metrics on how well our tool performs. Otherwise optimizations are "hit or miss".
One option would be to directly integrate benchmarks into CI with github-action-benchmark.
This requires the benchmark to be set up with pytest-benchmark, the process is described here.

Sounds reasonable to me. I'd be happy to learn how to do it, but if you
don't want to wait for me to figure out how to do it first, fair enough :D
On Sat, 30 Jul 2022, 14:07 Balduin Landolt, @.***>
wrote:

In order to be able to optimize things (e.g. #105
https://github.com/arbeitsgruppe-digitale-altnordistik/Sammlung-Toole/issues/105
), we need metrics on how well our tool performs. Otherwise optimizations
are "hit or miss".
One option would be to directly integrate benchmarks into CI with
github-action-benchmark
https://github.com/benchmark-action/github-action-benchmark.
This requires the benchmark to be set up with pytest-benchmark
https://pytest-benchmark.readthedocs.io/en/stable/, the process is
described here
https://github.com/benchmark-action/github-action-benchmark/blob/master/examples/pytest/README.md
.
—
Reply to this email directly, view it on GitHub
https://github.com/arbeitsgruppe-digitale-altnordistik/Sammlung-Toole/issues/106,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANZXABCUU2GIGTFLYSJMAHDVWULHJANCNFSM55DD7VKQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
com>

This issue has no rush IMO, and I will have to learn just as much... but I'd be happy to tackle it together. (But I think we should try to focus on getting an acceptable 1.0 out there before fall, so maybe we shouldn't let our work be guided by what's most interesting... even though it's tempting ^^)

Another option is a tool I've been working on, Bencher: https://github.com/bencherdev/bencher
It has all of the benefits of github-action-benchmark plus quite a bit more.
I would be more than happy to help with any integration!

Another option is a tool I've been working on, Bencher: https://github.com/bencherdev/bencher It has all of the benefits of github-action-benchmark plus quite a bit more. I would be more than happy to help with any integration!

Thank you for the hint! Looks promising indeed.
We are currently not progressing much (all have other things on our plates), so I'm not sure how soon we tackle this issue. But once we do, we'll definitely look into bencher - and possibly ask you for help with the integration ;-)


possible feature to play more than 1 video

is it possible to make it capable of running more that 1 video, like putting more videos in secuence like bumper.sdf bumper1.sfd bumper2.sfd.....etc.
or adding a menu to select the videos?

I can add this. FFmpeg can merge multiple videos into one single video and then that would be converted and burned to a CD-R.


Add limits for Alterations

Add limits for Alterations. This is same as Custom Question Answering.
https://github.com/MicrosoftDocs/azure-docs/pull/90241

@nkgami : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@mrbullwinkle – Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
Before merge, the link will need to be changed to omit the ja-JP locale.
#label:"aq-pr-triaged"

@ShannonLeavitt : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Discord link is broken

I would mention it in the Discord but I can't even join
Discord link on this page is broken: https://python.useinstructor.com/

yes, i was also wondering  what the proper discord link is

The problem with the Discord link was addressed in #690. This issue can be closed.

Closing this issue since the discord link has been fixed, thanks @jlondonobo! :)

Closing this issue since the discord link has been fixed, thanks @jlondonobo! :)


Suggestion: Use the containment operator to speed up metadata equality

Hi @olirice, thanks for creating this library!
Using the containment operator (@>) for equality should speed up metadata filtering, as it'll allow the GIN index to used. From the docs: "The default GIN operator class for jsonb supports queries with the key-exists operators ?, ?| and ?&, the containment operator @>, and the jsonpath match operators @? and @@."
You can also use jsonb_path_ops for a smaller index size. "The non-default GIN operator class jsonb_path_ops does not support the key-exists operators, but it does support @>, @? and @@."
import json
import psycopg
import random

conn = psycopg.connect(dbname='vecs', autocommit=True)
conn.execute('CREATE TABLE docs (id bigserial PRIMARY KEY, metadata jsonb)')

for _ in range(100):
    metadata = []
    for _ in range(10000):
        md = {
            'a': random.randint(1, 10000),
            'b': random.choice(['value' + str(i) for i in range(1, 100)])
        }
        metadata.append(json.dumps(md))

    sql = 'INSERT INTO docs (metadata) VALUES ' + ','.join(['(%s)' for _ in metadata])
    conn.execute(sql, metadata)

conn.execute('CREATE INDEX ON docs USING gin (metadata jsonb_path_ops)')

# no index
res = conn.execute("EXPLAIN ANALYZE SELECT * FROM docs WHERE metadata -> 'a' = '1'").fetchall()
print('\n'.join([r[0] for r in res]))

# index
res = conn.execute("""EXPLAIN ANALYZE SELECT * FROM docs WHERE metadata @> '{"a": 1}'""").fetchall()
print('\n'.join([r[0] for r in res]))

# no index
res = conn.execute("""EXPLAIN ANALYZE SELECT * FROM docs WHERE metadata -> 'b' = '"value1"'""").fetchall()
print('\n'.join([r[0] for r in res]))

# index
res = conn.execute("""EXPLAIN ANALYZE SELECT * FROM docs WHERE metadata @> '{"b": "value1"}'""").fetchall()
print('\n'.join([r[0] for r in res]))

Edit: More from the docs on jsonb_path_ops

Although the jsonb_path_ops operator class supports only queries with the @>, @? and @@ operators, it has notable performance advantages over the default operator class jsonb_ops. A jsonb_path_ops index is usually much smaller than a jsonb_ops index over the same data, and the specificity of searches is better, particularly when queries contain keys that appear frequently in the data. Therefore search operations typically perform better than with the default operator class.

Using the containment operator (@>) for equality should speed up metadata filtering

thanks!

use jsonb_path_ops for a smaller index size. "The non-default GIN operator class jsonb_path_ops does not support the key-exists operators, but it does support @>, @? and @@."

we don't support key exists filters so that sounds great
--
how does #12 look, does that address both the way you expected?

Yeah, looks great!

Just realized it probably shouldn't use containment if the query is a list or dict, since both the following are true but not equal:
SELECT '{"a": [1, 2, 3]}'::jsonb @> '{"a": [1]}';
SELECT '{"a": {"b": 1, "c": 2}}'::jsonb @> '{"a": {"b": 1}}';

makes sense, thanks
I added tests for the various edge cases to make sure its all working properly
The trouble with re-checking for equality in python after the results are returned is that it makes it difficult to get the exact number of results the user requested without potentially fetching more matches from the database. To avoid that, the containment optimization is limited to when the user provides a single value e.g. 6 or "abc".


No notification when exceeding Anthropic rate limit

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain.chat_models import ChatAnthropic
from langchain.schema import HumanMessage
chat = ChatAnthropic()
Simulate exceeding rate limit by making multiple rapid requests
for _ in range(100):
messages = [HumanMessage(content="Hello, how are you?")]
response = chat(messages)
print(response.content)
No clear error or notification is raised when rate limit is exceeded
Error Message and Stack Trace (if applicable)
No exeption display
Description
Description
Problem: When the API rate limit of Anthropic is exceeded while using LangChain, no notification or clear error is received, and the application just stays connected, causing the entire API to crash.
Expected Behavior: LangChain should throw a specific exception (e.g., AnthropicRateLimitExceeded) or provide a clear error message when the Anthropic rate limit is reached.
Current Behavior: The application continues running without any indication of the rate limit issue, making it difficult to implement proper error handling and manage API usage effectively. When switching to OpenAI, the API functioned perfectly, and this issue never occurred. The problem seems to be specific to Anthropic. Upon investigation, I found that the issue occurred after exceeding 40k tokens within a 1-minute period. However, there is no alert or error to handle this, the connection to the endpoint remains active, and the program continues running but stops responding. This is a critical issue in my opinion.

You can avoid the issue partially by specifying a rate limiter: https://python.langchain.com/v0.2/docs/how_to/chat_model_rate_limiting/

After that, the next step would be sending a string to Anthropic, which acts as the master model or controller for my other agents. This is where the issue occurs, but no error is displayed in the terminal. The program doesn’t crash; it just gets stuck, which means I can no longer send any requests to my endpoint. The only solution is to restart the server to get it working again.
I decided to reproduce the error on my local server to rule out any misconfiguration in NGINX. Upon testing locally, I found that the issue could be replicated there as well, so I ruled out NGINX as the cause. (Although I did increase the timeout response on my server, thinking it might be related to the response time.)
I added several debug statements to track down the error and noticed that all the information is processed correctly. However, in the final step, when the system should deliver the final response, the server freezes, waiting indefinitely. At this point, nothing can recover the situation; the server must be restarted.
Considering possible causes, I suspected it might be due to Anthropic’s service tier being lower than OpenAI's concerning the workload I’m handling. After trying to force the error again, I was able to reproduce it and concluded:
After spending a considerable amount of time investigating, I am almost certain that this is a rate limit issue. However, it occurs silently and leaves the server waiting indefinitely.

Update:
After further investigation, I’ve discovered that the issue isn’t limited to Anthropic—it also occurs with OpenAI. However, the problem is more easily reproducible with Anthropic. It typically takes no more than 2 or 3 attempts for the error to occur. The specific line where the issue arises is:
response = await llm_with_tools.ainvoke({"messages": final_result})

Here’s the relevant part of my code where I create the LLM chain:
if user_agent.is_anthropic():
    llm = ChatAnthropic(model=model, temperature=temperature)  # noqa
else:
    llm = ChatOpenAI(model=model, temperature=temperature)  # noqa
primary_assistant_prompt = ChatPromptTemplate.from_messages(
    [
        SystemMessage(content=f"<today>\nToday is: {get_mex_time()}\n</today>\n" + user_agent.assistant_prompt),
        MessagesPlaceholder(variable_name="messages")
    ]
)
llm_with_tools = primary_assistant_prompt | llm.bind_tools(tools)

The models I’m using are Anthropic Claude v3.5 and GPT-4, with the temperature set to 0.
To aid in debugging, I’ve temporarily modified my code to include additional prints:
async def _chatbot(state: State):
    messages = state['messages']
    final_result = messages if user_agent.is_anthropic() else self._custom_filter(messages)
    try:
        ic(final_result)
        response = await llm_with_tools.ainvoke({"messages": final_result})
        return ic({"messages": [response]})
    except Exception as e:
        ic(f'FATAL ERROR: {e}')
        raise Exception('BIG ERROR HERE with the quoter')

This issue persists and causes the server to hang indefinitely, requiring a restart. It seems likely that this is related to rate limits, but the error occurs silently and without any clear indication in the logs.
 Por favor, proporcione '
                      'el producto exacto que está buscando para que pueda ayudarle mejor.')
ic| final_result: [HumanMessage(content='hola'),
                   AIMessage(content='Hola! Bienvenido a Power Depot. Soy AquaExpert,

Was this solved? Unfortunately I get the same issue.


Update README.md

The google color picker is falsely labelled as a HSL picker. Its a HSV.
If it was HSL it would be all white on top:

Thanks!


Pin config change implies another pin config change

We need to decide how to handle this case...
See https://github.com/andrewdavidmackenzie/pigg/issues/45
When I2C1 function is selected on either of these two pins:
SDA: BCM GPIO 2 (physical pin 3)
SCL: BCM GPIO 3 (physical pin 5)
Then the selection for the other pin should be changed too.
Either that, or we leave them separate and have an incorrect config that should be flagged in the UI, or used alerted to it when trying to apply to HW or save...
I think probably changing both at once would be best?
But open to other ideas how to handle it.

Got any ideas on how to handle this @sundaram123krishnan ?

ig incorrect config flag would be better?


Colemak Mod-DH issues on ansi pixel c keyboard.

This is very cool!  I'm trying to use Colemak Mod-DH layout on a Pixel C keyboard which appears to be ansi style, but have some issues. M and K are reversed, and Z is missing. Z should be where the 'B' key is on an ansi keyboard.
Here's a photo showing characters typed for each keyboard row and _ where the B key is and Z was missing.
Also, any idea if there is a way to type the missing ~, |, [, ], {, } keys on this funky tablet keyboard?

Indeed, I did not include the ANSI layouts.
Here is a project where they are included: https://github.com/ColemakMods/mod-dh/tree/master/android
But you can't type any bracket as far as I can tell unless you add them to the altgr layer and recompile yourself.

Thanks!  I stumbled across that repo shortly after trying this one and have a working layout now.  I'll have to look into the altgr layer stuff as you recommend.   I'll mark this issue closed.


Submit translations on Ox-Dev

Background
In https://github.com/phetsims/rosetta/issues/262, we discussed an issue where the translations of certain translators weren't being correctly submitted. We're not sure why it's happening for some translators and not others. We think it might be platform-specific.
Before the Test
I need to:

[x] make sure phet-server-dev (Ox-Dev) is running the same version of Rosetta as phet-server,

d37d0da36df42a81cb5dc2020425c12342261af4 (phet-server-dev)
d37d0da36df42a81cb5dc2020425c12342261af4 (phet-server)

[x] make sure phet-server-dev/Rosetta's Babel branch is the tests branch, and
[x] let QA know when this test is ready.

Instructions for QA
For those of you with Windows 10 (preferably version 2004) and Chrome (latest), please do the following:

Go to https://ox-dev.colorado.edu/translate/.
Log in with one of the QA translator accounts. (If you have access to level 3 credentials you should be able to do this. If you have questions, ping me or @KatieWoe.)
Change as many strings as you want (they can be anything, none of this will be public) and submit a translation for the following:

Build a Molecule

[x] locale = ca (Catalan) Day/time = 7/8/2021 2:53 pm
[x] locale = ko (Korean) Day/time = 7/12/2021 10:33 am

Collision Lab

[x] locale = ca (Catalan)
[x] locale = ko (Korean)
[x] day/time = 7/12/2021 9:48 am

Energy Skate Park

[x] locale = ca (Catalan)
[x] locale = ko (Korean)
[x] day/time = 7/12/2021 10:04 am

Number Line: Integers

[x] locale = ca (Catalan)
[x] locale = ko (Korean)
[x] day/time = 7/12/2021 10:17 am

Number Line: Operations

[x] locale = ca (Catalan)
[x] locale = ko (Korean)
[x] day/time = 7/12/2021 10:26 am

@liam-mulhall I believe this is done


Reformat JSON

Currently the JSON format is not uniform. Each key should be conformed to either camel case or underscores, not both. A sample JSON on server mode:
{
  "self": "http://localhost:8888/cd?url=http://www.cnn.com/2017/07/04/politics/us-officials-meet-north-korea-missile-launch/index.html",
  "URI": "http://www.cnn.com/2017/07/04/politics/us-officials-meet-north-korea-missile-launch/index.html",
  "Estimated Creation Date": "2017-07-04T15:10:24",
  "Archives": {
    "Earliest": "2017-07-04T15:28:32",
    "By_Archive": [
      {
        "URI": "http://web.archive.org/web/20170704152832/http://www.cnn.com/2017/07/04/politics/us-officials-meet-north-korea-missile-launch/index.html",
        "memento_datetime": "2017-07-04T15:28:32",
        "pubdate": "2017-07-04T15:10:24"
      },
      {
        "URI": "//wayback.archive-it.org/all/20170704185254/http://www.cnn.com/2017/07/04/politics/us-officials-meet-north-korea-missile-launch/index.html",
        "memento_datetime": "2017-07-04T18:52:54",
        "pubdate": "2017-07-04T23:59:59"
      },
      {
        "URI": "http://archive.is/20170704205543/http://www.cnn.com/2017/07/04/politics/us-officials-meet-north-korea-missile-launch/index.html",
        "memento_datetime": "2017-07-04T20:55:43",
        "pubdate": "2017-07-04T20:55:43"
      }
    ]
  },
  "Backlinks": "2017-07-04T15:28:32",
  "Bing.com": "",
  "Bitly.com": "2017-07-04T15:10:25",
  "Google.com": "2017-07-04T23:59:59",
  "Last Modified": "",
  "Pubdate tag": "2017-07-04T15:10:24",
  "Twitter.com": "2017-07-04T19:28:34"
}

The "Estimated Creation Date" date key can be transformed to "estimated_creation_date" using underscores.
Another idea discussed with @ibnesayeed, is to use grouping for services leaving the first three keys, "self," "uri" and "estimated_creation_date," outside of this group. Earliest archive date could be removed since estimated_creation_date is the overall earliest date that we're concerned with. It could be as follows:
{
  "self": "",
  "uri": "",
  "estimated_creation_date": "",
  "services": {
    "backlinks": "",
    "bing": "",
    "bitly": "",
    "google": "",
    "last_modified": "",
    "pubdate": "",
    "twitter": "",
    "archives": [
        {
          "uri": "",
          "memento_datetime": "",
          "pubdate": ""
        }
      ]
  }
}

I was going to say that there is some utility in keeping the earliest archival date, but I guess it's not necessary.
I wonder if we should add a note saying which service "won"?  In the example above, all the services have variations of "2017-07-04" and it takes a sec to scan and see that pubdate "wins".

If I were to organize it, I would put it something like this:
{
	"self": "http://example.com/cd?url=http://www.cnn.com/",
	"uri": "http://www.cnn.com/",
	"estimated-creation-date": "2017-07-04T15:10:24",
	"earliest-sources": ["pubdate", "web.archive.org"],
	"sources": {
		"backlinks": {
			"earliest": "2017-07-04T15:28:32"
		},
		"last-modified": {
			"earliest": ""
		},
		"pubdate": {
			"earliest": "2017-07-04T15:10:24"
		},
		"bing.com": {
			"earliest": ""
		},
		"bitly.com": {
			"earliest": "2017-07-04T15:10:25"
		},
		"google.com": {
			"earliest": "2017-07-04T23:59:59"
		},
		"twitter.com": {
			"earliest": "2017-07-04T19:28:34"
		},
		"web.archive.org": {
			"uri-m": "http://web.archive.org/web/20170704152832/http://www.cnn.com/",
			"memento-datetime": "2017-07-04T15:28:32",
			"pubdate": "2017-07-04T15:10:24",
			"earliest": "2017-07-04T15:10:24"
		},
		"wayback.archive-it.org": {
			"uri-m": "//wayback.archive-it.org/all/20170704185254/http://www.cnn.com/",
			"memento-datetime": "2017-07-04T18:52:54",
			"pubdate": "2017-07-04T23:59:59",
			"earliest": "2017-07-04T18:52:54"
		},
		"archive.is": {
			"uri-m": "http://archive.is/20170704205543/http://www.cnn.com/",
			"memento-datetime": "2017-07-04T20:55:43",
			"pubdate": "2017-07-04T20:55:43",
			"earliest": "2017-07-04T20:55:43"
		}
	}
}

I have only put the summary and meta information at the top level. Everything else goes into the sources block that is uniformly formatted with each containing a mandatory attribute earliest, even if it is empty. Each source block can have additional attributes, such as title or attributes that are shown in the archival sources like uri-m or memento-datetime (note that they still have the earliest attribute in them). Naming convention here is to keep every attribute name in downcase letters and use hyphen - for joining words. I have also added a top-level earliest-sources attribute in response to @phonedude's idea, but I kept the value in an array because there might be cases where more than one source are pointing to the same creation date. So it is better to keep it consistent and always put them in an array, even if there is only one winner source. The array contains the keys of corresponding sources block.
With such organization and conventions, it is very easy and predictable for clients to consume the service. Also, it streamlines the kind of response you expect from each module while adding more sources in future. Just by looking at uri, estimated-creation-date, and earliest-sources attribute an application can state something like this very easily:
Based on {earliest-sources}, {uri} first came to life at {estimated-creation-date}.


Angular开发异常和解决方法

AngularJS : Prevent error $digest already in progress when calling $scope.$apply()
解决方法：http://stackoverflow.com/questions/12729122/angularjs-prevent-error-digest-already-in-progress-when-calling-scope-apply
The digest cycle is a synchronous call. It won't yield control to the browser's event loop until it is done. There are a few ways to deal with this. The easiest way to deal with this is to use the built in $timeout, and a second way is if you are using underscore or lodash (and you should be), call the following:
$timeout(function(){
    //any code in here will automatically have an apply run afterwards
});

or if you have underscore:
_.defer(function(){$scope.$apply();});

We tried several workarounds, and we hated injecting $rootScope into all of our controllers, directives, and even some factories. So, the $timeout and _.defer have been our favorite so far. These methods successfully tell angular to wait until the next animation loop, which will guarantee that the current scope.$apply is over.

Error: [ngModel:nonassign] Expression is non-assignable
使用ng-value替换ng-model
http://stackoverflow.com/questions/24177707/error-ngmodelnonassign-expression-is-non-assignable
WARNING: Tried to load angular more than once.