Daemon crashed after restart with -run-snark-worker

ran the daemon on the testnet with coda daemon -peer $SEED1 -peer $SEED2
stopped it after some hours with coda client stop-daemon
restarted with coda daemon -peer $SEED1 -peer $SEED2 -run-snark-worker $CODA_PUBLIC_KEY -snark-worker-fee 0.05 &> ./daemon.log
observed this in the log

        dir: "/home/etsvigun/.coda-config/coda_genesis_3e9b1748_5e70c20db27df299/ledger"
        error: "(Failure\n \"IO error: While lock file: /home/etsvigun/.coda-config/coda_genesis_3e9b1748_5e70c20db27df299/ledger/LOCK: Resource temporarily
unavailable\")"
...
include the last 50 lines from .coda-config/coda.log

log_23_51_59.gz

Commit 3e9b174 on branch feature/bump-genesis-timestamp
etsvigun@coda-testnet:~$ uname -a
Linux coda-testnet 4.9.0-12-amd64 #1 SMP Debian 4.9.210-1+deb9u1 (2020-06-07) x86_64 GNU/Linux

Another one:
{"timestamp":"2020-06-12 00:37:17.210123Z","level":"Info","source":{"module":"Snark_worker__Functor","location":"File \"src/lib/snark_worker/functor.ml\", line 139, characters 59-66"},"message":"SNARK work received from $address. Starting proof generation","metadata":{"address":"<IP_ADDRESS>:8301","pid":14653,"process":"Snark Worker"}}
2020-06-12 00:37:18 UTC [Info] Running async shutdown handler: "Close transition frontier, if exists"
(monitor.ml.Error
 (Sys_error
  "/home/etsvigun/.coda-config/coda_crash_report_2020-06-12_00-36-19.352230/coda_status.json: No such file or directory")
 ("Raised by primitive operation at file \"stdlib.ml\", line 316, characters 29-55"
  "Called from file \"stdlib.ml\" (inlined), line 321, characters 2-74"
  "Called from file \"write.ml\", line 470, characters 11-24"
  "Called from file \"src/app/cli/src/init/coda_run.ml\", line 265, characters 2-40"
  "Called from file \"src/deferred1.ml\", line 17, characters 40-45"
  "Called from file \"src/job_queue.ml\" (inlined), line 131, characters 2-5"
  "Called from file \"src/job_queue.ml\", line 171, characters 6-47"))

Moved everything from ~/.coda-config except wallets, tried again - a new crash.
coda_crash_report_2020-06-12_00-51-06.365405.tar.gz

Another one:
2020-06-12 01:02:15 UTC [Info] Coda daemon is now doing ledger catchup
2020-06-12 01:03:02 UTC [Fatal] after lock transition, the best tip consensus state is out of sync with the local state -- bug in either required_local_state_sync or frontier_root_transition.
2020-06-12 01:03:02 UTC [Fatal] Unhandled top-level exception: $exn
Generating crash report
        exn: "(monitor.ml.Error\n  (Failure \"local state desynced after applying diffs to full frontier\")\n  (\"Raised at file \\\"stdlib.ml\\\", line 33, characters 17-33\"\n    \"Called from file \\\"src/lib/debug_assert/debug_assert.ml\\\" (inlined), line 1, characters 25-33\"\n    \"Called from file \\\"src/lib/transition_frontier/full_frontier/full_frontier.ml\\\", line 606, characters 4-1023\"\n    \"Called from file \\\"src/lib/transition_frontier/transition_frontier.ml\\\", line 292, characters 4-93\"\n    \"Called from file \\\"src/lib/transition_handler/processor.ml\\\", line 57, characters 10-68\"\n    \"Called from file \\\"src/lib/rose_tree/rose_tree.ml\\\", line 79, characters 18-24\"\n    \"Called from file \\\"src/deferred_or_error.ml\\\", line 101, characters 23-33\"\n    \"Called from file \\\"src/lib/transition_handler/processor.ml\\\", line 285, characters 21-570\"\n    \"Called from file \\\"src/result.ml\\\", line 151, characters 9-15\"\n    \"Caught by monitor coda\"))"
2020-06-12 01:03:02 UTC [Fatal] Unhandled top-level exception: $exn
Generating crash report
        exn: "(monitor.ml.Error\n  (Failure\n    \"traced task `R&transition_handler_processor` failed, exception reported to parent monitor\")\n  (\"Raised at file \\\"stdlib.ml\\\", line 33, characters 17-33\"\n    \"Called from file \\\"src/lib/pipe_lib/strict_pipe.ml\\\", line 152, characters 28-37\"\n    \"Called from file \\\"src/deferred0.ml\\\", line 56, characters 64-69\"\n    \"Called from file \\\"src/job_queue.ml\\\" (inlined), line 131, characters 2-5\"\n    \"Called from file \\\"src/job_queue.ml\\\", line 171, characters 6-47\"\n    \"Caught by monitor coda\"))"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:02 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:03 UTC [Error] Unconsumed item in cache: $cache
        cache: "cached item was not consumed (cache name = \"Transition_handler__Unprocessed_transition_cache\")"
2020-06-12 01:03:29 UTC [Info] Running async shutdown handler: "Close transition frontier, if exists"
(monitor.ml.Error
 (Sys_error
  "/home/etsvigun/.coda-config/coda_crash_report_2020-06-12_01-03-02.910500/coda_status.json: No such file or directory")
 ("Raised by primitive operation at file \"stdlib.ml\", line 316, characters 29-55"
  "Called from file \"stdlib.ml\" (inlined), line 321, characters 2-74"
  "Called from file \"write.ml\", line 470, characters 11-24"
  "Called from file \"src/app/cli/src/init/coda_run.ml\", line 265, characters 2-40"
  "Called from file \"src/deferred1.ml\", line 17, characters 40-45"
  "Called from file \"src/job_queue.ml\" (inlined), line 131, characters 2-5"
  "Called from file \"src/job_queue.ml\", line 171, characters 6-47"))

it didn't finish compressing the report this time

Dropped the whole ~/.coda-config, imported the key, started the snarker with coda client set-snark-worker -address $CODA_PUBLIC_KEY
now it works.

4th crash covered by #5059

Closing this issue as it is stale for a long time.


Operator Hub: switch to bundles

Going forward we'll use the bundle operator meta data. For that we also need to update the entry for PMEM-CSI 0.9.

Submitted a PR(https://github.com/operator-framework/community-operators/pull/3922) to community-operators.

That was merged.


gopls root_dir add dirname for support single go file.

for #1278

I'm trying to remove these fallbacks (or at least not proliferate them). Lspconfig only supports projects for now.


Themes from Bootswatch

Hello, can you break down the steps involved in incorporating a theme from bootswatch into our spree app, after we installed this frontend extension? I would much appreciate it.

If your trying to use a custom bootstrap.css. This is what I did.
Placed the custom css into /app/assets/stylesheets/custom-bootstrap.css
Open /vendor/assets/stylesheets/spree/frontend/all.css
Change it to this!
/*

This is a manifest file that includes stylesheets for spree_frontend
Spree_bootstrap_frontend overrides all spree’s default css and resets here in favour of bootstrap
*= require spree/frontend/bootstrap_frontend
*= require custom-bootstrap
*= require_self
*= require_tree .
*/

You'll have to do some edits to the bootstrap_frontend.css.scss as the background image is there still. A start for anyone who looks at this!


Fix gov proposal end-of-voting-period consensus failure

Description
In the begin and end blockers, wrap the gas meter as a fee gas meter if it isn't already a fee gas meter.
In the new gov stuff, when voting ends on a gov proposal, the end block triggers handles the sending of successfully voted-on proposals, and does so through the message service router. Our custom message service router expects the gas meter to be a fee gas meter though. So, in order for the new gov stuff to have the right gas meter, it needs to be wrapped in our end block so that it's wrapped in the gov module's end block which can then call our message service router without blowing up.
closes: #1099
Note: I put together this PR without any integration tests on it because writing such an integration test is going to take quite a bit longer. I am working on it though.
I did test this by make clean build run-config && ./scripts/upgrade-config.sh && make run, then in another window, I ran the script from #1097 with the color "brown". Before this PR, the chain halted at the end of the voting period (with the not-a-fee-gas-meter error). After this PR, it got past the end of the voting period and instead (expectedly) halted at the upgrade height.

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[x] Targeted PR against correct branch (see CONTRIBUTING.md)
[x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[x] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[x] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

I'm going to cancel this PR and do it a different way.
After discussions in our channel, I (we) think it's better to bypass the fee consumption (in the message service router) if the gas meter is not a fee gas meter.


Add a rescue for parser errors

for those not infrequent times recently when SurveyGizmo has an nginx failure and barfs with web content instead of a parseable API response
@jarthod

Thanks, released 6.2.6


Draw attention to clipboard and scaling capabilities of Meadow in tutorial

See feedback regarding Meadow:
The buttons apple-C, apple-V, apple-+, apple-- all work as expected. Excellent! The tutorial should draw attention to this.

added


feat(api): add handler for user info

Tests

Created docker image with docker build . in root folder
Created Docker Service with docker service create userInfo
Tested the API endpoint with curl

In particular I tested:
$ curl -X GET http://localhost:8080/api/users/123

We can get user info.
curl -X POST -H "Content-Type: application/json" -d '{"name":"John<EMAIL_ADDRESS>http://localhost:8080/api/users

The API endpoint properly creates a new user with the provided name "John Doe" and email<EMAIL_ADDRESS>curl -X PUT -H "Content-Type: application/json" -d<EMAIL_ADDRESS>http://localhost:8080/api/users/123

The API endpoint successfully updates the email address of the user with the ID "123" to<EMAIL_ADDRESS>curl -X DELETE http://localhost:8080/api/users/123

The API endpoint correctly deletes the user with the ID "123"
Deployed
Deployed in staging environment dev-60, when this is merged it will be automatically deployed to prod-19.

This was an example PR for Bee's blog post. 😅
https://skerritt.blog/writing-better-commit-messages/


Matchday Scrolling

Enable the routing to a different matchday (1-7) with the .../bet/<matchday> thingy. Each matchday should be loaded correctly.

Partially solved.


The other cars have random speed and angle. Other methods also got re…

othercars.ino has been refined. The following changes occurred:

Now each car will behave differently while facing an obstacle. Their speed and angle will be assigned to them randomly after compiling each sketch.
There is a range for both angles and speeds of the cars. The upper and lower bounds were obtained after hours of trying and error of various speeds and angles and number of the cars to make sure they will not get stuck behind an obstacle or need to reset position. (If they are stuck in somewhere they are able to get out after a short while now)
Some methods are added and some other are refined. Comments are added too to clarify the code.
Other sensors were investigated and tested and now we have only the useful ones!
The cars are ready to be hunted by Tank

You can compile 6 cars and enjoy watching them wandering around for hours :)

Don't close the issue #75 though. I might have something I can add later today


Clear service menu logging

Is this something that could be added? You know when you hold MENU for 5 seconds then choose software/updates. You will basically see a list of all updates made to the Discover Pro unit, including firmware, patches and map upgrades. Can this list be cleared?

Yes, I'm currently working on a privacy menu that has the ability to clear this.

Superb, keep up the good work!

thanks! :-)
Added this with ebea36f32e60489c643c0812468ba4683aaabfeb


browser.services is private despite being part of the public API documentation

Hey,
First of all thank you for your commitment to this library.
I'd like to access the field browser.services in my application, but unfortunately the field is set to private. However, the API documentation indicates that the field should be accessible publicly.
Thus, I wanted to ask if I could contribute a PR to fix this issue? For now, I'll just use a workaround for accessing the field, but it'd be nice to be able to remove this hack in the future.
I am looking forward to hearing from you!

@agentS Hi, yes, open to PR. Maybe worth ensuring the field is get only.

#7 Closed by PR 7


allow for tasks to be ran automatically on more events and conditionally

The use case i have is, we have a branch that migrates an application from angular to angular X which has a different startup process.  i am usually on angular 1 branch but if i want to toggle between i have to modify the tasks file or manually kill and start the other.
on way to get around this is to add a new runOn "onBranch" and then a  regex or glob to match the branch.
it would be awesome to get the original process terminated with a property called "terminateOn": "branchSwitch" or "branchRegexUnmatch"
so in my case i would have two tasks, each with a runOnBranch, a branch regex and "terminateOn": "branchSwitch" or "branchRegexUnmatch"

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Onboarding

User Story
As a new user, I need to configure my AWS Account(s) before using Cost Management
Impacts
UI
Role
User/Admin
Acceptance Criteria

[ ] Onboarding is displayed for users without a configured AWS account
[ ] Onboarding is not displayed for users have a configured AWS account

Issue #48 is the "design user story" that is connected to this implementation issue.

We were not able to use the PF3 wizard from frontigrade due to issues with fonts, styles, margins, etc. -- the wizard also needed a lot of refactoring for the Koku UI. It would have taken some effort to refactor and make PF3 look decent.
As a solution, we decided to implement the PF4 modal with simple text inputs for provider name, bucket name, and provider resource name. This will be replaced when the PF4 wizard becomes available.
Merged via #97.


Typo: ...being delete. -> ...being deleted.

Found and fixed a minor typo.  ...being delete. -> ...being deleted.

LGTM

LGTM

LGTM

LGTM


Release a new alpha build.

The release 1.0.0-alpha-4 is currently blocked by a change made in #2665.
When I run ./gradlew release --info, I get the following error message:
> Task :vavr:vavr:initializeSonatypeStagingRepository FAILED
Caching disabled for task ':vavr:vavr:initializeSonatypeStagingRepository' because:
  Build cache is disabled
Task ':vavr:vavr:initializeSonatypeStagingRepository' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
No stagingProfileId set, querying for packageGroup 'io.vavr'
:vavr:vavr:initializeSonatypeStagingRepository (Thread[Execution worker for ':vavr:vavr' Thread 11,5,main]) completed. Took 0.661 secs.

> Task :vavr:runBuildTasks FAILED
:vavr:runBuildTasks (Thread[Execution worker for ':vavr',5,main]) completed. Took 4.099 secs.

> Task :release FAILED
Release process failed, reverting back any changes made by Release Plugin.
Running [git, checkout, gradle.properties] in [/Users/cafebab3/git/vavr-io/vavr]
Running [git, checkout, gradle.properties] produced output: []
:release (Thread[Execution worker for ':',5,main]) completed. Took 40.157 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':vavr:vavr:initializeSonatypeStagingRepository'.
> Failed to load staging profiles, server at https://oss.sonatype.org/service/local/ responded with status code 401

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 41s
1 actionable task: 1 executed

Ok, we are one step further, the oss sonatype credentials are now found by the build after we merged #2691.
Now there are problems signing the release...
> Task :vavr:vavr:signMavenPublication FAILED
Caching disabled for task ':vavr:vavr:signMavenPublication' because:
  Build cache is disabled
Task ':vavr:vavr:signMavenPublication' is not up-to-date because:
  Task has failed previously.
:vavr:vavr:signMavenPublication (Thread[Execution worker for ':vavr:vavr',5,main]) completed. Took 0.004 secs.

> Task :vavr:runBuildTasks FAILED
:vavr:runBuildTasks (Thread[Execution worker for ':vavr',5,main]) completed. Took 16.91 secs.

> Task :release FAILED
Release process failed, reverting back any changes made by Release Plugin.
Running [git, checkout, gradle.properties] in [/Users/daniel/git/vavr-io/vavr]
Running [git, checkout, gradle.properties] produced output: []
:release (Thread[Execution worker for ':',5,main]) completed. Took 1 mins 0.738 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':vavr:vavr:signMavenPublication'.
> Cannot perform signing task ':signMavenPublication' because it has no configured signatory

All ssh signatures are in place, I will take a look why this does not work anymore (after migrating to Gradle 7)...

Done! Many thanks to all helping hands!
https://search.maven.org/artifact/io.vavr/vavr/1.0.0-alpha-4/jar


RFC: Modular GraphQL Backends

https://gist.github.com/thebigredgeek/3e5d9757b054b642062569424f7fee03

@stubailo @helfer

https://github.com/apollographql/graphql-server/issues/371

Thanks @thebigredgeek! As I mentioned in the other issue, a bit of context would be nice. I don't think most people want to click on links before at least having some idea of what they'll find there.

Sure.  Didn't have time tonight.  Will write a brief pitch tomorrow

Well, I clearly lied.    Will try to find some time to clarify this tonight

Hope your family are doing better. There's nothing more important than the family! 😃
Scott

Gonna start working on a prototype here

This is a great design but there doesn't seem to be an action item here! Excited to see movement in this direction :)

Thanks @stubailo .  I hope to publish something in Q1.  We are currently shipping our first GraphQL system at Collective so I've been crazy busy.  Working a lot with Java these days too which is no bueno haha


AOC-2023-D14-P2 - How does FindSequence work?

Is there a name for your FindSequence algorithm I could look up? If not, would you mind explaining how / why your FindSequence method works for 2023 Day 14, Part 2 ?

It might have a name, but I'm not sure what it might be called. I just put together a basic check for 2 identical sequences in the passed in list starting from the end. I also start with at least a sequence length of 5 in this case to avoid any small sequences.
So if you had the following list passed in:
Vals: 1,2,3,4,5,6,1,2,3,4,5,6
Ind: 0,1,2,3,4,5,6,7,8,9,10,11
It starts i at ind 6 = val 1
since 1 != 6, i moves to 5
since 6 = 6 now, it then checks further down at both ends to make sure everything is the same
5 = 5
4 = 4
3 = 3
and so on, and since it finds the same sequence twice it returns the starting index and length

Yeah I think I understand what you you did now. Can I also ask how you know that `loads[position + leftOver - 1] will give you the load you're looking for? Without some algorithm it feels a bit random to the untrained eye.

So at this point FindSequence has returned saying it found a duplicate sequence with a starting (position) and (length).
You then try to figure out what position in that sequence you would be at cycle<PHONE_NUMBER>. This is just the modular math in the code. You then just do your current position + the remainder from the modular math.
The only reason for the -1 is that Im indexing from 0 instead of 1. You can avoid that by just using 999999999 instead, but I just opted to use<PHONE_NUMBER>.

Awesome thanks for taking to time to explain. Makes sense now :)

No problem, if you haven't already. the semi-official discord for AoC is a good place for code discussion as well :)
https://discord.gg/QbMbFhTdbW


"Albești", "Arsa" are listed in the drop-down menu

Description:
When user writes "Albești", "Arsa" the location appears in the drop-down menu next to the comune and the county witch is a part according to the law
Precondition:
The website is up an running.

Step 1
Write in the search bar "Albești"

Expected results
The location appears in the drop-down menu as "Sat Albești, CONSTANȚA, (Albești)"

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Arsa"

Expected results
The location appears in the drop-down menu as "Sat Arsa, CONSTANȚA, (Albești)"


Recreate missing folders of project structure when you run "serverpod generate"

I've deleted the serverpod_server/lib/src/generated folder for cleaning older files.
On "serverpod generate" it retrieves
Running serverpod generate.
Failed to generate lib/src/generated/soggetto_entity.dart
Yikes! It is possible that this error is caused by an internal issue with the Serverpod tooling. We would appreciate if you filed an issue over at Github. Please
include the stack trace below and describe any steps you did to trigger the error.
https://github.com/serverpod/serverpod/issues
FileSystemException: Cannot create file, path = 'lib/src/generated/soggetto_entity.dart' (OS Error: No such file or directory, errno = 2)
#0      _File.throwIfError (dart:io/file_impl.dart:635:7)
#1      _File.createSync (dart:io/file_impl.dart:273:5)
#2      ClassGenerator.generate (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/class_generator.dart:61:20)
#3      performGenerateClasses (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/class_generator.dart:23:19)
#4      performGenerate (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/generator.dart:36:3)
#5      _main (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:201:13)
<asynchronous suspension>
#6      main.<anonymous closure> (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:33:9)
<asynchronous suspension>
#7      main (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:30:3)
<asynchronous suspension>

Yikes! It is possible that this error is caused by an internal issue with the Serverpod tooling. We would appreciate if you filed an issue over at Github. Please
include the stack trace below and describe any steps you did to trigger the error.
https://github.com/serverpod/serverpod/issues
FileSystemException: Cannot create file, path = 'lib/src/generated/protocol.dart' (OS Error: No such file or directory, errno = 2)
#0      _File.throwIfError (dart:io/file_impl.dart:635:7)
#1      _File.createSync (dart:io/file_impl.dart:273:5)
#2      ClassGenerator.generate (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/class_generator.dart:74:13)
#3      performGenerateClasses (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/class_generator.dart:23:19)
#4      performGenerate (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/generator/generator.dart:36:3)
#5      _main (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:201:13)
<asynchronous suspension>
#6      main.<anonymous closure> (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:33:9)
<asynchronous suspension>
#7      main (file:///Users/andry/.pub-cache/hosted/pub.dartlang.org/serverpod_cli-0.9.19/bin/serverpod_cli.dart:30:3)
<asynchronous suspension>

recreating the serverpod_server/lib/src/generated folder fixes the problem. When someone accidentally delete optional folders that are not directly correlated to the core function of serverpod, serverpod should recreate them.
BTW I love Serverpod <3

@AndryHTC Ideally they must create if they don't exist. Can you able to recreate this with the latest version?

@yahu1031 I didn't try but I didn't see any change about it either. Just follow the 2 steps to reproduce the issue and if it's fixed I will close the issue 👍

Okie, lemme try.

@AndryHTC I have tested it and it was fixed.
mr.minnu@Tradelabs-MacBook-Pro-2 hey_server % cd lib/src/
mr.minnu@Tradelabs-MacBook-Pro-2 src % rm -rf generated/
mr.minnu@Tradelabs-MacBook-Pro-2 src % cd ../..
mr.minnu@Tradelabs-MacBook-Pro-2 hey_server % serverpod generate
Running serverpod generate.
Done.
mr.minnu@Tradelabs-MacBook-Pro-2 hey_server % ls lib/src/
endpoints    future_calls generated    protocol     web
mr.minnu@Tradelabs-MacBook-Pro-2 hey_server %

So the thing when you are using this and the errors happened is that we don't have file/dir creation on recursion mode. Now the latest have the recursion everywhere. So, even if the folder doesn't exist, due to the recursion, the parent will be automatically generated.

@yahu1031, your contribution is greatly appreciated. Thank you for taking the time to test and verify the issue. I will now proceed to close this issue.


Fix #25625: Memory leaks in ossl_quic_calculate_retry_integrity_tag

Several error paths return 0 directly instead of going to err to clean up the objects.
CLA: trivial

OK with CLA: trivial

This pull request is ready to merge

Merged (with slightly tweaked commit message) to the master, 3.4, 3.3 and 3.2 branches. Thank you for your contribution.


MagStack.timeseries failing in get_chunk

I am initializing a MagStack and trying to take a time series from it.  Here is the script:
## Set up combined hdf5 stack
fpath='/home/ehultee/Stack/'
hel_stack = ice.MagStack(files=[fpath+'vx.h5', fpath+'vy.h5'])

## Extract time series at a point
xy_1 = (305.0, -2578.0) #polar stereo coordinates of a point near Helheim 2009 terminus
d = hel_stack.timeseries(xy=xy_1)
print(type(d))

When I run this, I get a TypeError:
Traceback (most recent call last):
  File "helheim-extract_ts.py", line 14, in <module>
    d = hel_stack.timeseries(xy=xy_1)
  File "/home/ehultee/.conda/envs/ehu-gld/lib/python3.7/site-packages/iceutils/stack.py", line 329, in timeseries
    dsum += (stack.timeseries(xy=xy, coord=coord, key=key, win_size=win_size))**2
TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and 'int'

I suspect that one of the slices is empty for one or more of the time points.  I will check at other extraction coords and using only a single Stack (rather than a MagStack).  If I am correct, this means that we need to do error handling for points close to termini where there may be data only before (or after) a certain time horizon.

That can also happen if a point is requested outside of the bounds of the Stack, e.g. check lines 158-161 in stack.py. You might want to try converting the values in xy_1 to meters. Either way, better error handling is definitely in order.


Fix table styles for notebook>=5.0.0

table-layout: auto fixes the main issue (overrides new default of table-layout: fixed), and adding white-space: nowrap; for cells with "N more positive/negative" makes the table more tidy.
here is how it looks for full 20 newsgroups dataset:

Fixes #178

Does it still look ok in IPython < 5.0.0?

Yep, I checked, it looks the same in notebook < 5.0.0 apart from nowrap, and nowrap works fine both in old and new versions.

Codecov Report

Merging #183 into master will decrease coverage by 10.57%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master     #183       +/-   ##
===========================================
- Coverage    97.3%   86.73%   -10.58%
===========================================
  Files          41       41
  Lines        2487     2487
  Branches      469      469
===========================================
- Hits         2420     2157      -263
- Misses         35      299      +264
+ Partials       32       31        -1

Impacted Files
Coverage Δ

eli5/formatters/html.py
100% <ø> (ø)
:arrow_up:

eli5/lightgbm.py
4.23% <0%> (-94.92%)
:arrow_down:

eli5/xgboost.py
5.55% <0%> (-93.75%)
:arrow_down:

eli5/_decision_path.py
43.47% <0%> (-56.53%)
:arrow_down:

eli5/_feature_names.py
94.33% <0%> (-2.84%)
:arrow_down:


Remove MAX_CAPACITY limit

Fixes https://github.com/rpgp/rpgp/issues/158.
I believe MAX_CAPACITY was useful in previous implementations, e.g. when we needed to grow a buffer:
https://github.com/rpgp/rpgp/blob/bbd38f679a1cd3a6cddb4dbe9d4e9cd2eb4ad742/src/packet/mod.rs#L180-L186
However in the Iterator implementation, there shouldn't be any reason to enforce a limit anymore?
It might be slow to parse > 1GiB, however I'd argue it is on users to decide if they are willing to accept the slowness (as opposed to have rpgg enforce an arbitrary limit).

The purpose of these "max size" checks is to protect applications from (potentially maliciously caused by a sender) allocations that could cause their application to be OOM-killed.
That said, size limits are of course generally undesirable to have.
Ideally, I'd want to see streaming support in most parts of rPGP, to eliminate the possibility of excessive allocations within rPGP. In cases where this is not possible, caller-configurable size limitations might be the next best thing.
Do you have a specific need/use case, where the current size limit is a problem for you? If so, can you share some background?

mmap is one way of doing this, but still brings risk in regards to memory exhaustion. The only real fix is to implement proper streaming support, for which these limits are not needed anymore, everything else can still pose a security risk when using the library on untrusted input


feat: script to filter helmfile selector by release name

Create get-selectors-by-release-and-clean.sh, which allows HELMFILE_GLOBAL_FLAGS to select by releases that have changed per-namespace. Useful for PRs where only a small number of releases are changed at-a-time.
In particular, the script compares changes to the local branch and main, making cautious decisions about what to select based on files that have changed. i.e.:
Changes outside base helmfiles/ directory:

Action: Perform a full cluster-wide with a "bare" helmfile template command
Cleanup command(s): rm -rf ${OUTPUT_DIR}/*/*/
helmfile selector: "" (empty string)
Explanation: Any changes outside of helmfiles/ are treated with caution by default - do a full regen

Changes within helmfiles/{$namespace}/helmfile.yaml outside of specific helm release versions:

Action: Regenerate the selected namespace.
Cleanup command(s):

rm -rf ${OUTPUT_DIR}/cluster/namespaces/${namespace}.yaml
rm -rf ${OUTPUT_DIR}/cluster/resources/${namespace}
 rm -rf ${OUTPUT_DIR}/customresourcedefinitions/${namespace}
rm -rf ${OUTPUT_DIR}/namespaces/${namespace}

helmfile selector: --selector namespace=${namespace}
Explanation: If there are changes within helmfiles.yaml in a given namespace, trigger a full namespace regen. This is cautious by design but also justified if, say, one adds a new chart repo under repositories:

Changes to helm release versions within helmfiles/{$namespace}/helmfile.yaml:

Action: Selective regeneration based on changed release versions
Cleanup command(s):

rm -rf ${OUTPUT_DIR}/cluster/resources/${namespace}/${release}
rm -rf ${OUTPUT_DIR}/customresourcedefinitions/${namespace}/${release}
rm -rf ${OUTPUT_DIR}/namespaces/${namespace}/${release}

SELECTOR: --selector namespace=${namespace},name=${release} for each changed release
Explanation: Increments to a release version should only template the release itself
Example for helmfiles/dev/helmfile.yaml:

releases:
[...]
- chart: dev/foo
  name: foo
<  version: 0.1.0
>  version: 0.1.1
[...]

Generates: SELECTOR=--selector namespace=dev,name=foo
Changes to chart value configs within helmfiles/{$namespace}/configs/*.yaml:

Action: Selective regeneration based on releases these configs apply to within the namespace
Cleanup command(s):

rm -rf ${OUTPUT_DIR}/cluster/resources/${namespace}/${release}
rm -rf ${OUTPUT_DIR}/customresourcedefinitions/${namespace}/${release}
rm -rf ${OUTPUT_DIR}/namespaces/${namespace}/${release}

SELECTOR: --selector namespace=${namespace},name=${release} for each changed release.
Explanation: Changes to chart config should only template the affected releases
Example: A change to helmfiles/dev/configs/baz.yaml; a dependency of releases foo and bar:

releases:
[...]
- chart: dev/foo
  version: 1.1.1
  name: foo
  values:
  - configs/baz.yaml
  - jx-values.yaml
- chart: dev/bar
  version: 1.0.2
  name: bar
  values:
  - configs/baz.yaml
  - jx-values.yaml
[...]

Generates SELECTOR = --selector namespace=dev,name=foo namespace=dev,name=bar
Activating the script can be done by overriding the parent Makefile
override HELMFILE_USE_SELECTORS := true
override HELMFILE_GLOBAL_FLAGS += $(if $(findstring true,$(HELMFILE_USE_SELECTORS)),$(shell versionStream/src/get-selectors-by-release-and-clean.sh $(OUTPUT_DIR)),)

include versionStream/src/Makefile.mk

Using this script can significantly increase the execution speed of the helmfile $(HELMFILE_GLOBAL_FLAGS) --file helmfile.yaml coommand within the fetch step, especially for PRs where only single releases are promoted.

/cc @msvticket @Skisocks @mentlak0 @tomhobson

/ok-to-test

/retest

/retest

/test gsm

/retest

/retest

The pipeline has succeeded, but it was started again. I think due to a race condition between the webhooks and keeper components of lighthouse.
/override gsm


Converting jupyter notebooks with treescope renderings to html files

By default, treescope visualizations in .ipynb files do not render at all when said notebooks are converted to html files. I see in the treescope docs that there's some javascript that's inserted into cell outputs to ensure it works properly with the docs, but its embedded in the Sphinx machinery. Is there a simple solution to get treescope visualizations to render without having to go through Sphinx?

Closing this as html conversion works when using treescope.basic_interactive_setup(), just not with the  penzai alias pz.ts.basic_interactive_setup()

Hm, this still shouldn't be happening even through the Penzai alias.
What version of Penzai do you have installed? This sounds like could be caused by a rendering bug that was introduced in Penzai 0.1.4 and fixed in Penzai 0.1.5.
(Note that before Penzai 0.2.0, Treescope was part of Penzai, so Penzai 0.1.x has its own copy of Treescope and isn't compatible with the separate treescope package. On the other hand, Penzai 0.2.0 uses the separate treescope package directly for its pretty-printing, so there should be no rendering differences between Treescope 0.1.0 and Penzai 0.2.0.)

Yep - there must have been a version mismatch! This works no matter how you import with Penzai 0.2.0, but fails with Penzai 0.1.4.


Check if an Event is handled or not?

Hi there, i am completely new to this library, but it looks very promising and i love it so far!
Sorry if this is the wrong place to ask, but the Readme suggested to ask here:
Basically I want to check if a postponed event is handled by someone else. Example:
struct WindowCloseEvent{};

void listen() {
    bus->listen([](const WindowCloseEvent& event) {
        // Custom handling code
    });
}

void update() {
    bus->postpone(WindowCloseEvent{});
    bus->process();

    if (!bus->handled()) {     // <- Does not exist
        window.close();
    }
}

listen() may or may not be called, depending on how the user uses the library.
The update part of the code should dispatch the event. If there is no custom listener attached it should execute a default handler, which just closes the window. But if a custom handler for this event is attached, it should execute the user implementation and not the default code.
My intuition would have been to call if (bus->postpone(...)) or similar, but none of these functions seem to return the correct information. I also tried to override the same event again with another implementation, but this throws an exception.
Thank you for any help!

Hello, thank you for question
There is no way currently to determine if someone/something handled the event. Also there is no way to add a "default" handler for any event or unhandled event. I'm thinking about it for debug purposes.
bus->process(); return only number of processed events (no matter if there was a listener for such an event or not)
bus->postpone() just return true/false if the event was successfully added to the queue
I will update documentation for that. (I will keep open this issue)
In my opinion you should store your state on your own. I mean the state of your listener as a listener could be called by "Event" or by simply callback in your code. Another approach would be to create 2 listeners. First one would handle logic as in your example, other will just be by some kind of state handler which will collect information but I would not recommend this approach.
Handling state by this library if something was called isn't a good idea (I think but we may argue). Simply you need information about if it was called, someone wants information how many times it was called and another person needs information how long ago it was called, so as you can see there are many use cases for handling such a state.

Thank you for your response!
I also tried this library: https://github.com/DeveloperPaul123/eventbus, but it also has currently no way of doing that, and because it is slightly more complicated i'm back to your library. The only difference i noticed is that this library allows to register multiple handlers for the same type which are then called in order, while your library throws an exception, but still no way to check if it was handled.
It works fine as i only call it in a wrapper function and never expose it directly, so i can do whatever i want in the wrapper.
Actually I don't need any of the information you listed, the only information i need is if a handler was attached or not, so i can decide wether or not to execute the default behaviour. My current workaround is:
Have a std::vector<std::string> as a list of registered handlers and whenever a handler is registered in the wrapper, i call listen and store typeid(T).name() in the list. When an event is called in the second wrapper, i check if typeid(T).name() is in the list and if so, call postpone and process, if not, not.
I think the easiest and most beautiful solution to this challenge would be to add an EventBus function that simply checks if the handler is registered, since you surely must have an internal representation of all the registered handlers. It would be called like if (bus->is_listening<WindowCloseEvent>()), maybe a similar name.
I think this would work amazingly well since you have the exceptions in place that allow only a single handler per type, meaning you can have zero or one handlers per type, which aligns perfectly with the is_listening function, making the library more complete.
Thank you for your consideration!

1 listener ID can have only 1 event handler thats because you need to distinguish which listen + unlisten pair you would like to dismiss. Of course you can have multiple listeners for same event. But this is only for your safety to have 1 listener that can listen for 1 event type.
Remember that you can register lambda as listener. Then you need a way to unregister such lambda. Thats why listenerID is returned.
You shouldn't relay on order of calling of your handlers this is some kind "UB" for library. Currently it works that way.
You should be using Listener so it will handle for you when to listen and unlisten in RAII style.
For now I may suggest such approach:
template <typename Event>
class MyListener
{
public:
	MyListener(std::shared_ptr<EventBus> bus)
		: _listener(bus)
	{
	}

	void listen(std::function<void(const Event&)> callback)
	{
		assert(isListening() == false);
		_listener.listen(callback);
		_isListening = true;
	}

	void unlisten()
	{
		_listener.unlistenAll();
		_isListening = false;
	}

	~MyListener()
	{
		_isListening = false;
	}

	bool isListening() const
	{
		return _isListening;
	}

private:
	EventBus::Listener _listener;
	bool _isListening = false;
};

struct WindowCloseEvent{};

int main(int argc, char* argv[])
{
	auto eventBus = std::make_shared<EventBus>();
	MyListener<WindowCloseEvent> listener{eventBus};

	if(false /*you decide when*/ && listener.isListening() == false)
	{
		listener.listen([](const auto& event)
						{
							//...
						});
	}

	if(true && listener.isListening() == false)
	{
		listener.listen([](const auto& event)
						{
							// other listener logic
						});
	}

	if(listener.isListening())
	{
		//...
	}
...
}

This is just simple implementation approach. Maybe better is to implement "protocol" of your listener in such way to not use lambda and use just methods. Eg:
struct WindowCloseEvent{};

template <typename Base, typename Event>
class MyListener
{
public:
	MyListener(std::shared_ptr<EventBus> bus)
		: _listener(bus)
	{
	}

	void listen()
	{
		assert(isListening() == false);
		_listener.listen([self = static_cast<Base*>(this)](const Event& e)
						 { self->callback(e); });//better than std::bind
		_isListening = true;
	}

	void callback(const Event& event)
	{
		//Generic callback for any event not implemented
		std::cout << "generic" << __PRETTY_FUNCTION__ << std::endl;
	}

	void unlisten()
	{
		_listener.unlistenAll();
		_isListening = false;
	}

	~MyListener()
	{
		_isListening = false;
	}

	bool isListening() const
	{
		return _isListening;
	}

private:
	EventBus::Listener _listener;
	bool _isListening = false;
};

class WindowCloseListener : public MyListener<WindowCloseListener, WindowCloseEvent>
{
public:
	void callback(const WindowCloseEvent& event)
	{
		// Logic specific for this event
		std::cout << "WindowClose" << std::endl;
	}
};

int main(int argc, char* argv[])
{
	auto eventBus = std::make_shared<EventBus>();
	WindowCloseListener listener{eventBus};

	eventBus->postpone(WindowCloseEvent{});
	eventBus->process();

	if(false /*you decide when*/ && listener.isListening() == false)
	{
		listener.listen();
	}

	eventBus->postpone(WindowCloseEvent{});
	eventBus->process();

	if(true && listener.isListening() == false)
	{
		listener.listen();
	}

	eventBus->postpone(WindowCloseEvent{});
	eventBus->process();

	if(listener.isListening())
	{
		//...
	}
...
}

Just example.

And of course you may add some bool flag was it called or not.

Thanks! This is almost what i am currently using, but not quite. It would be pretty much impossible to have a listener class for every event i have as this defies the point of having the event bus in the first place. I currently have one bus and one listener attached to each other, and then I simply call listen() on the listener object for every event i want to catch. This is why i can't set a bool variable as the listener is used for an arbitrary amount of events. This is why i use typeid(T).name(), as this is a string-representation of the type, allowing me to differentiate between types without having a variable for each and every one.
Here are a few code snippets from my usage:
std::vector<b::string> m_registeredEvents;
std::shared_ptr<b::event_bus> m_eventbus;
b::event_listener m_eventListener;

b::event_listener and b::event_bus are aliases to EventBus, to better fit into my library.
template<typename T, typename... TArgs>
bool dispatchEvent(TArgs&&... args) {
    if (std::find(m_registeredEvents.begin(), m_registeredEvents.end(), typeid(T).name()) == m_registeredEvents.end()) {
        return false;
    }
    m_eventbus->postpone<T>({ std::forward<TArgs>(args)... });
    m_eventbus->process();
    return true;
}

template<typename T>
void listenEvent(std::function<void(const T&)> listener) {
    m_eventListener.listenToCallback(listener);
    m_registeredEvents.emplace_back(typeid(T).name());
}

sf::Event event {};
while (m_sfmlWindow.pollEvent(event)) {
    ImGui::SFML::ProcessEvent(m_sfmlWindow, event);

    switch (event.type) {             // a gazillion events are dispatched here
        case sf::Event::Closed:
            if (!dispatchEvent<b::events::WindowCloseEvent>()) {
                m_sfmlWindow.close();
            }
            break;
        .....

As you can see, this is what allows me to call m_sfmlWindow.close() if no handler is attached, and just the handler if one is attached (where the user can call m_sfmlWindow.close() or not, based on a condition). This is done for like 30 different event types with one event bus and one listener.
It does work perfectly like this, but having isListening() would make everything a lot more conscise.

Checkout ProtectedEventStream this is why there is limitation 1 listener can have 1 event handler.
Each Listener just holds single ID.
Of course this lib allows to add your own implementation of EventStream sure no wants to do this :).
In my opinion your approach is correct and I should expose isListening in future.
For now what I could just recommend to inherit/compose from EventBus::Listener and just add there your map of tracked state of listened events. In future it will be more easy to upgrade as such interface such show up in Listener (Listener::isListening()).
Thank you that you describe here your use case. I will add it as example and consider best approach I can find.
Just hint as typeid(T).name() may be not unique if I remember correctly. So just be safe with that and double check. Maybe use just hash_code?

No problem, i did not mean that the limitation of 1 listener is bad, i just said it because it was the most apparent difference i found when comparing it to the other library. It's a difference that i noticed, that unfortunately does not help in my situation. The rest is mostly identical, the other one is just another implementation and a bit more complicated.
Anyways, i just looked into it and per cppreference.com, both typeid(T).name() and hash_code have the exact same implications: Both only guarantee that an invocation with the same type always produces the same result. But both might produce the same result with different types and are completely implementation defined, and both might produce different results between invocations of the same program, it is only the same throughout 1 lifetime of 1 process. MSVC produces human-readable names, GCC does not. So, actually both are flawed for this purpose.
As i currently use mainly MSVC which prints human-readable type names (eg. "struct WindowCloseEvent") and i use them with distinct unique names, i think it should not be an issue most of the time. I will keep my implementation just like it is until isListening is supported ;)
Thank you for your time, i really apprechiate it!

Limitation is only because I wanted to make it simple with RAII and avoid mistakes by someone with listen-unlisten match count. I will improve soon. Thanks. Give ma 1-2 weeks for that.

This maybe will interest you: https://github.com/gelldur/gcpp/blob/master/src/gcpp/type_name.h

Ah, very interesting, i saw such an implementation many times but never wanted to use it. is it cross-platform on MSVC and on GCC/Clang on Linux? I assume it if does work on all platforms and compilers, it does not produce equivalent results, just unique ones, right?

I mostly used it for debug so far is working.

Thank you, i still have to try this when i get back to it

This works perfectly, it simplifies my code significantly. Thank you so much for the change!


missing llama-7b-hf and llama-13-hf urls.

the urls in the readme are unreachable

This may be the case that the author who originally shared the model weights has deleted the repository. You can convert the corresponding checkpoint on your own.

Due to the lack of updates for a long time, your issue has been temporarily closed. If you still have any problems, please feel free to reopen this issue.


[Flaky] flaky test in test_operator_gpu.test_convolution_multiple_streams

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-14324/3/pipeline#step-682-log-1293
@DickJC123

@mxnet-label-bot Add [Test, Flaky]

Looking at this now.  The problem occurs during shutdown of the system when MXNET_ENGINE_TYPE=ThreadedEngine (so the ThreadedEnginePooled).  I'll first try to determine if it's related to the dual stream work, or whether it's just that the dual-stream PR, in expanding the meager testing of the ThreadedEnginePooled, uncovered an issue with it.

I believe I've isolated the problem.  I started by checking out the commit that introduced the dual stream facility and test (5f32f32e8).  I could run the test 20 times without failure.  I then examined behavior under the other 19 commits to master that were made since then.  The problem behavior first appears with:
053ffc7f8  2019-03-03  Wang Jiajun            fix memory-related issues to enable ASAN tests (#14223)

I ran the test_convolution_multiple_streams test with the engine list pruned down to just 'ThreadedEngine' (so ThreadedEnginePooled) for speed of testing.  It failed with the /usr/bin/python3': double free or corruption (out) abort at the 6th iteration.  I then backed up my testing to the previous commit:
0e23a18da  2019-03-02  Yuxi Hu                add symbolic link to mkldnn header files in include (#14300)

...and saw no failures after 40 repetitions of the full test_convolution_multiple_streams test. I even left the mshadow and dmlc-core versions the same as the 053ffc7f8 commit, which advances their versions.
In inspecting the change introduced by 053ffc7f8, I see changes to threaded_engine.cc as well as many free/delete operations added.  From this I conclude that 053ffc7f8 is the likely culprit and is not compatible with the ThreadedEnginePooled.  @szha @eric-haibin-lin @ptrendx

http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-gpu/branches/PR-14192/runs/5/nodes/273/log/?start=0
======================================================================
FAIL: test_operator_gpu.test_convolution_multiple_streams
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/work/mxnet/tests/python/gpu/../unittest/common.py", line 173, in test_new
orig_test(*args, **kwargs)
File "/work/mxnet/tests/python/gpu/test_operator_gpu.py", line 570, in test_convolution_multiple_streams
{'MXNET_GPU_WORKER_NSTREAMS' : num_streams, 'MXNET_ENGINE_TYPE' : engine})
File "/work/mxnet/tests/python/gpu/test_operator_gpu.py", line 540, in _test_in_separate_process
assert p.exitcode == 0, "Non-zero exit code %d from %s()." % (p.exitcode, func.name)
AssertionError: Non-zero exit code 255 from _conv_with_num_streams().
-------------------- >> begin captured logging << --------------------
common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=997087021 to reproduce.
--------------------- >> end captured logging << ---------------------

Thanks, @DickJC123. Does it reveal an actual problem or is it specific to the CI environment? I'd prefer to see this fixed instead of reverting ASAN enforcement as that change fixed a number of other problems.
cc @arcadiaphy

To debug this further, I checked out the ASAN PR commit, reverted my dual-stream PR, then reinserted the test_operator_gpu.py file version from that PR.  Then running the test_convolution_multiple_streams test multiple times generated segmentation violations and python 'double free' aborts (see below) on the ThreadedEngine only.  The test passes solidly when the ThreadedEngine is omitted.  I conclude that the dual-stream facility is not part of the issue, only that the PR came with a ThreadedEngine test that is perhaps more stressful than others in the CI.
@szha, if your preference is not to revert the ASAN commit, I could PR a change to the test_convolution_multiple_streams where it tests against only the NaiveEngine and ThreadedEnginePerDevice, and prints out that the ThreadedEngine is skipped.  I could then file an issue to correct the ASAN PR for the ThreadedEngine that @arcadiaphy should respond to.  The ThreadedEngine should be considered 'broken' until that is done.
OK?
[INFO] 3 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=87934937 to reproduce.
FAIL

======================================================================
FAIL: test_operator_gpu.test_convolution_multiple_streams
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/opt/mxnet/tests/python/gpu/../unittest/common.py", line 173, in test_new
    orig_test(*args, **kwargs)
  File "/opt/mxnet/tests/python/gpu/test_operator_gpu.py", line 570, in test_convolution_multiple_streams
    {'MXNET_GPU_WORKER_NSTREAMS' : num_streams, 'MXNET_ENGINE_TYPE' : engine})
  File "/opt/mxnet/tests/python/gpu/test_operator_gpu.py", line 540, in _test_in_separate_process
    assert p.exitcode == 0, "Non-zero exit code %d from %s()." % (p.exitcode, func.__name__)
AssertionError: Non-zero exit code -11 from _conv_with_num_streams().
-------------------- >> begin captured logging << --------------------
common: INFO: Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=124093393 to reproduce.
common: INFO: 3 of 10: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=87934937 to reproduce.
--------------------- >> end captured logging << ---------------------

After pruning the engine list to only include ThreadedEngine:
MXNET_TEST_COUNT=10 nosetests --verbose -s tests/python/gpu/test_operator_gpu.py:test_convolution_multiple_streams
[INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=549472273 to reproduce.
test_operator_gpu.test_convolution_multiple_streams ... [22:27:47] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:27:50] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
[22:27:52] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:27:55] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
[22:27:56] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:27:59] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
[22:28:01] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:28:04] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
[22:28:05] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:28:08] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
[22:28:10] src/engine/engine.cc:55: MXNet start using engine: ThreadedEngine
[22:28:13] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (se
tting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
*** Error in `/usr/bin/python': double free or corruption (out): 0x00007f80d04e77d0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f817c0897e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f817c09237a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f817c09653c]
/opt/mxnet/lib/libmxnet.so(_ZN5mxnet7storage23GPUPooledStorageManager10ReleaseAllEv+0x2f1)[0x7f816b78cef1]
/opt/mxnet/lib/libmxnet.so(_ZN5mxnet7storage23GPUPooledStorageManagerD0Ev+0x1a)[0x7f816b79507a]
/opt/mxnet/lib/libmxnet.so(_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE10_M_releaseEv+0x47)[0x7f81684d0f47]
/opt/mxnet/lib/libmxnet.so(_ZN5mxnet11StorageImpl10DirectFreeENS_7Storage6HandleE+0x84)[0x7f816b790ab4]
/opt/mxnet/lib/libmxnet.so(_ZNSt17_Function_handlerIFvN5mxnet10RunContextEEZNS0_8resource19ResourceManagerImpl17ResourceTempSpaceILNS0_15ResourceRequest4Ty
peE1EED4EvEUlS1_E_E9_M_invokeERKSt9_Any_dataOS1_+0x10f)[0x7f816b8ca51f]
/opt/mxnet/lib/libmxnet.so(+0x40da8ba)[0x7f816b7768ba]
/opt/mxnet/lib/libmxnet.so(_ZN5mxnet6engine14ThreadedEngine15ExecuteOprBlockENS_10RunContextEPNS0_8OprBlockE+0x956)[0x7f816b770c06]
/opt/mxnet/lib/libmxnet.so(_ZNSt17_Function_handlerIFvSt10shared_ptrIN4dmlc11ManualEventEEEZN5mxnet6engine20ThreadedEnginePooled5StartEvEUlS3_E_E9_M_invoke
ERKSt9_Any_dataOS3_+0x17d)[0x7f816b77173d]

@DickJC123 this plan sounds good. Thanks.

Rather than open up a new issue to track any follow-up PR from @arcadiaphy  I thought it best to continue this one to keep the history.
My PR https://github.com/apache/incubator-mxnet/pull/14338 removes the ThreadedEngine from the engine list of the test_convolution_multiple_streams, but also enables easy testing of that problem case.  After the PR is merged, I would recommend that @arcadiaphy test with:
MXNET_TEST_COUNT=50 MXNET_ENGINE_TYPE=ThreadedEngine nosetests --verbose -s tests/python/gpu/test_operator_gpu.py:test_convolution_multiple_streams

...to prove that the fixes he comes up with cure the present segfaults.  Also, to ensure that the previous engines still work, I recommend:
MXNET_TEST_COUNT=50 nosetests --verbose -s tests/python/gpu/test_operator_gpu.py:test_convolution_multiple_streams

@DickJC123 OK. Let me find out the problem behind the threaded engine crash.

I need to point out that a recently merged "op bulking" PR saw a segfault in either the NaiveEngine or the ThreadedEnginePerDevice.  I could not reproduce this on my internal machines after hundreds of trials.  I'm hoping that what @arcadiaphy learns in his follow-up work may have relevance past the ThreadedEngine problem.  See: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-14200/9/pipeline

Failed for me today: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/job/unix-gpu/job/PR-14855/5/display/redirect

Failed for me as well: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-14638/6/

Failed again! http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-15883/40/pipeline/302

Unable to recreate the failure locally with
MXNET_TEST_COUNT=10 nosetests --verbose -s tests/python/gpu/test_operator_gpu.py:test_convolution_multiple_streams

Can anyone try to see if they can reproduce it locally?
@ChaiBapchya Could you also try it once?

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-17325/23/pipeline#step-875-log-1043

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-17508/2/pipeline/
#17508

Unable to recreate the failure locally with
MXNET_TEST_COUNT=10 nosetests --verbose -s tests/python/gpu/test_operator_gpu.py:test_convolution_multiple_streams

Can anyone try to see if they can reproduce it locally?
@ChaiBapchya Could you also try it once?

Ran the above command. Test passed
Finished engine ThreadedEnginePerDevice with 2 streams.
ok

----------------------------------------------------------------------
Ran 1 test in 958.201s

I don't think you will be able to repro this failure on its own unfortunately - I believe the problem with this test is that it is affected by other tests. What I believe happens is the other tests use a lot of GPU memory that is cached in the caching memory allocator. Then this tests (to test the effect of env variables that are read only during init of MXNet) launches a new MXNet process. That process then tries to allocate memory and hit OoM condition - this is not a problem for other tests, because they would just free the memory stored inside the caching allocator, but this process does not have that option, so it just fails, which is then reported as a failure.
I believe the proper way of fixing this would be to move this test to its own command during testing (so that there is no memory allocated by other tests when it runs).

@ChaiBapchya Thanks for confirming.
@ptrendx That makes sense. On digging a bit deeper, I found that test_bulking had similar failures. https://github.com/apache/incubator-mxnet/issues/14970
Both test_bulking and test_convolution_multiple_streams spawn a new process for the test.
Do you think it is a good idea to move both of them into a new test file?

They should be fine, yes, but I don't think the new test file is enough - it actually needs to be its own nosetests instance so that they are the only thing launched from that process.

@kshitij12345 since the fix is identified do you mind leading it? I can take it up otherwise. :)

@ChaiBapchya I'll be busy in the coming weeks. So it'll be great if you can fix this one.

Another one - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-gpu/detail/PR-17475/8/pipeline #17475


Capture only first row of terraform --version output

Terraform version returns "Terraform v0.5.0
Your version of Terraform is out of date! The latest version
is 0.5.1. You can update by downloading from www.terraform.io"
When not using latest version. Need to suit script for capturing v0.5.0


Revert "[GitHub Actions] - Updates in Java Tests to runs-on Self-hosted runners"

Reverts apache/beam#23160
This broke the workflow - https://github.com/apache/beam/actions/runs/3660849278/jobs/6188476823

R: @elink21 @dannymartinm


Broken logic while parse 'Today' type of string with the time offset

Hi. I found some bug while parsing the date. I will try to explain with examples.
In every parse, we use time offset (important, TIMEZONE must be more than your own).
For example, right now 23:00 and our timezone is +0300.
we parse the string "26.10.2018 05:28 PM" with timezone +0400 to our timezone. This time dateparser work right:

datetime.datetime(2018, 10, 26, 16, 28)

But if we will parse the string "Today  05:28 PM" with timezone +0400 to our timezone, dateparser will make mistake:

datetime.datetime(2018, 10, 27, 16, 28).

I guess, that this mistake appears from using datetime.datetime.now() when converting "Today" and converting timezone from our server time. It is not correct.
Please, can you fix this mistake?

Comment on the offending code.


Fix years in OpenDR copyright

Ditto.

@passalis Could you please give write access to this repository to @ad-daniel?
Thank you!


Being a good Xref citizen

Re-report from https://github.com/elixir-lang/elixir/issues/9083

Fixed in latest release. It says:
warning: invalid association `omg` in schema Friends.Person: associated schema Unknown does not exist
  lib/friends/person.ex:1: Friends.Person (module)


refactor: reduce the error logs of ENOENT

Description

reduce the error logs of ENOENT.

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Please describe this motivation. The code diff looks good to me.

I have made changes to log all caught errors.
apm calls this function many times to update listings, etc. So if apm.json was not present in the initial installation, etc., a lot of errors were logged even though it was not an error.
I tried to fix that, but now I think it would be better to check for its existence and return in try.

OK. There will be another way to check apm.json exists using path.existsSync. However, this way is simple.


Mount wheel not visible in Silverwastes map

After I installed the latest version (warclaw update), the wheel works normally on most maps.
Only on Silverwastes the whell does not show. It also always chooses the favorite mount, no matter in what direction I press.
Going back to the old Version (prior warclaw) brings back the wheel on all maps.

You need to download the latest hotfix which already fixes this issue.

Sorry that I didn't see the hotfix was already available.
Thanks for the great work!


Create Multiverse Plugin

Instead of creating your own okhttp client, inject OkHttpClient.

Closing due to inactivity.


Sync Backward in Time a.k.a Reverse Sync

We have a desired feature of doing syncing by grabbing chunk going backwards in time.
This would allow for a user to see their current positions/markets etc quickly, while still getting the full time-series information in charts and history after awaiting a full sync.

Do you either of you have thoughts on how this could work?
What data integrity issues could we run into (and could they be mitigated)?

do eet

do eet

Adding this issue to the Performance Testing epic to decide if / how we need to do this.

@pgebheim @joeykrug I think this is largely not needed now that we're developing a warp sync strategy as the primary method of reducing sync time. Thoughts?

yeah it never made sense to me to begin with

Agreed, closing.


Shai: Add ping RPC to services

Closes #2192
Adds a no-op RPC call that can be used to ping a service to see if it is available. This will be used on the client side to make it easy to wait until the server is up an running in the likely case that the users of the library is starting an apalache server alongside their application.
NOTE: Will need to rebase after #2186 is in, only https://github.com/informalsystems/apalache/pull/2193/commits/59b4f278e9b0547673746f88c26e7532252e7e6d is added here, and review only needs to consider that commit.

I will need to rework this likely because of the gnarliness of python's protbuf support: https://github.com/grpc/grpc/issues/9575

Codecov Report

Merging #2193 (4f9a5cc) into main (fb972ee) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #2193      +/-   ##
==========================================
- Coverage   80.62%   80.61%   -0.01%
==========================================
  Files         426      426
  Lines       13927    13929       +2
  Branches     1840     1854      +14
==========================================
+ Hits        11228    11229       +1
- Misses       2699     2700       +1

Impacted Files
Coverage Δ

.../forsyte/apalache/shai/v1/CmdExecutorService.scala
87.50% <100.00%> (+0.54%)
:arrow_up:

...orsyte/apalache/shai/v1/TransExplorerService.scala
86.20% <100.00%> (+0.24%)
:arrow_up:

...a/at/forsyte/apalache/tla/lir/TlaLevelFinder.scala
93.75% <0.00%> (-2.09%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Since the underlying protocol is handshaking via TCP, doesn't the client establishing a connection already imply that the server is up?

Even gooder question: in theory we should be able to use to the channel state API to check whether the server is up and healthy. However, in practice -- as I discovered through hours of trial and error a couple months back -- if we try to move forward with RPC calls as soon as the channel state reports "READY", the request can actually be missed, leaving the client waiting indefinitely. I haven't looked at the implementation-level source of this problem, but presumably there transition between having an TCP connection up and ready and the gRPC plumbing to actually handle requests is not atomic.
I guess the potential gap between having a ready connection and the gRPC system actually up and ready to handle requests is also indicated by https://stackoverflow.com/a/51250184/1187277, which notes:

Note that the API does not actively monitor the service's health; it just informs whether the server is currently known to be running and reachable.

I'd be quite happy to switch to some built in mechanism if we can find one!

presumably there transition between having an TCP connection up and ready and the gRPC plumbing to actually handle requests is not atomic.

Interesting, thanks for explaining!


Comment specific line diff with error

Pre-requisite: #324
Given that we are already detecting which PRs and which line diffs have console.logs, we should coment those line diffs with "please remove this console log".

Closed via https://github.com/watermelontools/watermelon/pull/331


Timeout update

This should be merged after PR #165 A few changes to timeouts and some mechanics for checking names of remote brokers

Actually you can probably just merge the timeout one I think that is based on the echo app

Sent from VMware Boxer
On February 22, 2018 at 1:17:52 PM PST, Dheepak Krishnamurthy<EMAIL_ADDRESS>wrote:
@kdheepak approved this pull request.
There's a lot of common commits between this PR and #165https://github.com/GMLC-TDC/HELICS-src/pull/165. I recommend doing a squash merge for both these PRs. Alternatively, I can merge manually locally. Let me know if you think we should do that.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/GMLC-TDC/HELICS-src/pull/167#pullrequestreview-98731166, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ATtbEXLH_bEpaPbBkguaLc4jAza4P6lzks5tXdl0gaJpZM4SOqv0.


Handling New removeBannerForLiveChatCommand Item in YouTube API

When parsing and scraping each of the types of messages I noticed an error when I encountered the following,
{'removeBannerForLiveChatCommand': {'targetActionId': 'ChwKGkNMZWVyYWIwdjRFREZSXzRsQWtkTTAwTTRR'}}
it seems like there is a new item in YouTube api's it seems or something but I just did the following to fix it.
def output_messages(self):
        """"build a messages list that contains all the chat messages"""
        messages = []
        for content in self.contentSet:
            payload = content[nc.ACTIONS_NODE][0]
            if nc.TICKER_ITEM_ACTION_NODE in payload or "removeBannerForLiveChatCommand" in payload: # Notice that I just added an or with "removeBannerForLiveChatCommand"
                pass
            # ... (rest of the code)

Can you share the URL of the video that had this?

I think it was this one, XS-S0SnLhHk, but dont hold my word for it, I've actually modified a lot of thing on my end so I dont actually remember which it was, but it seems to be like some kind of LiveChat action by the live video host, maybe they removed a specific banner and it gets logged as removeBannerForLiveChatCommand ? Let me know if thats the video, if it's not I can provide you with a list where of videos from where I was getting the error, but is about 400 videos...

Found one, in this video F6Ep5q2ymzo you can get the removeBannerForLiveChatCommand  object when parsing the comments.

Neat, if it's just another Message node, just needs to be created under messages, inheriting from Message and adding build_message() and generate_content() methods.

Do you already have a markdown explaining all the different types of Message nodes saved anywhere?
Actually I just modified the message factory module like so,
#!/usr/bin/env python3
# Importing necessary libraries and modules
from livechat_scraper.constants import node_constants as nc
from livechat_scraper.messages.chat_message import ChatMessage
from livechat_scraper.messages.membership_gifted_message import MembershipGiftedMessage
from livechat_scraper.messages.membership_message import MembershipChatMessage
from livechat_scraper.messages.pinned_message import PinnedMessage
from livechat_scraper.messages.superchat_message import SuperChatMessage
from livechat_scraper.messages.message import Message
from livechat_scraper.messages.membership_redeemed_message import MembershipRedeemedMessage
from livechat_scraper.messages.purchased_sticker_message import PurchasedSticker

class messageFactory:
    """A factory class for creating Message objects based on the provided payload.

    This class provides a method to create Message objects of the correct type
    based on the live chat item type specified in the payload.

    Attributes:
        build_from_items (dict): A mapping from live chat item types to lambda
            functions that create Message objects.
        build_from_root (dict): A mapping from root node types to lambda functions
            that create Message objects.
    """

    def __init__(self):
        """Initialize the factory with mappings for building Message objects."""
        self.build_from_items = {
            nc.LIVECHAT_PAID_MESSAGE_NODE: lambda pl: SuperChatMessage(pl),
            nc.LIVECHAT_MEMBERSHIP_NODE: lambda pl: MembershipChatMessage(pl),
            nc.LIVECHAT_MEMBERSHIP_GIFT_PURCHASED_ANNOUNCEMENT_NODE: lambda pl: MembershipGiftedMessage(pl),
            nc.LIVECHAT_TEXT_MESSAGE_RENDERER_NODE: lambda pl: ChatMessage(pl),
            nc.LIVECHAT_MEMBERSHIP_GIFT_RECEIVED_ANNOUNCEMENT_NODE: lambda pl: MembershipRedeemedMessage(pl),
            nc.LIVECHAT_PAID_STICKER_RENDERER : lambda pl: PurchasedSticker(pl)
        }

        self.build_from_root = {
            nc.ADD_BANNER_NODE: lambda pl: PinnedMessage(pl)
        }

    def build(self, payload) -> Message:
        """Create a Message object from the provided payload.

        This method iterates through the mappings defined in build_from_root and
        build_from_items to find a matching live chat item type, then creates and
        returns a Message object of the correct type.

        If no matching live chat item type is found, the method prints the payload
        and a message indicating that the payload has a message type that is not
        currently being tracked. This can be useful for identifying new or
        unsupported message types.

        Args:
            payload (dict): The payload containing the live chat item data.

        Returns:
            Message: A Message object of the correct type, or None if no matching
            live chat item type is found.

        Examples:
            A payload that can be parsed:
                {
                    "addChatItemAction": {
                        "item": {
                            "liveChatPaidMessageRenderer": {
                                ...  # other data
                            }
                        }
                    }
                }

            A payload that cannot be parsed:
                {
                    "unknownNode": {
                        ...  # other data
                    }
                }

            Usage:
                factory = MessageFactory()
                message = factory.build(payload)
        """
        for m_type, builder in self.build_from_root.items():
            if m_type in payload:
                return builder(payload)

        for m_type, builder in self.build_from_items.items():
            item_node = payload.get(nc.ADD_CHAT_ITEM_ACTION_NODE, {}).get(nc.ITEM_NODE, {})
            if m_type in item_node:
                return builder(payload)

        # Print statements below are for debugging purposes:
        # They alert the developer to payloads with untracked message types,
        # which could indicate new or unsupported message types in the YouTube API.
        unrecognized_type = next(iter(item_node), "Unknown type")
        print(f"Unrecognized message type: {unrecognized_type}")
        print(f"Payload content: {payload}")
        print("The payload above has a message type that is not currently being tracked.")

        return None  # Explicitly return None for clarity

This are the modifications,

Shebang Line:

Added: #!/usr/bin/env python3 at the top of the script.
Benefit: Specifies the interpreter for executing the script, making it executable directly from the command line, adhering to good practice for script files.

Class Name:

Changed: class messageFactory(): to class MessageFactory:.
Benefit: Adheres to the PEP 8 naming convention for classes, which suggests using CamelCase.

Docstrings:

Added: Detailed docstrings to the class and methods, following the PEP 257 docstring conventions.
Benefit: Provides clear, structured documentation that explains the purpose and usage of the class and its methods, improving code readability and maintainability.

Payload Parsing:

Modified: Extracted item_node before the second loop to avoid code duplication.
Benefit: Improves code efficiency and readability by reducing redundancy.

Unrecognized Type Handling:

Added: Code to identify and print the unrecognized message type and payload content.
Benefit: Provides clearer debug information when encountering unrecognized message types, aiding in identifying new or unsupported message types for future development.

Explicit Return:

Added: return None statement at the end of the build method.
Benefit: Makes it explicit that the method returns None if no matching message type is found, adhering to the principle of making code self-explanatory.

Examples Section in Docstring:

Added: Examples section in the build method docstring.
Benefit: Provides example usage and input for the method, aiding in understanding how to use the method and what kind of payloads it can handle.

With these modifications I managed to find liveChatModeChangeMessageRenderer and liveChatDonationAnnouncementRenderer because it now prints it out like this,
Unrecognized message type: liveChatDonationAnnouncementRenderer
Unrecognized message type: liveChatModeChangeMessageRenderer

Which can be helpful for debugging and managing unrecognized/unknown new message types that YouTube might add in the future.
Cheers!

ohhh nice, that's much more detailed than my quick factory. Feel free to add a PR for it if you want.


failed to convert special folder: errno=42

librealsense
2.55.1 RELEASE

OS
Windows

Intel RealSense Viewer / Depth Quality Tool has crashed with the following error message:
RealSense error calling rs2_create_context_ex(api_version:25501, json_settings:nullptr):
    failed to convert special folder: errno=42

Hi @Biojio  Advice about the errno=42 error can be found at https://github.com/IntelRealSense/librealsense/issues/13037

Hi @Biojio  Do you require further assistance with this case, please?  Thanks!

Case closed due to no further comments received.

My Intel RealSense colleagues have provided some further advice about the errno=42 error.
Given that the problem seems to be related to certain charcters being used in a Windows user account name, I asked whether creating a secondary Windows user account with a name that has no special characters might help.  This was their response.

You can try to run it as Administrator instead of switching the account.  On Windows 11, it also has a feature to run the app as a different user.  That option can be enabled with the instructions at the link below:
https://winaero.com/how-to-run-apps-as-different-user-in-windows-11/


reverse functionality on validatePage buttons for macOS

The PageWalletSeed::validatePage() msgbox buttons when creating a new wallet seems to be reversed on Mac M1. (Ventura 13.3.1a)
Clicking "I understand"  takes you back, and "Go back"  or pressing escape takes you forward. This happens for me on both the v2.4.5 Intel and Apple Silicon builds.

I think for custom qt buttons on macOS you need to do something like this instead:
// new example
if (seedWarning.clickedButton() == rejectButton) {
    return false;
}
...
// old way
if (res == QMessageBox::Rejected) {
    return false;
}

Thanks, fixed in 4b43e49c6faeafc394970a8bcce6cbbad0cd5829


5.0.2 rel notes

Docs Build status updates of commit 6c388ca:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/release-notes/NuGet-5.0-RTM.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add support for config

Feature request
Add support for config arg
Configurable options

Radius of confetti
Number of confetties for each side of screen

Merged #6


Refactor build pipeline

I don't know a good way to avoid checking in these boilerplate scripts. Most of them are subject to tinkering for individual repos, so importing them in a way that makes them readonly to you is counter-productive. The few files that tend to be exactly the same across repos are few enough and slow moving enough that I haven't crafted a way to share just that subset.

But the source of truth for these is https://github.com/aarnott/library.template.

@AArnott  build failed with error message:  Response status code does not indicate success: 403 (Forbidden - User '6d3b3c1a-123d-454c-a78b-b4a426164711' lacks permission to complete this action. You need to have 'ReadPackages'. (DevOps Activity ID: 2CF1D764-FEDC-446D-BA87-7C1FEC70684D)) at https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4104702&view=logs&j=2d2b3007-3c5c-5840-9bb0-2b1ea49925f3&t=92eaab8a-d904-52ba-6bd2-a55f3ebce252&l=36 . Looks like the build passed when you manually queue it. It is the build failure due to PR build doesn't have the right permission to access internal DevOps feed?

That failure is very odd. I've never seen it before, but I believe you're right: secrets are not exposed to PRs from forks as mine is. Once you accept my PR it should work in yours.
However since yours is an OSS repo, you may want to copy another pattern that we use for our OSS repos: skip the microbuild signing step for PRs, or at least PRs from forks. I'll prepare that change.

It's fixed. :)


ORC-1602: [C++] limit compression block size

This is a clone of

#1774

This is resolved via https://github.com/apache/orc/commit/6d2c4767f71147027096b447dd49460d43e3deec


Proper marshal response metadata timestamps

Marshal response metadata timestamp as RFC3339Nano

Codecov Report

Merging #261 (47c3f27) into main (ce5eeda) will decrease coverage by 0.08%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #261      +/-   ##
==========================================
- Coverage   32.36%   32.28%   -0.09%
==========================================
  Files          44       44
  Lines        4514     4526      +12
==========================================
  Hits         1461     1461
- Misses       2861     2873      +12
  Partials      192      192

Impacted Files
Coverage Δ

api/server/v1/marshaler.go
0.00% <0.00%> (ø)

server/services/v1/api.go
0.00% <0.00%> (ø)

server/services/v1/query_runner.go
0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce5eeda...47c3f27. Read the comment docs.


Refer to the title in the same markdown

Describe the bug
markdown example
123
456
456
When I click on 456 under the title 123, the webpage can not automatically scroll to the title 456
Reproduction
N/A
System Info
valaxy version：0.14.56

Used Package Manager
pnpm
Validations

[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A, please open a GitHub Discussion instead.
[X] The provided reproduction is a minimal reproducible of the bug.

Works well, just upgrade your valaxy version.


xSQLServerSetup: SQLSvcAccount credentials does not work after a reboot

Details of the scenario you try and problem that is occurring:
When I set SQLSvcAccount and run xSQLServerSetup everything works fine and sql server is installed and the correct account is set for the sql server service.
But when I reboot the machine, none of the sql services will start. And I can't start it from services UI either.
The only way is to set the password for the sql service again and click ok. And also set a blank password for the sql agent service and click ok. Then I can start the services.
When I have done this manually it works to reboot the server.
But If I run the DSC once more and reboot I have to do the procedure above again to be able to start the services.
I have only tested with creating credentials with clear text password and have not yet setup a certificate store. But it should work anyhow.
The DSC configuration that is using the resource (as detailed as possible):
        xSQLServerSetup SqlServerSetup
        {
            SourcePath = $ConfigurationData.SoftwarePath + "sql_server_2016"
            InstanceName = "MSSQLSERVER"
            Features = "SQLEngine,FullText,RS,AS,IS"
            SetupCredential = new-object -typename System.Management.Automation.PSCredential -argumentlist ($ConfigurationData.InstallerCredential.User, (ConvertTo-SecureString -String $ConfigurationData.InstallerCredential.Password -AsPlainText -Force))
            SourceCredential = new-object -typename System.Management.Automation.PSCredential -argumentlist ($ConfigurationData.InstallerCredential.User, (ConvertTo-SecureString -String $ConfigurationData.InstallerCredential.Password -AsPlainText -Force))
            SQLSvcAccount  = new-object -typename System.Management.Automation.PSCredential -argumentlist ($ConfigurationData.SqlSQLSvcAccountCredential.User, (ConvertTo-SecureString -String $ConfigurationData.SqlSQLSvcAccountCredential.Password -AsPlainText -Force))
            SecurityMode = "SQL"
            SAPwd = new-object -typename System.Management.Automation.PSCredential -argumentlist ("sa", (ConvertTo-SecureString -String $ConfigurationData.SqlSAPassword -AsPlainText -Force))
            SQLCollation = "Finnish_Swedish_CI_AS"
            SQLUserDBDir = "D:\data\"
            SQLUserDBLogDir = "L:\data\"
            SQLTempDBDir = "T:\data\"
            DependsOn = "[WindowsFeature]NETFramework45Core"
        }

Version of the Operating System, SQL Server and PowerShell the DSC Target Node is running:
Windows 2016 Server UI
Version of the DSC module you're using, or 'dev' if you're using current dev branch:
dev

That is strange, I will look into this. The $ConfigurationData.InstallerCredential you are referencing in the configuration. Is that a Credential object or a another typ of object?

No, it's just a custom object holding a username and a password. I'm still in the testing phase of the configuration and I create the credentials with clear text passwords for now. I haven't had the time to setup the certficate store yet.
$ConfigurationData.SqlSQLSvcAccountCredential.User => user name in clear text
$ConfigurationData.SqlSQLSvcAccountCredential.Password => password in clear text


Language server error caused by Malformed Mach-o file

Check for existing issues

[X] Completed

Describe the bug / provide steps to reproduce it
Hi team,
I got a language server error for multiple languages like JavaScript, Python, JSON, etc.
Language server error: JavaScript

failed to run npm info

Caused by:
    Malformed Mach-o file (os error 88)

Step to reproduce:

Open a JS file.

Environment
Zed: v0.81.1 (stable)
OS: macOS 12.6.1
Memory: 16 GiB
Architecture: x86_64
If applicable, add mockups / screenshots to help explain present your vision of the feature
No response
If applicable, attach your ~/Library/Logs/Zed/Zed.log file to this issue.
If you only need the most recent lines, you can run the zed: open log command palette action to see the last 1000.
2023-04-16T02:41:25 [INFO] ========== starting zed ==========
2023-04-16T02:41:25 [INFO] set environment variables from shell:/bin/zsh, path:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
2023-04-16T02:41:25 [INFO] Opening main db
2023-04-16T02:41:25 [INFO] Opening main db
2023-04-16T02:41:25 [INFO] open paths ["/Users/zartre/Git/algo-exercises"]
2023-04-16T02:41:26 [INFO] Opening main db
2023-04-16T02:41:26 [INFO] Opening main db
2023-04-16T02:41:26 [ERROR] no cached binary
2023-04-16T02:41:26 [ERROR] failed to run npm info

Sounds like something went awry while downloading the bundled binary of Node. Could you please try the following steps:

Quit Zed completely.
In a terminal of your choice run rm -rf ~/Library/Application\ Support/Zed/node
Open Zed again.

Is the issue resolved?

Your solution worked. Thanks a lot! @hovsater


Bounded process pool?

Originally reported by: jaddison (Bitbucket: jaddison, GitHub: jaddison)

Is gipc compatible with a pool scenario?  I want to take web requests to resize images - the resizing of images likely fits well into a separate process, given the GIL.
That said, I do not want uncontrolled process spawning. Is it possible to achieve something like gevent.pool.ThreadPool like upper limit pool sizing? Along the order of:
On initialization, create a process pool size of 20.
Handle incoming web request, ask the pool to perform image resizing function and cooperatively wait for the result
Once the result is returned from the process, return it to the web client
Handle many incoming requests, but do not perform more than 20 image resizes at a time because the process pool is bounded
If this is possible, can the processes be reused to save on process initialization overhead?
The docs make it seem like gipc is great for spawning processes and waiting, but in an uncontrolled fashion? Or am I misunderstanding? Thanks!

Bitbucket: https://bitbucket.org/jgehrcke/gipc/issue/24

CC @jaddison

@jgehrcke I believe the combination of ProcessPoolExecutor with asyncio does exactly this - of course, in the async/await style of asyncio - which is where I am focusing my attention.
So, at the moment, I do not need this particular feature in gipc, although I have another project that would benefit if this should get implemented.
Thanks for picking up the reins again! 🚀

Any update on this?

To add detail to my use case:
I am using gipc to create a split-run-join paradigm. Within a gevent WSGI environment, I need to spawn some number of jobs that are long running that need to all return results to the original spawn. My current wrapper looks something like this:
def wrapper(cend):
    """
    Wrapper that makes functions act normally.
    """
    fn, args = cend.get()
    cend.put(fn(*args))

def gipc_starmap(fn, arg_list):
    """
    Driver function that maps each arg in arg_list on to fn, and waits for the functions to finish.
    """
    workers = []
    for args in arg_list:
        cend, pend = gipc.pipe(duplex=True)
        p = gipc.start_process(wrapper, (cend,))
        pend.put((fn, args))
        workers.append([p, pend])
        log.debug("Spawned job")
    results = []
    log.debug("Joining jobs")
    for worker, pend in workers:
        worker.join()
        results.append(pend.get())
        pend.close()
    return results

So, gipc_starmap is given a function and a list of arguments, these functions are ran in parallel, and the return values are returned once all jobs are done.
I am looking for a way to cap the number of simultaneous greenlets this would spawn, because as-is it is unbounded.


When will Client-side caching be supported in cluster mode?

Currently, client caches support standalone. How can Redis cluster be supported? There is no such function at present, can you support it? We really need this feature

I don't know. See #1380 for further details. Drafting a pull request would be a good first step.


UG Bug Fixes

Still doing

Codecov Report

Merging #163 (45f78ec) into master (83edc0c) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #163   +/-   ##
=========================================
  Coverage     58.68%   58.68%
  Complexity      602      602
=========================================
  Files           121      121
  Lines          2561     2561
  Branches        251      251
=========================================
  Hits           1503     1503
  Misses          966      966
  Partials         92       92

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 83edc0c...45f78ec. Read the comment docs.


Update to 3.9, rerender

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@pitrou, looks like OProfile has bitrotted in llvm 3.9 (see the circle build log). You mentioned an interest in having it enabled when this first got put together. I don't know much about that code.
Huh: Just saw this mailing list message from you. Did you learn anything since then?

Well, I guess OProfile support can be ditched, then. I haven't heard back about it.

Removed. Thanks for getting back to me so quickly.

FWIW, we can easily revert that if the support comes back.


Null RecyclerView after Fragment recreation on a ViewPager

RecyclerView get Null after recreation of fragment.
I bind view in OnCreateView().

Butter Knife is just calling findViewById so either the view is absent or you're not calling bind() at the right time during the lifecycle.