Non-recoverable error

Someone reported this with AngelBob
I have this with Py when drilling down a product chain, latest Factorio 0.18.
latest pysuite no PyAL
`12391.656 Error MainLoop.cpp:1202: Exception at tick 297461: The mod Recipe Book caused a non-recoverable error.
Please report this error to the mod author.
Error while running event RecipeBook::on_gui_selection_state_changed (ID 58)
The mod Recipe Book caused a non-recoverable error.
Please report this error to the mod author.
Error while running event RecipeBook::Custom event (ID 215)
RecipeBook/gui/info-base.lua:194: attempt to index local 'object_data' (a nil value)
stack traceback:
RecipeBook/gui/info-base.lua:194: in function 'update_contents'
RecipeBook/gui/info-base.lua:217: in function 'open_or_update'
RecipeBook/control.lua:433: in function 'handler'
RecipeBook/lualib/event.lua:86: in function <RecipeBook/lualib/event.lua:26>
[C]: in function 'raise_event'
RecipeBook/lualib/event.lua:263: in function 'raise'
RecipeBook/control.lua:65: in function 'handler'
RecipeBook/lualib/event.lua:86: in function <RecipeBook/lualib/event.lua:26>
stack traceback:
RecipeBook/lualib/event.lua:263: in function 'raise'
RecipeBook/control.lua:65: in function 'handler'
RecipeBook/lualib/event.lua:86: in function <RecipeBook/lualib/event.lua:26>
stack traceback:
[C]: in function 'raise_event'
RecipeBook/lualib/event.lua:263: in function 'raise'
RecipeBook/control.lua:65: in function 'handler'
RecipeBook/lualib/event.lua:86: in function <RecipeBook/lualib/event.lua:26>`

This should be fixed in the most recent version. Can you confirm?

Can you confirm if this has been fixed?

As far as my installation goes neither does the game crash nor do i get another error.
so it's fixed for me.

Sweet, thanks!


Link to the searching guides on the specific widgets/connectors doc

I had a live-chat today, where the user had an issue with the pagination because it did not calculate the correct number of pages.
In the end, it was a limitation from Algolia API documented here (https://www.algolia.com/doc/guides/searching/pagination/#pagination-limitations) in which its automatically limit the max nbPages to 34.
The user suggested to link this kind of searching guides to the connectors/widgets pages to learn more on how it works behind and avoid this kind of frustration thinking that Algolia is broken.
What do you think @Haroenv @bobylito ?

Yep, that seems perfect to me, a button to "read more on the API documentation" or something would fit.
Things like hierarchicalMenu should then be linked to the helper documentation maybe?

That seems fair to do that (and of great help for our users). Let's have that in mind for all the use cases, but here I don't see more links to add.


Show the correct legend for the currently selected period in the version breakdown chart

In the version breakdown chart, the default legend (no period selected, hence it's the last period) is always showing the values for the 1 day interval, even if the user chooses a different interval.
How to reproduce:

Go to the group extended view and check the legend showing for the default interval (1 day);
Click on the 7 day selector: the legend remains the same as before (and it's the same when selecting other intervals).

Fixed in master.


chore: Update work manager version

Fixes #1225
Checklist:

[ ] I have checked for PMD and check-style issues
[x] I have read the Contribution & Best practices Guide and my PR follows them.
[x] My branch is up-to-date with the Upstream development branch.

@Masquerade0097 @mishuvs @iamareebjamal The app no longer seems to be crashing on toggling attendees. Kindly test it in your device as well

@sridharjajoo Yes, the app is not crashing now on toggling checkin.


Change "server install" links to point to readthedocs.

(Right now this is a 404! PR pending to add the doc this would point to.)
Also it'd be cool to make https://zulip.org/server.html actually give an HTTP redirect to the new page, for the benefit of anyone following links there from outside. Not sure how best/easiest to do that.

The page this links to is up now (and I've amended the commit message accordingly), so this is ready to merge.


problem with Select an Identifier Mapping Database on PathVision

I install PathVision on my window computer.
I am following this: https://pathvisio.org/tutorials/getting-started.html
I Installed the WikiPathways plugin but when trying to 'select gene Database' I got an error: Failed to open gene database; could not connect to database.
I download Hs_Derby_Ensembl_105.bridge from https://bridgedb.github.io/data/gene_database/
I could not find any help on this issue.
Thanks,

I think we have the same question. I was following the tutorial  but couldn't find GeneProduct ACO2. I downloaded only the homo sapiens from the file you mentioned, loaded it via Data > Select Gene Database (note that the PathVisio Book tutorial here is outdated as there is no "Load Identifier Mapping" option under "Files"... & still cannot find my GeneProduct.
Pity. The tutorial was going so swimmingly, but stumped at the first practice.

My apologies! This issue somehow skipped my inbox :(.
There were some issues with the identifier mapping database version 105 which should be resolved now. Please let me know if this error still occurs!

@oddlyaware the PathVisio Book is only for PathVisio 4 which is not yet fully released but in a testing phase.


az devops login failed

This is the command that's failing.
az devops login --organization https://msazure.visualstudio.com
ERROR: Failed to unlock the collection!
Traceback (most recent call last):
  File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 206, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 603, in execute
    raise ex
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 661, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 652, in _run_job
    cmd_copy.exception_handler(ex)
  File "/home/cyl/.azure/cliextensions/azure-devops/azext_devops/dev/common/exception_handler.py", line 26, in azure_devops_exception_handler
    reraise(*sys.exc_info())
  File "/opt/az/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 631, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 306, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 485, in default_command_handler
    return op(**command_args)
  File "/home/cyl/.azure/cliextensions/azure-devops/azext_devops/dev/team/credentials.py", line 42, in credential_set
    set_credential(organization=organization, token=token)
  File "/home/cyl/.azure/cliextensions/azure-devops/azext_devops/dev/common/_credentials.py", line 33, in set_credential
    cred_store.set_password(key, token)
  File "/home/cyl/.azure/cliextensions/azure-devops/azext_devops/dev/common/credential_store.py", line 31, in set_password
    old_token = keyring.get_password(key, self._USERNAME)
  File "/home/cyl/.azure/cliextensions/azure-devops/keyring/core.py", line 57, in get_password
    return _keyring_backend.get_password(service_name, username)
  File "/home/cyl/.azure/cliextensions/azure-devops/keyring/backends/SecretService.py", line 64, in get_password
    collection = self.get_preferred_collection()
  File "/home/cyl/.azure/cliextensions/azure-devops/keyring/backends/SecretService.py", line 58, in get_preferred_collection
    raise KeyringLocked("Failed to unlock the collection!")
keyring.errors.KeyringLocked: Failed to unlock the collection!
WARNING:

I went to https://msazure.visualstudio.com/_usersSettings/tokens and generated a new token. Still doesn't work.
This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az devops login Extension Name: azure-devops. Version: 0.12.0.
Errors:
Failed to unlock the collection!
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 206, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 603, in execute
    raise ex
cli/core/commands/__init__.py, ln 661, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
...
azure-devops/keyring/backends/SecretService.py, ln 64, in get_password
    collection = self.get_preferred_collection()
azure-devops/keyring/backends/SecretService.py, ln 58, in get_preferred_collection
    raise KeyringLocked("Failed to unlock the collection!")
keyring.errors.KeyringLocked: Failed to unlock the collection!

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az devops login --organization {}

Expected Behavior
Environment Summary
Linux-5.0.0-31-generic-x86_64-with-debian-buster-sid
Python 3.6.5
Shell: bash

azure-cli 2.0.76

Extensions:
azure-devops 0.12.0

Additional Context

Hi @ironyman ,
The fix to avoid storing credentials in keyring in case it has issue is already in master and will be available in next release.
Meanwhile, you can try following solutions.
Do you have an application named "Sea horse" on your system? Or was there a pop-up to enter a password which you might have dismissed by mistake while running this command?
The problem here is, you need to set-up a password to access seahorse which will be inturn used by keyring, and if its already locked, keyring asks you to enter the same password. Without this password, keyring can't store your credentials in Seahorse application.
Thus, setting up of Seahorse is required in this case.
You can check more details in a similar issue on keyring #762


Add more debug_value tests

Add more debug_value tests now that debug_value is correctly preserved by deabstraction.

@swift-ci please test tensorflow

Please delete test/TensorFlow/debug_scope.swift: the only reason it exists is to test that deabstraction passes debug scope verification without -O.
Since test/TensorFlow/debugging.swift runs without -O, test/TensorFlow/debug_scope.swift is no longer necessary.


Add ahk icon

Type

[X] File icon
[ ] Folder icon

Context and associations
file : ".ahk"
language : "AutoHotkey"
References

You should probably change the title to "Add AutoHotkey icon" to be more clear :)


solution presented is not working

suggested fix doesnt work. 1: there is no Advanced Save Options. (it's hidden under Save As -> small arrow)
2: Even after save in UTF-8 it still complains. blabla...current code page (932). Save the file in Unicode...
Try to add /utf-8 then it's even worse "warning C4828: The file contains a character starting at offset 0x3eaf that is illegal in the current source character set (codepage 65001)."
Finally what indeed does work is save with signature (I imagine this means with BOM) + add /utf-8 option.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f677989c-03ac-f741-48f5-c5924441a7f5
Version Independent ID: 16584d74-1a02-22b4-14ed-355371fbc148
Content: Compiler Warning (level 1) C4819
Content Source: docs/error-messages/compiler-warnings/compiler-warning-level-1-c4819.md
Product: visual-cpp
Technology: cpp-diagnostics
GitHub Login: @corob-msft
Microsoft Alias: corob

@siliconvoodoo
Thanks for bringing this to our attention. There's more than one way to fix this issue, so the instructions have been expanded. Visual Studio 2019 even automates some of the process for you.


The exception will cause the application to exit

CoreCLR Version: 6.0.1122.52304
.NET Version: 6.0.11
Description: The process was terminated due to an unhandled exception.
Exception Info: System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
at System.Convert.FromBase64String(String s)
at EasyCaching.Bus.FreeRedis.DefaultFreeRedisBus.OnMessage(String topic, Object body)
at FreeRedis.RedisClient.<>c__DisplayClass433_0.b__0(String p, String k, Object d)
at FreeRedis.RedisClient.PubSub.OnData(String pattern, Boolean ssub, String key, Object data)
at FreeRedis.RedisClient.PubSub.<>c__DisplayClass18_0.b__3()
at System.Threading.Thread.StartHelper.Callback(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.Thread.StartCallback()

@xubinhua888  Thanks for your interest in this project.
在pub时，我们内部会将消息进行封装，并且会进行base64处理，所以message格式应该为base64格式，sub才能正常解析


Unable to send hipchat room notifications

Hi,
I have the following rule with a Hipchat alert:
name: ErrorRule
type: frequency
index: logstash-*
num_events: 10
timeframe:
    minutes: 5
filter:
- query:
    query_string:
      query: "level: ERROR"
alert:
- "hipchat"
hipchat_auth_token: my_token
hipchat_room_id: my_id

but when the filter matches I always get the following exception:
ERROR:root:Error while running alert hipchat: Error posting to HipChat: 400 Client Error: Bad Request for url: https://api.hipchat.com/v2/room/my_id/notification?auth_token=my_token

I am sure the token and room_id are valid because I tried to send a POST request with message, notify and color in the payload, from within a REST tool, and it works perfectly.
Only thing I could think of is that an empty message is used in the payload...
Is there any way I could control this message?
Regards,
Geert

According to the elastalert codebase,

HipChat sends 400 bad request on messages longer than 10000 characters

But the code specifically truncates the body to a length of 9980 to avoid that problem. Perhaps that length needs to be made smaller. Any ideas @Qmando ?

@geertvanheusden
Technically HipChat should give a 401 if your authentication was wrong--and I am not saying it is...but if you have your hipchat instance hosted at another location, your credentials wouldn't exist at https://api.hipchat.com. You would need to set the hipchat_domain key in your rule (I have to do this).

@bean5 I am pretty sure the credentials are correct. We are also using the public Hipchat service, not the self hosted one.
I think it has something to do with the length of the message. Although it's strange that truncating the message won't help.


Revert "BLD: Add const to Cython signatures for BLAS/LAPACK (#18247)"

Should fix the problem mentioned in #18377 introduced by #18247 (cc @lysnikolaou). It would be good if the pip-pre wheels were usable again by downstream libraries ASAP, and we can decide a better plan moving forward in #18377, hence why I'm not marking that to be closed by this PR.
Maybe some more fine-grained approach could be used but a git revert of the PR commit seemed the most straightforward.

The failing CI jobs are all unrelated, let's get this in. I'll manually start a nightly wheels build job too.


RedeliveryCount is 0 on redelivery

Describe the bug
I have a pulsar server with a single topic and a message produced to the topic.
It is a standalone server.
I have a client app having a single consumer.
Subscription type is Shared.
When I receive the message, and then stop the client app before ack-ing, and then start the client app again, then the message is redelivered as expected, but with redeliveryCount=0 (expected 1).
Similarily, when the consumer reconnects due to temporary lack of connectivity to pulsar server before ack-ing, then the redelivered message has the redeliveryCount=0.
To Reproduce
Steps to reproduce the behavior:

Receive a message.
Block connectivity. I did it like this:
sudo iptables -I INPUT -p tcp -m tcp --dport 6650 -j REJECT
Acknowledge the message while there's no connectivity
Wait longer than ping timeout (keepaliveIntervalMillis?, default is 30sec.)
Unblock connectivity:
sudo iptables -D INPUT -p tcp -m tcp --dport 6650 -j REJECT
Reconnect will occur and message will be redelivered (as expected), but, it's redeliveryCount will be 0 (not 1).

OR:

Receive a message.
Stop the client app without ack-ing the message.
Start client app again.
Redelivery occurs with redeliveryCount=0.

Expected behavior
Redelivered message's redeliveryCount value should be 1 instead of 0.
Desktop (please complete the following information):

OS: Ubuntu 18
Using Java pulsar client.

Hi @BewareMyPower.
I suppose it's not really related to C++ client. It happened also with Java client and Python client. I wonder why this should be handled at client side at all.

@MichalKoziorowski-TomTom It's because the original issue was tagged with client-cpp label, when I checked the stale issues in the original pulsar repo, I moved it here.


ndb crashes after launching script

Steps to reproduce:
I'm launching ndb with ndb bot.js and it launches, starts the script and crashes, the same when i click "start" inside ndb and for example toggle "pause on exceptions" or change tab.

ndb version: 1.0.37
Platform / OS version: Microsoft Windows [Version 10.0.17134.407]
Node.js version: 10.14.1

There seems to be no problem with code since it works on normal node, but when i open it in ndb, then it crashes.
What happens instead?
ndb throws:
events.js:167
throw er; // Unhandled 'error' event
^
Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
at ChildProcess.target.send (internal/child_process.js:628:16)
at Rpc.routeMessage_ (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\rpc\rpc.js:379:9)
at page_.exposeFunction.data (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\lib\carlo.js:327:71)
at Page._onBindingCalled (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:516:56)
at CDPSession.Page.client.on.event (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:134:54)
at CDPSession.emit (events.js:182:13)
at CDPSession._onMessage (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:216:12)
at Connection._onMessage (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:99:19)
at PipeTransport._dispatch (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:65:24)
at Socket.PipeTransport._eventListeners.helper.addEventListener.buffer (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:30:64)
Emitted 'error' event at:
at process.nextTick (internal/child_process.js:632:35)
at process._tickCallback (internal/process/next_tick.js:61:11)
I've tried re-installing ndb, reinstalling all dependencies, reinstalling ms build tools, different flags and stuff. It did work, but lately stopped.
I can add that when this started occuring, the ndb got slow, instead of showing my files immediately, it showed them after 10s, and there are windows popping up, which weren't before (node.EXE and npm)
Thanks in advance.

popping up windows are fixed in<EMAIL_ADDRESS>

What version of Chrome do you have locally installed?

Version 70.0.3538.110 (Official) (64-bit)

maybe it's a problem with my system specifically, tell me what to check if there are any possible causes.

sorry for the multiple comments but here's a list of things I'd want to add, because they might be helpful in finding the cause.

My windows C drive has recently got some files NTFS compressed. Idk why, I've reverted most of them and disabled it. Maybe some critical ndb files got affected still? (appdata/npm isn't compressed anymore)
When launching ndb, it takes AGES to show the project folder despite it being not so large (3k lines of code in main .js file)
When ndb launches and i wait those AGES for the folder to show up, the CMD takes 70% of CPU, which is pretty much a lot for a script editor, it stops when the folder appears in ndb.

My best guess would be windows NTFS compression since the crashes started occurring randomly and about a week later i found about 25GB of compressed files, so that may be the cause.
If it can be, tell me which files does ndb use for me to check if they aren't compressed.
De-compressed and anti-compress locked folders that may relate to ndb as of right now are:

project folder
AppData/roaming/npm
AppData/roaming/npm-cache
whole AppData/Local

Hope we can find the cause,
cheers!

Thanks a lot for your diagnostic, I will take a look on Windows as soon as I get my windows machine back, it should happen in next day or two.

i have the same issue and same message with:

ndb 1.0.42
chrome Version 71.0.3578.80 (Build officiel) (64 bits)
node 10.14.1 and previous

`events.js:167
throw er; // Unhandled 'error' event
^
Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
at ChildProcess.target.send (internal/child_process.js:628:16)
at Rpc.routeMessage_ (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\rpc\rpc.js:379:9)
at page_.exposeFunction.data (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\lib\carlo.js:380:71)
at Page._onBindingCalled (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:516:56)
at CDPSession.Page.client.on.event (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:134:54)
at CDPSession.emit (events.js:182:13)
at CDPSession._onMessage (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:216:12)
at Connection._onMessage (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:99:19)
at PipeTransport._dispatch (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:65:24)
at Socket.PipeTransport._eventListeners.helper.addEventListener.buffer (C:\Users\user\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:30:64)
Emitted 'error' event at:
at process.nextTick (internal/child_process.js:632:35)
at process._tickCallback (internal/process/next_tick.js:61:11)`
i tried to debug till event.js:167 with nbd, but when i added a breakpoint to this line, ndb added lots of breakpoints of other node modules... so i could not track the bug...

reverted to<EMAIL_ADDRESS>- everything's fine.

yeah，reverted to<EMAIL_ADDRESS>，  everything is OK

ndb 1.0.44
chrome 73.0.3683.103
node v10.11.0
Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
ndb 1.0.26 works, including breakpoints

I noticed that any kind of error, and I think warning would cause ndb to
crash. I would run normally with 'node your_server.js' to be able to see
the errors/warnings, fix any errors or warnings, then run with ndb.
On Mon, Apr 15, 2019 at 12:35 PM Mehuge<EMAIL_ADDRESS>wrote:

ndb 1.0.44
chrome 73.0.3683.103
node v10.11.0
Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
ndb 1.0.26 works, including breakpoints
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GoogleChromeLabs/ndb/issues/204#issuecomment-483326329,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIEUTQcNwqm-_6FoRcK5M862ouQA5YLdks5vhKpHgaJpZM4Y87HJ
.

node v10.15.3
ndb v1.0.44
chrome 74.0.3729.169
I'm installing ndb for the first time and having the same issue, any suggestions on a fix?

I have the same issue. it was working fine with node v10.16.0 and then I removed it and installed 12.4.0 and now after I open it and try to run a script it crashes. I removed it and installed it a couple of times but that didn't work.
events.js:177
      throw er; // Unhandled 'error' event
      ^

Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
    at ChildProcess.target.send (internal/child_process.js:678:16)
    at Rpc.routeMessage_ (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\rpc\rpc.js:379:9)
    at C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\carlo\lib\carlo.js:388:71
    at Page._onBindingCalled (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:548:56)
    at CDPSession.<anonymous> (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Page.js:136:54)
    at CDPSession.emit (events.js:200:13)
    at CDPSession._onMessage (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:200:12)
    at Connection._onMessage (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\Connection.js:112:17)
    at PipeTransport._dispatch (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:59:22)
    at Socket.<anonymous> (C:\Users\artan\AppData\Roaming\npm\node_modules\ndb\node_modules\puppeteer-core\lib\PipeTransport.js:30:64)
Emitted 'error' event at:
    at internal/child_process.js:682:35
    at processTicksAndRejections (internal/process/task_queues.js:82:9)

not an ideal fix but as someone noted above rolling ndb back seems to work,  I went back to ndb v1.0.26 and it has been working with node v10.15.3

One of the possible reasons of this crash was a broken native terminal dependency. I updated it in<EMAIL_ADDRESS>and it should at least not to crash.
Another possible reason is reusing installed Chrome by carlo. I am going to force carlo to download latest Chromium to address this problem. I will release version with this fix soon (https://github.com/GoogleChromeLabs/ndb/pull/260).

Could someone with reproducible issue check out<EMAIL_ADDRESS>Most problems related to ndb crashes should be fixed there.
This build includes nice way to profile your app as well, just add --prof right after ndb, e.g. ndb --prof index.js.

Since the ECONNRESET issues point here, I'll add another datapoint that I just updated and get ECONNRESET with 1.0.48+.  1.0.47 works ok.  Is there any particular debug info that would help?

@davidlehn could you try to clean install ndb? npm uninstall -g ndb && npm install -g ndb.
The biggest change after 1.0.47 is that ndb fetches chromium by itself and do not reuse installed Chrome. I can assume that something went wrong when ndb was fetching Chromium.
What OS do you use?
Another useful information can be get by following steps:

mkdir test && cd test
npm init -y && npm install ndb - it creates new node package and install ndb locally,
try to run ./node_modules/.bin/ndb
if it does not work try to start fetched by ndb Chromium, you can find it inside ./node_modules/carlo/lib/.local-data/linux-624492/, on linux you need to run ./chrome-linux/chrome from this folder. Does it start?

I'm on 1.0.48 and it's still the same error.
First time you try to launch a script nothing happens, when you click again
Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed

I'm having the same error as well, any updates on this?

@ZeldOcarina and @ForsakenHarmony could you try steps mentioned in https://github.com/GoogleChromeLabs/ndb/issues/204#issuecomment-505988945 ?
Does puppeteer package work in your environment our of the box? Could you share more details about your environment?
I am going to release next version soon and it will include some other improvements as well.

@ak239 I've tried executing all of your instructions.
With the latest clean version installed I just get a crash:

With earlier versions as suggested earlier I get the code to be runned but the debugger does not stop on checkpoints, thus useless.
My environment is Windows 10 64bit, I've runned everything with Powershell with admin privileges.
I've installed the Windows toolbox as well.
Can you better explain the puppeteer point? I don't get what you mean? It opens Chromium that should have puppeteer built in by default, right?
Thanks!

@ZeldOcarina, thank you. I was able to reproduce it on windows machine yesterday and I know the root of this issue - I will publish fix this week.

@ZeldOcarina, thank you. I was able to reproduce it on windows machine yesterday and I know the root of this issue - I will publish fix this week.
I found that carlo spawns new process for backend services with detached: true flag and when one of the services tries to do process.stderr.write - it fails. It surprisingly works on mac and linux just fine.

Thanks man for your incredible job and what you guys do for the community!

@ZeldOcarina could you check<EMAIL_ADDRESS>(I do not have access to windows machine right now - I will check it this evening).
Thank you for your help! It is critical with platform specific bugs like these.

I've taken my time to test it in different ways, with both global and local installs.
Now the debugger is not crashing but it's still not stopping on errors as you can see in the screenshot.

Thanks, @ZeldOcarina. Does debugger statement work? I have some idea why breakpoints can be broken on windows, I will debug it this evening. I definitely need to get windows machine and test ndb much better there.

@ak239 Yes Aleksey with debugger keyword in place the code actually stops. It's only the manual green flags that aren't working!

@ZeldOcarina what node version do you use?
I built some similar with your's script and set couple breakpoints there. All of them works as expected. At the same time if you would like to trigger pause on error or exception you can activate pause on exception DevTools feature. If you'd like to restart your app - you can use Ctrl + R. If you'd like to kill current process without restarting you can hover it in Node processes section and click Kill button.
You snippet contains a little mistake - it does not require url module but uses url.parse - I am not sure is it expected in the test or not.

Thanks @ak239. I've corrected the bug in the code. The breakpoints set with the green flag are not working for me but it's fine, I can use "debuger" keyword to handle this.
Thanks for the tips on Ctrl + R and killing the current process.
Now the program is functional and that is what matters the most!

Hi, @ak239 I am using<EMAIL_ADDRESS>and<EMAIL_ADDRESS>.
I am getting the same issue, the breakpoints are not working but the debugger keyword is working.

@hritique thank you. I believe that I detected problem with breakpoints. It is all about spaces in paths on Windows, DevTools frontend does not replace space with %20 properly. I will fix it.

Thanks @ak239 , I hope it will get fixed.

@hritique please try 1.1.2, it should contain breakpoints fix.

This one needs better fix, once and for everything. I will work on it this week. For now I reverted fix from 1.1.2 and publish 1.1.3 without it.
Thanks for your feedback @ZeldOcarina , I think we are close to make windows users happy!

Thanks @ak239 I'm starting to realize why Windows isn't that popular amongst developers!

I had same problem on Debian, and found I had to run:
 sudo sysctl -w kernel.unprivileged_userns_clone=1

https://github.com/GoogleChrome/puppeteer/blob/ff6d2a33f3edb8837bde3ab687c6d06510b5d4de/docs/troubleshooting.md#setting-up-chrome-linux-sandbox
In my case it is for a dev machine, so I am not too worried about changing this option.

I'm also having this issue. It fails in the same way after a few seconds. I've been using ndb with this project for a while and have never seen this issue. Could be our project just grew past some limit?
Environment:

ndb version: 1.1.5
Platform / OS version: macOS 10.14.5
Node.js version: 10.16.0

Problem:
➜  ndb
/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/lib/process_utility.js:14
    throw error;
    ^

Error: Error: CharacterIdMap ran out of capacity!
Error: CharacterIdMap ran out of capacity!
    at Common.CharacterIdMap.toChar (https://domain/ndb.js:1:141718)
    at https://domain/ndb.js:3:160732
    at Array.map (<anonymous>)
    at Persistence.PathEncoder.encode (https://domain/ndb.js:3:160711)
    at Persistence.Automapping.FilePathIndex.addPath (https://domain/ndb.js:3:142687)
    at Persistence.Automapping._onUISourceCodeAdded (https://domain/ndb.js:3:137592)
    at https://domain/ndb.js:3:135752
    at Workspace.Workspace.dispatchEventToListeners (https://domain/ndb.js:1:91157)
    at Persistence.FileSystemWorkspaceBinding.FileSystem.addUISourceCode (https://domain/ndb.js:1:658133)
    at Persistence.FileSystemWorkspaceBinding.FileSystem._fileChanged (https://domain/ndb.js:3:134868)
    at Rpc.dispatchMessageLocally_ (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/carlo/rpc/rpc.js:423:16)
    at Rpc.routeMessage_ (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/carlo/rpc/rpc.js:371:12)
    at process.emit (events.js:198:13)
    at emit (internal/child_process.js:832:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
    at ChildProcess.target.send (internal/child_process.js:636:16)
    at Rpc.routeMessage_ (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/carlo/rpc/rpc.js:379:9)
    at page_.exposeFunction.data (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/carlo/lib/carlo.js:388:71)
    at Page._onBindingCalled (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/Page.js:548:56)
    at CDPSession.Page.client.on.event (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/Page.js:136:54)
    at CDPSession.emit (events.js:198:13)
    at CDPSession._onMessage (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/Connection.js:200:12)
    at Connection._onMessage (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/Connection.js:112:17)
    at PipeTransport._dispatch (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/PipeTransport.js:59:22)
    at Socket.PipeTransport._eventListeners.helper.addEventListener.buffer (/Users/ishaantaylor/.nvm/versions/node/v10.16.0/lib/node_modules/ndb/node_modules/puppeteer-core/lib/PipeTransport.js:30:64)
Emitted 'error' event at:
    at process.nextTick (internal/child_process.js:640:35)
    at process._tickCallback (internal/process/next_tick.js:61:11)

i had same problem with npm (os fedora 32 workstation)
solution worked for me : delete node_modules and related stuff.
use yarn to re install, and run ndb with yarn.

I think https://github.com/GoogleChromeLabs/ndb/issues/290 is the same as this. I'm having this issue on MacOS and ndb v1.1.5 as well. I put some of my pokings in the other thread.

Does anyone solve that?

Does anyone solve that?

Both ndb 1.1.4 and 1.1.5 work for me on macOS Catalina

@Mehuge I have v1.1.5 but looks like this current issue is not related to ndb itself. But I hope to get some info from people that solved it somehow.

I am getting this issue after upgrading to macOS Big Sur from macOS Catalina. No problems on Catalina. I'm running ndb v1.1.5, node v14.16.0, macOS v11.2.3, and Chromium 73.0.3679.0.

@att14 try this solution.


Bot is building sentences but not tweeting

Hi - I'm quite new to this bot and I think I've nearly got it working.
The problem I'm having is that the bot only seems to build sentences/statements on my configured schedule; however, it never seems to actually tweet anything.  Code in question and output produced is below. Any thoughts?
Code in question:
##############
def on_startup
load_model!
scheduler.every '1m' do
  puts "I'M HERE NOW"
  statement = model.make_statement(140)
  puts "ABOUT TO TWEET"
  tweet(statement)
  puts "SHOULD HAVE TWEETED"
end

end
###################
Output produced:
pi@raspberrypi:~/tBot/testBot $ sudo ebooks start
@handle: User information updated
@handle: Loading model model/testuser.model
@handle: starting tweet stream
@handle: Online!
I'M HERE NOW
Building: sentence 0 of 90911
Building: sentence 10000 of 90911
Building: sentence 20000 of 90911
Building: sentence 30000 of 90911
Building: sentence 40000 of 90911
Building: sentence 50000 of 90911
Building: sentence 60000 of 90911
Building: sentence 70000 of 90911
Building: sentence 80000 of 90911
Building: sentence 90000 of 90911
'M HERE NOW
Building: sentence 0 of 90911
Building: sentence 10000 of 90911
Building: sentence 20000 of 90911
Building: sentence 30000 of 90911
Building: sentence 40000 of 90911
Building: sentence 50000 of 90911
Building: sentence 60000 of 90911
Building: sentence 70000 of 90911
Building: sentence 80000 of 90911
Building: sentence 90000 of 90911

I've modified the code and have had some success getting the bot to tweet; however it is only working intermittently:
def on_startup
load_model!
scheduler.every '1m' do
puts "I'M HERE NOW"
#statement = model.make_statement(140)
tweet(model.make_statement)
puts "ABOUT TO TWEET"
puts "SHOULD HAVE TWEETED"
end
end

Please close this issue.
I rebuild the model using a .json extension and all is working well now.
Cheers!


Fix URL generation

There are several reports of issues with the URL returned by Inertia as a result of #333.
#333 was created to support the X_FORWARDED_PREFIX header value which can be added by reverse proxies that serve a web request under a sub-path that isn't normally visible to the underlying webserver.
Unfortunately, it seems to have created a scenario where a sub-path can be doubled up. I haven't been able to figure out exactly what scenario triggers this. There are several proposed PRs to fix it, but nothing I can reproduce. In any case, I'm assuming both $request->baseUrl() and $request->getRequestUri() can contain the same sub path, which we concatenate.
Some solutions propose to use $request->fullUrl(), which constructs a full URL from the various pieces (without concatenating those specific methods). The problem is that this method includes the HTTP scheme, host, and port, and there may be users that depend on Inertia's URL not having those elements (e.g. styling active links). Other solutions combine various other methods to construct the URL, but they need to get into the weeds of adding query strings, etc.
This PR proposes to let Laravel/Symfony handle the URL construction intricacies. Unfortunately, there doesn't appear to be a method that gives us exactly what we need without the scheme and host, so I've just stripped those from the resulting URL.
Alternatives:

#360 (This one manually constructs the URL but misses some logic from fullUrl(). It has a test that replicates the new issue, but not the potential regression.)
#372 (This just restored the old behaviour, which had the proxy issue)
#566 (this is very similar to mine, and given the existing tests pass, I'm happy to accept it)
#576

Closes #359
Closes #421

Glad to see this is being addressed!
I'm just wondering why we can't just make the URL generation configurable with a sensible default (probably the implementation of this PR)? Currently we have to either override both the ResponseFactory and Response classes, or create a custom middleware to do this.

@rojtjo I'm happy to look at making it configurable. May I ask your use case for configuring it differently from how this PR does it?

The only project I ran into the double path issue was when I mounted the app on a subpath using an nginx location block, but that has since been migrated to a subdomain. So unfortunately I cannot confirm if this PR would've actually fixed the issue.
The main issue in my opinion is that there is currently no easy way to override the behavior on app level. At the time I created #503 which solved the issue for me using closures, but it would've probably been easier if we had the concept of a UrlResolver contract so we could just swap the implementation through the container.
Anyways, if this PR does actually fix the issue for everyone there is no need to go through all these hoops to retrieve the correct URL. So I guess it's best to just hold off of making it configurable for now.

I can confirm that this PR fixes the issue. I'm no longer having duplicated URLs, But now i'm getting some weird query params on the URLs.

My nginx conf:

Can confirm the behaviour here is changed. Visiting /?value=te/st in current Inertia works, no redirect happens, the History is updated correctly. But with this change visiting above URL would perform the visit normally, however the GET parameters are now escaped, resulting in the history pushstate updating the URL to /?value=te%2Fst. And I'm testing this on Valet, so no special config or subdirectories. Don't think this is desired behaviour?

So the \Illuminate\Http\Request::fullUrl method uses the Symfony\Component\HttpFoundation::getQueryString method, which in turn passes the query string through Symfony\Component\HttpFoundation::normalizeQueryString.
The Symfony\Component\HttpFoundation::getRequestUri method we previously used returns the path and query string "raw". However, it doesn't include the X_FORWARDED_PREFIX, which is why it was prefixed with Symfony\Component\HttpFoundation::getBaseUrl, but that resulted in URL double-ups.
This feels like a game of whack-a-mole.
The only thing I can think of is to manually construct the URL, similar to how fullUrl does, but without the scheme and host and without query string encoding.

People, just want to note that Laravel does not and will never support projects in sub directories and this should be avoided at all cost: https://laravel.com/docs/10.x/installation#directory-configuration

People, just want to note that Laravel does not and will never support projects in sub directories and this should be avoided at all cost: laravel.com/docs/10.x/installation#directory-configuration

That's just talking about not using a document root on a Laravel app. For example you have a example.com pointed to /home/example and then your Laravel app located at /home/example/laravel-app, which would result in a URL like example.com/laravel-app, that's unsupported yes. But setting a document-root to /home/example/laravel-app or using proxy with the x-forwarded-prefix should work without issues. So with that you could host your app at example.com/laravel-app without issues.

@RobertBoes nope, sorry not supported.

@driesvints That's not what the text says tho? Along with https://github.com/laravel/framework/blob/10.x/src/Illuminate/Http/Middleware/TrustProxies.php#L22 it properly sets the prefix path.

I'm sorry @RobertBoes but Laravel has never intended to host an app other than at the document root and root url. The routing system just isn't intended to be used with a prefix.

So you're saying Request::HEADER_X_FORWARDED_PREFIX does in fact not work in TrustProxies? Why is it an option in the middleware then?

@RobertBoes look, all I'm saying is that if you expect laravel-app of example.com/laravel-app to be the root of your Laravel app that that is not something we support. Url or server wise.

Supported by Laravel, or not, this PR fixed the double path issue. Only issue is that it introduced a bug where the query string is being encoded.


Long sequence pronunciation issue of pre-train model for custom language

Checks

[X] This template is only for bug reports, usage problems go with 'Help Wanted'.
[X] I have thoroughly reviewed the project documentation but couldn't find information to solve my problem.
[X] I have searched for existing issues, including closed ones, and couldn't find a solution.
[X] I confirm that I am using English to submit this report in order to facilitate communication.

Environment Details

same as your requirements

Steps to Reproduce

same as you defined steps

✔ Expected Behavior
I have trained custom language around 500000 steps and vocab size 134 but the inference audio not clear on long sequence, not generated as text have. short sequence text is well generated but issue on long sequence around 100 character.
❌ Actual Behavior
What is the issue of long sequence round above 150 character in a sentence.

the released base model is trained on up-to-30-sec audio dataset.
and the inference code will receive reference audio and clip to max 15 sec if longer
then an up-to-15 sec generation is supported
so if you got a dataset with up to 20 sec samples and train from scratch, the model never see samples longer to 30 seconds
thus, an up-to-5-sec generation is allowed with default setting if you provide up-to-15 sec ref_audio
try modify code in these places:
https://github.com/SWivid/F5-TTS/blob/20aa6a110325c1915b238ba1af03a87494a7ea02/src/f5_tts/infer/utils_infer.py#L288-L317
https://github.com/SWivid/F5-TTS/blob/20aa6a110325c1915b238ba1af03a87494a7ea02/src/f5_tts/infer/utils_infer.py#L61

@SWivid Thank you for your response, but i want to know how many 30 second data need to contain into the custom dataset for long sequence handle. Basically the duration frequency like 30 second data how many time appear in your dataset and that's why long sequence  generation quality is clear and natural.

we use Emilia dataset which is an open-source one, you could just check out everything you want from https://huggingface.co/datasets/amphion/Emilia-Dataset
and also amphion's github repo for the pipeline they use for organizing this corpus

@SWivid another queries regarding long sequence issue, you have already mention  base model will solved the long sequence issue, according you observation i am working on it generate data for custom dataset. but if i trained small variant like f5-tts small model, it will solve the long sequence issue if long data appear into my dataset.

the released base model is trained on up-to-30-sec audio dataset.


where is the codes of failure detecting for master or slave?

I want to learn the logic of failure detecting for master or slave.
thanks very much!

Hehe, welcome to the "mega query". It's a bit daunting at first sight (and on second sight, too).
https://github.com/openark/orchestrator/blob/master/go/inst/analysis_dao.go#L55-L515

thanks very much!
The sql is so complicated ^^

The table has a parent-child relationship with itself. So it's a self join, where on the left you have a mysql server, and on the right you have all the mysql servers replicating from it.


Proposal: update rllib and pettingzoo examples to current release versions

I'm planning to do some testing with the newly released ray 2.3.0 and I noticed the pettingzoo example is 9 months old and still uses gym rather than gymnasium, both should probably be updated. Will create a PR if I can get them to work but figure I'd create an issue first. Appreciate the repo, very excited to use it.

That sounds very useful thank you! I don't know the Ray stuff etc that well, so @duenez is best to comment on any details.

Hello Elliot,
That would be fantastic! Please let me know if there's anything I can do to help.

Hello Elliot,
That would be fantastic! Please let me know if there's anything I can do to help.

Appreciate it, will reach out if I run into any issues. I got a good bit of progress today, although a major blocker to updating the entire pettingzoo tutorial is that stable-baselines3 doesn't officially support gymnasium yet (pettingzoo has migrated as of October 2022). There's some open pull requests which may get merged in the future (stable-baselines3 stable-baselines3-contrib but the only way to get it working currently is manually installing the feature branch: pip install git+https://github.com/Stable-Baselines-Team/stable-baselines3-contrib@feat/gymnasium-support. I can do this in the meantime for development but I'd imagine merging the tutorial should probably wait on an official release.
Edit: looks like there's an API incompatibility with pettingzoo and the sb3 gymnasium support feature branch (link) so the pettingzoo tutorial will have to wait until that is fixed.
I got most of the pettingzoo code adapted though, I chose to use the SB3's frame stacking functionality rather than supersuit because the latter required casting types to float, stacking frames, and then casting back to uint8 for SB3. I've also heard that supersuit will be closed down and its functionality migrated directly into gymnasium and pettingzoo, so I figure it's best not to rely on it too much if possible.
https://github.com/elliottower/meltingpot/commit/64b38e99515ec3b9b7d08ea7211d1aa1abe7cbf9
Luckily ray/rllib 2.3.0 does have gymnasium support so that shouldn't be a blocker, planning to work on that tomorrow.

One minor issue I had with this repo was running the pylint script, didn't find any documentation on it and for some reason running pylint --rcfile .pylintrc didn't work. I'm currently running the pytest suite but it seems to be going very slowly and says it has 4787 items. The .sh install scripts worked great though and the tests ran pretty quickly there.

Yes, some of our tests are very slow (the full substrate tests). I believe we don't run all the tests in the install.sh script.

Is there a reason the pettingzoo env is defined with a separate superclass class _ParallelEnv? I got some issues when trying to use wrapper functions from pettingzoo/gymnasium/SB3 because the base class when you do env.unwrapped is _ParallelEnv.
Moving the EzPickle init line into the regular class (which is of the correct type parallel pettingzoo env) led to an error saying it can’t load the lab2d object from pickle. I don’t have experience with lab2d or the EzPickle function, imagine the specific definition of this ParallelEnv with the two superclasses (one for the env one for EzPickle) is what allows it to load properly?
I’ve gotten most of the functionality for pettingzoo working with melting pot the integration between SB3’s and pettingzoo has been difficult. Seems like there are a few issues that still need to be resolved with defining parallel envs and converting between APIs, so I’m planning to try RLlib on top of pettingzoo instead. RLlib can be used without pettingzoo as your other example shows, but using pettingzoo can be useful as some multi agent models are built on top of it (e.g., RLlib’s implementation of leelazero).

Any idea on how to deal with these warnings? Getting them when running pettingzoo API tests, thinking they may be corner cases that the pettingzoo wrapper doesn't check for @jagapiou @duenez
W @/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/prefab_utils.lua:83] Character	x	not found in the charPrefabMap. Ignoring.
W @/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/base_simulation.lua:225] GameObject 'scene' did not have a Transform component, but explicitly specifying one is strongly preferred. Using a default.
W @/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/base_simulation.lua:219] GameObject 'scene' did not have a StateManager component, but explicitly specifying one is strongly preferred. Using a default.
Also PR is up: https://github.com/deepmind/meltingpot/pull/117, running final tests and will then make a PR on shimmy with melting pot compatibility. I think using shimmy is best as it has comprehensive tests and is used for other API conversions, so I can update this PR to use shimmy once that gets added (as opposed to examples/pettingzoo/utils.py)

Those warnings are not critical. They happen when a substrate doesn't
define a scene, it when their ASCII map contains a character that is not
present in the prefab map. Which substrate is this for? I could just add
those and fix the warnings.
Shimmy seems like a reasonable approach to unify APIs. Thanks.
Finally, regarding the EzPickle, this is needed, I believe, to serialise
the environment. I'm particular, the EzPickle class has a set_state and
get_state pair of functions (
https://github.com/openai/gym/blob/master/gym/utils/ezpickle.py) that I
imagine allow for saving the environment. If you can modernise the code and
not need that, it should be fine.
On Sat, 4 Mar 2023, 00:49 Elliot Tower, @.***> wrote:

Any idea on how to deal with these warnings? Getting them when running
pettingzoo API tests, thinking they may be corner cases that the pettingzoo
wrapper doesn't check for @jagapiou https://github.com/jagapiou @duenez
https://github.com/duenez
W
@/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/prefab_utils.lua:83]
Character x not found in the charPrefabMap. Ignoring.
W
@/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/base_simulation.lua:225]
GameObject 'scene' did not have a Transform component, but explicitly
specifying one is strongly preferred. Using a default.
W
@/Users/elliottower/Documents/GitHub/meltingpot//meltingpot/lua/modules/base_simulation.lua:219]
GameObject 'scene' did not have a StateManager component, but explicitly
specifying one is strongly preferred. Using a default.
Also PR is up: #117 https://github.com/deepmind/meltingpot/pull/117,
running final tests and will then make a PR on shimmy
https://github.com/Farama-Foundation/Shimmy with melting pot
compatibility. I think using shimmy is best as it has comprehensive tests
and is used for other API conversions, so I can update this PR to use
shimmy once that gets added (as opposed to examples/pettingzoo/utils.py)
—
Reply to this email directly, view it on GitHub
https://github.com/deepmind/meltingpot/issues/113#issuecomment-1454306494,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAY7XEO4XO7EPPHRXV6CDJ3W2KGR5ANCNFSM6AAAAAAVLTHO6M
.
You are receiving this because you were mentioned.Message ID:
@.***>

Will get the list of substrates later today when I rerun the tests.
@duenez Do you happen to know if it’s possible to do rendering using PIL or something other than matplotlib? Not sure exactly how the rendering works internally but on the shimmy PR they raised concerns about not wanting to have to import matplotlib (most farama code uses pygame afaik and/or PIL, I think it might be a speed concern?)
And fwiw the pickle code is still used in other farama games like Atari and their devs said it looks fine.

Yes, of course. The observation is produced as a raw numpy array of height X width X channels (3, RGB). Si you can render this to anything you want. One possibility is doing imshow in matplotlib, but equally possible is using PIL.

@duenez thanks, I ended up searching through the code more and finding an example using pygame rendering, which is what is used by all other pettingzoo envs, and mirrored that functionality in the shimmy wrapper.

@duenez is it possible to reset the environment using a specific seed rather than re-initializing a new environment with a specified seed? The new gym API requires that the reset() function take a seed argument, I've been trying to adapt the code but it doesn't seem to be possible without initializing the environment again, which I also have had trouble figuring out how to do.
The only python file I see using seeding is builder_test.py but all of the example scripts and documentation seem to indicate that the higher level substrate.build() or substrate.build_from_config() are preferred to manually calling builder.builder.
I think the correct code should be something like this, but it says that the config is missing they key simulation.
config = meltingpot.python.substrate.get_config(substrate_name)
env = builder.build(config, env_seed=seed)

Have been having a lot of difficulty understanding the inner workings of this code and if it's even possible to do what I want to do. Maybe it's best to change the way the reset() function works internally to allow a seed to be specified?

You need to get the factory:
https://github.com/deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/substrate.py#L70
config = meltingpot.python.substrate.get_config(substrate_name)
factory = meltingpot.python.substrate.get_factory_from_config(config)
env = factory.build(config.default_player_roles)

If you need to explicitly set the seed, we currently don't expose it in the builder through the canonical way to build substrates. The reason we need a factory is because now the config doesn't need to know the number of players ahead of time. This means that we must have an intermediate step when we have the player roles to know what we need to build.
If this is critical, we can think about piping a seed through. What do you think?

You need to get the factory:
https://github.com/deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/substrate.py#L70
config = meltingpot.python.substrate.get_config(substrate_name)
factory = meltingpot.python.substrate.get_factory_from_config(config)
env = factory.build(config.default_player_roles)

If you need to explicitly set the seed, we currently don't expose it in the builder through the canonical way to build substrates. The reason we need a factory is because now the config doesn't need to know the number of players ahead of time. This means that we must have an intermediate step when we have the player roles to know what we need to build.
If this is critical, we can think about piping a seed through. What do you think?
Thanks for the quick reply, makes sense about the number of players. So I take it using the factory doesn’t allow seeding either? It’s a core functionality of the current gymnasium and pettingzoo APIs so if it’s possible to make the reset method take a seed argument that would be awesome. In gymnasium/pettingzoo you have to call reset() on the env before using it and set the seed that way, but if your code doesn’t do that maybe it’s better to pipe the seed through the factory build method and I can make the reset method rebuild the substrate using that.

We are also planning to make a shimmy wrapper for lab2d and gymnasium, do you happen to know if that would run into the same issue?

It depends on the shimmy API. We usually don't have a way to reset substrates without rebuilding the whole thing anyway, because some stochastic choices occur at the creation stage. This is why we have a reset wrapper that does this: https://github.com/deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/utils/substrates/wrappers/reset_wrapper.py

It depends on the shimmy API. We usually don't have a way to reset substrates without rebuilding the whole thing anyway, because some stochastic choices occur at the creation stage. This is why we have a reset wrapper that does this: https://github.com/deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/utils/substrates/wrappers/reset_wrapper.py

Makes sense, I did some more testing and saw that the underlying dm_env code defines _rng to be a certain seed so I tried setting env._rng in the shimmy wrapper but then noticed what you mentioned about the stochastic environment creation. So is it possible to modify the reset wrapper to take in a seed? If you could help out with adding support for seeding that it would be greatly appreciated.
I could look into it myself as well (and potentially make a PR) but I’d need a better idea of where to look and which files to modify.  Might be simpler for someone more familiar with the repo though.

I'll take a look tomorrow.
On Wed, 8 Mar 2023, 17:16 Elliot Tower, @.***> wrote:

It depends on the shimmy API. We usually don't have a way to reset
substrates without rebuilding the whole thing anyway, because some
stochastic choices occur at the creation stage. This is why we have a reset
wrapper that does this:
https://github.com/deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/utils/substrates/wrappers/reset_wrapper.py
Makes sense, I did some more testing and saw that the underlying dm_env
code defines _rng to be a certain seed so I tried setting env._rng in the
shimmy wrapper but then noticed what you mentioned about the stochastic
environment creation. So is it possible to modify the reset wrapper to take
in a seed? If you could help out with adding support for seeding that it
would be greatly appreciated.
I could look into it myself as well (and potentially make a PR) but I’d
need a better idea of where to look and which files to modify. Might be
simpler for someone more familiar with the repo though.
—
Reply to this email directly, view it on GitHub
https://github.com/deepmind/meltingpot/issues/113#issuecomment-1460526107,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAY7XENH7ZTHPYD2KCTSROTW3C5GJANCNFSM6AAAAAAVLTHO6M
.
You are receiving this because you were mentioned.Message ID:
@.***>

Fundamentally, there's a API incompatibility here: a gym reset(seed) cannot be forwarded to a dmenv reset(). So you'll have to rebuild the Substrate/Scenario on reset as you say.
FYI right now you can change the Substrate seed at training-time:
config = meltingpot.substrate.get_config(name)
config.seed = 42
substrate = meltingpot.substrate.build(config, roles)

But that won't help with Scenarios, and they'd need a seed to also be passed to the background bots when building them.
There may also be other sources of randomness (e.g. calls to random in Wrappers) or other sources of non determinism (e.g. python set iteration order varies on each process) that might make things hard.
Finally, rather than adding an optional seed to all build methods, I think it might be better to add a seed (optional so we're still DMEnv compatible) to dmlab2d.Lab2d.reset (https://github.com/deepmind/lab2d/blob/main/dmlab2d/init.py). Since this is where the seed is actually randomized and set. Then our Substrate.reset method can then inherit and use that (and I think we can kill our ResetWrapper).
So it's not a small job. Alternatively, is there a quick hack?

Ignore the seed argument and just let the environment be stochastic (trigger a warning or whatever).
If reset(seed) accepts a default or a "randomize seed" sentinel, raise a NotImplementedError if a fixed seed has been requested.

Sorry update here: it's not currently possible to change the substrate seed. builder takes an env_seed that allows this but it is not currently wired in.

I have locally updated the rllib example to use ray==2.5.1 instead of 2.0.0, should I submit a pull request?

Amazing, yes please!

Just fwiw when I get a chance I’m planning to do a PR updating the pettingzoo example to use shimmy and current pettingzoo, I’ve created official SB3 tutorials for pettingzoo in the most recent release so it would be cool to show this on our docs site as another example using SB3

Fantastic, let me know if there's anything on our side that's needed.

FYI: @dimonenka's PR updating rllib: https://github.com/deepmind/meltingpot/pull/153

Hi, I found a minor typo in the file "example/pettingzoo/sb3_train.py" on line 86.
The environment name should be commons_harvest__open instead of commons_harvest_open. Could you please take a look? Thanks.

Hey all, I’ve been planning on it for a while but finally have time to make a PR updating this tutorial now that PettingZoo has official SB3 tutorials, was thinking about other possible tutorial options such as AgileRL, which now supports PettingZoo directly.
Also would be nice to demonstrate the Shimmy conversion wrapper I made, which was based on this example but is more full featured and tested against every underlying environment in our CI, supports serialization via pickle, etc. Maybe @duenez has thoughts on the best way to proceed?

It looks like I'm late to the party! Is it now possible to make commons_harves__open deterministic? The 'wired in' hyperlink seemed like it was useful, but it looks like that page has been taken down at this point.

It should be possible to make the substrates deterministic. You can see how the seed is set here:
https://github.com/google-deepmind/meltingpot/blob/7de41d2db0e5eca31107312d405e20ff3a7da39e/meltingpot/python/utils/substrates/builder.py#L176-L189
It should work if you just change that logic to pass a fixed seed.

Even with overwriting a seed, the agents spawn in different areas:

Actually, that's where you were doing this. Don't use the zero seed, because that means to resample a seed. Use a fixed, non-zero one (like 42 :) ). If that fails, try setting you python random seed too, as some objects are built on the python side, but I don't think avatars would.

Still no dice:

In builder.py I'm directly setting seed = 42, as well as setting resetting random.seed(42) each time I call random.random().


Wrong datetime on saving to gallery

When saving to the device, the date and time of the image are wrong (not the current datetime)

For example, the one above was saved to the device today

how can I change the date in a file?

Hey @DariusBuhai can you show me source code how did you save it?
Because I have the opposite, I try to save the image with the modified datetime, but it is always the current today date :D


Cilium refuses to start with a network interface ifindex that exceeds max(uint16)

Bug report
If Cilium is started with the --devices parameter set to a specific network interface that has an ifindex higher than max(uint16), Cilium refuses to start by crashing with a fatal error message. Temporary workaround for this is to restart the server where the cilium-agent is trying to run on, but this is obviously not acceptable in the long run.

General Information

Cilium version:

Client: 1.10.3 4145278 2021-07-15T16:11:03+02:00 go version go1.16.6 linux/amd64
Daemon: 1.10.3 4145278 2021-07-15T16:11:03+02:00 go version go1.16.6 linux/amd64

Kernel version

Linux dev1 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:14:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Linux dev2 5.8.0-59-generic #66~20.04.1-Ubuntu SMP Thu Jun 17 11:14:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Linux dev3 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Orchestration system version in use

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"archive", BuildDate:"2021-08-19T21:28:12Z", GoVersion:"go1.17", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.2+k3s1", GitCommit:"5a67e8dc473f8945e8e181f6f0b0dbbc387f6fca", GitTreeState:"clean", BuildDate:"2021-06-21T20:52:44Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/amd64"}

Link to relevant artifacts (policies, deployments scripts, ...): Not applicable.
Generate and upload a system zip: Not applicable.

How to reproduce the issue

You start Cilium with the --devices parameter set to a specific network interface that has an ifindex that exceeds max(uint16)
Cilium crashes with the following message:

level=fatal msg="tailscale0 link ifindex 93715 exceeds max(uint16)" subsys=daemon

(tailscale0 being the network interface in question)

It seems that check was introduced in https://github.com/cilium/cilium/commit/4709f0ba3f1ccfdad5741093ee7e9ea5375d0a16 and the longer term fix is described in the commit message:

The first restriction will go away once we introduce the surrogate id => ifindex BPF map.

So I'm guessing the limitation here is from the ifindex field of ct_entry and ct_state. /cc @brb

I do wonder, why has uint16 been chosen when an ifindex can exceed uint16?

git blame can probably answer that. I suspect the map entry was sized before we started using the interface index.

@clrxbl The ifindex needs to be stored in ct_entry.ifindex. Unfortunately, the ct_entry struct only had u16 available (we cannot easily change the struct, as it would interrupt open connections after the upgrade).

Closing as it duplicates https://github.com/cilium/cilium/issues/16260.


new vagrantfile

This commit allows a user to run dnflow on virtualbox locally. It is most useful in helping me design the docker containers where they will finally rest. Others may find it useful for development purposes.

updated Vagrantfile for rapid prototyping.

@kayiwa is this typical to intersperse packer configuration, etc with the actual application that is being packaged -- in the same repository?  I have definitely seen Vagrant files combined with application repositories, but not Packer.

good question that I haven't an answer for at the moment @edsu I think of them as no different from the Vagrantfile (in that both create a reproducible lightweight environment), but will spend some time seeing what others have done.

created a new docnow-ansible repo.


Reset filters doesn't work properly for array filters

Calling resetFilters from useListingFilters will set every filter key to null which breaks UI if the filter is multiselect.
Proposal
const resetFilters = () => {
    Object.keys(filters).forEach((key) => {
        if (Array.isArray(filtersForm[key])) {
            filtersForm[key] = [];
        } else {
            filtersForm[key] = null;
        }
    });
  };

This was fixed in v1.1.17


add default value functions

Add default value setters for some additional types.
/assign @mfojtik

@soltysh I think you added the linter.  Any idea what's up with it?

@deads2k they've changed the download URL: fix in https://github.com/openshift/library-go/pull/44


[VC] cleanup: add cache test

cleanup: more test and comments
fix minor language trick
Signed-off-by: zhuangqh<EMAIL_ADDRESS>

/lgtm
/approve


lxqt needs upower for support of "powerManagement.enable = true;" in configuration.nix

Issue description
lxqt battery monitor doesn't work even with "powerManagement.enable = true;" enabled in the configuration
Steps to reproduce
On a laptop with battery power, in configuration.nix, add the lines:
services.xserver.lxqt.enable = true;powerManagement.enable = true;
Run nixos-rebuild switch
After starting up lxqt, enable the battery watcher under main menu/Preferences/LXQt settings/Power Management
Battery watcher will complain no battery exists.
Technical details
lxqt is unable to display battery usage without upower. Most other desktop managers tie upower to "powerManagement.enable = true;" by including the following line in their service:
services.upower.enable = config.powerManagement.enable;

$ grep services.upower.enable nixos/modules/services/x11/desktop-managers/*
nixos/modules/services/x11/desktop-managers/enlightenment.nix:    services.upower.enable = config.powerManagement.enable;
nixos/modules/services/x11/desktop-managers/gnome3.nix:    services.upower.enable = config.powerManagement.enable;
nixos/modules/services/x11/desktop-managers/mate.nix:    services.upower.enable = config.powerManagement.enable;
nixos/modules/services/x11/desktop-managers/plasma5.nix:      services.upower.enable = config.powerManagement.enable;
nixos/modules/services/x11/desktop-managers/xfce.nix:    services.upower.enable = config.powerManagement.enable;

I think nixos/modules/services/x11/desktop-managers/lxqt.nix should have this line, too.
Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

system: "x86_64-linux"
host os: Linux 4.14.29, NixOS, 18.09.git.56fb68d (Jellyfish)
multi-user?: yes
sandbox: no
version: nix-env (Nix) 2.0
channels(tim): ""
channels(root): "nixos-18.09pre133621.56fb68dcef4"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs

fixed in pull request.


Maestro studio suddenly stops working

I'm getting this error after interacting few times in meters studio. This error is happening on Android (device and emulator) and iOS
assertion failed [found_x86_offset]: no instruction entry found for jit breakpoint at arm address 0x10c0ec8c0
(TranslationCacheJit.cpp:300 handle_jit_breakpoint)
[1]    23628 trace trap  maestro studio

Thanks for taking the time to raise this issue! There has been a lot of improvements and fixes on Maestro Studio since this was raised so we’ll go ahead and close this one out as part of an issue cleanup - if you still experience issues after upgrading to the latest version of Maestro, please open a new issue with reproduction steps. Thanks again for using maestro!


Complete the getting-started documentation after the build-system upd…

…ate.
JerryScript-DCO-1.0-Signed-off-by: Robert Sipka<EMAIL_ADDRESS>

LGTM

LGTM

I've rebased it with master.


Pyleniumio doesn't install drivers

Hey there, just trying this lib out, looks pretty good, but I don't think it's installing anything in terms of driver or browser (as the docs claims)
You don't need to worry about installing any driver binaries like chromedriver. Pylenium does this all for you automatically :)

So I'm installing this on a docker image (CentOS 8) via pip install pyleniumio but then when I run the tests it cannot find chrome, I ssh-d in to the container and confirmed that chrome is indeed not installed.
>           raise ValueError(f'Could not get version for Chrome with this command: {cmd}')
E           ValueError: Could not get version for Chrome with this command: google-chrome --version || google-chrome-stable --version

You need to make sure your machine (aka the docker container) has the actual browser installed. Pylenium handles installing and managing the driver binaries for you, but the drivers require the their corresponding browsers so they can drive them

I'm closing this issue since there's been no response and this is not a Pylenium issue

Thanks, I didn't notice the reply until now - I eventually gave up on Pyleniumio


CDI-627 test vetoing alternative.

I am waiting on https://github.com/cdi-spec/cdi/pull/320

Merged.


Connects to #291. Button UI in modal to download analysis reports.

Key Changes:
Added button in the multi-omics working group resources modal for users to download the dawg-pac data analysis reports (HMTL files).

lgtm


Support Llama3

Hi @byerose , I think there is no default system prompt for Llama-3 model. I think by default we can just ignore the chat_template args and let it default to Hugging Face's tokenizer.apply_chat_template?


Fix B3LYP functional.

Closes #1480

I think it is acceptable to change the default B3LYP functional. I added an option to configure the default B3LYP. @fishjojo @gkc1000 what do you think? Can we break the backward consistency?

I think it is acceptable to change the default B3LYP functional. I added an option to configure the default B3LYP. @fishjojo @gkc1000 what do you think? Can we break the backward consistency?

@sunqm, I think it is fine as long as one can restore the previous convention.


Regression of artifact URL due to change since pip 23.3

Looks like there is a "breaking" change from pip in this commit, which applies the build_tracker conditionally. The result is that there will be no log info from the build tracker module, including the item like this that is used in pex. It seems as a result of this change, the URL for artifacts from private python package index hit the same issue as raised in https://github.com/pex-tool/pex/issues/2156.
It affects pip >= 23.3. Lower versions seem fine with pex generating proper URLs for those artifacts.
As an example, this is the log from pip 23.2.1 for installing requests:

and this is from pip 23.3:

@tachikoma-li most useful would be log text (instead of screen shots) and an example of the lock file generated with both versions of Pip. Even better, just the full pex command line (and input requirements files, if any). I'm not exactly sure what problem you are encountering. The #2156 issue was moot, since the querystring that user needed was dynamic and changed over time (an auth token with an expiry). Pex currently absolutely does not support dynamic URLs. Whatever artifact URL is written down in the lockfile when it is created must be good forever after or the lockfile must be regenerated. If your lockfile artifact URLs are missing static querystring data they need, that's different and we can use this issue to track a fix. I look forward to hearing more details about your case.

Hi @jsirois, we have some internal Python packages (in both wheel and sdist format) host in the private PyPI. When we try to generate the lockfile with pex, we have some issues with the URLs for those internal packages. In particular, Pex uses the impermanent URLs for those packages which causes the installation to fail later, and the issue only happens with pip >= 23.3.1. With pip 23.2 (or any lower version), the lockfile can pick up the URL pointing to the private PyPI (instead of the impermanent URLs that have the token for authentication), which allows us to use .netrc to access the PyPI for downloading. For example, this would generate the expected lock
pex3 lock create --use-pip-config <internal_pkg> --pip-version 23.2
but the following fails
pex3 lock create --use-pip-config <internal_pkg> --pip-version 23.3.1
From what I understand, pants needs static URLs for downloading packages in parallel when resolving dependencies. But the behavior seems to be inconsistent with different configurations:
pants can generate expected lock for pip <23.3.1, but fail with higher version of pip
however, with higher versions of pip, pants can also generate the expected lock for sdist, but fail for wheel.
For example,  if we run
pex3 lock create --use-pip-config <internal_pkg> --pip-version 24.0 --only-build <internal_pkg>
we can also get the expected lock.
It seems Pex relies on log from pip to get the information of the artifacts. One thing we notice is this change from pip can affect the log. For example, all logs related to build tracker are only available for sdist (which is part of the release of pip 23.3), but not for wheel, which seems to explain the inconsistence mentioned above.
Just wondering, is this the intended behavior, it there any plan to fix the issue for wheel (or the make the behavior for sdist similar to wheel?

@tachikoma-li the Pip change you identify may or may not have to do with the issue you see. That said, it should only affect sdists (and VCS URLs and local project paths) and not wheels - exactly the opposite of the problem you're having if I understand correctly.

Just wondering, is this the intended behavior, is there any plan to fix the issue for wheel (or the make the behavior for sdist similar to wheel?

The Pex integration tests are extensive and include locks of wheels, sdists, VCS URLs and even local projects across the full range of supported Pip versions. As such, you're hitting a corner here that almost certainly involves the specifics of the private PyPI setup you use. Note that Pex CI itself uses a "private" devpi mirror instead of PyPI directly - although there is no auth used.
More to the point, I really need to be able to reproduce your issue to solve it. Can you please provide all of the details from either the "Indirect" or "Direct" lists below?
Indirect:

The full Pip log (add --preserve-pip-download-log to your pex3 lock create command line, observe a line like: pex: Preserving pip download log at /tmp/pex-pip-log.1zil7ou6/pip.log and then grab that fil and search / replace any sensitive info you need to redact.
The full before (good) and after (bad) lock files - again redacted as needed.

Direct:

The private PyPI server implementation you use and its setup.
How you think you're passing credentials to the private PyPI server (you mention ~/.netrc but you also pass --use-pip-config to Pex which is potentially another source of auth info).

@jsirois Unfortunately, we can't provide the full pip log for now. Our internal server will redirect to AWS to fetch the packages when recieving download requests, which seems to have the same behavior as devpi. We tried devpi today and did find something similar to our case, which hopefully can help clarify the issue.
Steps to reproduce:

define the index in ~/.pip/pip.conf as

[global]
index-url=https://m.devpi.net/root/pypi/+simple/

run pex3 lock create --use-pip-config wheel --pip-version 23.3.1 which outputs

{"allow_builds": true, "allow_prereleases": false, "allow_wheels": true, "build_isolation": true, "constraints": [], "locked_resolves": [{"locked_requirements": [{"artifacts": [{"algorithm": "sha256", "hash": "55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81", "url": "https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl"}], "project_name": "wheel", "requires_dists": ["pytest>=6.0.0; extra == \"test\"", "setuptools>=65; extra == \"test\""], "requires_python": ">=3.8", "version": "0.43.0"}], "platform_tag": ["cp310", "cp310", "macosx_14_0_arm64"]}], "only_builds": [], "only_wheels": [], "path_mappings": {}, "pex_version": "2.3.1", "pip_version": "23.3.1", "prefer_older_binary": false, "requirements": ["wheel"], "requires_python": [], "resolver_version": "pip-2020-resolver", "style": "strict", "target_systems": [], "transitive": true, "use_pep517": null}

Note that the URL is the redirected one.

run pex3 lock create --use-pip-config wheel --only-build wheel --pip-version 23.3.1 for sdist, which outputs:

{"allow_builds": true, "allow_prereleases": false, "allow_wheels": true, "build_isolation": true, "constraints": [], "locked_resolves": [{"locked_requirements": [{"artifacts": [{"algorithm": "sha256", "hash": "465ef92c69fa5c5da2d1cf8ac40559a8c940886afcef87dcf14b9470862f1d85", "url": "https://m.devpi.net/root/pypi/%2Bf/465/ef92c69fa5c5d/wheel-0.43.0.tar.gz"}], "project_name": "wheel", "requires_dists": ["pytest>=6.0.0; extra == \"test\"", "setuptools>=65; extra == \"test\""], "requires_python": ">=3.8", "version": "0.43.0"}], "platform_tag": ["cp310", "cp310", "macosx_14_0_arm64"]}], "only_builds": ["wheel"], "only_wheels": [], "path_mappings": {}, "pex_version": "2.3.1", "pip_version": "23.3.1", "prefer_older_binary": false, "requirements": ["wheel"], "requires_python": [], "resolver_version": "pip-2020-resolver", "style": "strict", "target_systems": [], "transitive": true, "use_pep517": null}

Now note the URL is not the redirected one.

run with pip 23.2 pex3 lock create --use-pip-config wheel --pip-version 23.2  which outputs

{"allow_builds": true, "allow_prereleases": false, "allow_wheels": true, "build_isolation": true, "constraints": [], "locked_resolves": [{"locked_requirements": [{"artifacts": [{"algorithm": "sha256", "hash": "55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81", "url": "https://m.devpi.net/root/pypi/%2Bf/55c/570405f142630/wheel-0.43.0-py3-none-any.whl"}], "project_name": "wheel", "requires_dists": ["pytest>=6.0.0; extra == \"test\"", "setuptools>=65; extra == \"test\""], "requires_python": ">=3.8", "version": "0.43.0"}], "platform_tag": ["cp310", "cp310", "macosx_14_0_arm64"]}], "only_builds": [], "only_wheels": [], "path_mappings": {}, "pex_version": "2.3.1", "pip_version": "23.2", "prefer_older_binary": false, "requirements": ["wheel"], "requires_python": [], "resolver_version": "pip-2020-resolver", "style": "strict", "target_systems": [], "transitive": true, "use_pep517": null}`

Note that the URL is not the redirected one.
This is very close to our issue where the redirected URL is impermanent with auth tokens and causing trouble, while the original URL would work. And this would only affect pip version >= 23.3.1 (excluding the sdist case).

@tachikoma-liI thanks, I think your last post starts to help reveal the issue here - its use of ~/.pip/pip.conf / --use-pip-config. The --use-pip-config flag prevents Pex from running Pip with --isolated, so Pip can read its ~/.pip/pip.conf config, but, importantly, Pex does not read ~/.pip/pip.conf. So when you tell Pip (and not Pex), that the index URL is https://m.devpi.net/root/pypi/+simple/, Pex still uses the default index URL of PyPI. As things stand, you need to duplicate that piece of config information so Pex knows. Like so:
pex3 lock create --use-pip-config --no-pypi --index https://m.devpi.net/root/pypi/+simple/ ...

This is a bug, but a long standing one (you can also specify extra indexes in a requirements.txt file and Pip will use those but Pex will not known about them - same bug). The idea of introducing --use-pip-config (which was introduce in September 2023 in v2.1.147) was to enable things like Pip keyring support.
@tachikoma-li now that you know you must tell Pex directly about the indexes to use, does doing so fix your problem?

To explain what I think the issue really is here:
You're using Pants to run pex and Pants does this:

https://github.com/pantsbuild/pants/blob/aaf94df44063168771ade199da52a8fc6c73a84f/src/python/pants/backend/python/util_rules/pex_requirements.py#L354-L360
https://github.com/pantsbuild/pants/blob/aaf94df44063168771ade199da52a8fc6c73a84f/src/python/pants/backend/python/subsystems/repos.py#L15-L25

As such Pants ends up launching Pex like so:
pex3 lock create --no-pypi --index https://pypi.org/simple/ --use-pip-config ...

Which mean Pex ends up launching Pip like so:
pip download --index-url https://pypi.org/simple/ ...

So now Pip is configured with 2 --index-urls - the one from the command line and the other from ~/.pip/pip.conf. When faced with multiple indexes, Pip is notorious for not defining a rule for the precedence they will be used in and that precedence has, in fact, shifted over time (see https://github.com/pypa/pip/issues/9484#issuecomment-764470437 for example). This shifting of precedence over Pip releases (and not the logging patch you suspected) likely explains why things worked for you under one Pip version and not another.

Hi @jsirois, thanks for the quick reply!
 That is, it seems the issue remains for a single index.

@tachikoma-li --index is not single index, it adds an index to the default (PyPI). Use --no-pypi --index https://m.devpi.net/root/pypi/+simple/.
Perhaps this is not clear from the --help:
pex --help
...
 --pypi, --no-pypi, --no-index
                        Whether to use PyPI to resolve dependencies. (default:
                        True)
  -f PATH/URL, --find-links PATH/URL, --repo PATH/URL
                        Additional repository path (directory or URL) to look
                        for requirements. (default: None)
  -i URL, --index URL, --index-url URL
                        Additional cheeseshop indices to use to satisfy
                        requirements. (default: None)
...

@jsirois, thanks. Just tried with --no-pypi --index, but still got the same result as reported above.
That is, pex3 lock create --no-pypi --index https://m.devpi.net/root/pypi/+simple/ wheel --pip-version 23.3.1 --only-build wheel outputs the right URL, but pex3 lock create --no-pypi --index https://m.devpi.net/root/pypi/+simple/ wheel --pip-version 23.3.1 outputs the redirected one.

@tachikoma-li Ok, great - thanks. I repro that at the 23.2 / 23.3.1 boundary. This is enough to work with now.

The fix is released in 2.3.3: https://github.com/pex-tool/pex/releases/tag/v2.3.3


Add desktop release on tag

Desktop app must work properly so that we can include the desktop app in the "Quick start" documentation (instead of just telling users to build the app themselves)

Fix desktop app deadlock when looking for processes
Fix desktop app style different from web app
Add github action to create release + upload MacOS/Windows/Linux binaries on tag

Example:

Uploaded binaries seem to work fine on Windows :)
Not sure why a terminal pops up with the app but that an improvement for another day

As for macos binaries I am clueless as of what is the executable ??

Ah ok you need to start it from terminal with ./DrDotnet.Desktop and then it works. A .dmg would be a nice improvement for later. It seems there is not icon also in the carousel on macos. Fine on Windows.


CI: lint, test and deploy to pub.dev

login pub.dev
verify pub.dev publisher with my domain
dart pub login
cat ~/Library/Application\ Support/dart/pub-credentials.json, copy to GitHub
submit ci yaml

error

docs: https://dart.dev/tools/pub/pubspec#sdk-constraints

https://github.com/marketplace/actions/publish-dart-flutter-package
skip_test ?

error

https://github.com/sinkcup/dart-pretty-number/actions/runs/10902916896/job/30255861452

error

You must have a LICENSE file in the root directory.
An open-source license helps ensure people can legally use your code.
Published packages should have an upper bound constraint on the Dart SDK (typically this should restrict to less than the next major version to guard against breaking changes).
See https://dart.dev/tools/pub/pubspec#sdk-constraints for instructions on setting an sdk version constraint.
line 1, column 256 of credentials.json: Potential leak of Google OAuth Refresh Token detected.

I will remove CI, and publish manually

Message from server: Package name pretty_number is too similar to another active package: pretty_numbers (https://pub.dev/packages/pretty_numbers).
I must change the package name?

new name: trim_zero


Merge atmos specific and terraform help documentation

Draft Reason
This pr is just to spin up conversation about which commands and how we need to handle things.

Is the current wrapping okay? (i don't think so but because the ticket mentions we want a simple solution)
I am checking the commands we support but the better one could be which one we support post the release of next version. So help here would be appreciated.

what
Current:

After this change (this is based on in progress build not final):

Merge help for terraform and atmos
Fade out not supported commands (In progress)

why

We want to help customers show both native terraform commands and atmos terraform commands

references

https://linear.app/cloudposse/issue/DEV-2821/atmos-terraform-help-should-also-show-terraform-help

I wonder why your terminal is not in color

Oh, I see now I didn't realize we were not using cobra's built-in mechanism. Can you see why we are not using that because that means we have a very different implementation for this one off command. I like what you're doing more than what we had. However, it looks nothing like atmos --help

Is the current wrapping okay? (i don't think so but because the ticket mentions we want a simple solution)

We have solved the wrapping issues in Cobra, so if we can get the menu back in there, that would be ideal.


src/butil: correct assign atomic pointer member to be NULL

memset assign value to var without considering var type.
memset can't guarantee atomic operation

Signed-off-by: Liu, Changcheng<EMAIL_ADDRESS>

@changchengx Do you have any problem using memset ? (eg. compile error or unexpected behavior at runtime)

@wwbmmm No. From semantic perspective, memset doesn't consider variable type at all and guarantee atomic operation.

Could you please submit the application scenario of brpc in your company?
See #1640