[1.7.9] High memory uses often killed due to OOM. Then timeout on writes, potentially due to high memory usage. Pretty much unusable.

Steps to reproduce:
List the minimal actions needed to reproduce the behavior.

Issue writes through a CPP client which basically uses writev call.
Using tsi1 index.

Expected behavior:
Low memory usage and no timeouts. Infrequent timeouts already handling through backoff.
Actual behavior:
High memory usage(7.6GB/8GB) followed by timeouts and then so damn slow that it makes me wait for more than half an hour for a 5 minute execution time. However, at the end it does come back to around 2 GBs.
Environment info:

System info: Run uname -srm and copy the output here
Linux 4.4.0-142-generic x86_64
InfluxDB version: Run influxd version and copy the output here
InfluxDB v1.7.9 (git: 1.7 23bc63d43a8dc05f53afa46e3526ebb5578f3d88)
Other relevant environment details: Container runtime, disk info, etc
Only application running on a VM. 8 GM RAM allocation with 4 CPUs. ESXi is not loaded either.

Config:
Copy any non-default config values here or attach the full config as a gist or file.
Only thing I changed is index from inmem to tsi1. Rest are defaults.

Logs:
Include snippet of errors in log.
Nov 11 15:04:09 ub influxd[12959]: ts=2019-11-11T20:04:09.223591Z lvl=info msg="Cache snapshot (end)" log_id=0J2~r7M0000 engine=tsm1 trace_id=0J30uDEG000 op_n
ame=tsm1_cache_snapshot op_event=end op_elapsed=6733.914ms
Nov 11 15:04:12 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:01 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 67987233-04be-
11ea-8188-000c292620c0 10780042
Nov 11 15:04:12 ub influxd[12959]: ts=2019-11-11T20:04:12.437295Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:20 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:01 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 204 0 "-" "-" 67c1877f-04be-1
1ea-8189-000c292620c0 18080219
Nov 11 15:04:27 ub influxd[12959]: ts=2019-11-11T20:04:27.909117Z lvl=info msg="Cache snapshot (start)" log_id=0J2~r7M0000 engine=tsm1 trace_id=0J30vlXG000 op
_name=tsm1_cache_snapshot op_event=start
Nov 11 15:04:29 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:10 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 6d0ed13b-04be-
11ea-818a-000c292620c0 19034349
Nov 11 15:04:29 ub influxd[12959]: ts=2019-11-11T20:04:29.855898Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:30 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:11 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 6d3d943e-04be-
11ea-818b-000c292620c0 19423632
Nov 11 15:04:30 ub influxd[12959]: ts=2019-11-11T20:04:30.551596Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:34 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:14 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 6f783cd0-04be-
11ea-818c-000c292620c0 19972154
Nov 11 15:04:34 ub influxd[12959]: ts=2019-11-11T20:04:34.840122Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:38 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:20 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 72ba2909-04be-
11ea-818d-000c292620c0 18158716
Nov 11 15:04:38 ub influxd[12959]: ts=2019-11-11T20:04:38.491763Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:46 ub influxd[12959]: ts=2019-11-11T20:04:46.144027Z lvl=info msg="Snapshot for path written" log_id=0J2~r7M0000 engine=tsm1 trace_id=0J30vlXG000
 op_name=tsm1_cache_snapshot path=/var/lib/influxdb/data/demo_new/autogen/31 duration=18234.938ms
Nov 11 15:04:46 ub influxd[12959]: ts=2019-11-11T20:04:46.144947Z lvl=info msg="Cache snapshot (end)" log_id=0J2~r7M0000 engine=tsm1 trace_id=0J30vlXG000 op_n
ame=tsm1_cache_snapshot op_event=end op_elapsed=18235.851ms
Nov 11 15:04:55 ub influxd[12959]: [httpd] <IP_ADDRESS> - - [11/Nov/2019:15:04:39 -0500] "POST /write?db=demo_new&u=&p= HTTP/1.1" 500 20 "-" "-" 7e1783c9-04be-
11ea-818e-000c292620c0 16037474
Nov 11 15:04:55 ub influxd[12959]: ts=2019-11-11T20:04:55.437408Z lvl=error msg="[500] - \"timeout\"" log_id=0J2~r7M0000 service=httpd
Nov 11 15:04:59 ub influxd[12959]: ts=2019-11-11T20:04:59.256131Z lvl=info msg="Cache snapshot (start)" log_id=0J2~r7M0000 engine=tsm1 trace_id=0J30xfz0000 op
_name=tsm1_cache_snapshot op_event=start

Performance:
Generate profiles with the following commands for bugs related to performance, locking, out of memory (OOM), etc.
# Commands should be run when the bug is actively.
# Note: This command will run for at least 30 seconds.
curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all?cpu=true"
curl -o vars.txt "http://localhost:8086/debug/vars"
iostat -xd 1 30 > iostat.txt
# Attach the `profiles.tar.gz`, `vars.txt`, and `iostat.txt` output files.

iostat.txt
profiles.tar.gz
vars.txt

There, have some activity.

I am having the same issue. Is there a fix for this


Make lovejs builds work on Windows

Fix 'KeyError: "There is no item named
'love.js-master\\src\\compat\\index.html' in the archive"'
ZipFile accepts strings for names, but doesn't understand Windows paths
and they get excessively escaped. Force all paths to be Posix paths.
Test (win10)
export my game to lovejs and it is able to build a page!
I haven't tried testing on a non Windows platform.

I've used pathlib more over the past few months and learned about as_posix() which is an easier way of doing what I was doing before.
Still works on windows as it did before!


In-app media control keys doesn't work after switch from different app until manual 'click'

OS: Linux neko 4.17.3-1-ARCH #1 SMP PREEMPT Tue Jun 26 04:42:36 UTC 2018 x86_64 GNU/Linux
GPMDP Version: gpmdp 4.6.0-2 (ArchLinux AUR)
Issue Descriptions: upon switch to the app via window switcher (alt+tab) or via clicking app icon in the dock (I use latte-dock) media control keys such as space, arrow buttons are not recognized (don't produce any result) until I manually click in the app window with mouse.
Steps to Reproduce:

Open GPMGP
Start any song
Minimize the app/Switch to any other app
Switch back to GPMDP via alt+tab or dock
Press space/arrow keys

This is like 90% reproducible, there are times when they actually work, but I have no idea why. Probably some focus related issues on KDE.
Additional info
Qt: 5.11.1
KDE Frameworks: 5.47.0
Qt: 4.8.7
KDE Development Platform: 4.14.38
Thanks for the great app =).

I think you are probably correct in that the window doesn't gain focus so you are unable to send keys to the window. However I'm fairly certain that's the responsibility of the window manager to focus the window on Alt+tab, etc. I'm not sure we can fix this behavior as the GPM keyboard shortcuts are bound to the window/document, which should be forwarded.

Hmm, @jostrander could you please point me at the code that does this, I may try to debug it if I have time, as gpmdp is the only app that have the problem.

This works as expected on macOS and Windows, I'm going to put this down to a weird WM thing on your linux setup.
The closest thing we have to relevant code for this is our focus proxy logic:
https://github.com/MarshallOfSound/Google-Play-Music-Desktop-Player-UNOFFICIAL-/blob/82c84871163e0949172e6dde7550f6844fb88749/src/renderer/ui/pages/PlayerPage.js#L80


[android] Remove demo mode

Demo mode is starting to really get in the way.  Let's axe it.

hooray :raised_hands:

LGTM.


Fix Daily

I replaced the UNIX timestamp with the star count.

Please rebase this branch

I have now rebased this branch and changed the commit message.


Favor 'go install' over 'go get' for installation

go get is deprecated in terms of installing executables. $ is removed so that folks can just copy the whole block and run it.
Not opening an issue first because this PR is small. Signed the CLA. Waiting for approval.

@cweill I can see the signed CLA in my account now, and I'm wondering if there's a way to re-trigger all the checks again, thanks!

Can you please check why the tests are failing? Sorry I don't have much time to investigate myself.

@cweill Please take a look at #168, thanks!


Provides whitelisting mode

All my attachments (images, PDFs, office files, etc.) are stored in one directory, and only need to clean up attachments in this one folder. Is it possible to provide a whitelist mode instead of an excluded folder mode?

Hi, this should indeed be possible.
I remember this was requested in the original plugin as well.
https://github.com/Johnson0907/obsidian-file-cleaner/issues/11
I'm going to think about the implementation details over the weekend and get back to you.

@chenmaoran this has now been implemented and is available in version 0.8.0


Have you tested how much video memory will be used when using the GPU?

HI,Have you tested how much video memory will be used when using the GPU?

What do you mean video?

HI,Have you tested how much video memory will be used when using the GPU?

video memory 显存

I remember it would cost around 9000MiB GPU memory when testing a 4K image. As for the training, around 11000MiB for 2 batch_size * (768,768,3) resolution image.
Not sure for this, just a rough estimation.

ok,thanks


GitHub Raffle at GitHub + Vercel AI Strikes at AWS re:Invent - Leave a comment to enter

Random Name Picker for GitHub prize draw at the AI Strikes reception at AWS re:Invent!

:pray: First, huge shout-out to @icelam for a great and simple name picker we could fork and use for the GitHub booth!
:rocket: This site is deployed to GitHub Pages using GitHub Actions. The Actions workflow pulls entrants from commenters of a GitHub Issue.
:ticket: To enter the draw, add a comment at the bottom of this issue 👇
🏆  The prize will be drawn toward the end of the event on 11/28
Check out the original README

A couple of T&C's:

Entry into this raffle will NOT constitute your consent to be contacted for anything else
We will not be adding you to our marketing database (but if you want to subscribe to our newsletter, you can do so here)
The raffle prize cannot be swapped or changed,
The winner of the prize at the AI Strikes reception at AWS re:Invent MUST be present at the time of drawing to claim their prize, otherwise we will redraw!

Good luck!

Techtaka

GitHub is outstanding

Keep them coming people, you're all in the draw!

GitHub is great!!!

Let's go github

Github is cool

give me the shoe pleeeeeeeaaase

Thx for the fun event!

Awesome sneakers!

Good times!

Lfg github

@mariolpgz @snewman1977 @josephyi @canzhen @alainajaffer @Antoniorios17 @judisyau @yongjip @Amal-David @adrika912 CONGRATS!!! You are the winners of the custom GitHub Converse sneakers and bowling shirt from the AI Strikes bowling reception on 11/28. If you are still at Brooklyn Bowl, please come to the hostess stand to get your shirt and so we can collect your info for the shoes. If you are not, please visit the GitHub booth during AWS re:Invent (1196) before Thursday at 12pm to collect your prizes.

Sneakers please


Add language selector

Is your feature request related to a problem? Please describe.
Related to #11, we need a way to define current session language.
Describe the solution you'd like
A combo with all available languages will be ideal.
Also think if registered users should store and reuse their language.

Maybe it's better to just provide a combo to change session language for now :)


Bootstrap modal show instance method is called when datetimepicket field is focused

Hi there,
I've tracked down to the show method
347     this.element.trigger({
348        type: 'show',
349        date: this.date
350      });

If I comment these lines all seems to work well again.
Any clue where what I'm doing wrong.
Thank you.
Bootstrap 3.36
JQuery 2.2.3

266
This issue seems like resolved this problem already, but I don't know why the code changed again. Could anybody help? :(

Should be fixed in the latest bootstrap version


fix(esco-content-menu): use isFavorite to determine if portlet is favorited

resolves #405

Thanks @jonathanmtran for the issue a the fix, but I completed your change with the #409

Thanks @jonathanmtran for the issue a the fix, but I completed your change with the #409


Tree model sort

#503 Should be done afaik.
At least the TreeModelSort is working now :)

and generally, please check if return value is nil and then put the return value into glib.Take(unsafe.Pointer(c))

before storing it into a new pointer to a new struct, or
when calling a wrap* function

examples:
// AssistantNew is a wrapper around gtk_assistant_new().
func AssistantNew() (*Assistant, error) {
	c := C.gtk_assistant_new()
	if c == nil {
		return nil, nilPtrErr
	}
	obj := glib.Take(unsafe.Pointer(c))
	return wrapAssistant(obj), nil
}

// BuilderNew is a wrapper around gtk_builder_new().
func BuilderNew() (*Builder, error) {
	c := C.gtk_builder_new()
	if c == nil {
		return nil, nilPtrErr
	}
	obj := glib.Take(unsafe.Pointer(c))
	return &Builder{obj}, nil
}

I hope I've got every point. If not please tell me what's still not ok :)

Looks good to me, unless the travis pipeline tells you otherwise ;)

looks good to me as well
still not 100% sure when to use runtime.SetFinalizer(), though...

still not 100% sure when to use runtime.SetFinalizer()

When there is something to free/unref/cleanup.
Any of these criteria:

it is a glib.Object of some sort
it is a C-style array (e.g. char *)
it has a custom free() function (e.g. TreePath or glib.List)

TreeIter is just a plain struct, and similarly a lot of types like GdkWindowEdge are enums, i.e. integers - these don't need a finalizer. Those are taken care of automatically, should they ever escape into the heap memory.

@mfulz: the discussions should be resolved, no? if so, can you please mark them as such?
then @andre-hub can remove the labels and do a merge

i merged this now and all open questions and justified objections should be done on the common code base. there are some good mappings here, it would be a pity if they are not available to everyone.


How to parse out dormant addresses?

I'm hoping to parse through the BTC blockchain and parse out every address that hasn't sent funds for at least 8 years. Does anyone have any recommendations for how to go about this? Thanks in advance!

Is this for fun or profit? You could use rust-bitcoincorce-rpc to get blocks and then parse each transaction in each block and build up a data base then query the database. At that point you've basically written an indexer so you could look at electrs and see if it can be coerced into doing what you want. Just thoughts, I've never done such a thing.

Using https://github.com/RCasatta/blocks_iterator should be quite easy and fast.

This is where I started using rusty blockparser - my concern is that bigger addresses receiving dust will eliminate a lot of addresses as being too new. Am I thinking about this wrong?

Is this for fun or profit? You could use rust-bitcoincorce-rpc to get blocks and then parse each transaction in each block and build up a database then query the database. At that point you've basically written an indexer so you could look at electrs and see if it can be coerced into doing what you want. Just thoughts, I've never done such a thing.

Its a personal interest project to dip my toes in working with the blockchain! I'm working on testing the speed of this method - the bottleneck I'm seeing is that I need to call the bitcoin-cli for each transaction to call getrawtransaction and see input addresses.

This is where I started using rusty blockparser - my concern is that bigger addresses receiving dust will eliminate a lot of addresses as being too new. Am I thinking about this wrong?

You are :) receiving funds has nothing to do with the age of any other funds, even if they were sent to the same address.
This may be helpful: there is no such thing as a "from address"


Filters

Are the filters reggex ?
It would be really usefull to filter a really too large all-world iptv list !
if not, is it possible to add the feature ?
Thank you for the good work

Ah,
After some trys and reading the source code, it's a simple search java.
So the search work for what i need but ...
Would it be possible to
-> Or leave the m3u8 channel number when found ?
-> Or put an increasing channel number when found ?
TX !

Regarding the similar name feature you are asking; having an example will help me to implement a better approach.


[Bug] Performance Improvement through prompt optimization

There are some problems when using the tool:

Sometimes the task is not started with nmap.
Use Ctrl-C on the Windows terminal does not load status bar.
The test generation session cannot capture the right task. Example: https://chat.openai.com/chat/1a0f5b9f-9e94-4ac8-8db6-d8d1b88951c0

Further verification shows that the reasoning module has to be run with GPT-4. Will optimize the prompts.


Update timezone on myProject page

Purpose
Update timezone and locale for users when they visit the myProjects page.
Changes
Use the user profile endpoint to update locale and timezone if they are different than the user's once they visit the my projects page
Side effects
more accurate timezone and locale for our users
Ticket
https://openscience.atlassian.net/browse/OSF-6538

Since the user is usually redirected to the Dashboard, it would make sense to call ensureUserTimezone on the Dashboard rather than My Projects (note: the name of the page modules is confusing at the moment).

I can make the above change before I merge.


Publishing to chrome web store responds with 400

Failed job run:
https://github.com/stcmz/video-hotkeys/actions/runs/3923067062/jobs/6706428632
No configuration changes since last successful run:
https://github.com/stcmz/video-hotkeys/actions/runs/3519977831/jobs/5941082543
Run wdzeng/chrome-extension@v1
  with:
    extension-id: nhddoifhjbelkmbddllomdnblibdanfe
    zip-path: ./extension.zip
    tester-only: false
    client-id: ***
    client-secret: ***
    refresh-token: ***
Start to publish extension to Web Store.
Start to refresh access token.
Error: Web Store API server responses with error code: [4](https://github.com/stcmz/video-hotkeys/actions/runs/3923067062/jobs/6706428632#step:3:4)00
Error: {"error":"invalid_grant","error_description":"Bad Request"}
Error: Request failed with status code 400

Turned out the refresh token needs to be updated often. No problem after I replaced it with a new one.

I encounteted the same problem too, but have no idea why we need to frequently update the refresh token. Accroding to the doc the refresh token should never expire 🙄🙄

I encounteted the same problem too, but have no idea why we need to frequently update the refresh token. Accroding to the doc the refresh token should never expire 🙄🙄

According to https://developers.google.com/identity/protocols/oauth2#expiration the refresh token expires in 7 days if the GCP project with an OAuth consent screen is configured for an external user type and the publishing status is "Testing".
So I published my GCP project to Production and hopefully it will never expire again.
The publishing should take effect immediately and requires no review since it is actually an empty project.


[15721] Query Rewrite

This PR adds support for query rewriting optimizations that are oblivious to the cost model in Peloton.

Transitive predicates (optimized in OptimizeLoop)
Constant folding (optimized in PostgresParser).

We implement query rewrite rules using the current optimizer framework:

Transitive predicates: we implemented this rule by adding a new step to the beginning OptimizerLoop using a TopDownRewriter. We added rules to take predicates from nodes, generating all possible predicates by taking a transitive closure and add the new ones;

Trivial predicate simplification: we used the same framework as in the transitive predicates rewrite. This method removes trivial predicates like a = a, some of which are generated by the transitive predicates rule;

Constant folding can transform queries like select * from table where t.column1 = 3*(2-1) to select * from table where t.column1 = 3. These transformation are done in parser level instead of optimizer level because of implementation simplicity. And we don't really lose any information.
It support 3 type of constant folding

Operators like +-*/
Comparison operators like >=, NOT, is null,
Conjunctions : AND OR.

Closing. We will have @Zeninma build this in the fall with a proper rewrite layer.


Multiple event bindings

Hi, i was checking out http://viewmodel.meteor.com/
In Blaze Hooks Example 2 everytime you show/hide "Blaze Button" it stacks another event handler.

Very good catch. It was adding the blaze_events every time the template was created. It's fixed on 1.7.3.
Thanks for the help.


longRunningRecognize and pollUntilComplete doesn't work with really long files

I'm using this code https://github.com/GoogleCloudPlatform/php-docs-samples/blob/master/speech/src/transcribe_async_gcs.php to upload and transcribe a 1 hour audio file, and the $operation->pollUntilComplete(); function ends but both $operation->operationSucceeded() and $operation->operationFailed() are false, and $operation->getError() is empty.
Tested it in multiple platforms, always using the options to transcribe the file from a Google Cloud Storage bucket. Same code for other audio lengths works correctly.

@iglesiaspablo A few things to try here:

What is the return of pollUntilComplete?

Do you know if your PHP environment has the grpc.so PHP extension installed? You can find this out by running php -i | grep grpc and seeing if there is any config output.

If you're using REST instead of gRPC, try seeing the result from the API. It's possible some important response is getting buried

It's possible that because the audio file is so long (1 hour), the pollUntilComplete function hits a quota limit. Although I'd expect this to show up in getError

Hi @bshaffer. I'll answer according to the numbers you posted in your reply:

$operation->pollUntilComplete() return is false. $operation->operationSucceeded() return is false. $operation->getResult() is null. $operation->operationFailed() return is false. $operation->getError() is null.

php -i | grep grpc result is Additional .ini files parsed => /etc/php/conf.d/grpc.ini, so I think I'm using it.

I'm not sure how to check which one I'm using. Is there a way to be sure of that?

Regarding the other message, I'm not seeing any quota errors in the console, and every other operation I'm running with files that aren't that long is successful. Is there a way to get error logs or something like that for the operations that aren't successful?

This issue may no longer be relevant due to its age. Feel free to re-open.


When Traffic Stats panics, it leave systemd in a weird state

If traffic stats panics and dies, it leaves systemd in a weird state where the process still says its running even though it is dead.  This can make monitoring the service tricky.  If traffic stats panics and dies (which is shouldn't) then the service should be updated to reflect this status.
 traffic_stats.service - LSB: start and stop Traffic Stats
   Loaded: loaded (/etc/rc.d/init.d/traffic_stats; bad; vendor preset: disabled)
   Active: active (exited) since Tue 2018-07-31 20:44:15 UTC; 2 weeks 1 days ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1058 ExecStart=/etc/rc.d/init.d/traffic_stats start (code=exited, status=0/SUCCESS)

Jul 31 20:44:15 cdn-trafficstats-osch2h-01.cdn.comcast.net systemd[1]: Starting LSB: start and stop Traffic Stats...
Jul 31 20:44:15 cdn-trafficstats-osch2h-01.cdn.comcast.net traffic_stats[1058]: Starting traffic_stats:
Jul 31 20:44:15 cdn-trafficstats-osch2h-01.cdn.comcast.net systemd[1]: Started LSB: start and stop Traffic Stats.```

reviewed in 7/13/2021 working group. leaving as is.


Status of Cabot itself

I'd like to run a separate monitoring utility against Cabot just to make sure that Cabot is running OK. I can ping the api endpoint / source of the dashboard for some keywords, which gives me a general state of health of the web/api components.
Is there a way to expose / test the overall health of Cabot, meaning web, beat, worker, MQ and database overall?
Thanks

I believe the /status endpoint should work. Please re-open if you think there should be another kind of healthcheck


Ensure LightClientUpdate returned by REST API do not exceed MAX_REQUEST_LIGHT_CLIENT_UPDATES

Is your feature request related to a problem? Please describe.
In the specification there is a requirement that results returned from /eth/v1/beacon/light_client/updates do not exceed MAX_REQUEST_LIGHT_CLIENT_UPDATES.

The response MUST NOT contain more than min(MAX_REQUEST_LIGHT_CLIENT_UPDATES, count) results.

Has been implemented as part of #4365


[stable-1.11] virtcontainers: Don't set Ctty

The https://go-review.googlesource.com/c/go/+/231638/ commit on Golang
introduced a failure on Kata Containers when the runtime is built with
golang 15.2+.
Fixes: #2982
Signed-off-by: Fabiano Fidêncio<EMAIL_ADDRESS>(cherry picked from commit c56af73d3d142125e0712028be0b9e179e0ff957)

/test

/test

Please do not merge, testing CI


[feature]: custom Group Key API for minting

Related to #343 .
We have support for group keys that include tapscript trees, but no way for users to actually provide a tapscript tree root during the minting process. Specifically:

[ ] Providing a specific group key for an asset during minting.
[ ] Providing a custom witness for group key membership in a batch when finalizing a batch.
[x] Storing the tapscript tree for future use (tracked in #343)

For the first point, we'll need to add a new field to the Seedling and MintAssetRequest structs for custom group keys. We likely also want to mark these seedlings as using a custom group key, so that we require an external witness when finalizing the batch.
For the second point, we'll need to extend the FinalizeBatch call to accept a list of witnesses for each seedling. Otherwise we shouldn't have many changes wrt. creating the minting proofs.
The last point is being tracked in #343.

Depends on #768.
There are a few points in the minting flow where this feature could be exposed:

AddSeedling call, when providing other details about the asset that will become the group anchor
Finalize call, when specifying other batch-wide parameters like feerate
A new call, Sign, to provide a witness for a group key in the batch

Separately, there is the option to mandate that a tapscript tree is stored with tapd before minting begins (or least, with an RPC unrelated to minting). Then the tree could be referenced by root hash in minting-related calls.
More generally, right now we don't support an externally-managed key as the internal key for an asset group key. However, we also can't create a group key without an anchor asset. So we likely want to expand the AddSeedling call to be able to specify a custom internal key.
We'll also need more validation logic around group witnesses, to fail early for incorrect user-provided witnesses, but also to reject batch finalization if tapd can't produce a group witness directly (because a group internal key is externally managed).

During initial implementation, investigation, @jharveyb uncovered, a few areas in which necessary changes will need to be implemented.  An corresponding issue to describe the necessary prerequisites before this issue is started is currently being written.
Roughly the prerequisite are:

[ ] Reordering minter logic
[ ] Exposing group key creation phases

We'll also need more validation logic around group witnesses, to fail early for incorrect user-provided witnesses, but also to reject batch finalization if tapd can't produce a group witness directly (because a group internal key is externally managed).

IIRC, we can only actually validate the group witness once we know the true asset/ID, which right now is when we go to finalize (do the coin selection, etc) and obtain the genesis prev out. As a result, I think we're forced to either just accept all the witnesses in finalize, or accept it in AddSeedling, then only validate later on once we've done the necessary in selection. With the latter option, we could start to lock that genesis prev out sooner in the process, but need to make sure the lease is properly renwed.

Moved discussion on this requirement to #820.

From discussion in  #820, user flow would be:

Submit 1+ assets via MintAsset. Some assets may specify an internal key their asset group, or a tapscript root to be included in their asset group key.
Explicitly fund the batch via FundBatch, with an optional feerate and anchor tapscript tree sibling.
Call ListBatches to display the group virtual TXs associated with each seedling that will be a group anchor or grouped asset.
Select seedlings to produce custom witnesses for and generate tapscript leaves that satisfy the tapscript tree referenced in a seedling submitted earlier.
Construct the correct signing descriptor and pass that, along with the relevant script leaves, trees, and group virtutal TXs, to an external signer that satisfies the lnd SignOutputRaw RPC (or equivalent).
Submit the produced witnesses, along with the assetID of the associated seedling, to the minter via SealBatch.
If all witnesses are validated successfully, call Finalize to finish the minting process.

https://github.com/lightninglabs/taproot-assets/blob/649425831ce5310ff2cca50a7ca85e5245000f4e/asset/asset.go#L1108-L1217 stipulates the properties of the fields that a user, who is creating an external group key, will need to satisfy.  Previously this would have been handled by lnd but now user would need to ensure conformance.

Implemented in #827 , being exposed in #866 .


nrf/saadc: make PositivePin sealed.

What it says on the tin.

I believe this can be closed?

Yup!


Remove region validation support from Observability instances DA

We seem to be validating against a manually maintained list of regions in the DA: https://github.com/terraform-ibm-modules/terraform-ibm-observability-da/blob/0de73b2c79f12ffabfef94e40c670f287e1d414f/solutions/instances/variables.tf#L48
We also seem to be maintaining a list of regions in the ibm_catalog.json: https://github.com/terraform-ibm-modules/terraform-ibm-observability-da/blob/0de73b2c79f12ffabfef94e40c670f287e1d414f/ibm_catalog.json#L133-L167
⚠ We are missing ca-tor from that list (and maybe others?)
This does not scale as it requires a version of the DA to be released any time a support for a new region is added. We should remove this validation and allow service api to fail when incorrect region is passed.
For the ibm_catalog.json, we should use the out of the box region selection picker so we don't have to maintain a list of regions

https://github.com/terraform-ibm-modules/terraform-ibm-observability-da/pull/197


Max records size has been increased to 30k

I think we need to update this document for max records returned limit.
The new limit is 30k

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1ca0191e-5d1b-44a1-afb3-71463164b53e
Version Independent ID: 53952dc6-d596-389c-e1d2-2f845bb59b86
Content: Azure Monitor service limits - Azure Monitor
Content Source: articles/azure-monitor/service-limits.md
Service: azure-monitor
Sub-service: general
GitHub Login: @bwren
Microsoft Alias: bwren

@sachinpanwar Thanks for your comment. We will review the issue and get back to you shortly.

@sachinpanwar I have made changes to the document, once the changes are approved they will live. We will close the issue for now. If there are further questions , please revert and we will be glad to assist you.


Fail to deploy latest version 1.1.2

Hi,
the latest version 1.1.2 that was released 2 days ago fails to deploy from scratch. I've tested the previous version 1.1.1 and it works fine.
Here is the output:
$ terraform plan
╷
│ Error: Unsupported block type
│
│   on .terraform/modules/aft/modules/aft-account-request-framework/backup.tf line 26, in resource "aws_backup_selection" "aft_controltower_backup_selection":
│   26:   condition {}
│
│ Blocks of type "condition" are not expected here.
╵
╷
│ Error: Unsupported argument
│
│   on .terraform/modules/aft/modules/aft-account-request-framework/backup.tf line 27, in resource "aws_backup_selection" "aft_controltower_backup_selection":
│   27:   not_resources = []
│
│ An argument named "not_resources" is not expected here.

According to the comment on the line 25 it seems to be caused by https://github.com/hashicorp/terraform-provider-aws/issues/22595.
Francisco.

Hello, we're actively investigating this issue.
Could you paste your output of running terraform -v?

Sure!
$ terraform -v
Terraform v1.1.4
on linux_amd64

Thank you for the details. Can you please also post the output of terraform providers?

Hey @fjromerom, thanks for helping catch this. We were able to reproduce this, and discovered it was an out of date minimum version for the hashicorp/aws provider.
We've updated the minimum hashicorp/aws provider version in 1.2.1 which should fix this issue. Can you retry with 1.2.1?

Hi @balltrev! I can confirm the new version 1.2.1 works fine and there are no issues.
Thanks for the prompt response :)


Black Screen

Hi,
After build is done and uploaded to the TestFlight in App Store Connect, there is always a black screen in my IAP shop scene only.
Black screen blocks UI, but I still can click behind the Black screen on the button (not seeing the button) and test payment works well.
I tried to disable parts of code to see when black screen appears and it looks like it appears when store.init( transactionListener) happens.
In transactionListener I also had only simple code for tests
local function transactionListener( event )
	local transaction = event.transaction
        if ( transaction.state == "purchased" ) then
       end
end

In an old app (last built on 13 mar 2023) I also have the same IAP code and everything works fine.
In Discord and Forum, I didn’t find any solution.
Tried with build .3697, .3698, .3699
Black screen seen on iOS 17 and 16
Any idea where to dig?

It is almost certainly some Lua exception is happening. Do you have access to a mac? If you do, you can connect your phone with cable to it, then in Console.app you'll see your phone and you can stream messages from it. It's gonna be a lot. You can filter them by your app process

Nope, no mac yet. Tha's why I'm using this App store automation. As i can understand there is no other way to debug this without a mac, right?

I think you can use https://itools-for-windows.en.softonic.com/ iTools for that.

Never really done that personally

Ok, thank you, will try that today

Borrowed a mac to debug this, iTools didn’t work. I see an error in Console.app on my side indeed. Thank you for the tips and for the fast reply.


[add-resource] New Plugin: grav-plugin-tabs

I would like to add my new plugin to the Grav Repository.
Here are the project details: grav-plugin-tabs
The plugin provides four components:

blueprints/partials/tabs.yaml, for inclusion in a theme's blueprint
templates/partials/tabs.html.twig, for inclusion in a theme's template
js/tabs.js, enables the Tab functionality
js/tabs.css (optional), simple built-in CSS

With these, the plugin enables easy creation of tabbed content, no matter the page type (regular or modular). It is aimed mainly at theme developers, as it requires the partial template to be included in another page template to function.
There can be multiple tab sets on any given page and all tabs can retain their state (active/hidden) via URL fragments, so sharing links to a page with tabs will re-open the tabs accordingly. The plugin differs from the Shortcode UI plugin in that it only provides tabs, plus it does not do so via shortcodes.

Added to GPM for next update


Fix inputfield in ff

de facto fixes #414

Deployed in integration servers and checked in firefox: cursor is not jumping but the published needs,a dn their messages, are not displayed. In crome those created both in firefox and in chrome are displayed.


Made various changes to the react snippets.

removed "import React from 'react" as its no longer needed since oct 2020 (React 17)

made rfc export the function created aswell (95% of the time you want to export the function you just created)

created a react component where you can specify the name in case you want to use it inside another component

added the settername to both fields of the useState function (unfortunately I use LuaSnip which does not support regex transformations), so I couldnt test to have the second field capitalized. It should be hard to do but I didnt want to change something I couldnt test myself, should be TODO though

Added $0 to the end of the useState to be in line with the other snippets

Divided useEffect into two parts, one with the return statement and one without. Most of the time youre not using the return statement, so I made this one into "uer" and kept the one without the return statement as "ue"

EDIT: I just removed "import React from 'react'" in a few more places I missed (in places where multiple things were imported)

hi man your pull not the best changes that because i use typescriptreact and i will get diagnostic to import react

hi man your pull not the best changes that because i use typescriptreact and i will get diagnostic to import react

Hmm what specifically are you talking about? I think I just removed react from the imports. Also these changes are only for reactjs, the react for typeacript is in another file, which I don't touch

I am sorry for that because new in react world 😂 i do not see any difference between them

It's okay 👌

thanks!


I18N

I'm planning two points for I18N support:

Configurable static strings (defined in the source code) for templates and stuff that doesn't change.
I18n support in the database from the ground up.

I've been following a common pattern for localized databases, which includes a Locale table to manage locale data (translations, currency, etc).
This will have a slight impact on Respect\Relational queries, which will look like:
$mapper->Product_Model(
    $mapper->Product,
    $mapper->Model(
        $mapper->Model_Locale($mapper->Locale)
    )
);

Hopefully we will automate locale-based queries in the future so we don't have to write them explicitly.

:+1: for localized database. This approach is not explored by e-commerces of today, all work with static strings. But we need to review this, I think this approach require a lot of servers.


Typo in argument name - tires/tries

Hi,
In sftpretty/init.py:716 there is a typo in the tries argument which causes the call to putfo() to raise an exception

retry() got an unexpected keyword argument 'tires'

It's a wheel shame I missed this. Been meaning to compensate for the lack of non-local upload testing in the current suite. This also underscores some additional holes in the current tests the need to be filled. I will dedicate a PR to this in the near future. Thanks for finding this!

Happy to help!


Create e2e tests

A major enhancement of complex tools need e2e tests to make updates of the component, dependencies  and depending workloads not error prone.
I remember for example we had an issue that a wrong metrics object let controller-manager panic. I would like to see an error before we update for example Kubernetes. :)

Are there any particular testing tools or frameworks used in general for k8s controllers or operators? I have some basic test setup I'm putting together here (https://github.com/kedacore/keda/tree/master/tests) just using shell scripts. But I'm curious if there's a more conventional way for kubernetes projects in general.

@ahmelsayed We have some of our tooling tested in https://github.com/zalando-incubator/kubernetes-on-aws/tree/dev/test/e2e. I think it would be quite nice to have more projects adapt the kubernetes e2e tests, such that users can have test infra that actually tests all the tools they use.

Thanks @szuecs. After looking at few options I decided to go with Typescript for tests and using the node client for kubernetes or shelljs+kubectl. I added a readme here https://github.com/kedacore/keda/tree/master/tests and an E2E test for azure-queues scalers. Will slowly add tests for the rest of the event sources next.


Ansible request for OpenSSL 1.1.1k on Win build machines

ref: #2083

[x] Missing install
[ ] Bug in ansible playbook
[ ] Request for new playbook addition

Details:
As the title suggests. Checklist:

[ ] build-alibaba-win2012r2-x64-1
[x] build-alibaba-win2012r2-x64-2
[x] build-azure-win2012r2-x64-1
[x] build-azure-win2012r2-x64-2
[x] build-azure-win2016-x64-1
[ ] build-ibmcloud-win2012r2-x64-1
[x] build-ibmcloud-win2012r2-x64-2

Having an issue with build-ibmcloud-win2012r2-x64-1:
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ChunkedEncodingError: ("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer'))
fatal: [build-ibmcloud-win2012r2-x64-1]: FAILED! => {"msg": "Unexpected failure during module execution.", "stdout": ""}

Had this error twice, once whilst building VS2013 32-bit and VS2013 64-bit.
I'll delete those failed build directories and try the next machines.

i've deployed on the two alibaba machines via AWX. How's the ibmcloud machine looking?

Still trying. Getting some timeout issues too - thought that may be due to me closing the lid of my laptop

It failed again - Just doing it manually, it'll be easier.

LMK when it's ready - would be good to verify before the nightly pipelines

All machines done, closing the issue


Code completion behaviour differs for exported records

Given the following record in point.res:
type point = {
    x : int,
    y : int,
}
let p = {x : 5, y : 10}

I see the following completions when typing p. in point.res:

But when I got to use p in another file, it doesn't should any completions.  When I force the completions to show up it shows me a list of all identifiers VSCode knows about:

Opening Point has no effect on this behaviour.

This issue is not exclusive to records, objects suffer from it too.

It looks like the server calls out to bsc.exe.  I'll try to see if I can repro the issue with bsc.exe itself.

Thanks for the report.
Indeed, currently Point.p. is transformed to p. by determineCompletion and the results are not found.
Some internal logic is missing to handle these cases.


unify floating point properties usage

From @bob-carpenter on February 4, 2015 20:53
We are currently mixing our own versions of isinf and isnan with ones called from Boost.  What I think we should do is define our own version of isnan and isinf in stan::math.  These will delegate to Boost's floating-point classifiers for primitive values and use value_of_rec to pull out primitives from our autodiff types.
The reason to call Boost for primitives is that it's more portable:
http://stackoverflow.com/questions/2249110/how-do-i-make-a-portable-isnan-isinf-function
At the same time, we don't just want to use Boost's with our own numeric_limits, because that will potentially overallocate var and fvar instances:
http://www.boost.org/doc/libs/1_57_0/libs/math/doc/html/math_toolkit/fpclass.html
This also clarifies that Boost uses the C99 macros if available, otherwise it uses std::numeric_limits.   The problem we've run into with std::numeric_limits<var> is that it puts an extra variable on the stack;  see issue #1263
Copied from original issue: stan-dev/stan#1267

From @bob-carpenter on February 28, 2015 17:39
We should skip the Boost versions and just use our own isnan and isinf functions that will pull out values with templating rather than trying to go through numeric_limits.
This one's going to be important for managing memory, especially in forward mode, where there's this
 fvar(const TV& val, const TD& deriv) : val_(val), d_(deriv) {
        if (unlikely(boost::math::isnan(val)))
          d_ = val;
      }


Definition of network interface

closes #24

Some open questions:
How to relate  CoreConfiguration with NetworkConfiguration ?
We can put one inside the other or write a type like
data SeathConfiguration a b c d e f = SeathConfiguration {core::CoreConfiguration a b c d e f, network,NetworkConfiguration}

and then use it as base for the general SeathMonad or we can restrict the SeathMonad to just network or delete it an just pass around NetworkConfiguration (but we are boing to update it a lot in seath.
We need to include a Leader query?
Shall we add the IP of the user to the UserAction to simplify the work of storing it?

The latest are related to this comment
Also, part of the intention is to:

have a variable number of participants in plutip test.
Define a replacement for Seath.Test.Examples.Addition.ContractSeath(mainTest) that uses the new interface.
Tried to avoid changes that conflict with the branch misha/demo

Some changes that I still want to do:

Change the name of Seath.Core.HandleActions to something like ActionsProcessing or ChainBuild
Define how to use mix CTL and Quickcheck (most probably getting rid of Quickcheck or running it and lift the result to Contract, but we won't be able to run property testing, that can be a problem to solve later in stage 3)

Remaining work :

Solve all TODOs
fully write Seath.Examples.Addition.ContractSeath.newMainTest
Remove Seath.Examples.Addition.SeathSetup

Something I still don't know how to do is the interaction between the seath network part and the CTL part.
Using a SeathMonad we can have
-- env = LeaderNode | UserNode
newtype SeathMonad env a = ReaderT env Aff a

And then move between them by converthing SeathMonad and Contract to Aff and then use liftAff
Another option is to:
newtype SeathMonad env a = ReaderT env Contract a

Then we can drop some parameters of the configuration for the network. A problem with this is that we would misuse Contract as we principally would want it


Update jdm's ssh key

This change is

Yay for stronger keys!
@bors-servo r+

:pushpin: Commit 02d7f42 has been approved by aneeshusa

:hourglass: Testing commit 02d7f4225ac204109c747db4e9b1e459c55a3ddd with merge 97c5ba5f5e588f7e32c85ae8f7bf3ec23285d561...

:sunny: Test successful - status-travis
Approved by: aneeshusa
Pushing 97c5ba5f5e588f7e32c85ae8f7bf3ec23285d561 to master...

This is now deployed on all machines.


Upgrade Jenkins LTS Version to fulfill Jenkins Security Advisory 2023-03-08

To fix the issues from Jenkins Security Advisory 2023-03-08, Jenkins needs to be upgraded to LTS 2.375.4 or 2.387.1.
In the master branch, the version is already set to 2.387.1 (commit) .
Do you have any idea when it will be released to https://catalog.redhat.com/software/containers/openshift4/ose-jenkins/?

https://nvd.nist.gov/vuln/detail/CVE-2023-27898
• Jenkins >= 2.394
• Jenkins LTS >= 2.375.4

Done.


num.absolute() modifies the original value

Describe the bug
When method absolute() is called without arguments, it does return the proper value, but also modifies the original value. Found by Michal Portes.
custom_numbers 1.1.3
To Reproduce
Steps to reproduce the behavior:

Define numeral system "abc" and a number "-bc".
Observe on print - number is "-bc".
call num.absolute()
Observe correct return value: "bc"
Observe on print: num value now is "bc"

Expected behavior
Num value must be the same as it was initialized: "-bc".

Fixed and published in version 1.2.0

Fixed and published in version 1.2.0


Invalid content /en/testnets/cardano/get-started/installing-and-running-the-cardano-node/building-the-node-using-nix/

Describe the issue
--nix-extra-conf-files is wrong
./install-nix.sh --daemon --nix-extra-conf-files nix.conf
correct: ./install-nix.sh --daemon --nix-extra-conf-file nix.conf

Thank you for your feedback, resolved in PR #437

Thank you for your feedback, resolved in PR #437


Request: Allow anonymous read-only access to Kargo demo

Checklist

[x] I've searched the issue queue to verify this is not a duplicate feature request.
[ ] I've pasted the output of kargo version, if applicable.
[ ] I've pasted logs, if applicable.

Proposed Feature
I tried to access to Kargo demo page (https://kargo.demo.akuity.io/) but received the following error after authorizing GitHub OAuth app for Dex
Internal Server Error
Failed to authenticate: github: user "mubarak-j" not in required orgs or teams

Motivation
Similar to Argo CD demo page (https://cd.apps.argoproj.io/) it would help interested new users view Kargo capabilities and get familiar with it and what to expect before diving into implementation.
Suggested Implementation
Unless, this is inationally restricted for security reasons (which I'd understand) I think show casing Kargo potential to new users would help drive adoption, interest and more community discussion :)

@mubarak-j that URL is indeed for Akuity's own, private use.


build error for sdk21

there ia a "wasm-ld: error: duplicate symbol: accep" error happends when excute NINJA_FLAGS=-v make package
platform: aarch64
eulerosv2r12
/home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/bin/llvm-ar crs build/wasm32-wasi/libc.so.a
/home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/bin/clang -nodefaultlibs -shared --sysroot=/home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/share/wasi-sysroot \
-o /home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc.so -Wl,--whole-archive build/wasm32-wasi/libc.so.a -Wl,--no-whole-archive /home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/lib/clang/17/lib/wasi/libclang_rt.builtins-wasm32.a
wasm-ld: warning: creating shared libraries, with -shared, is not yet stable
wasm-ld: error: duplicate symbol: accept
>>> defined in build/wasm32-wasi/libc.so.a(accept.pic.o)
>>> defined in build/wasm32-wasi/libc.so.a(accept-wasip1.pic.o)

wasm-ld: error: duplicate symbol: accept4
>>> defined in build/wasm32-wasi/libc.so.a(accept.pic.o)
>>> defined in build/wasm32-wasi/libc.so.a(accept-wasip1.pic.o)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [Makefile:514: /home/shulin/17/wasi-sdk-wasi-sdk-21/build/install/opt/wasi-sdk/share/wasi-sysroot/lib/wasm32-wasi/libc.so] Error 1
make[1]: Leaving directory '/home/shulin/17/wasi-sdk-wasi-sdk-21/src/wasi-libc'
make: *** [Makefile:111: build/wasi-libc.BUILT] Error 2

please help me so solve this problem

Are doing an incremental rebuild in an existing build directory? If so, could you try deleting the build directory and trying again?

Are doing an incremental rebuild in an existing build directory? If so, could you try deleting the build directory and trying again?

I've try'ed make clean which delete the build folder, but the same result .

@sunfishcode thank you .  I found there'is a build folder under wasi-libc， delete it, and it worked .

solved


Change header order to "Concepts, Tasks, Tutorials"

https://kubernetes.io/docs/home/ suggests:
Concepts, Tasks, and Tutorials

The Kubernetes documentation contains a number of resources to help you understand and work with Kubernetes.
Concepts provide a deep understanding of how Kubernetes works.
Tasks contain step-by-step instructions for common Kubernetes tasks.
Tutorials contain detailed walkthroughs of the Kubernetes workflow.

This is a logical order and I've rearranged the header to match.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For 1.7 Features: set Milestone to 1.7 and Base Branch to release-1.7
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NOTE: Please check the “Allow edits from maintainers” box below to allow
reviewers fix problems on your patch and speed up the review process.
Please delete this note before submitting the pull request.

This change is

@steveperry-53 I agree w/ this. Do you concur?

Ok, I'm going to merge this.


Rlp v2 rebase

I did a merge myself, and came up with some differences, will leave comments inline

To others interested in revieweing this PR:
git diff remotes/origin/rlp-v2-update-main..remotes/KevFan/rlp-v2-rebase


Define "default" (and only) command to execute

It'll be nice to have a "default" global command that will be execute, if there are no other commands defined in the interface.
We have ton of utilities that are a "single purpose" ones. They receive a bunch of flags but have only one command to execute

I personally want a mix of the two.
Ideally I want to be able to take in a wildcard argument without passing anything else in.
Like I'm defining shortcuts that can be used later as a wildcard.
gli WILDCARD
gli standard-cmd

Is something like that possible in GLI?

What does WILDCARD mean in this context?
All that said, GLI really is designed to always be given at least one subcommand.

WILDCARD would be an argument, any text that is passed in that doesn't match a predefined command

Ah, ok.  It's not really designed for that but you might be able to implement it yourself.  For reference, here is how GLI locates a command based on the command line:
https://github.com/davetron5000/gli/blob/dd79e12a041aed8791a4ba9c8ec9a875075914d1/lib/gli/command_finder.rb#L15-L36
You could rescue UnknownCommand and do something else.  If you implement the on_error handler, you can check to see what exception you got. If you can handle whatever the value is, kick off your code and return false.
The one thing you couldn't do easily is detect the name of the unknown command without parsing the message of that exception.  I suppose UnknownException could be modified to include an attribute that held the command name it could not find.

closing as not a bug in GLI (though I totally understand the identified need - GLI just isn't the right lib for this)


Version incorrect in the instructions

The .net 5 specific instructions, under creating the solution indicates in step 4 that the user should select version 3.0 and not leave it as version 5.0.
Do the instructions really require version 3 and a downgrade of the .net version, or should the instructions be updated?
Specific text reads:
"Select the .NET Core target framework and ASP.NET Core 3.0. Select the API project template, and select Create."

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5240866d-d0a1-f0dd-7b2c-203e1d69bc4b
Version Independent ID: 83a5cb3b-23bc-4f0c-cd1c-1d1ffeb1131a
Content: Create a web API with ASP.NET Core and MongoDB
Content Source: aspnetcore/tutorials/first-mongo-app.md
Product: aspnet-core
Technology: aspnetcore-tutorials
GitHub Login: @prkhandelwal
Microsoft Alias: scaddie

@Smith-Rick The document needs to be updated.

Filters "83a5cb3b-23bc-4f0c-cd1c-1d1ffeb1131a" is:open

@Smith-Rick  any chance you can update this doc?


chore: consistently include code in block error metadata

Motivation
Follow up to https://github.com/ChainSafe/lodestar/pull/6146, noticed we are omitting error code in some block errors but those add more context to the error message itself without having to look into the stack trace which does not show up when filtering logs by slot in grafana.
Description
Consistently include code in block error metadata
By default would render type which always include code
https://github.com/ChainSafe/lodestar/blob/9475b8d40224848a6cc193ded54a1d4a3ef42a20/packages/beacon-node/src/chain/errors/blockError.ts#L152-L153
This aligns the implementation with what is currently done for AttestationError
https://github.com/ChainSafe/lodestar/blob/9475b8d40224848a6cc193ded54a1d4a3ef42a20/packages/beacon-node/src/chain/errors/attestationError.ts#L169-L170

:tada: This PR is included in v1.13.0 :tada:


Duplicated global styles

🐞 Bug report

[ ] new
[X] build
[ ] serve
[ ] test
[ ] e2e
[ ] generate
[ ] add
[ ] update
[ ] lint
[ ] extract-i18n
[ ] run
[ ] config
[ ] help
[ ] version
[ ] doc

Description
The styles that are in styles.scss are duplicated only in production.
🔬 Minimal Reproduction
Add styles in styles.scss and then run "ng build" you see that the styles are duplicated in document

This is working as expected.
Critical styles from the global stylesheets are inlined in the HTML file to reduce render blocking requests and improve FCP and LCP.


Simplify Combine/CombineN vector construction

Currently we have Combine and CombineN. We should have just one (called Merge) but we need to check that it does not affect the performance anywhere.

Close as issue has been addressed


pmbootstrap: move to by-name, add update script, update to 2.3.1

Description of changes
Replaces #312969

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 315082 run on x86_64-linux 1

2 packages built:

pmbootstrap
pmbootstrap.dist

Result of nixpkgs-review pr 315082 run on x86_64-linux 1

2 packages built:

pmbootstrap
pmbootstrap.dist

Thank you!


rspec-puppet test coverage for Debian

PR #69 added an initial version of Debian support. It was merged to the deb_fork branch, but needs to be cleaned up before it's merged to master.
This issue will track adding rsepc-puppet test coverage.

Lots of good progress on the deb_fork branch.

It took many not-linked commits, but as of c5f696bdc32f2bc547c606495cf264c513bbdbd7 I think the refactoring of rspec-puppet tests to accommodate Debian-based OSs is solid.


[9.0.1xx/preview2] R2R test failures: a valid runtime package was not found

The Microsoft.DotNet.SourceBuild.SmokeTests.BasicScenarioTests.VerifyScenario tests are failing for R2R scenarios:
System.InvalidOperationException : Failed to execute /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/.dotnet/dotnet publish --self-contained true -r linux-x64 /p:PublishTrimmed=true /p:PublishReadyToRun=true /bl:/vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/logs/BasicScenarioTests_Console_CSharp-publish-self-contained-linux-x64-trimmed-R2R.binlog -o /vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/projects-202402272116299402/BasicScenarioTests_Console_CSharp/bin/publish
Exit code: 1
Determining projects to restore...
/vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/.dotnet/sdk/9.0.100-preview.2.24127.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(90,5): error NETSDK1094: Unable to optimize assemblies for performance: a valid runtime package was not found. Either set the PublishReadyToRun property to false, or use a supported runtime identifier when publishing. When targeting .NET 6 or higher, make sure to restore packages with the PublishReadyToRun property set to true. [/vmr/test/Microsoft.DotNet.SourceBuild.SmokeTests/bin/Release/net9.0/projects-202402272116299402/BasicScenarioTests_Console_CSharp/BasicScenarioTests_Console_CSharp.csproj]

This was introduced by https://github.com/dotnet/installer/pull/18792
Example pipeline

Interestingly, these test failures are intermittent. I see two different pipeline runs with different results.
Build 1 (internal only):
Fails in CentOSStream9_Offline_MsftSdk_x64 and Ubuntu2204_Offline_MsftSdk_x64
Build 2:
Fails in Ubuntu2204Arm64_Offline_MsftSdk_arm64

[Triage] The bootstrap was reverted.  If this surfaces with the next bootstrap we should open a new updated issue.


Continue working on Player Details

Add The Following

Matchmaking Stats
COTD Stats (win percentage, num rounds, simple stuff basically)
Flag (???)
Club Tag (gotta figure out the colour codes, prob Hex so should be simple)

done in db345c4ade6c8e7365965104d092bc16a352ac84


Expose Application to Payday & models

Properly speaking, we should have zero global configuration. Everything should hang from a Gratipay object that we instantiate for gunicorn and in scripts. But for https://github.com/gratipay/gratipay.com/pull/4346#discussion_r102598587 we at least need to make the application object accessible from within Payday.

Also for https://github.com/gratipay/gratipay.com/pull/4346#discussion_r102598587 we need to find a way for models to have access to the application object.

This is escalating into a refactor of all of gratipay.wireup. 😳

The drive here is towards eliminating global configuration.

Okay, I think I've pulled up from the 🐰 :hole:. Close one! 😳

Rebased, was f2d40d456c54db6d7a4b0ff3f0afa68f8e60a7dd.

Rebased and switched base to master, was f8d4078e5a82f98ec464f52cf75767d94b11f44f.

This one's up next, @nobodxbodon, et al.

Bump.

Kinda wish I'd merged this one before running payday today (https://github.com/gratipay/inside.gratipay.com/issues/1022) so that we could test it in relative isolation. I expect we'll have a number of other PRs merged between now and next payday.


Update MB deps

Fixes https://github.com/maputnik/editor/issues/648
From the GL JS v1.10.0 release notes:

Avoid throwing errors when calling certain popup methods before the popup element is created

Demo: https://2446-67726921-gh.circle-artifacts.com/0/artifacts/build/index.html

👍


Provider reconnection topic

Currently, all our providers implement their back-off for the stream connection. But in a buggy way, it only recreates the stream; GRPC handles the reconnection independently. see https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md
This might lead to false assumptions about our configuration and paint the wrong picture regarding what people can configure and what to expect.
languages
GRPC usually already offers a battle-tested reconnection logic. However, the configuration option is not the same for everyone.
For example, sometimes, we can set 3 out of the five defined options, but multiplier and jitter seem to be something we can't correctly set in all of them.
Java
Java has an exponential backoff, which is not exposed for modification like in other languages see https://github.com/grpc/grpc-java/issues/9353
private long initialBackoffNanos = TimeUnit.SECONDS.toNanos(1);
  private long maxBackoffNanos = TimeUnit.MINUTES.toNanos(2);
  private double multiplier = 1.6;
  private double jitter = .2;

https://github.com/grpc/grpc-java/blob/3b39a83621626c844b16e64ec6389511903ee075/core/src/main/java/io/grpc/internal/ExponentialBackoffPolicy.java#L40-L43
findings

notifyWhenStateChanged(...)  registers a one-off callback and you need to re-register again

C++, Python, Ruby, Objective-C, PHP, C#
Python uses the C++ library in the background and, therefore, offers a min and max backoff, and it will be a random value in that range. Values are mapped; see https://github.com/grpc/grpc/issues/25540#issuecomment-793441156. Seems like every language is in sync, but the docs would not suggest this behavior. I found the comment, but I am still looking to find the proper documentation.

GRPC_ARG_INITIAL_RECONNECT_BACKOFF_MS i.e. "grpc.initial_reconnect_backoff_ms" corresponds to INITIAL_BACKOFF from the algorithm.
The other configurable args are -
GRPC_ARG_MIN_RECONNECT_BACKOFF_MS "grpc.min_reconnect_backoff_ms" which corresponds to MIN_CONNECT_TIMEOUT
GRPC_ARG_MAX_RECONNECT_BACKOFF_MS "grpc.max_reconnect_backoff_ms" which corresponds to MAX_BACKOFF

from the grpc c++ docs (maybe i am misintrepreting this docs):
#define GRPC_ARG_MAX_RECONNECT_BACKOFF_MS   "grpc.max_reconnect_backoff_ms"
The maximum time between subsequent connection attempts, in ms.

#define GRPC_ARG_MIN_RECONNECT_BACKOFF_MS   "grpc.min_reconnect_backoff_ms"
The minimum time between subsequent connection attempts, in ms.

#define GRPC_ARG_INITIAL_RECONNECT_BACKOFF_MS   "grpc.initial_reconnect_backoff_ms"
The time between the first and second connection attempts, in ms.

subchannel defaults
https://github.com/grpc/grpc/blob/782814e28006f132880c0aebbc3d0833d515fa61/src/core/client_channel/subchannel.cc#L75-L79
javaScript
deprecated  implementation wrapping C++ client
utilizes C++ in the background, the same as python
grpc-js
Js implementation, here is a list of config properties: https://github.com/grpc/grpc-node/blob/master/packages/grpc-js/README.md#supported-channel-options. It only supports two out of the three in Python used, based on docs

grpc.initial_reconnect_backoff_ms
grpc.max_reconnect_backoff_ms

const INITIAL_BACKOFF_MS = 1000;
const BACKOFF_MULTIPLIER = 1.6;
const MAX_BACKOFF_MS = 120000;
const BACKOFF_JITTER = 0.2;

https://github.com/grpc/grpc-node/blob/263c478c9a0216e1c864850248ce8efffd7c2da5/packages/grpc-js/src/backoff-timeout.ts#L18C1-L21C28
Summary
Ultimately, all of them offer a reliable backoff mechanism but not a unified one. They're somewhat unified but different to configure, if even possible. Hence, whether we want to offer a custom backoff is questionable, especially as it is currently not behaving as it should.

language
initial (grpc.initial_reconnect_backoff_ms)
max (grpc.max_reconnect_backoff_ms)
timeout  (grpc.min_reconnect_backoff_ms)
jitter
multiplier

C++, Python, Ruby, Objective-C, PHP, C#, js(deprecated)
✅
✅
✅
0.2
1.6

js
✅
✅
❌
0.2
1.6

java
❌
❌
❌
0.2
1.6

Implementation-wise, it would be good to delegate reconnection to GRPC as it reduces our complexity and code to maintain. On the other hand, people might rely on it (this ticket is a bug, and we need to fix it).
If we delegate reconnection to GRPC, we should try to configure it, if possible (see table). Furthermore, we should utilize connection listeners (java: notifyWhenStateChanged, python: subscribe ...) to handle our connection state and Wait-For-Ready for our streams to wait for a connection.
Generally, we should define a general approach to handling the connection topic, e.g., reconnect, error state, and best practice, so we sync all our implementations where possible and especially highlight provider implementations that differ from the recommendation.
sidenote
I created this ticket because there have already been many discussions regarding this. Documenting this is quite essential; otherwise, we will talk about this again in a year and start from scratch.

It's a bummer that it's inconsistently implemented in gRPC, but I agree that we should try and leverage as much of their reconnection logic as possible.

update: it seems like although misleading by the naming, python follows the connection strategy define here https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md. but the values are mapped strangely:
https://github.com/grpc/grpc/issues/25540#issuecomment-793441156:

a note worthy link for the defaults of subchannel timeouts https://github.com/grpc/grpc/issues/23861#issuecomment-675607892

so contrary to my first comment, they all implement the same thing, in a well defined manner, but our initial problem, of race condition, due to custom implementation on top still exists.
hence we should maybe drop our own configuration, and support the settings, based on our values, except where it is not workin, but clearly state that this is not possible

so we can't modify the jitter and the multiplier, but it seems like everywhere

language
initial
max
timeout

C++, Python, Ruby, Objective-C, PHP, C#, js(deprecated)
✅
✅
✅

js
✅
✅
❌

java
❌
❌
❌

So my information was most likely due to mixup of retry and reconnect and a recent change of grpc https://github.com/grpc/proposal/pull/452

So we can't modify the jitter and the multiplier, but it seems like we can modify some of the other properties, depending on the language
language	initial	max	timeout	jitter	multiplier
C++, Python, Ruby, Objective-C, PHP, C#, js(deprecated)	✅	✅	✅	0.2	1.6
js	✅	✅	❌	0.2	1.6
java	❌	❌	❌	0.2	1.6
Maybe i misread something, and somebody should cross-check
To summarize:

We should remove our custom backoff mechanism for connection and add those properties to the connection.
We might want to implement retryConfig for RPC mode
We want to configure the GRPC connection backoff, where we can. (seems like everywhere except java)
We want to document the behavior of our providers.

The good part is that after this weekend's exploration. We can keep our configuration and definition, as the defaults are the same as those used by GRPC. We can remove our own backoff and we can use Wait-For-Ready for our streams.

@aepfli I read the docs and did checked some of the inline ones as well and I can confirm your conclusions.
I agree with @beeme1mr that I think the best path forward is to leverage the underlying reconnect mechanism and sort of "piggyback" on that to re-establish the stream by listening to the connection state. This will mean that our "stream-re-establishment" algo will match gRPC's re-connection , and will be configurable everywhere gRPC's re-connection params are.

i think we can close this issue now, as we also documented this in the spec. wdyt?


Multi streams optimization under indexing?

In ORC Spec:

For columns with multiple streams, the sequences of positions in each stream are concatenated. That was an unfortunate decision on my part that we should fix at some point, because it makes code that uses the indexes error-prone.

ORC v2.0 will be a good opportunity to solve the issue.
cc @dongjoon-hyun @wgtmac @guiyanakuang @pavibhai

Do you have any proposal?

Do you have any proposal?

Yes, I will add it here in a few days.

Yes, I will add it here in a few days.

If it makes sense. I would recommend that we also pen down the specific challenges that arise from this to ensure that we have addressed all of these with the proposal.

Well, it's possible, but I don't think it's worth to try in Apache ORC 2.0.0.
Basically, I hope we can split Apache ORC Format Version from Apache ORC Library Version like Apache Arrow or Apache Parquet project.

Thank you @dongjoon-hyun . Please let me know once we start to split the ORC Format.


HTML5: Add cache buster for html export

Html export works fine for the first time. You can use this for publishing your game on a webserver. But when you publish the game for a second time on your website the end users will not get your new game, instead it just loads the old version from the cache (even if you refresh the page) This is often something which you do not notice during development because during debugging the browser is configured to hard refresh your website.  For this problem I have implemented a cache buster. It is a simple change which adds a parameter with a timestamp of the export moment to the javascript and wasm urls. This way the game is updated when you  place a new version on your website. The fix is for normal html exports and not for progressive web app exports

Thank you for your quick review. I agree with your changes. Yes it will not happen often that a second export is within I second. But these days with automated scripts you never know. It is such a simple low impact change and it makes it a lot saver, so I included it.

Thank you for your commet. Yes I agree it is not the clock time. But that does not matter for a cache buster. It does not have to be a valid time. It just needs to be unique within the second.
Tijmen

On 16 Jan 2022, at 19:25, Rémi Verschelde @.***> wrote:
@akien-mga commented on this pull request.
In platform/javascript/export/export_plugin.cpp https://github.com/godotengine/godot/pull/56841#discussion_r785478499:

@@ -142,12 +143,18 @@ void EditorExportPlatformJavaScript::_fix_html(Vector<uint8_t> &p_html, const Re
head_include += "\n";
}

else {

const String date = Time::get_singleton()->get_date_string_from_system();

const String time = Time::get_singleton()->get_time_string_from_system();

const uint64_t ms = Time::get_singleton()->get_ticks_msec() % 1000;

That's not the clock time in ms, that's the amount of time since the editor process was started. It's pretty arbitrary so I don't think it's relevant to export.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#pullrequestreview-853706900, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NSXVVJIC6VVKGZSTK3UWMETFANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

Yes I agree it is not the clock time. But that does not matter for a cache buster. It does not have to be a valid time. It just needs to be unique within the second.

I'm not sure it's correct though, as the only realistic way to have two exports done in the same second is to do it from two separate processes (e.g. two command line exports on the same machine, or exports done from two different machines and ending at the same time). In this case, get_ticks_msec will be process specific and there's no guarantee than a bigger value actually identifies an export that is newer, it just means that the editor instance that did this export had more uptime.

When you write an automated UI test and you test the export by clicking on it. It is possible to click two times within one second on the export button and compare the results and since the cache bustor has the same value in when it does not add mili seconds the unit test will fill because it sees the same result.
Tijmen

On 16 Jan 2022, at 19:42, Rémi Verschelde @.***> wrote:
Yes I agree it is not the clock time. But that does not matter for a cache buster. It does not have to be a valid time. It just needs to be unique within the second.
I'm not sure it's correct though, as the only realistic way to have two exports done in the same second is to do it from two separate processes (e.g. two command line exports on the same machine, or exports done from two different machines and ending at the same time). In this case, get_ticks_msec will be process specific and there's no guarantee than a bigger value actually identifies an export that is newer, it just means that the editor instance that did this export had more uptime.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1013929704, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NTR6IVHVPQENGWGI43UWMGQDANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

I'm not fully convinced that this is meaningful, but ok, it shouldn't hurt. I'd rename the variable to ticks though to make it clear that the code author understood that they're handle engine ticks (i.e. multiple seconds, possibly dozens of minutes) and not the milliseconds that would be expected after date and time.

Good point. Yes tick is a better name

On 16 Jan 2022, at 19:59, Rémi Verschelde @.***> wrote:
I'm not fully convinced that this is meaningful, but ok, it shouldn't hurt. I'd rename the variable to ticks though to make it clear that the code author understood that they're handle engine ticks (i.e. multiple seconds, possibly dozens of minutes) and not the milliseconds that would be expected after date and time.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1013932459, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NWYXYNOM7XSTKHITU3UWMISDANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

I changed the variable name and I have also added a comment in the code to explain the code buster formatting.

On 16 Jan 2022, at 19:59, Rémi Verschelde @.***> wrote:
I'm not fully convinced that this is meaningful, but ok, it shouldn't hurt. I'd rename the variable to ticks though to make it clear that the code author understood that they're handle engine ticks (i.e. multiple seconds, possibly dozens of minutes) and not the milliseconds that would be expected after date and time.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1013932459, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NWYXYNOM7XSTKHITU3UWMISDANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

I have been able to move the CACHE_BUSTER in the javascript file as requested. I have double checked if every thing still works, and it works (It works because the load trigger is initiated by startGame and at that moment the CACHE_BUSTER constant is declared)
So every I have now fixed every thing you asked. Or is there still more?
Tijmen

On 16 Jan 2022, at 19:29, Rémi Verschelde @.***> wrote:
@akien-mga commented on this pull request.
In misc/dist/html/full-size.html:

@@ -133,7 +133,9 @@

-
+

There seems to still be a style issue reported by CI (trailing whitespace probably): https://github.com/godotengine/godot/runs/4844786586?check_suite_focus=true
Otherwise squashing the commits would indeed be welcome, you can do so as described in https://docs.godotengine.org/en/latest/community/contributing/pr_workflow.html#the-interactive-rebase
I was waiting for @Faless (HTML5 platform maintainer) to review before suggesting you do more work, but you're welcome to do it now so that it's ready to merge if approved.

I squashed it to one commit. It does not yet run the automated tests again. But maybe I have to wait.

On 18 Jan 2022, at 20:16, Rémi Verschelde @.***> wrote:
There seems to still be a style issue reported by CI (trailing whitespace probably): https://github.com/godotengine/godot/runs/4844786586?check_suite_focus=true https://github.com/godotengine/godot/runs/4844786586?check_suite_focus=true
Otherwise squashing the commits would indeed be welcome, you can do so as described in https://docs.godotengine.org/en/latest/community/contributing/pr_workflow.html#the-interactive-rebase https://docs.godotengine.org/en/latest/community/contributing/pr_workflow.html#the-interactive-rebase
I was waiting for @Faless https://github.com/Faless (HTML5 platform maintainer) to review before suggesting you do more work, but you're welcome to do it now so that it's ready to merge if approved.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1015742206, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NU72XIMLNL3XQN63LDUWW4CFANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

Sorry, I didn't have the time to fully investigate this yet, but I don't think we need it, at least not this way.
If your server is properly configured (via E-Tag, or "must-revalidate") you don't need a cache buster, additionally, this might even force re-downloading of some assets that hasn't changed.
If anything, we seem to have to opposite problem, i.e. that the PWA app does not cache enough (#56103 and there we might need to invalid cache see https://github.com/godotengine/godot/compare/3.x...Faless:js/3.x_pwa_prefer_cache which is still WIP )

If your server is properly configured (via E-Tag, or "must-revalidate") you don't need a cache buster, additionally, this might even force re-downloading of some assets that hasn't changed.

Unfortunately, those headers can't be overridden on itch.io or GitHub Pages.

Yes I have also the problem that I can not override these headers, and you do not want to browser re-evaluate each time the page is loaded (costs to many requests), The only way a browser can see without a server request that a resources is changed is by changing it (so add a cache buster). And yes this can result in loading resources which where not changed because, this will happen not too often and is not in comparison with browser which always needs to check if resources are changed, this will delay loading the page.
Tijmen

On 19 Jan 2022, at 01:31, Hugo Locurcio @.***> wrote:
If your server is properly configured (via E-Tag, or "must-revalidate") you don't need a cache buster, additionally, this might even force re-downloading of some assets that hasn't changed.
Unfortunately, those headers can't be overridden for all files on itch.io or GitHub Pages.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1015959596, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NTJRZNQZJ6CUPOEKR3UWYA6PANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

The best solution would be not to use one date time as cache buster but to determine the hash of each file and add the hash to the url. This way the file only gets downloaded when it is actually changed. This would be more work, and if we choose to do this a more experienced godot developer should pick this up. However a timestamp is small simple and will work just fine as short term solution which can be back ported to the 3.x because it is a small change.
Tijmen

On 19 Jan 2022, at 01:31, Hugo Locurcio @.***> wrote:
If your server is properly configured (via E-Tag, or "must-revalidate") you don't need a cache buster, additionally, this might even force re-downloading of some assets that hasn't changed.
Unfortunately, those headers can't be overridden for all files on itch.io or GitHub Pages.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1015959596, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NTJRZNQZJ6CUPOEKR3UWYA6PANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

It is appended for the wasm in the javascript script file
return fetch(file + cacheBuster).then(function (response) {

When you are going to do effort to add guid you may also consider to use file hashes in this case. The advantage of this is that when the file file is not changed the hash is also not changed.
But I gave up on this pull request, I know the code works fine but it just takes too long to get it merged into the main repository. I have created my own work around to get it fixed.
Tijmen

On 9 Feb 2022, at 17:08, Jordan Schidlowsky @.***> wrote:
I'm also a bit confused... In the PR I see the cache buster extension to the .js filename, but i don't see it appended anywhere else. Where is the filename changes to exported .pck and .wasm? To be fair I haven't tried it yet, but from the PR I cant where or tell if this happens
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1033927845, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NUXZOZKHYPXJUW6B5LU2KGQLANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

If the cache headers from the server are wrong, the html page returned will be the old one, with the old hash, and cache will not be busted.  This is exactly the problem this would be addressing.  We currently have this issue, and this sucks for cached clients because our game is multiplayer and our servers will not let old clients connect due to need to update.
And I can confirm that unless the web request chain receives a very short expires time with 'must-validate' in cache-control header (which would revalidate and receive a 304), there is going to be problem.  This is also the case for our cluster's reverse proxy, as caching may be done at the proxy as well.
Deployments aren't just limited to itch.io or GitHub Pages as some html5 gaming portals do not implement caching properly.  This is a partial reason why we are attempting to serve our app in an iframe on portals if it's an option, as we have more control there.
Servers can respond with wrong cache headers, which may or may not be out of your control and different exported URIs may help in those scenarios.  I'm not saying this really needs this to be merged, but there is a use-case here.

This is exactly the problem this would be addressing.

As I explained, I don't see how this PR addresses this problem, since if the cache headers are wrong, the html returned will have the old timestamp, making the whole process pointless.

This is exactly the problem this would be addressing.

As I explained, I don't see how this PR addresses this problem, since if the cache headers are wrong, the html returned will have the old timestamp, making the whole process pointless.

Which is why I asked about the cache buster on the *.wasm and *.pck exports.    Here's a very real scenario that is happening right now:
*.html files served from a webserver with correct cache control (revalidates), while *.pck (unknown file extension to most webservers) are served with say "Cache-control: max-age=604800"  What happens in this case when re-exported projects now have a different .pck file with the same URI?  The web client in this case doesn't even make the request for a new pck, and just plucks it out of it's local cache.

In general, the response to this PR seems to be that as long as http servers are configured properly, and reverse proxies and clients are doing what they are supposed to, this PR is uncessary.  But that is often not the case in many real-world scenarios.  We have seen de-sync plenty of times among our exported game specifically when *.pck resources are plucked from http client's local cache while the other http resources are re-validated and potentially updated.  A guaranteed different exported URI for each resource imho is a pretty good way to bust this up and guarantee client reload on re-export.  So I do support this although there are a couple small issues with the implementation.

In general, the response to this PR seems to be that as long as http servers are configured properly, and reverse proxies and clients are doing what they are supposed to, this PR is uncessary.

I think there is a misunderstanding here. Adding a cache buster when the servers are configured correctly is detrimental. This is the main reason why I am against this PR.

But that is often not the case in many real-world scenarios. We have seen de-sync plenty of times among our exported game specifically when *.pck resources are plucked from http client's local cache while the other http resources are re-validated and potentially updated.

If you feel this can help a common scenario, I think we can add an optional cache buster, but I would still recommend fixing the proxy and servers configurations (again, the goal of CDNs and proxy caching is to provide proper caching headers, if that doesn't happen, something is wrong in the infrastructure).
And again, I still don't think this will fix it in the end, and it will just trigger other problems.
So, in case we want this:

Needs a rebase
The cache buster should be added as a config option and not as a separate function.
Should also be correctly added to the localFile function.
Must be added as an export option (off by default).

Good point. I agree it should be optional.

On 9 Feb 2022, at 21:01, Fabio Alessandrelli @.***> wrote:
In general, the response to this PR seems to be that as long as http servers are configured properly, and reverse proxies and clients are doing what they are supposed to, this PR is uncessary.
I think there is a misunderstanding here. Adding a cache buster when the servers are configured correctly is detrimental. This is the main reason why I am against this PR.
But that is often not the case in many real-world scenarios. We have seen de-sync plenty of times among our exported game specifically when *.pck resources are plucked from http client's local cache while the other http resources are re-validated and potentially updated.
If you feel this can help a common scenario, I think we can add an optional cache buster, but I would still recommend fixing the proxy and servers configurations (again, the goal of CDNs and proxy caching is to provide proper caching headers, if that doesn't happen, something is wrong in the infrastructure).
And again, I still don't think this will fix it in the end, and it will just trigger other problems.
So, in case we want this:
Needs a rebase
The cache buster should be added as a config option https://github.com/godotengine/godot/blob/master/platform/javascript/js/engine/config.js#L109-L115 and not as a separate function.
Should also be correctly added to the localFile https://github.com/godotengine/godot/blob/master/platform/javascript/js/engine/config.js#L294-L307 function.
Must be added as an export option (off by default).
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-1034142583, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NULLLW3Y7CJDKSYGSLU2LBY5ANCNFSM5MC6I5JA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.

If you append the file contents hash to the exported filename (with the exception of the .html  root)  I'm not sure how this can be considered detrimental as you can actually be more aggressive on your webserver cache settings...  Here is an example output from say our current export:
index.html
index.audio.worklet.js
index.js
index.pck
index.wasm

If you want this merged what I would propose is output like:
index.html
index-FILECONTENTSHASH.audio.worklet.js
index-FILECONTENTSHASH.js
index-FILECONTENTSHASH.pck
index-FILECONTENTSHASH.wasm

Since all the other URI's are loaded via the first request to index.html, *.html is the only resource type that needs to be configured to revalidate.  All other files under this directory you could even cache indefinitely without revalidation as it's guaranteed their contents will never change (and if their content does, it is exported as a different URI).
From a caching perspective, this is actually the opposite of detrimental and more cache efficient because right now this is what happens if configured correctly:

Every URI needs to be re-validated because the content MAY change at any time, which mean a bunch of additional round-trips and a bunch of 304's if configured properly ( and really annoying de-syncs across wasm, js, and pck files if not).
If you implement exports like this:
index.html
index-FILECONTENTSHASH.audio.worklet.js
index-FILECONTENTSHASH.js
index-FILECONTENTSHASH.pck
index-FILECONTENTSHASH.wasm

you could configure your webserver to aggressively instruct clients to cache .js, .wasm, and .pck as their contents at their respective URI's are guaranteed to never change.  In this scenario there is only a single request and single validation +304  response required for the initial request to index.html, as every other dependency is derived from that first request.
At any rate, from a potential cache 'getting it wrong' perspective, I think the above export example is less error prone.  I think it's going to be a good option for the engine to provide devs because it really removes a lot of potential issues whether it's considered a cache buster or not really.  Definitely useful for devs who are deploying to environments that aren't under their direct control, or devs who are not familiar with what their actual server cache settings should be.  However, I think in order for this to make it into main, I would want to the see a file-contents hash on the exported filename though which is definitely more work than what is included in this PR.
The only problem / downside I can see to implementing this is local filesystem issues & confusion if you choose to export to the same output directory as a previous export.  In which case I think there could/would be multiple copies of the same dependency with different hash depending on how implemented, but I can lready think of a few different solutions for that.

I've honestly never seen cache busting by changing file name, I would definitely go for using a query string, though using the hash instead of timestamp sounds good.

If it's implemented with file contents hash instead of a timestamp, this would address your concerns about being 'detrimental' correct?

Yes, I would still make it optional or at least off in the PWA export since it's irrelevant there.

Seems like some consensus was reached, but this was never finished. Would be nice to revive this for a future 4.x release.

May be the fix for https://github.com/godotengine/godot/issues/89455 and https://github.com/godotengine/godot/issues/69414.
This needs to be pushed up, and 3.x cherry picked.

Hi,
Cache buster fix the caching problem for normal websites. For PWA’s you need more. I do not use the PWA version because the cache problem is even worse. You will also need to correct the pwa service worker for this which is in my opinion not correctly written for on and offline caching. I have given up on this change. I have simply modified the html template with extra cache busters and wrote a script to update it on every release.
Tijmen

On 19 Mar 2024, at 03:46, PeterMarques @.***> wrote:
May be the fix for #89455 https://github.com/godotengine/godot/issues/89455 and #69414 https://github.com/godotengine/godot/issues/69414.
This needs to be pushed up, and 3.x cherry picked.
—
Reply to this email directly, view it on GitHub https://github.com/godotengine/godot/pull/56841#issuecomment-2005647522, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAVG7NRGY42OOJLJ2AXQDMTYY6RGTAVCNFSM5MC6I5JKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBQGU3DINZVGIZA.
You are receiving this because you authored the thread.


Allow programmatic file deletion

The current API design supports just a single way of deleting a file
files: {
  "file/to/delete.txt"" null
}

However, it would be useful to trigger a file deletion based on specific conditions, such as the example below:
files: {
  "file/to/delete.txt": ({ exists, encoding, content }) => {
    // do not create the file if it does not exist
    if (!exists) return null;

    const fileContent = Buffer.from(content, encoding).toString("utf-8")

    if (fileContent.includes('abc')) {
      // trigger file deletion
      return ???
    }

    // do not alter file content
    return null;
  }
}

Since both null and undefined already have different functionalities, it is unclear what return value could be used to instruct the library to initiate a file deletion. What are your thoughts?

Since both null and undefined already have different functionalities, it is unclear what return value could be used to instruct the library to initiate a file deletion. What are your thoughts?

I think the correct way to do this is to use Symbols. See how mapObjectSkip is implemented in map-obj. I don't mind releasing a breaking change for this.
What do you think?


Print success/failure msgs at the end of each operation

This is addressed in https://github.com/ge-high-assurance/RITE/commit/6e558bde80af8ff65b74f599c86de3c228985956