[ODS-5767] Add the Standard version to packages that reference Ed-Fi-TPDM-Artifacts repository

Rebuild Database Templates workflow works again
initdev works with and without the tpdm plugin

@simpat-jesus  Rebase with latest main ,so that this issue get resolved https://github.com/Ed-Fi-Alliance-OSS/Ed-Fi-ODS-Implementation/actions/runs/4613463010/jobs/8155459560?pr=644#step:14:65


I'm trying to use Your prompt for fish, but I get some errors.

Hello.
I'm trying to use Your prompt for fish, but after applying it, I get these errors in the output. I used the recommended font.

System details:

What is the output of which logname?

If logname is installed you can permanently disable this by removing these lines https://github.com/manilarome/fishblocks/blob/022d18bb6a8762b2eb691fe117d1bc48300e06f3/fish_prompt.fish#L167-L168

You can fix this locally by changing  (logname) to $LOGNAME in the fish_prompt.fish file.


Create Date scalar

Date scalar should be string matching YYYY-MM-DD format. It should throw exception in case of format is violated.
You can follow example defined in https://www.apollographql.com/docs/apollo-server/schema/custom-scalars/

Assign me, please


OCI8

I had tries add oci8 PHP extension, and always unsuccessful, u have test things like this?
thanks

I have not tried oci8 (in fact, I don't even know what that is). Feel free to post your findings here or provide more details about the "unsuccessful" attempts if you need help.


chore: Pull in slightly older version of regex

Because transitive deps suck. We need this so we can update the OCI deps in the main host

Yep, yanking shall occur


Design Analytics Person View

Design the person view for the analytics application.

The same as the normal library view with a filter applied.


Version support check is ineffective in createSandbox

Description of problem
In cloudHypervisor#createSandbox, the check against major and minor supported versions cannot handle either version being 0.
Expected result
The check should cover either major or minor version being 0.
Actual result

I've added the "backport" label as it may be material for. at least 1.11.0 stable release.


Not Found, This page doesn’t exist

Hi there,
The following link on your website shows "Not Found, This page doesn’t exist.".
It's a link from additional resources on your Meta Front-End Developer Certificate on Coursera on "Rules of Hooks and Fetching Data with Hooks, week 2 of Advanced React" as well.
https://react.dev/apis/react/useEffect#fetching-data-with-effects

Hi, Mason. I don't think the link exists actually

At one point during the we used /apis route but it was changed to /reference before release, so the correct link is https://react.dev/reference/react/useEffect#fetching-data-with-effects. The reference will need to be updated from the source. on the Coursea page (which the React team doesn't own/manage).


clone wiki no longer exists

The instructions reference a "clone wiki" option in the menu that no longer exists.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 172eecd7-1a6b-30d0-96c0-fa6ae56b521c
Version Independent ID: adc2c741-a41f-68d9-cdaf-dac696e68b9f
Content: Clone and update wiki content offline - Azure DevOps
Content Source: docs/project/wiki/wiki-update-offline.md
Product: devops
Technology: devops-collab
GitHub Login: @chcomley
Microsoft Alias: chcomley

@chcomley - I found a Clone wiki option after selecting the Wiki menu item, not immediately after selecting a project. The procedure implies that it's right there after you select the project.

#sign-off


please co-publish on open-vsx

Please make this extension available to Open VSX marketplace, this will also make it directly available for VSCodium and Gitpod.

Open VSX is a vendor-neutral open-source alternative to the Visual Studio Marketplace. It provides a server application that manages VS Code extensions in a database, a web application similar to the VS Code Marketplace, and a command-line tool for publishing extensions similar to vsce.

There is a guide for publishing and after the initial setup it is just a matter of osvx publish (similar to the MS marketplace).

Thanks for the idea, I published the extension here: https://open-vsx.org/extension/JacquesLucke/gcov-viewer
Hope that works for you.

Can you please update it to the current version?

Ping @JacquesLucke - you may want to adjust your build process to either use a CI action to publish to both marktplaces, or script that for a local deployment.


Allow for datetime related type such as DATE (YYYY-MM-DD)

Before suggesting a new feature, please make sure this hasn't already been suggested by searching through the past issues and the PR tracker.
Presentation of the new feature
Currently, if you define date in Pydantic BaseModel, the regex you get will be invalid as it is not enclosed with a string.
For example, when defining Pydantic BaseModel:
from datetime import date

class Weapon(str, Enum):
    sword = "sword"
    axe = "axe"
    mace = "mace"
    spear = "spear"
    bow = "bow"
    crossbow = "crossbow"

class Wizard(BaseModel):
    name: str
    age: float
    weapon: Weapon
    birth_date: date

Now that we have define date, the regex that comes out is:
'\\{[\\n ]*"name"[\\n ]*:[\\n ]*"(?:[^"\\\\\\x00-\\x1f\\x7f-\\x9f]|\\\\.)*"[\\n ]*,([\\n ]*"age"[\\n ]*:[\\n ]*((-)?((0|[1-9][0-9]*))(\\.[0-9]+)?([eE][+-][0-9]+)?|null)[\\n ]*,)?[\\n ]*"weapon"[\\n ]*:[\\n ]*("sword"|"axe"|"mace"|"spear"|"bow"|"crossbow")[\\n ]*,[\\n ]*"birth_date"[\\n ]*:[\\n ]*(?:\\d{4})-(?:0[1-9]|1[0-2])-(?:0[1-9]|[1-2][0-9]|3[0-1])[\\n ]*\\}'

The part we need to focus on is:
(?:\\d{4})-(?:0[1-9]|1[0-2])-(?:0[1-9]|[1-2][0-9]|3[0-1])'

When datetime related types are detected, we should add  "...." enclosing quotations so that output JSON is valid to load.
Where does it fit in Outlines?
Allow JSON mode to be used for datetime related types.
Are you willing to open a PR?
Yes. It will be great if you can provide some pointers to help accelerate my development.

Example output that is an invalid JSON:

Saw the code, recent update fixed this issue. Nice one.
For anyone with the same problem, just update to 0.0.40


New command: m365 pp solution publisher list

Usage
m365 pp solution publisher list
Description
Lists Microsoft Power Platform publishers in the specified Power Platform environment.
Options

Option
Description

-e, --environment <environment>
The name of the environment

--asAdmin
Run the command as admin for environments you are not a member of

Examples
List publishers in the environment named Default-d87a7535-dd31-4437-bfe1-95340acd55c5.
m365 pp solution publisher list --environment Default-d87a7535-dd31-4437-bfe1-95340acd55c5

List publishers in the environment named Default-d87a7535-dd31-4437-bfe1-95340acd55c5 as admin.
m365 pp solution publisher list --environment Default-d87a7535-dd31-4437-bfe1-95340acd55c5 --asAdmin

Additional Info
Depends on #3684 for the PowerPlatform util and Dynamic Permissions.
Can be selected using following snippet:
https://DYNAMICSULR/api/data/v9.0/publishers?$select=publisherid,uniquename,friendlyname,versionnumber,isreadonly,description,customizationprefix,customizationoptionvalueprefix&$filter=publisherid ne d21aab70-79e7-11dd-8874-00188b01e34f&$orderby=friendlyname asc
Since it is a list command, only the publisherid,uniquename,customizationprefix should be shown in text output

One more 😄 . I'm not saying that i'm addicted to solving issues for Microsoft 365 Cli, i just need it

Hi @appieschot , in your snippet there is following filter publisherid ne d21aab70-79e7-11dd-8874-00188b01e34f where the id d21aab70-79e7-11dd-8874-00188b01e34f references to the Microsoft publisher.
Is it supposed to be filtered out of the results when listing the publishers or do we list all publishers?

Awesome, all yours @nicodecleyre

Is it supposed to be filtered out of the results when listing the publishers or do we list all publishers?

The UI filters those out, so it made sense to filter them out in cli as well (at least for me it did)

Is it supposed to be filtered out of the results when listing the publishers or do we list all publishers?

The UI filters those out, so it made sense to filter them out in cli as well (at least for me it did)

Fair point!
What if a user goes via make.powerapps.com, clicks on advanced settings.
After that clicking on settings and customizations and finally on publishers. There all the publishers are listed including the Microsoft Publisher. Wouldn't that confuse the user that the Microsoft publisher is listed there but not in the cli?

Hmm, there you have a good point. It marks how I am never going to dynamics 😂. How about we add an additional parameter --includeMicrosoftPublishers to also include those? Matches our spo site list --includeOneDriveSites approach.

Awesome! I'll update the pr!

--includeMicrosoftPublishers added to the pr 👀


One more test for TRY block

one more test
package dependency changed

@lyriccoder this PR will pass tests after merge of https://github.com/acheshkov/program-graphs/pull/15


fix: valibot peer dependency version to support dist-tags

Looks like the npm install broke with the upgrade to Valibot v1. This PR should fix that.

Thank you


[Bug] SHA-1 and MD5 hashes are never calculated

For some reason no ROM that I try with ROM64 seems to calculate the SHA-1 or MD5 hash. It just says "not calculated".
This happened with big-endian, byte-swapped, and little-endian ROMs.
All of the tested ROMs were No-Intro verified, or checked using romjudge and work on emulators and flash carts.
Side note, the text output puts the CRC1 and CRC2 twice in the outputted text, which is redundant I believe.

Please include the command you were running when you had this problem.

The CRC1/2 do appear twice in --output text mode but they don't come from the same source. Under the Checksums heading, this is the CRC1/2 as calcualted by rom64. Under the ROM heading, these are the CRC1/2 as reported by the ROM file itself.

Sorry, I was using the ls command and with --output text was what I noticed was having the issue specifically.
I didn't realize there were multiple ways of acquiring the CRC1 and CRC2, so that's my bad. Is it common for the CRC1 and CRC2 to be different based on what method you use to calculate or retrieve it?

Ah, now I see how you were able to encounter "Not Calculated". Checksum calculation is not done unless explicitly needed since it creates an appreciable delay and CPU work with large directories of roms. Originally ls only output in table format, so it would only calculate those checksums if the columns were asked for with the --columns flag, The text, json, and xml formats are "static" in that you can't pick the fields you want returned like with table, tab, and csv format, so we end up with a confusing issue here.
The --columns flag can actually be used in this case to get the checksums to calculate.
rom64 ls ~/Downloads/n64 -o text -c file_md5,file_sha1,file_crc1,file_crc2

This also works for json and xml formats to get the checksums to calculate.
One option is to document this case and instruct users to use the --columns flag to get the calculations they want. Another is to always calculate them in text, json, and xml formats. The downside with that is creating a slowdown when users might not even want them.

For now I've updated the readme with a note about checksums and how to request them to be calculated.
As for the CRC1 and CRC2, in my experiences they have matched. Endianness shouldn't cause an issue since it's swapped to the native format internally when doing these calculations. If you have the calculated and reported CRCs mismatching, I'd be interested to see what other tools make of it.
When I was developing this I used another simple C program to compare the CRC results and I also compared to what I saw on EverDrive 64.

Ah, now I see how you were able to encounter "Not Calculated". Checksum calculation is not done unless explicitly needed since it creates an appreciable delay and CPU work with large directories of roms. Originally ls only output in table format, so it would only calculate those checksums if the columns were asked for with the --columns flag, The text, json, and xml formats are "static" in that you can't pick the fields you want returned like with table, tab, and csv format, so we end up with a confusing issue here.
The --columns flag can actually be used in this case to get the checksums to calculate.
rom64 ls ~/Downloads/n64 -o text -c file_md5,file_sha1,file_crc1,file_crc2

This also works for json and xml formats to get the checksums to calculate.
One option is to document this case and instruct users to use the --columns flag to get the calculations they want. Another is to always calculate them in text, json, and xml formats. The downside with that is creating a slowdown when users might not even want them.

For now I've updated the readme with a note about checksums and how to request them to be calculated.
As for the CRC1 and CRC2, in my experiences they have matched. Endianness shouldn't cause an issue since it's swapped to the native format internally when doing these calculations. If you have the calculated and reported CRCs mismatching, I'd be interested to see what other tools make of it.
When I was developing this I used another simple C program to compare the CRC results and I also compared to what I saw on EverDrive 64.

Ah, that makes sense. I was a bit confused about why it wasn't calculating them since it didn't seem there was a way to set that as an option when using ls. 😄
As for the CRC's, I was a bit worried about CRC clashing, but so far I haven't encountered any issues in the real world. There have been examples of bad dumps of certain software titles having the same CRC1 and CRC2 and causing issues in game databases who use that info to tell games apart, but that's the closest thing to that I've come.
Thanks for all your help and information! 😄

Now I'm not sure if I've understood your question about CRCs and if my answer added more confusion.
When a ROM dump is good, the CRC1 and 2 calculated by rom64 of the File should match what's in the ROM header. That's what we see with production game dumps anyway:
rom64 info ~/Downloads/Super\ Mario\ 64\ \(USA\).n64
File:
  Name:    Super Mario 64 (USA).n64
  Size:    8 MB
  Format:  v64 (Byte-swapped)
  Checksums:
    MD5:     165a9558b7539507e73e7afecffb200c
    SHA1:    1002dd7b56aa0a59a9103f1fb3d57d6b161f8da7
    CRC 1:   635A2BFF
    CRC 2:   8B022326

ROM:
  ID:        NSME
  Title:     SUPER MARIO 64
  Media:     Cartridge
  Region:    USA
  Video:     NTSC
  Version:   1.0
  CIC:       6102
  CRC 1:     635A2BFF
  CRC 2:     8B022326

By clash, do you mean two ROM files with different contents having the same CRC values? It's entirely possible that two different ROM files would have the same values in the header. There's nothing stopping anyone from just writing whatever CRC values into the header that they want. Just like how you could overwrite the title on a ROM with any data. Then sure, you could have different files with the same values in the headers.
As for the calculated CRCs of the file, those are highly unlikely to be the same for different files since it's a checksum of about ~1 MB of data in the file.
The SHA-1 and MD5 will never be the same between files unless those files have identical contents. The ROM validation data file built into rom64 uses SHA-1 validation.

Now I'm not sure if I've understood your question about CRCs and if my answer added more confusion.
When a ROM dump is good, the CRC1 and 2 calculated by rom64 of the File should match what's in the ROM header. That's what we see with production game dumps anyway:
rom64 info ~/Downloads/Super\ Mario\ 64\ \(USA\).n64
File:
  Name:    Super Mario 64 (USA).n64
  Size:    8 MB
  Format:  v64 (Byte-swapped)
  Checksums:
    MD5:     165a9558b7539507e73e7afecffb200c
    SHA1:    1002dd7b56aa0a59a9103f1fb3d57d6b161f8da7
    CRC 1:   635A2BFF
    CRC 2:   8B022326

ROM:
  ID:        NSME
  Title:     SUPER MARIO 64
  Media:     Cartridge
  Region:    USA
  Video:     NTSC
  Version:   1.0
  CIC:       6102
  CRC 1:     635A2BFF
  CRC 2:     8B022326

By clash, do you mean two ROM files with different contents having the same CRC values? It's entirely possible that two different ROM files would have the same values in the header. There's nothing stopping anyone from just writing whatever CRC values into the header that they want. Just like how you could overwrite the title on a ROM with any data. Then sure, you could have different files with the same values in the headers.
As for the calculated CRCs of the file, those are highly unlikely to be the same for different files since it's a checksum of about ~1 MB of data in the file.
The SHA-1 and MD5 will never be the same between files unless those files have identical contents. The ROM validation data file built into rom64 uses SHA-1 validation.

Yeah, I understand what you're saying, I think I just worded it poorly or wasn't specific enough. CRC1 and CRC2 being the same is sort of common among the same official release, but every once in a while there is homebrew or something that will have the same CRC1 and CRC2 as a retail game which is confusing from my perspective of wanting to create a database for all the ROMs.

I think this can be closed since the flag is available and this isn't really a bug. 😄

@DerekTurtleRoe Cool! Glad we were able to sort these issues out. Let me know if there's anything the program could do to make life easier. :) I'm thinking about making one for other systems next...

@DerekTurtleRoe Cool! Glad we were able to sort these issues out. Let me know if there's anything the program could do to make life easier. :) I'm thinking about making one for other systems next...

Can I ask what systems? I feel like more complete tools like this for other systems would be awesome! I would love to help in any way I can. 😄

Also are Nintendo 64 Disk Drive, iQue, and Aleck64 games fully supported or only cartridge conversions? That could be something  to add. I suppose just adding the No-Intro database entries for them should be somewhat easy, not sure if the images themselves would need to be read differently for iQue and Aleck64. N64DD images are very different though, so it might be ambitious.


Play with Eve: Error during WebSocket handshake

WebSocket connection to 'ws://play.witheve.com/ws' failed: Error during WebSocket handshake: Unexpected response code: 404
EveClient	@	client.ts:145
(anonymous)	@	client.ts:455
I see this error in the JavaScript console when loading examples at http://play.witheve.com/#/examples/quickstart.eve. Specifically, I can't get the quickstart example to work properly unless I mess around randomly checking checkboxes, the reset button, and refreshing the page. I would expect that when I check the checkboxes in the quickstart example, that the view would update automatically, but this does not occur.
Chrome OS Version 58.0.3029.140 (64-bit)

Hey @philmaker1,
The websocket message is expected. Eve attempts to raise a connection to the server in the event that you'd like to remotely execute code. If the server isn't up (as is the case with play.witheve.com, for security reasons), it'll harmlessly failover to local-only execution mode.
W.r.t. getting the quickstart examples to work:
The checkboxes allow you to enable/disable blocks of code from evaluating in the current state of the running program. For blocks of code which commit values, disabling a block will not remove the values it asserted, so you'll need to reset the program (ctrl+shift+enter or cmd+shift+enter) or write a block to remove those values from the running program. Refreshing the page will also reset the program state, which is likely what caused things to intermittently work for you.
Unfortunately, we don't explain this at all in the QS, and didn't engineer the QS to avoid this gotcha. We initially had a hack that would reset the program for you when you checked/unchecked a block in the QS (since in the QS you don't really care about maintaining your live state), but found that transitioning to authoring their own programs was confusing for the user.
We've rewritten the quickstart for 0.3 very differently to account for this, but if you have any specific suggestions on how we can better the UX we're always looking for ways to improve.


[zh-cn] sync nodes garbage-collection

content/zh-cn/docs/concepts/architecture/garbage-collection.md
content/zh-cn/docs/concepts/architecture/nodes.md

/lgtm


 Go's block profiling would be useful for this. We should add a flag to enable it.
c.f. https://golang.org/pkg/runtime/#SetBlockProfileRate

Flag is now in Scope master. I need to start focusing on other things, so I'll leave the actual experimentation up to @2opremio


Update to use gradle-wiremock-extension-plugins.

Encourage extensions to use gradle-wiremock-extension-plugins by showing an example of its use.
References

https://github.com/wiremock/gradle-wiremock-extension-plugins/pull/12
closes https://github.com/wiremock/wiremock-extension-template/issues/7

Submitter checklist

[x] Recommended: Join WireMock Slack to get any help in #help-contributing or a project-specific channel like #wiremock-java
[x] The PR request is well described and justified, including the body and the references
[x] The PR title represents the desired changelog entry
[x] The repository's code style is followed (see the contributing guide)
[x] Test coverage that demonstrates that the change works as expected
[ ] For new features, there's necessary documentation in this pull request or in a subsequent PR to wiremock.org

Thanks for your contribution 👍


Usage of Docker Hub

L: Uhm, Docker Hub is apparently discontinuing its free tier for teams (is that the same as organizations?): https://hacky.town/@rtyler/110023261891172425
Now, currently the container image is hosted on Docker Hub, and it says that https://hub.docker.com/u/unifiedpush is an organization, so this might be uh, an issue, unless you're paying for that? So, you might want to move to quay.io or GHCR, for example?

It is wonderful to see you moving not sideways to another Big Tech silo but to a proper Free Software home! Thank you!
https://codeberg.org/UnifiedPush/common-proxies


A regex in RoundCubeLogin.class seems to be outdated

roundcubemail-0.9.5
ownCloud 9.0.0
OC app: RoundCube Mail 2.6.1
Autologin enabled
The errors I first saw are these:
Error	roundcube	OC_RoundCube_App.class.php->showMailFrame(): There were login errors
Error	roundcube	OC_RoundCube_App.class.php->showMailFrame(): Autologin is enabled. Seems that the owncloud and roundcube login details do not match
Error	roundcube	OC_RoundCube_App.class.php->showMailFrame(): RoundCube can not login to roundcube due to a login exception to roundcube
Error	roundcube	Not rendering roundcube iframe view due to errors
Error	roundcube	Got the following error code: 11

After debugging I realized that the problem might be in a regular expression that is not matching due to a simple comma ','. I removed it and some errors related to this stopped. There are some addDebug calls in the functions I provide next, that helped track the extra comma.
The file is lib/RoundCubeLogin.class.php and the regex is in two functions:
private function updateLoginStatus($forceUpdate = false) and
private function sendRequest($path, $postData = false)
Both lines 343 and 442 contain the regex:
if (preg_match('/"request_token":"([^"]+)",/mi', $response, $m)) {
and tyeh should be like this, without the comma ',' before the end of the regex:
if (preg_match('/"request_token":"([^"]+)"/mi', $response, $m)) {
The interesting portion of the $response string looks like this:
"contentframe":"messagecontframe","request_token":"55df9787e1693f1f0665b9e96b488434"});
Note that the not currently matched string is the last of an array, so there is no comma.
After this change, it got a bit better in my environment.
Can anyone else confirm this?

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/36815911-a-regex-in-roundcubelogin-class-seems-to-be-outdated?utm_campaign=plugin&utm_content=tracker%2F308690&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F308690&utm_medium=issues&utm_source=github).

i think the comma is added due most parts of and set of array.. since lasted commit that able to work with oc 9.X also works with oc 7
do u test it in others enviroments?
we need to test in others enviroment! please some one can test also (i can do it but not right now, in few weeks)

do u test it in others enviroments?

I only have a production env :/

so then, need to test in others enviroment! please some one can test also (i can do it but not right now, in few weeks), and thanks for the report

sorry for the late, the next week iĺl free of some task and test this ! i see that the request not currently matched string is the last of an array, so must not thereś a comma here.. in teory

big outdate and busy, i'll test this in oc 9 and oc 7 in these days, sorry for late

i installed the path and i can confirm the log "Unknow parameter ','" in the log files where gone when i use the patch..
but now i get the following error: Undefined index: defaultValue at /home/intranet/elfichero/core/ajax/appconfig.php#29 insteresting also that in oc <IP_ADDRESS> works and in <IP_ADDRESS> does not work anymore the roundcube login
so i cannot determine if this patch are working or not! due the new log entry, if u can confirm or somebody that whit "," removed does not cause errors in oc 7 and oc 6 good by me!

umm its worked.. this patch must be integrated, my problem was a cache proxy reverse problem, the rare log its a core defect.. when enabled manually a plugin app, i'll integrate this in few hours


Remove all the dead code about iOS and iOS Simulator in flutter_tools

In particular, any iOS code in 'start' and 'listen' is apparently entirely dead.

I'd rather we make them work.  But obviously we can add them back when they do.  @chinmaygarde


creepers should have a low chance of exploding into fireworks

ref:

https://github.com/Jishuna/MineTweaks/blob/master/src/main/java/me/jishuna/minetweaks/tweaks/mobs/FireworkCreeperTweak.java
https://github.com/sachingorkar102/Tweakin/blob/master/src/main/java/com/github/sachin/tweakin/modules/confetticreepers/ConfettiCreepers.java

https://github.com/globau/ClodMC/commit/c27ab70df5ec1ff2bad28acde29f1fb74fe36932


[front] - fix(vault): CreateOrEditVaultModal

Description
This PR aims at fixing the state and the endpoint used to update the vaults members within the CreateOrEditVaultModal component.
Risk
Low
Deploy Plan
Deploy front

@Fraggle I am creating a new /search endpoint on members that will search on emails. PR is almost ready, so let's use that right?


Initiating a consumer with multiple strategies not working

I am trying to subscribe to 2 different topics, but have the handler be different.
So basically i am defining a strategy array with 2 objects as following:
var strategies = [
   {
        subscriptions: ['test'],
        handler: dataHandler
    },
    {
        subscriptions: ['test1'],
        handler: dataHandler1
    }
];

What happens is that only the 'test1' topic is being consumed.
This forces me to subscribe with both topics to the same handler atm.
Is this a bug? What's the point of having the init function receive an array then?

Multiple strategies in Kafka are only for rolling updates of consumer groups, you can't use them simultaneously. Only a single strategy will be selected by group coordinator for all group members. So in order to use different handlers for different topics just add a split by topic logic in your single handler.


ninja performance regressed ~7% since v1.1.0

I noticed that empty chrome builds that don't use the new deps feature are slower with ninja head than with ninja v1.1.0 (which we currently use). I wasn't sure if that's because I'm using a different clang version, or if something in ninja changed. Today I realized I could just checkout v1.1.0 and build it with the same clang I'm using for ninja head.
This is an empty build with 1.1.0 with the clang that was current when I built it 3 months ago:
$ time -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.898s
user    0m0.642s
sys 0m0.253s
$ time ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.891s
user    0m0.641s
sys 0m0.245s
$ time ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.898s
user    0m0.645s
sys 0m0.248s

Here's the same with v1.1.0 and Chromium's current clang:
$ time ~/src/ninja2/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.878s
user    0m0.628s
sys 0m0.245s
$ time ~/src/ninja2/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.883s
user    0m0.634s
sys 0m0.244s
$ time ~/src/ninja2/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.891s
user    0m0.639s
sys 0m0.247s

It's the same speed (or, if anything, faster). Here's head ninja with Chromium's current clang:
$ time ~/src/ninja/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.951s
user    0m0.701s
sys 0m0.246s
$ time ~/src/ninja/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.995s
user    0m0.725s
sys 0m0.265s
$ time ~/src/ninja/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.954s
user    0m0.697s
sys 0m0.252s
$ time ~/src/ninja/ninja -C out_64_nodeps/Release/ chrome
ninja: Entering directory `out_64_nodeps/Release/'
ninja: no work to do.

real    0m0.941s
user    0m0.692s
sys 0m0.248s

Since I can reliably repro, I'll just bisect this tomorrow.

It's been quite a long time so I'll close this. Feel free to reopen or open new issues with performance regressions.


Create Peer Message View

Prerequisite

[ ] Self-review :monocle_face:
[ ] No Crashes ☠

Root cause

// Explain what you have done

Solution

// Explain how you fix it

Impact

// List of affected components/modules

Screenshots (Optional)

// Capture affected screens

          1 Warning

:warning:
Big PR

[ ] I understand that "DirectMessages.Placeholder" is now existed in Localized service

  Generated by :no_entry_sign: Danger


Exploration forge cli against katana branch

These are the changes to have the cli forge pass against this branch of katana: https://github.com/jobez/dojo/tree/custom-steps-and-get-block-potential-fix
katana is started with the following cli command from the dojo/crates/kata
cargo run -- --validate-max-steps 16777216 --invoke-max-steps 16777216 --gas-price 0

kakarot rpc is started with
make run-dev STARKNET_NETWORK=katana

Forge is ran with the following command from the root of this project
RUST_LOG=debug forge script --rpc-url http://<IP_ADDRESS>:3031 --broadcast --delay 30 --retries 10 -v scripts/PlainOpcodes.s.sol:PlainOpcodesScript -vvvvv --legacy

Please check the type of change your PR introduces:

[X] Exploration
[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Testing

What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

https://github.com/dojoengine/dojo/pull/622 now the katana branch basically only needs configurable vm steps, as the changes i had to make around blocks was pulled into main

Closed to differentiate PRs for ease of merge--updating and test refactoring in #292 and small changes based off explorational efforts in #forge cast #295


spiffs: enable temporal cache for spiffs v 0.3.6

In spiffs 0.3.6, it is recommended to enable the temporal fd cache feature. This optimizes the opening of frequently opened files. The cost is 6 extra bytes per file descriptor.

Thanks @pellepl for the PR.
I looked into spiffs wiki for this feature details. Really impressive numbers for the esp8266 web server example. And as a web server is the most common use case for spiffs in esp8266 it is good to have the temporal cache enabled.
On my hardware everything works fine, so I will merge it right away.


[Unexpected Behavior] Possible to softlock Soundwaves profile select

Using Stepmania-Outfox_Alpha-4.6-amd64-date-20200711 (on Linux).
Steps to reproduce:

Start from Outfox main menu
Press start (Game Start)
Press enter to validate caution screen.
Press enter a second time, before the caution screen fades to black entirely. The timing is a bit strict.
=> profile select is softlocked. Up/down arrows to select a profile don't do anything. At that point, if you press ESC once you take control back

I have recorded a short video, but since I didn't include an input overlay, it doesn't show much, except the fact I have to press ESC once too much: https://youtu.be/Hwm-oaEqVFk
(ps. I'm Kaede on Discord btw)

Thanks for this, we'll fix the input issue on the theme.

It looks like this is now fixed (fixed back in Stepmania-Outfox-Alpha-4.8.1-amd64-date-20200814, it seems).
Thanks !


Redisson 3.31.0 couldn't detect nodes IP changes after an Azure Redis failover

Azure Redis cache had a failover which resulted in lost connection and change of IP addresses for nodes.
Our client app tried to connect to Redis nodes three times (retryAttempts are set to 3) having following exceptions:
org.redisson.client.RedisTimeoutException: Unable to acquire connection! java.util.concurrent.CompletableFuture@7d303a35[Completed exceptionally: java.util.concurrent.CancellationException]Increase connection pool size. Node source: NodeSource [...], command: (EVALSHA), params: [...] after 3 retry attempts|
Afterwards, client tried to connect using stale IP addresses, hence couldn't recover from a failover. In parallel we made the CLUSTER NODES request from a console which was showing a different set of IPs in contrast to what our client tried to use for a connection. For around an hour we were seeing below exceptions until we decided to manually restart our service.
org.redisson.client.RedisNodeNotFoundException: Node: NodeSource [slot=13622, addr=rediss://OLD_IP+PORT_BEFORE_A_FAILOVER, redisClient=null, redirect=MOVED, entry=null] hasn't been discovered yet. Increase value of retryAttempts and/or retryInterval settings.
Btw, our client provides cluster host:port through nodeAddresses config parameter.
Is it something what was fixed in a recent version?

I need Redisson logs with trace logging level

Unfortunately, It happened on production, so there were no time to change log level and collect traces.
The https://github.com/redisson/redisson/issues/5931 looks similar to me. Could you elaborate more on what was fixed there?

Unfortunately, It happened on production, so there were no time to change log level and collect traces.
The #5931 looks similar to me. Could you elaborate more on what was fixed there?

In some cases topology may return empty nodes list which causes next updates impossible.

Can you try 3.32.0 version?

We have updated to 3.36.0. I'll let you know if we find something


Update sbt, scripted-plugin to 1.9.2

About this PR
📦 Updates

org.scala-sbt:sbt
org.scala-sbt:scripted-plugin

from 1.7.3 to 1.9.2
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.scala-sbt" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, version-scheme:early-semver, commit-count:1

Superseded by #114.


Merge 1.13 release branch back to master

Description
Merges the 1.13 release branch back to the master branch.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close:
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[x] Code compiles correctly
[x] Created/updated tests
[ ] Extended the documentation

Self-reviewing as this is simply a merge of mostly previously-reviewed commits.


backport BSD support for rel-0.1

This branch backports https://github.com/rustls/rustls-platform-verifier/pull/55, targetting the rel-0.1 branch. This will unblock preparing a 0.1.1 release that adds BSD support.

Do we also want to cherry-pick back the documentation fixes made in https://github.com/rustls/rustls-platform-verifier/pull/57 and https://github.com/rustls/rustls-platform-verifier/pull/54 as well? Otherwise the documentation for 0.1.1 will be broken on docs.rs too.

Do we also want to cherry-pick back the documentation fixes

Good thinking. I'll pull those in.


Model Repository Parser for UML Diagrams

Парсер репозитория моделей для создания UML-диаграмм

@ismonnar, my friend! There is something you should fix.

In the file uml/uml.py:

Line 31:7: E111 indentation is not a multiple of four
Line 31:7: E117 over-indented
Line 32:7: E111 indentation is not a multiple of four
Line 32:121: E501 line too long (126 > 120 characters)
Line 33:7: E111 indentation is not a multiple of four
Line 34:7: E111 indentation is not a multiple of four
Line 35:7: E111 indentation is not a multiple of four
Line 36:23: E128 continuation line under-indented for visual indent
Line 37:7: E111 indentation is not a multiple of four
Line 37:121: E501 line too long (143 > 120 characters)
Line 43:47: W292 no newline at end of file


chore: cosmetics

adds a generic author and email address for the OpenAPI schema contact
changes token default validity to 12 hours, enough for development workflow but not more

I converted this to a draft because I still want to add to it.

I'll leave the UUID in the tests, and post a new PR for the tag lists change we discussed.


classes should not show for autocomplete if they've already been added

https://github.com/aarctan/schedubuddy-web/commit/73a5d100eb3cb18067f7d3075da4f84faa9877bb


Internal sidebar links should open in the same window

Hello @onweru
In the latest update I noticed that sidebars links open in new tabs. However this is not good from a UX standpoint.
Please remove the attribute target="_blank" from internal links.

@onedrawingperday,
Thank you for raising this one. I will update it and notify you

@onedrawingperday,
The problem is that "https://themes.gohugo.io/theme/" wasn't being recognized as the baseURL by this snippet of javascript code (parentURL is equal to whatever the baseURL is). Hence the target=_blank attribute was being added.
I'm not sure if the commit I just referenced above 47b40fe41e5f58d7368 will do, but it's the only thing I can think of currently being amiss.
Could it be the issue?

@onweru
I don’t think that using JS is necessary for this since in a Hugo template you can perform a check and conditionally add the target="_blank" attribute.
For example within the context of a Hugo menu try something like:
{{ if strings.HasPrefix .URL "http" }} target="_blank"{{ end }}>{{ .Name }}</a>

@onedrawingperday
You make a good point. I used js as a one-solution-fits-all. This way too, links inside markdown content files are attributed correctly.
I could limit the scope of the js script to the element housing {{ .Content }}. But then I would need to add this beautiful Hugo conditional {{ if strings.HasPrefix .URL "http" }} in at least one other place.
I'm curious if this 👇 was the issue. If it's not, I will need to explore other remedies.

No the issue is not fixed with your latest commit @onweru
I updated your theme on the Hugo Themes website -just now- and the links still open in a new tab.
see: https://themes.gohugo.io/theme/compose/docs/

@onweru
I confirm that your recent commits have fixed this issue.
The theme was updated in https://github.com/gohugoio/hugoThemes/commit/abfc371b865ccff1bc201bde7710efdd96c10ffb and the changes will be published once the Hugo Themes deploy concludes.
Thanks.

@onedrawingperday
Thanks a lot for your support


[MeshModel] Server: Support multi-zone use of capability registry assets

Current Behavior
Currently, integrated systems that Meshery allows users to configure have their individual system (e.g. Kubernetes) components (e.g. Pod) registered into Meshery’s Database (into the Capabilities Registry tables) upon boot. Part of this registration process includes a depositing of component-specific icons (as light and dark mode SVGs) onto Meshery’s filesystem here - https://github.com/layer5io/meshery/tree/master/server/meshmodel/components.
This path isn’t accessible by Meshery UI as a client of the Nextjs “web server” under its exposed path, /ui/public/static here - https://github.com/layer5io/meshery/tree/master/ui/public/static - where other images are made available.
Desired Behavior
Rather than replicating the SVG files (or splitting them out from their respective component packages), ideally, we can configure Nextjs to also expose the images under the /server/meshmodel/components folders.
Implementation
Use Nextjs's multi-zones capability to expose capability registry assets.
Acceptance Tests

/server/meshmodel directory contents are accessible over http under <meshery-server-host>/static/meshmodel`

Mockups

Contributor Guides and Handbook

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

Currently working through this, hopefully will have a solution ASAP.

Closing this as we decided to do a different route.
May revisit this in the future.


Update poison dep to be more flexible

Hi @sldab I was working on couple of projects and had different versions of poison. This PR changes the dependency spec.

Thanks. Merged.


new System.noConflict() function to restore state of the global System

Hi,
here is a little function you can use to restore the previous state of the global System var if you are using more than one System Instance.

Thanks for posting. I agree it makes sense to have a more cautious polyfill option and have created an issue for this in https://github.com/ModuleLoader/es6-module-loader/issues/443. I don't want to extend the API though as we need to adhere to the spec.


[Feature] Router Allow to configure the TLS certificate and the TLS truststore

Willingness to contribute
Yes. I can contribute a fix for this bug independently.
Feature Request Proposal
Add TLS configuration on the Router
Motivation

What is the use case for this feature?

In order to configure properly TLS we need a way in the Router Service to fully configure TLS.
This is required in order to configure properly authentication that is based on TLS client mutual auth
Details
No response
What component(s) does this bug affect?

[ ] Controller: This is the control-plane for Venice. Used to create/update/query stores and their metadata.
[X] Router: This is the stateless query-routing layer for serving read requests.
[ ] Server: This is the component that persists all the store data.
[ ] VenicePushJob: This is the component that pushes derived data from Hadoop to Venice backend.
[ ] Thin Client: This is a stateless client users use to query Venice Router for reading store data.
[ ] Fast Client: This is a stateful client users use to query Venice Server for reading store data.
[ ] Da Vinci Client: This is an embedded, stateful client that materializes store data locally.
[ ] Samza: This is the library users use to make nearline updates to store data.
[ ] Admin Tool: This is the stand-alone client used for ad-hoc operations on Venice.

Looks like this is "almost" supported today but it's a bit of an oversight.  When we deploy the services internally in LinkedIn all the server java objects are wrapped in logic which interfaces with our config system.  This then invokes the RouterServers constuctor which has utility for passing in an SSLFactory which would configure everything you need.
Unfortunately main doesn't seem to be doing that, it's building a generic SSLFactory.
I think the trick to make this work would be to make getVeniceLocalSsLProperties in SslUtils take a props argument and then change getVeniceLocalSslProperties do getOrDefault calls.
Should be a pretty straightforward patch.  Let me know if you have any questions (or if you are in a pinch you can reach out on slack and can get a patch through quickly).


[CT-1355] Add column type constraints as dbt native configs

Problem:

Data types are created after the fact(assumed by the database) and/or a dbt user has to explicitly cast data types in their SQL. Because of this, there isn't much incentive to assure dbt models have expected data types for consumers to depend on(it's tedious work :/). This may not be a problem for 50 dbt models, but becomes a big problem when you have 2000+ dbt models. There should be an easier way to configure data type constraints for any dbt models and provide options to enable/disable data type enforcement(think: what mypy does for the python static typing experience).

Research:

Postgres Constraints: https://www.postgresql.org/docs/current/ddl-constraints.html#id-<IP_ADDRESS>.6
Snowflake ONLY enforces not null constraints, the rest are nice to have metadata: https://docs.snowflake.com/en/sql-reference/constraints-overview.html#supported-constraint-types
BigQuery Macro Demo: https://www.loom.com/share/1f1f190e66254d12984962c613e8082d

Considerations:

have the configs embedded in dbt-core and then each adapter can take stable configs to adjust table materialization macros
Enforce column positions based on schema config
dbt-core contains the hub of native configs while adapters own specific implementations as each database has nuances to which enforceable constraints are valid.
think about not null and default values for a table
Work with Jon Neo from Canva
Include a check constraint as pgsql enables it?

Team:

@jonathanneo
Need one more person

Related discussion: https://github.com/dbt-labs/dbt-core/discussions/5244#discussioncomment-3880254

Example BigQuery Constraints Macro:https://gist.github.com/sungchun12/f7ea081773ae824a83294649530d6e41

What if constraints could be defined once within a subfolder path in dbt_project.yml and then in automatically creates not null constraints in the DDL?

Closing in favor of this: https://github.com/dbt-labs/dbt-core/issues/6079


[stable/mysql] ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

Describe the bug
I run helm install --name my-release stable/mysql and get the root password as described. When I attempt to login as root WITHIN the pod (kubectl exec -it pod/my-release-mysql-<pod_id> mysql -h <IP_ADDRESS> -u root -p<ROOT-PASSWORD>), I get: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
Note that I do not use any PV as in #9942 and I used a fresh minikube "cluster".
Version of Helm and Kubernetes:
Helm:
Client: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.12.1", GitCommit:"02a47c7249b1fc6d8fd3b94e6b4babf9d818144e", GitTreeState:"clean"}
Kubernetes:
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T09:18:22Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Which chart:
stable/minio

Same problem here.


How to use complex expressions

I'm trying to use expression to display a variable as another type. I'm trying to use the expression view for that but I can"t manage to get it working (I'm not sure if this is supported)
Let"s say I have
  int i = 1234;

I try to do the following in the expression window
 i int = 1234
 (float) i: Syntax error: (float) i
                                   ^
 (float*) &i: Syntax error: (float*) &i
                                   ^
>

These expressions work in the repl
dap> expr *((float*) &i)
dap>
(float) $1 = 1.7292023E-42
dap> expr (float*) &i
dap>
(float *) $2 = 0x00007fffffffd6ec
dap>

Any Idea how to get this to work?
I know that this has probably something to with nvim-dap and not nvim-dap-ui. So if you could tell what function is used under the hood I could use that to look more into it. Thanks :pray:

Evaluation is done by default with the hover context (see the evaluate request in the DAP spec) because hovering should have no side effect such as changing the value of a variable.
You can pass a different context e.g. dapui.eval(<expr>, { context = "repl" })  to change the behaviour and it should work as in the REPL.

Sorry I was being unclear. I was talking about the watches window (the one that per default says "No Expressions".
Is this possible there? Looks like it only supports add({expr})

Update:
after not even being able to use pui.eval(<expr>, { context = "repl" }) correctly I notices that I should probably not use (what I believe are) lldb commands e.g expr *((float*) &i) and instead use the syntax defined by the dap client. (e.g https://github.com/vadimcn/codelldb/blob/master/MANUAL.md#expressions)
So dapui.eval("?/nat (float) i", { context = "repl" }) works as expected.
Of course this works only for the hover windows and not for the watches

But changing the context here to repl already helps https://github.com/rcarriga/nvim-dap-ui/blob/master/lua/dapui/components/watches.lua#L91

of course regular hover expression won't work anymore (see first line).
Do you think it would be reasonable to add something like control symbol that switches the context from watch to repl


[Problem] How to deploy multiple schedulers on standalone mode but not docker

As the title.

Hi @smallzhongfeng. Do you want to deploy ballista with multiple schedulers outside of kubernetes? Standalone mode does not currently support multiple schedulers as all it does is spins up the scheduler server in a tokio task with a hardcoded bind address of localhost:50050. But if you just want to deploy a cluster without kubernetes or docker then it is possible (although I don't know of anyone who has done it that way).

Is there any yaml

There is a helm chart, and an example kubernetes.yaml.

I just want to be able to deploy multiple schedulers to ensure high availability of the scheduler.

I have not deployed Ballista in a production environment, much less HA so I will let others share their experience.
My understanding is that as long as you set shared storage (etd, sled, etc) that is all that is required. (And a load balancer, as mentioned before - I think the helm chart did that)

I just want to be able to deploy multiple schedulers to ensure high availability of the scheduler. @thinkharderdev

So you have two options:

Out of the box support for HA scheduler. As @avantgardnerio as long as you configure a storage backend that can be shared between the schedulers then this should work out of the box. There is already a storage backend implemented with etcd that you can use out of the box, but implementing a custom backend is relatively straightforward if you want to use some other DB or KV store. However the shared storage and distributed locking can add a significant amount of overhead.
If you need high throughput on task scheduling then you can implement an API layer in front of the scheduler that can route calls to the correct scheduler and then have schedulers use only in-memory state. The API layer would need to know which scheduler "owns" each query and route status requests to the correct scheduler.

Option 2 is what we have done in our deployment. We have multiple schedulers, each using an in-memory JobState and an API layer in front which routes calls to the appropriate scheduler. We also use a shared ClusterState based on redis (not yet upstreamed but it is relatively straightforward to implement). This gives all the schedulers a consistent view of the executor task slots and with a little bit of redis server-side scripting doesn't require any distributed locks.
One downside of this approach is that the job state is volatile so if a scheduler dies then all jobs running on it are lost. If you are running relatively short-duration queries then this is not a huge issue (at least for us) since the scheduler will try and complete any in-flight jobs before it shuts down so you can set up your deployment such that the schedulers have a shutdown grace period sufficient to complete any outstanding work.

Thanks for your patient reply. I got it. 👍
But I still have a question. If we use the push policy, will the task fail when the scheduler switches? Or will the running tasks before the switch fail?

Thanks for your reply again!
Last question. Can we extract the task scheduling part of the scheduler and make it a thread like the spark driver, so that when the scheduler hangs up, it will not affect the execution of the task.

local:
INFO tokio-runtime-worker ThreadId(02) ballista_scheduler::state::executor_manager: Reserved 0 executor slots in 588.435µs
etcd:
INFO tokio-runtime-worker ThreadId(02) ballista_scheduler::state::executor_manager: Reserved 0 executor slots in 77.631762ms

These days, I am testing the performance of etcd as a storage, but I found that the performance is very poor, especially when applying for and releasing resources. The time spent here is much longer than the local storage mode. Here I suspect this is the global distributed lock. Cause, do you have any suggestions here? @thinkharderdev @avantgardnerio

Hi @smallzhongfeng, could you explain the reason of using multiple schedulers? Is it just for HA or worried about the performance of single scheduler for task scheduling? If for HA, is it acceptable that all jobs scheduled on a scheduler fail when a scheduler is down?

local:
INFO tokio-runtime-worker ThreadId(02) ballista_scheduler::state::executor_manager: Reserved 0 executor slots in 588.435µs
etcd:
INFO tokio-runtime-worker ThreadId(02) ballista_scheduler::state::executor_manager: Reserved 0 executor slots in 77.631762ms

These days, I am testing the performance of etcd as a storage, but I found that the performance is very poor, especially when applying for and releasing resources. The time spent here is much longer than the local storage mode. Here I suspect this is the global distributed lock. Cause, do you have any suggestions here? @thinkharderdev @avantgardnerio

Yes, we found the same with etcd. I would suggest that if you don't need HA then use a single scheduler with in-memory state. If you need HA, what we did is implement ClusterState backed by redis. Haven't found the time to upstream the implementation yet but it's relatively simple. I will try to find some time to do that soon but the gist is that you use an hmap for holding the free task slots (so a map executor_id -> task_slots) and then a lua script for atomic reservation/freeing. Something roughly like:
const RESERVATION_SCRIPT: &str = r#"
local desired_slots = tonumber(ARGV[1])
local s = {}
for i=2, #ARGV do
    local exists = redis.call('HEXISTS', KEYS[1], ARGV[i])
    if( exists == 1 ) then
        local value = redis.call('HGET', KEYS[1], ARGV[i])
        local slots = tonumber(value)
        if( slots >= desired_slots ) then
            s[i - 1] = desired_slots
            local inc = -desired_slots
            redis.call('HINCRBY', KEYS[1], ARGV[i], inc)
            desired_slots = 0
        elseif slots == 0 then
            s[i - 1] = 0
        else
            s[i - 1] = slots
            local inc = -slots
            redis.call('HINCRBY', KEYS[1], ARGV[i], inc)
            desired_slots = desired_slots - slots
        end
    else
        s[i - 1] = 0
    end

    if( desired_slots <= 0 ) then
        break
    end
end
return cjson.encode(s)
"#;

const CANCEL_SCRIPT: &str = r#"
local cancelled = 0
for i=2, #KEYS do
    local exists = redis.call('HEXISTS', KEYS[1], KEYS[i])
    if( exists == 1 ) then
        local inc = tonumber(ARGV[i - 1])
        redis.call('HINCRBY', KEYS[1], KEYS[i], inc)
        cancelled = cancelled + inc
    end
end
return cancelled
"#;

const SLOTS_KEY: &str = "task-slots";

impl ClusterState for MyRedisState {
    async fn reserve_slots(
        &self,
        num_slots: u32,
        _distribution: TaskDistribution,
        executors: Option<HashSet<String>>,
    ) -> Result<Vec<ExecutorReservation>> {
        if num_slots == 0 {
            return Ok(vec![]);
        }

        if let Some(executors) = executors {
            let mut connection = self.get_connection().await?;

            let script = Script::new(RESERVATION_SCRIPT);

            let mut script = script.key(SLOTS_KEY);
            script.arg(num_slots);

            if !executors.is_empty() {
                let executor_ids: Vec<String> = executors.into_iter().collect();
                for executor_id in &executor_ids {
                    script.arg(executor_id);
                }

                let result: String = match script.invoke_async(&mut connection).await {
                    Ok(result) => result,
                    Err(e) => {
                        timer.stop_and_discard();
                        return Err(into_ballista_error(e));
                    }
                };

                let reservations = serde_json::from_str::<Vec<u32>>(&result).map_err(|e| {
                    BallistaError::General(format!(
                        "Error executing reservations, unexpected response from redis: {e:?}"
                    ))
                })?;

                let reservations: Vec<ExecutorReservation> = executor_ids
                    .into_iter()
                    .zip(reservations)
                    .flat_map(|(id, reserved)| {
                        (0..reserved).map(move |_| ExecutorReservation::new_free(id.clone()))
                    })
                    .collect();

                return Ok(reservations);
            }
        }

        Ok(vec![])
    }

    async fn cancel_reservations(&self, reservations: Vec<ExecutorReservation>) -> Result<()> {
        let mut connection = self.get_connection().await?;

        if !reservations.is_empty() {
            let script = Script::new(CANCEL_SCRIPT);
            let mut script = script.key(SLOTS_KEY);

            let reservations = reservations
                .into_iter()
                .group_by(|r| r.executor_id.clone())
                .into_iter()
                .map(|(key, group)| (key, group.count()))
                .collect::<HashMap<String, usize>>();

            for (executor_id, slots) in reservations {
                script.key(executor_id);
                script.arg(slots);
            }

            let cancelled: u64 = match script.invoke_async(&mut connection).await {
                Ok(cancelled) => {
                    cancelled
                }
                Err(e) => {
                    return Err(into_ballista_error(e));
                }
            };

            debug!("Cancelled {} reservations", cancelled);

            Ok(())
        } else {
            Ok(())
        }
    }
}

Not that this only supports TaskDistribution::Bias and for round-robin task distribution you would need a different lua script (which we have not implemented) but it could be done in principle.

could you explain the reason of using multiple schedulers? Is it just for HA or worried about the performance of single scheduler for task scheduling?

Hi @yahoNanJing  At present, we intend to support ha

If for HA, is it acceptable that all jobs scheduled on a scheduler fail when a scheduler is down?

We cannot accept the failure of all the jobs in the switching process, we are supporting the retry of the jobs in the switching process


keycloak should run on port 80 in the container

Exposing ports 80 and 443 as the default http/https ports avoids problems with token issuer

@leafty we'd like to be able to run keycloak on port 80 in the container so we don't have to do a redirect from 80 to 8080 outside the container -- see the docker-compose in #133. When I try to set the port using the jboss.http.port option, everything breaks -- could you please have a look?

Is this still an issue?

can close for now


Allow Creation of Clone and Restore snapshot from thick-to-thin and thin-to-thick

This issue is to track the feature support to allow the below operations

Allow Restore of snapshot from Thick PVC to Thin PVC
Allow Restore of snapshot from Thin PVC to Thick PVC

Note:- cloning might only be supported if one deleted the thick storageclass and recreates the storageclass with the same name with thick provisioned enabled.

csi-addons/spec#15 might be an alternative to support moving from thick to thin.
I do not know what a good reason is to use thick-provisioning and then clone thin volumes from it. What is the use-case?


How to define packet size for muxer ?

Hey ! First of all, thank you a lot for this repo, it's simply awesome :).
The question is. How can i set packet size for muxer ?
Using the native library there's a field for AVFormatContext - unsigned int packet_size. But how can i set it here ?
Since i'm using the muxer for an RTP output it requires a certain number of packets defined.
Thanks in advance!

Hi! Thank you.
You can set any option that would be normally accepted by the ffmpeg app using the MuxerBuilder::set_option() and Muxer::set_option() methods. In your case, it'd be probably something like:
let mut builder = Muxer::builder()
    .set_option("packetsize", 1234);

...

In future versions, I'll add methods for listing available options.

Hi! Thank you.
You can set any option that would be normally accepted by the ffmpeg app using the MuxerBuilder::set_option() and Muxer::set_option() methods. In your case, it'd be probably something like:
let mut builder = Muxer::builder()
    .set_option("packetsize", 1234);

...

In future versions, I'll add methods for listing available options.

Wow. I tried using pkt_size before and it didn't help. But for some reasons packetsize works great.
Thank you a lot !
Closing the issue :)

Yes. For some reason, FFmpeg devs decided to name this option differently than the struct field itself. Anyway, if you list all available ffmpeg options in command line, you'll find this:
$ ffmpeg -h full
...
  -packetsize        <int>        E......... set packet size (from 0 to INT_MAX) (default 0)
...

Yes. For some reason, FFmpeg devs decided to name this option differently than the struct field itself. Anyway, if you list all available ffmpeg options in command line, you'll find this:
$ ffmpeg -h full
...
  -packetsize        <int>        E......... set packet size (from 0 to INT_MAX) (default 0)
...

Oh, thanks a lot for the hint. Definitely gonna use it in a future.


Support deterministic relative include path in .clangd file

It seems there is no mechanism to support relative path expansion. For example
CompileFlags:
  Add:
  - -Iinclude/path/relative/to/.clangd

.clangd file can generally check into the repo, then the repo path will not be deterministic, adding a mechanism to dynamically determine the path will be greatly helpful.
For (a poor) example
Var:
  dot_clangd_path: dirname(__FILE__)
CompileFlags:
  Add:
  - -I${dot_clangd_path}/include/path/relative/to/.clangd

So when you open some subdirectory, those portion of config will still work correctly.
Note that I never have it work correctly with relative include path. So I do not have idea whether will it work and how it works.

Doesn't the compiler perform relative path expansion at the time it processes the -I argument?

Then relative to what? When you open a subdirectory of a project that might have some .clangd files nested deep, and red squiggle occurs, I bless you have time to figure it out.
So when a relative path occurs in the configuration, where does it relative to should be made clear in the document. If it is relative to the current working directory where the clangd server starts up, then there should be a way to specifiy where you exactly want it to relative to, regardless of the execution environment of the clangd server instance.

I expect it would be interpreted relative to the directory in which the compile command is invoked, i.e. the "directory" field of the entries in compile_commands.json.

The compile command is basically a pair {working directory, argv}. When argv are parsed by the driver, relative paths are resolved against the working directory.
First: what are you really trying to do?
If you have no compilation database, you should use compile_flags.txt instead of .clangd, this will set the working directory (as it sets rather than modifies the flags).
Are you trying to patch the compile commands from an existing CDB, to add extra includes?
The CompileCommands section in .clangd file modifies the argv textually, we can't reasonably say "relative paths are resolved against .clangd where they're specified, as the config parser doesn't even know which args specify paths (they're just strings).
Things we could possibly do:

provide a semantic config option like AddIncludeDirectory: foo that would be equivalent to CompileFlags: Add: -I/absolute/path/foo.
provide a config option to override the working directory for the compile command (though this would mess up relative paths specified in other places)
support some textual expansion like in your example

Sometimes I don't have CBD, because it requires extra work or the build system does not support generating it. I do it for patching a subdirectory (for example, opencl code). I find a .clangd file can be the last resort and can be very useful in this case. But not that useful if I cannot check it into the VCS. So to checkin, the relative path should be properly handled first.
So I think textual expansion just fit in this case. And it can be simplified as
CompileFlags:
  Add:
  - -I${__FILE__}/../include/path/relative/to/.clangd

I think this change will be minimal. After loading, expand every string value and then form a static config.

compile_flags.txt doesn't fit the case because I use If block to pattern match files.
Anyway, If someone can finialize a design, I can implement it myself and contribute a PR :)

Really need this if compilation database is not available.
Otherwise looks like need to adjust .clang file every time I move code to another location.

+1

+1

Would appreciate this functionality. For now I have added a bash script to generate the .clangd file for each user so the include paths are correct for their repository checkout path. Would be great to be able to commit the .clangd file to the repo and avoid requiring a setup script.

I made a review for a solution to this https://reviews.llvm.org/D142981

+1 to what @adenisonafifi said in the comment at https://github.com/clangd/clangd/issues/1038#issuecomment-1326557891

I expect it would be interpreted relative to the directory in which the compile command is invoked, i.e. the "directory" field of the entries in compile_commands.json.

Thanks.
At least that's solved it for me.
The problem is that the path is unintuitive.
Without a compilation database, it's relative to the .clangd file.
With it, it's relative to the compile_commands.json file.
And if you use CMake, it's usually a subdirectory.

I made a review for a solution to this https://reviews.llvm.org/D142981

Does this method also work for compile_commands.json? I have a compile_commands.json generated in Linux and I want it to work in Windows. I don't want to have to modify the common path of for the "directory" entries every time.

Does this method also work for compile_commands.json? I have a compile_commands.json generated in Linux and I want it to work in Windows. I don't want to have to modify the common path of for the "directory" entries every time.

No, compile_commands.json files are meant to be generated at build time on the machine on which they're consumed. They're not portable in any meaningful sense.

Is this possible yet?

+1 on having relative path supported in .clangd directly, e.g., relative to the location of .clangd.

It would be interesting to have this and make the config more portable.


Crash when attempting to launch Dolphin when using Wayland

I'm currently working on getting Dolphin to run with Wayland, using Sway 1.4 on wlroots 0.10.1. When running Dolphin with QT_QPA_PLATFORM=wayland and attempt to launch a game, Dolphin will error (as it has not yet implemented wayland-egl), then upon closing the error popup sway will crash. When running in debug mode, I only get the following output:
sway: types/wlr_surface.c:663: wlr_surface_set_role: Assertion `surface->role_data == NULL' failed.
2020-03-20 03:18:39 - [common/ipc-client.c:88] Unable to receive IPC response
[080 03:18:39.961701] [glfw error 65544]: Wayland: fatal display error: Broken pipe
fish: 'sway -d -c debug' terminated by signal SIGABRT (Abbruch)

This is a failure of the assertion assert(surface->role_data == NULL);

Can you post the WAYLAND_DEBUG=1 log for Dolphin?

A copy is here: https://gist.github.com/artemist/1b2388ca30876c48f9b577de6fdac6f2

Please provide a stack trace. You can do so by compiling from source, reproducing the crash and then running coredumpctl gdb and then bt full.
Here are some instructions to compile from source: https://github.com/swaywm/sway/wiki/Development-Setup#compiling-as-a-subproject

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007f043c7f9548 in __GI_abort () at abort.c:79
#2  0x00007f043c7f942f in __assert_fail_base (fmt=0x7f043c959508 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x7f043d0af5c3 "surface->role_data == NULL", file=0x7f043d0af56f "types/wlr_surface.c", line=671,
    function=<optimized out>) at assert.c:92
#3  0x00007f043c807ad2 in __GI___assert_fail (assertion=assertion@entry=0x7f043d0af5c3 "surface->role_data == NULL",
    file=file@entry=0x7f043d0af56f "types/wlr_surface.c", line=line@entry=671,
    function=function@entry=0x7f043d0af810 <__PRETTY_FUNCTION__.7294> "wlr_surface_set_role") at assert.c:101
#4  0x00007f043d093778 in wlr_surface_set_role (surface=<optimized out>,
    role=role@entry=0x7f043d0d1f50 <xdg_toplevel_surface_role>, role_data=role_data@entry=0x2ba8a90,
    error_resource=<optimized out>, error_code=error_code@entry=0) at ../subprojects/wlroots/types/wlr_surface.c:674
#5  0x00007f043d07d734 in create_xdg_toplevel (xdg_surface=0x2ba8a90, id=52)
    at ../subprojects/wlroots/types/xdg_shell/wlr_xdg_toplevel.c:509
#6  0x00007f043c152abd in ffi_call_unix64 ()
   from /nix/store/hfs34n1yww7ba1m8gbrwrwh6v1y269m6-libffi-3.3/lib/../lib64/libffi.so.7
#7  0x00007f043c15179c in ffi_call_int ()
   from /nix/store/hfs34n1yww7ba1m8gbrwrwh6v1y269m6-libffi-3.3/lib/../lib64/libffi.so.7
#8  0x00007f043ca22370 in wl_closure_invoke ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
#9  0x00007f043ca1e7f2 in wl_client_connection_data ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
#10 0x00007f043ca20402 in wl_event_loop_dispatch ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
#11 0x00007f043ca1e9e5 in wl_display_run ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
#12 0x0000000000419bb0 in server_run (server=<optimized out>) at ../sway/server.c:216
#13 0x000000000040e73d in main (argc=<optimized out>, argv=<optimized out>) at ../sway/main.c:409
(gdb) bt full
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {0,<PHONE_NUMBER>43644,<PHONE_NUMBER>, 46854592, 46854592, 46854592, 46854592, 46854692, 46854692,
            46854592, 46854692, 0, 0, 0, 0, 0}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007f043c7f9548 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x2caf1c0, sa_sigaction = 0x2caf1c0}, sa_mask = {__val = {
              46854592,<PHONE_NUMBER>94976,<PHONE_NUMBER>, 0, 0, 0,<PHONE_NUMBER>0,<PHONE_NUMBER>62984,<PHONE_NUMBER>62832,
<PHONE_NUMBER>51456,<PHONE_NUMBER>32712, 0,<PHONE_NUMBER>524091136,<PHONE_NUMBER>17572, 0,<PHONE_NUMBER>32712}},
          sa_flags =<PHONE_NUMBER>, sa_restorer = 0x7f043d0af5c3}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f043c7f942f in __assert_fail_base (fmt=0x7f043c959508 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
    assertion=0x7f043d0af5c3 "surface->role_data == NULL", file=0x7f043d0af56f "types/wlr_surface.c", line=671,
    function=<optimized out>) at assert.c:92
        str = 0x2caf1c0 " \363\312\002"
        total = 4096
#3  0x00007f043c807ad2 in __GI___assert_fail (assertion=assertion@entry=0x7f043d0af5c3 "surface->role_data == NULL",
    file=file@entry=0x7f043d0af56f "types/wlr_surface.c", line=line@entry=671,
    function=function@entry=0x7f043d0af810 <__PRETTY_FUNCTION__.7294> "wlr_surface_set_role") at assert.c:101
No locals.
#4  0x00007f043d093778 in wlr_surface_set_role (surface=<optimized out>,
    role=role@entry=0x7f043d0d1f50 <xdg_toplevel_surface_role>, role_data=role_data@entry=0x2ba8a90,
    error_resource=<optimized out>, error_code=error_code@entry=0) at ../subprojects/wlroots/types/wlr_surface.c:674
        __PRETTY_FUNCTION__ = "wlr_surface_set_role"
#5  0x00007f043d07d734 in create_xdg_toplevel (xdg_surface=0x2ba8a90, id=52)
    at ../subprojects/wlroots/types/xdg_shell/wlr_xdg_toplevel.c:509
        __PRETTY_FUNCTION__ = "create_xdg_toplevel"
#6  0x00007f043c152abd in ffi_call_unix64 ()
   from /nix/store/hfs34n1yww7ba1m8gbrwrwh6v1y269m6-libffi-3.3/lib/../lib64/libffi.so.7
No symbol table info available.
#7  0x00007f043c15179c in ffi_call_int ()
   from /nix/store/hfs34n1yww7ba1m8gbrwrwh6v1y269m6-libffi-3.3/lib/../lib64/libffi.so.7
No symbol table info available.
#8  0x00007f043ca22370 in wl_closure_invoke ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
No symbol table info available.
#9  0x00007f043ca1e7f2 in wl_client_connection_data ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
No symbol table info available.
#10 0x00007f043ca20402 in wl_event_loop_dispatch ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
No symbol table info available.
#11 0x00007f043ca1e9e5 in wl_display_run ()
   from /nix/store/2fxkfprccnc8r62k0arzjlmj4k37ifk2-wayland-1.18.0/lib/libwayland-server.so.0
No symbol table info available.
#12 0x0000000000419bb0 in server_run (server=<optimized out>) at ../sway/server.c:216
No locals.
#13 0x000000000040e73d in main (argc=<optimized out>, argv=<optimized out>) at ../sway/main.c:409
        verbose = 0
        debug = 1
        validate = 0
        allow_unsupported_gpu = 0
        long_options = {{name = 0x4584cb "help", has_arg = 0, flag = 0x0, val = 104}, {name = 0x45b929 "config",
            has_arg = 1, flag = 0x0, val = 99}, {name = 0x4584d0 "validate", has_arg = 0, flag = 0x0, val = 67}, {
            name = 0x4584d9 "debug", has_arg = 0, flag = 0x0, val = 100}, {name = 0x45842f "version", has_arg = 0,
            flag = 0x0, val = 118}, {name = 0x4575ad "verbose", has_arg = 0, flag = 0x0, val = 86}, {
            name = 0x4584df "get-socketpath", has_arg = 0, flag = 0x0, val = 112}, {
            name = 0x4584ee "unsupported-gpu", has_arg = 0, flag = 0x0, val = 117}, {
            name = 0x4584fe "my-next-gpu-wont-be-nvidia", has_arg = 0, flag = 0x0, val = 117}, {name = 0x0,
            has_arg = 0, flag = 0x0, val = 0}}
        config_path = 0x238be60 "/home/artemis/.config/sway/debug"
        usage = 0x458870 "Usage: sway [options] [command]\n\n  -h, --help", ' ' <repeats 13 times>, "Show help message and quit.\n  -c, --config <config>  Specify a config file.\n  -C, --validate         Check the validity of the config file, th"...
        c = <optimized out>

I'm seeing two cases of xdg_wm_base@21.get_xdg_surface(new id xdg_surface@45, wl_surface@31) there. This is a bug in your client.

This sounds like it may be a Qt bug, I'll look into fixing it upstream

Wouldn't the fact that sway itself crashes too make it a sway bug?

@FreeFull #2077 fixes the crash


High latency

b/8334662
I'm consistently seeing hight latency numbers when using the GCD - about 1.5 seconds on average for queries (10 items or less) and single item writes ( < 1kb).
My ping time to the API endpoint is about 50ms, so discounting a 100ms round trip, that still leaves more than a second to GCD latency. This simply won't work for a server environment, certainly not one that scales. This is very surprising because I was expecting latencies much closer to GAE: https://code.google.com/status/appengine/detail/hr-datastore/2013/06/05#ae-trust-detail-hr-datastore-query-latency
Can we get a dashboard like the GAE HR Datastore status board? Possibly measuring latencies from Google Compute Engine instances and a few different AWS Regions?

I'm measuring the latency between google cloud compute engine and google datastore.
I'm performing a simple lookup() using the python client library.
The google performance dashboard says that my requests are consuming aroud 18 milliseconds. I assume that this is a server side metric and not a round trip metric.
Can someone refer me to the service level agreements behind the minimum round trip response times I should expect currently between google compute engine and google datastore?
For a trivial lookup, i'm experiencing around 60 milliseconds. I would expect around 20 milliseconds.
Here's the code, not that some of these files are not strictly correlated. Each is intended to be indicative on its own. If someone can show me some code with better latencies, then that would be great.
https://github.com/peterrham/projects/blob/master/google_cloud/read.py
Here's an example output file:
https://github.com/peterrham/projects/blob/master/google_cloud/read.out
In this example, I'm getting 77 milliseconds. I'm not trying to be statistically significant here, I'm just looking for some indicative guidance.
Data store latency here is under 15 milliseconds:
http://code.google.com/status/appengine/detail/datastore/2015/02/02#ae-trust-detail-datastore-get-latency
I also have a sample tcpdump ascii text out put file:
https://github.com/peterrham/projects/blob/master/google_cloud/40ms.txt
using this command line: (the time stamps are the delta values in between the packet events)
/usr/sbin/tcpdump -r tcpdump.out -nnq -ttt > 40ms.txt
For example, the TCP initial SYN is acked in 1 millisecond, so network latency does not seem to be a problem.
However, the ack from the lookup() request is over 40 milliseconds after the request.
Server side latency from the app engine logs is 9 milliseconds:
2015-02-02 16:15:55.753 /datastore/v1beta2/Lookup 200 9ms 0kb module=default version=ah-builtin-datastoreservice
<IP_ADDRESS> - - [02/Feb/2015:16:15:55 -0800] "POST /datastore/v1beta2/Lookup HTTP/1.1" 200 136 - - "ah-builtin-datastoreservice-dot-glowing-thunder-842.appspot.com" ms=10 cpu_ms=18 cpm_usd=0.000015 app_engine_release=1.9.17 instance=00c61b117c76349d57bd7ae2e3c635edd5c994da
any ideas? Are their any buffering configurations to setting to get the minimum latency?

Looks like the 40ms is related to Nagle's algorithm, although I do not think that it accounts for the delayed response which is over 40ms, but i'm not sure.
http://neophob.com/2013/09/rpc-calls-and-mysterious-40ms-delay/

Hi - we are doing some app and backend testing (prior to lunch) and are seeing this exact problem (looking at the ah-builtin-datastoreservice) we see times form 10-20ms to 2000ms. Do you have any timescale on a fix (or when you will apply any such fix already made to the current Beta channel)? Else we'll just migrate to Dynamo. Thanks.

I second the motion to move to Dynamo.

I am also disappointed with the performance I'm seeing from GCD. A couple patterns I've noticed: 1) the first request from a new datastore connection is always slow, and 2) batch mutations (even a batch of 100 entities) is painfully slow.

In my experience it is better than it was when this issue was opened. I used to see a lot more response times between 500ms and 1000ms. Most of my requests are between 200ms and 500ms. Perhaps I've segmented my data better?

@cerdmann @gcjc I would advise against moving to dynamodb if you're concerned about keeping costs as low as possible. I did an evaluation after hitting this issue and in my opinion dynamodb has one major drawback. You have to provision your read/write capacity ahead of time. This makes it difficult to adjust to a sudden spike in traffic while keeping costs down. There are some auto scaling libraries that attempt to help with this but their adjustment time is not very fast and dynamo currently limits you to scaling back down only a few times a day.
I'm really hoping this issue gets fixed soon as there's a huge node.js community that could really take advantage of GCD.

@obeleh it is hard to predict when there will be problems (and it seems to be any operation).
We will likely knock up some longer running tests internally to see if we can spot any patterns. However, we just ran a very short test and it seems to have improved, most requests between 150-220ms. Anyone else seen any improvement, just wondering if any changes in the app engine front end have been made?
@hbizira we really do want to use GCD, moving to dynamo will be the last resort.

@hbizira, thanks for the advice. I'm in the same boat as @gcjc in that we really want to use GCD, but looking into other options as the latency is killing us.

Just started using gcloud via nodejs and it's a bit disappointing. Getting from 200ms - 500ms approx latency. This is running on RH OpenShift (really AWS) in the US and EU. Tried my home connection too, all roughly the same.
I hope I'm doing something wrong :/

I got better results than that, but still not great. What do you mean
"AWS"? My timings were from a google compute host to google cloud datastore.
By the way, I'm trying out google "bigtable" which is in Beta. It promises
a great SLA sub 10 ms for 99 percentile I think. I have tried it out, but
have not measured the latency, but I believe it!
On Wed, Jul 8, 2015 at 4:54 PM, jonface<EMAIL_ADDRESS>wrote:

Just started using gcloud via nodejs and it's a bit disappointing. Getting
from 200ms - 500ms approx latency. This is running on RH OpenShift (really
AWS) in the US and EU. Tried my home connection too, all roughly the same.
I hope I'm doing something wrong :/
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/google-cloud-datastore/issues/5#issuecomment-119761913
.

I was referring to Amazon Web Services EC2 which is the underlying infrastructure of OpenShift. Isn't BigTable more expensive and overkill?
I'll check my timings with wireshark. Also my datastore is empty, so it's not like it's got millions of items in it.

OK, it's hard to tell exactly how long it's taking via wireshark due to the TLS but you can kind of guess.
TLS connection SYN to FIN - total 460ms
- TLS Application data at 126ms
- TLS Application data at 201ms
- TLS Application data at 399ms
- TLS Application data at 400ms
- TLS Application data at 455ms
What is interesting is that for every query, a new connection is setup/destroyed. Is this correct? Am I doing something wrong? Would it not be better to keep some connection pool and reuse connections?
Thanks

I realise I'm being impatient but what's the plan for this? Is this just a nodejs lib problem?

@pcostell Will this be addressed with v1beta3?

That is the goal, but we are still working on benchmarking v1beta3.

@pcostell I am seriously considering this service. Please post an update about this issue since it has been a while. Thanks for the great work.

@pcostell I am seriously considering this service. Please post an update about this issue since it has been a while. Thanks for the great work.

We are seeing much better latency numbers with v1beta3. v1beta3 has been a complete rewrite in our infrastructure and as such we are being very cautious about rolling it out. We hope to have it ready early this year.

Great!
On Fri, Jan 8, 2016 at 2:09 PM, Patrick Costello<EMAIL_ADDRESS>wrote:

We are seeing much better latency numbers with v1beta3. v1beta3 has been a
complete rewrite in our infrastructure and as such we are being very
cautious about rolling it out. We hope to have it ready early this year.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/google-cloud-datastore/issues/5#issuecomment-170141652
.

@pcostell What numbers are you seeing? Anything below 20 ms? That is the performance goal that we have, and it is easily achievable in DynamoDB. This issue is a dealbreaker for us for moving to Google Cloud right now.

I thought I would share some instrumentation as I found this issue and have been following it with interest. Below is 95th percentile and mean (in milli-seconds) for PUT requests moving from v1beta2 to v1beta3. Both show an increase of ~10x (~25ms & ~10ms) - good work guys! It would also be nice to get it out of beta endpoints for us to use in production. One thing worth mentioning is that the v1beta3 seems to be missing the transaction endpoint.
95th

Mean

Hi @leonardaustin, can you share details about what service(s) you're using? We haven't actually launched v1beta3 yet, so I'm wondering if we're talking about two different things.

@eddavisson Sure, I forked https://github.com/GoogleCloudPlatform/gcloud-golang and changed the url from v1beta2 to v1beta3.

gcloud-python has "already" made the switch (in a branch)

v1beta2 in main branch
[v1beta3][3] in custom branch

All features from v1beta3 are present, though the URIs are different.

Is there a list of changes I should see or can I just change the url and I'm set?

The v1beta3 API is not yet available to customers. We will be sure to post announcement here when it is.

If possible I would like to sign up for the public beta of the beta?

@pcostell, is v1beta3 still on track to "be released this quarter" as per  https://github.com/GoogleCloudPlatform/google-cloud-datastore/issues/34? :)

@faizalkassamalisc, we're busy reticulating splines and hopefully will have an update for you soon. Thanks for your patience!
@obeleh, the beta will be public and just requires you using the appropriate API clients. There will be no other sign-up beyond the normal project creation you do currently.

https://cloudplatform.googleblog.com/2016/04/Google-Cloud-Datastore-gets-faster-cross-platform-API.html

Is there a more technical page with latencies expressed as milliseconds? Thanks!

We don't have a page with latency numbers since it's a moving target as we continually improve the platform as well as depending upon both the location your Cloud Datastore was set up for as well as from where you are accessing it.
Keep in mind you cannot compare DynamoDB with Cloud Datastore directly as we're a functionally different service and in most cases are serving customers in a Multi-Regional instance vs merely Regional.

Why the first request from a new datastore connection is always slow?
Is there a way to fix this.

Hello Guys, It is probably the wrong place for ask this question but I didn't find any better place.
Could datastore using local disks play the same role of redis in session cache? Everything would be much easier and cheaper to our company.

Is this ever solved? Any one share how much are the latencies we can expect now on data store(Firestore with datastore mode vs native mode)?


Add plugin whitelist

Pull Request Template
⚠ Before Submitting a PR, read the Contributing Docs in full!
Summary
Please provide a brief summary of your changes and the related issue. Include any motivation and context that is relevant to your changes. If there are any dependencies necessary for your changes, please list them here.
Change Type
Please delete any irrelevant options.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
[ ] Translation update
[ ] Documentation update

Testing
Please describe your test process and include instructions so that we can reproduce your test. If there are any important variables for your testing configuration, list them here.
Test Configuration:
Checklist
Please delete any irrelevant options.

[ ] My code adheres to this project's style guidelines
[ ] I have performed a self-review of my own code
[ ] I have commented in any complex areas of my code
[ ] I have made pertinent documentation changes
[ ] My changes do not introduce new warnings
[ ] I have written tests demonstrating that my changes are effective or that my feature works
[ ] Local unit tests pass with my changes
[ ] Any changes dependent on mine have been merged and published in downstream modules.
[ ] New documents have been locally validated with mkdocs

Hi. I don't know if I understand what you're is this PR about, but I believe there is already something implemented with "filteredTools" in the librachat.yaml file
Check this out: https://docs.librechat.ai/install/configuration/custom_config.html#filteredtools


feat: super factorial

Plan to build on this adding

rewriting as product
relation to double factorial (mod p)
relation to determinant of Vandermonde matrix

LGTM. It looks like comments from other reviewers are resolved, so let's merge it.
bors merge


Make fact object updatable in database

Summary

Make fact object updatable as it is already for products.

Checklist

[x] Categorize the PR by setting a good title and adding one of the labels:
bug, enhancement, documentation, change, breaking, dependency
as they show up in the changelog

I could nitpick a little and make a propose that we use INSERT .. ON CONFLICT UPDATE .. and simplify all this, but it's fine as is and the products already work the same way :)

We plan to do major refactoring of exoscale-metircs-collector and cloudscale-metrics-collector by merging them. I would add your valuable suggestions there and make sure they are implemented.


Add shape dropdown to SphereMesh to switch between Sphere, Spheroid, and Ellipsoid

Add a shape option to SphereMesh
This pull request introduces a new shape property to the SphereMesh class, allowing users to switch between three shape controls: Sphere, Spheroid, and Ellipsoid. I think this gives users greater flexibility when using primitives and provides more intuitive ways of creating specific spherical meshes.

Sphere: A regular sphere controlled only by radius.
Spheroid: A sphere with adjustable height and radius.
Ellipsoid: A sphere with adjustable height, width, and depth.

I would have preferred to make the Sphere the default, as I think that's the most intuitive when setting a meshinstance to SphereMesh, but I didn't want to break any compatibility with existing scenes, so with this pr the default is Spheroid as that is whats currently in the engine.

//Default
[sub_resource type="SphereMesh" id="SphereMesh_5fsbk"]

//Sphere
[sub_resource type="SphereMesh" id="SphereMesh_aw0sx"]
shape = 0
radius = 1.0

//Spheroid
[sub_resource type="SphereMesh" id="SphereMesh_xoc33"]
height = 2.0

//Ellipsoid
[sub_resource type="SphereMesh" id="SphereMesh_gw5of"]
shape = 2
height = 1.5
width = 2.0
depth = 0.5

Example Project:
spheres.zip

You put the enum binds in the wrong place

Thanks for opening a pull request :slightly_smiling_face:
Feature pull requests should be associated to a feature proposal to justify the need for implementing the feature in Godot core. Please open a proposal on the Godot proposals repository (and link to this pull request in the proposal's body).


test: update liechtenstein snapshot

TL;DR

fixes snapshot of liechtenstein requested by https://github.com/spatie/holidays/pull/33#issuecomment-1903793698

Thanks!


[SUGGESTION] Slight improvements to the exit builtin

The commits are very atomic, so you can pick and choose which ones you'd agree with.

Testcases to fix:

[ ]<PHONE_NUMBER>854775808  ->  numeric argument required
[ ] -9223372036854775809  ->  numeric argument required
[ ] -922337203685477580  ->  52
[ ]<PHONE_NUMBER>000000000000000009223372036854775807  ->  255


Intial API and implementation of ReactiveLoadBalancer

/cc @twicksell
The interfaces are in spring-cloud-commons.
An implementation, using reactor is in spring-cloud-loadbalancer.
For reference, original PR #227

Looks like a good start!

Squashed and merged via 1fa513194b07c949f21ab73af40f3ff5464d93f5


[bug] Cannot connect to remote mqtt broker

Describe the bug
Hi!
I'm creating a desktop app that will provide data for MQTT broker and I'm trying to connect with the broker using the MQTT.js library. Unfortunately, there is strange behaviour from my tauri environment. I had been reading a lot about it throughout Internet but didn't come across any actual solution. The problem is, when I try to connect to mqqt broker I provider an object like this:
const broker = mqtt.connect({         host: 'xxxxxxxxxxxxxxxxxxxxxx.s1.eu.hivemq.cloud', 	port: 8883, 	protocol: 'mqtts', 	username: 'user', 	password: 'secret', })
From nodejs environment, there is no problem in such configuration. The problem starts to occur while I'm trying to connect like this through tauri. I tried to modify "dangerousRemoteDomainIpcAccess" in config, to be honest I don't even know it is meant for it, but I tried at least, didn't work.
In console in application appears the following error: WebSocket connection to 'wss://localhost:8883/mqtt' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
It seems like rust and tauri change the url that is being used to the localhost, not allowing external addresses. With fetch everything works fine, when I try the mqqt/ws protocol it occurs. Any suggestions?
Thanks a lot in advance!
Reproduction
No response
Expected behavior
No response
Platform and versions
[✔] Environment
    - OS: Windows 10.0.22621 X64
    ✔ WebView2: 117.0.2045.47
    ✔ MSVC: Visual Studio Community 2022
    ✔ rustc: 1.72.1 (d5c2e9c34 2023-09-13)
    ✔ Cargo: 1.72.1 (103a7ff2e 2023-08-15)
    ✔ rustup: 1.26.0 (5af9b9484 2023-04-05)
    ✔ Rust toolchain: stable-x86_64-pc-windows-msvc (default)
    - node: 18.15.0
    - npm: 9.7.1

[-] Packages
    - tauri [RUST]: 1.5.1
    - tauri-build [RUST]: 1.5.0
    - wry [RUST]: 0.24.4
    - tao [RUST]: 0.16.4
    - tauri-cli [RUST]: not installed!
    - @tauri-apps/api [NPM]: 1.5.0
    - @tauri-apps/cli [NPM]: 1.5.1

[-] App
    - build-type: bundle
    - CSP: unset
    - distDir: ../dist
    - devPath: http://localhost:1420/
    - framework: React
    - bundler: Vite

Stack trace
No response
Additional context
No response

Describe the bug
https://discord.com/channels/616186924390023171/1228369933071814706
the app's hmr is ok on windows, but on android, annoying auto refresh is still reproducible, I have to turn down hot reload by hmr:false
Reproduction
I do a clean init pnpm create tauri --beta (use vue), then pnpm tauri android init, pnpm tauri android dev
Expected behavior
enable hmr on mobile while refresh the webpage only when it needs.
Platform and versions
[✔] Environment
    - OS: Windows 10.0.22631 X64
    ✔ WebView2: 120.0.2210.91
    ✔ MSVC: Visual Studio Community 2022
    ✔ rustc: 1.77.2 (25ef9e3d8 2024-04-09)
    ✔ cargo: 1.77.2 (e52e36006 2024-03-26)
    ✔ rustup: 1.27.0 (bbb9276d2 2024-03-08)
    ✔ Rust toolchain: stable-x86_64-pc-windows-msvc (default)
    - node: 20.11.1
    - pnpm: 8.15.6
    - npm: 10.2.4
    - bun: 1.0.30

[-] Packages
    - tauri [RUST]: 2.0.0-beta.14
    - tauri-build [RUST]: 2.0.0-beta.11
    - wry [RUST]: 0.39.0
    - tao [RUST]: 0.27.0
    - @tauri-apps/api [NPM]: 2.0.0-beta.7
    - @tauri-apps/cli [NPM]: 2.0.0-beta.12

[-] App
    - build-type: bundle
    - CSP: unset
    - frontendDist: ../dist
    - devUrl: http://localhost:1420/
    - framework: Vue.js
    - bundler: Vite

Stack trace

in the pnpm tauri android dev console, I see the following error spamming:

04-12 15:19:29.626  9990  9990 I Tauri/Console: File: http://tauri.localhost/@vite/client - Line 524 - Msg: [vite] server connection lost. polling for restart...
9990  9990 I Tauri/Console: File: http://tauri.localhost/@vite/client - Line 524 - Msg: [vite] server connection lost. polling for restart...
9990  9990 I Tauri/Console: File: http://tauri.localhost/@vite/client - Line 524 - Msg: [vite] server connection lost. polling for restart...
9990  9990 I Tauri/Console: File: http://tauri.localhost/@vite/client - Line 524 - Msg: [vite] server connection lost. polling for restart...
9990  9990 I Tauri/Console: File: http://tauri.localhost/@vite/client - Line 524 - Msg: [vite] server connection lost. polling for restart...

><EMAIL_ADDRESS>tauri D:\Documents\Code\rust\tauri-app
> tauri "android" "dev"

    Info Detected connected device: Pixel_3a_API_34_extension_level_7_x86_64 (sdk_gphone64_x86_64) with target "x86_64-linux-android"
    Info Using <IP_ADDRESS> to access the development server.
    Running BeforeDevCommand (`pnpm dev`)

><EMAIL_ADDRESS>dev D:\Documents\Code\rust\tauri-app
> vite

  VITE v5.2.8  ready in 1389 ms

  ➜  Local:   http://localhost:1420/
  ➜  Network: http://<IP_ADDRESS>:1420/
  ➜  Network: http://<IP_ADDRESS>:1420/
  ➜  Network: http://<IP_ADDRESS>:1420/
  ➜  Network: http://<IP_ADDRESS>:1420/

in the browser console, I see the following error:

WebSocket connection to 'ws://tauri.localhost:1421/' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
[vite] server connection lost. polling for restart...
client.ts:327

GET http://tauri.localhost:1421/ 400 (Bad Request)
已导航到http://tauri.localhost/

Additional context
It seems happening after I upgrade npm packages, rust crates, and rust itself by rustup.

Mine solved: clash vpn proxy conflict to websocket, turn down TUN mode