Prisma not properly set isolation level in MySQL and MariaDB

Bug description
When I start transaction with Prisma.TransactionIsolationLevel.ReadUncommitted:
Prisma runs SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED after START TRANSACTION, so it doesn't affect the current ongoing transaction as is stated in mysql manual https://dev.mysql.com/doc/refman/8.0/en/set-transaction.html#set-transaction-isolation-level
Similar sequelize issue and PR.
How to reproduce

Create Prisma instance with mysql dialect.
Run sample code.
See output and logs.

Expected behavior
A transaction isolation level is set correctly.
Prisma information
import { PrismaClient, Prisma } from '@prisma/client';

const prisma = new PrismaClient({
  log: [{ level: 'query', emit: 'event' }],
});

prisma.$on('query', (event) => {
  console.log(event.query);
});

await prisma.$transaction(async (tx) => {
    const isolationLevel = await tx.$queryRaw`SELECT @@transaction_isolation`;
    console.log(isolationLevel); /** REPEATABLE-READ (expected: READ-UNCOMMITTED) */
  },
  {
    isolationLevel: Prisma.TransactionIsolationLevel.ReadUncommitted,
  }
);

Environment & setup

OS: Debian
Database: MySQL, MariaDB
Node.js version: 21.6.1

Prisma Version
5.8.0

Thanks for the report @jonathanblade.
This sounds unexpected, it should be running it before START TRANSACTION: https://github.com/prisma/prisma-engines/blob/main/quaint/src/connector/mysql/native/mod.rs#L294-L296. I would also expect this to be caught by our tests.
We should reproduce it ASAP and see what the problem is.

Thanks for the report @jonathanblade.
This sounds unexpected, it should be running it before START TRANSACTION: https://github.com/prisma/prisma-engines/blob/main/quaint/src/connector/mysql/native/mod.rs#L294-L296. I would also expect this to be caught by our tests.
We should reproduce it ASAP and see what the problem is.

I found it here (if it's help you) when creating isolation level tests for prisma-client-py.

UPD: The engine seems to really check it.

@aqrln possibly related to this issue: setting the SERIALIZABLE transaction level on PlanetScale (via the driverAdapters preview feature) results in the following error:

Transaction characteristics can't be changed while a transaction is in progress (errno 1568) (sqlstate 25001) during query: SET TRANSACTION ISOLATION LEVEL SERIALIZABLE

(In fact, PlanetScale is excluded by this test).

I believe this is not a bug.
There are 3 scopes for isolation levels:

Global
Session
Next transaction

Prisma uses the following syntax:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
BEGIN;
--- transaction content
COMMIT;

Which sets the transaction level for the very next transaction of the session.
Unfortunately though, there is currently no way of querying this specific level. Regardless of where SELECT @@transaction_isolation is executed, it always returns the SESSION transaction level. It is equivalent to SELECT @@SESSION.transaction_isolation.
Sources:

https://bugs.mysql.com/bug.php?id=53341
MySQL docs (all the way at the bottom)

https://stackoverflow.com/a/50941611

To confirm what I'm saying is correct, you should be able to create a situation that proves the correct isolation level is used. In light of the MySQL bug reported above, I didn't take the time to craft such example.
I will now close this issue. Thanks for reporting 🙏


Add throw helpers for ObjectDisposedException

This should mimic the existing ArgumentOutOfRangeThrowHelper

@amdbrujin I have something similar on a draft PR. As soon as it is ready, I link it here


Reload search table for no data

This will reload the search table when there's no data. Added to fix a crash: search for keyword, cancel search bar, activate search again and enter any keyword; table appears with search results of old keyword and selecting any index causes the app to crash.
This should cover Cancel, Backspace and Cross in textfield as positive search results are already handled in the Dashboard and PullList controllers :)

Awesome, thanks a lot for this! Just tested and it works great.


Fix for movies with no release date

Fixing for adding movies with no release date (not release yet/ not available)
Here is the preview of the changes:

Now movies with no release date like dacoit can be added to top 5 movies list while onboarding
and prev they also cant be added to journal, but now they can be added to it as well
in movies page, where Invalid data was shown for release date, replaced it with Not released yet, and in year with - - to have better experience in frontend

fixes #204

Here is a new preview of changes based on this pr
@daccotta
https://github.com/user-attachments/assets/dc488471-af4d-4822-b233-779beeb461d3

@AshuKr22 @sid0000007
I have raised the pr. If it is fine, please label it with gssoc-ext and level


Package request: Houdoku

Project description

Metadata

homepage URL: https://github.com/xgi/houdoku
source URL: https://github.com/xgi/houdoku/releases/tag/v2.14.0
license: MIT
platforms: Linux

Add a :+1: reaction to issues you find important.

I've tried to package it, but i was stuck at local dependency, here is my gist
       > yarn install v1.22.19
       > [1/4] Resolving packages...
       > error Package "canvas" refers to a non-existing file '"/build/deps/houdoku/.erb/pkg/canvas"'.
       > info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
       >
       For full logs, run 'nix-store -l /nix/store/3y3l75pdi7dady9sqc4f5ahyhq0yi7k9-houdoku-modules-2.14.0.drv'.
error: 1 dependencies of derivation '/nix/store/53drysanjbln3p0rczn9ylphli8dxykw-houdoku-2.14.0.drv' failed to build

Feel free to take it and use it, maybe you know how to fix this build


Fix Chat Input Crashes on Android

The input was crashing when you sent messages that had modified the
native selection props. It crashed when you cleared the text because it
would set the text to empty, and reused a cached selection that was out
of bounds of an empty string.
For example, say you type "hello @" and autocomplete to "hello @foo". We
do some text selection magic to insert the @foo and put your cursor in
the correct spot. The Selection start and end indicies are both 11. When
you submit to message, we clear the text. When we clear the text, RN
keeps reusing those selection indices (of 11) and Android complains the
selection is outside of the bounds of the text ("" has a length of 0,
but you want to move the cursor to position 11, outside the bounds of
0).
The workaround here is to set the selection at the same time as the
text.
The RN Issue is: https://github.com/facebook/react-native/issues/25265
@buoyad since I think he has the most context here.
@keybase/react-hackers

I tried it and it seems fine on iOS too.


zookeeper cluster cannot communicate with each other

Hi all,
when I installing the seldon, I faced a problem.
After I installed kubernets, I run seldon-up.sh, the progress keep checking zookeeper status.
I investigate the problem, it was caused by that in the zookeeper pods, cannot get communication with each other with the hostname.
I try to using the service ip of each zookeeper, still failed. Because that the service ip cannot be accessed by the pod.
can you give some advise? thanks very much !!!

Hi,
Can you let us know how you are running kubernetes.
Are you using minikube or another method?

Gurminder

Hi Gurminder,
I think my kubernet is not the minikube
I am running kubernets like these(one mater, and one minion, only using minion node to run docker containers):
master node:
(flannel network <IP_ADDRESS>/16)
etcd, kube-apiserver, kube-controller-manager, kube-scheduler, and so on
minion node:
kube-proxy, kubelet, flannel
(flannel network <IP_ADDRESS>/24 (subnet), docker0: <IP_ADDRESS>)
when running seldon-up.sh   the services and pods created successfully,
there are three zookeeper pods, zookeeper-1, zookeeper-2, zookeeper-3
take zookeeper-1 for example, its cluster ip is <IP_ADDRESS> and endpoint address is <IP_ADDRESS>
run cmd on master : kubectl exec  zookeeper-1 -- bash -c "echo srvr | nc <IP_ADDRESS> 2181"
I can get the message "This ZooKeeper instance is not currently serving requests"
on the minion node, I run the command, echo srvr | nc <IP_ADDRESS> 2181 also get the reply
"This ZooKeeper instance is not currently serving requests"
this means that I can access the service via kube-proxy.
the problem is:
when I run command "kubectl exec zookeeper-2 -- bash -c "echo srvr | nc <IP_ADDRESS> 2181"
there is no reply, this means that in zookeeper-2 cannot access the address <IP_ADDRESS>
when I run command "kubectl exec zookeeper-2 -- bash -c "echo srvr | nc zookeeper-1 2181"
reply "cannot found hostname"
SO, my question is how can the zookeeper node communicate with each other?
Does there some iptables rule needed?
Another question, which seldon pods communicate with each other using kube-dns or using environment variable?
can you give me some guidethat you installed your kubernets?
thanks very much

Hi,
This looks like a dns issue.
The zookeeper servers are unable to find each other.
Latest version of kubernetes should have the dns service built in.
see http://kubernetes.io/docs/admin/dns/
Which version of kubernetes are you using?

Hi Gurminder,
thanks for you reply, I am using kubernets v1.2.0
Client Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"ec7364b6e3b155e78086018aa644057edbe196e5", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"2", GitVersion:"v1.2.0", GitCommit:"ec7364b6e3b155e78086018aa644057edbe196e5", GitTreeState:"clean"}
Indeed, there are two problems, cannot know hostname might be dns problem, but when I using cluster ip address, still not work, there might be network routing problems.
By the way, the kube-dns is required by seldon, right?
Thanks very much again~

Hi,
kubernets v1.2.0 would not have the dns built in, you would to manually add that.
It would be best to use the latest version of kubernetes or at least v1.3.

@gsunner
Hi Gurminder,
Thank you very much for your help, I have fix the problem that zookeepers cannot communicate with each other by services ip address. Just add some rules in iptables can resolve it. It's the configuration problem of k8s. The only problem is kube-dns now, thanks


classnames not found

Id like to run this and play with it, but I get this compile error in the browser.

npm install classnames fixes the problem. Looks like you just need to add this to package.json

@offsky I already sent Pull Request #1

@kuy looks like you beat me to it :)

Sorry for the late reply! My Github email notifications don't seem to be configured correctly.
Thanks for both catching this, both of you! Forgot to --save during development. I'll merge the fix in.


Images

Feature request: When uploading image through Slack, show url on IRC-channel.

Hi! See #22 for more information :)


Bugfix: Force download last workfile not triggered

Changelog Description
Fix force download last workfile not being triggered.
Additional note
The bug I'm fixing here doesn't occur in prod_test, but it does happen in WW.
Testing notes:

Use the force download last workfile feature (in WW prod).
It should download the last workfile with an incremented version number and downloaded subversion whether you have local workfiles or not.

The code change seems good, but can you please be more precise in the test procedure?

Use the force download last workfile feature (in WW prod).
Where is this feature ? How do I use it ?
It should download the last workfile with an incremented version number and downloaded subversion whether you have local workfiles or not.

How do I check that it download the last workfile, and the version number was incremented ? How do I check for downloaded subversion ?


Show how to disable validation on the server

https://docs.microsoft.com/en-us/aspnet/core/mvc/models/validation#disable-client-side-validation

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 39b6d0f3-9bc7-d091-0d45-9fc1e2e7aac8
Version Independent ID: 20e7eb53-215b-09ab-205d-d20177221278
Content: Add validation to an ASP.NET Core Razor Page
Content Source: aspnetcore/tutorials/razor-pages/validation.md
Product: aspnet-core
Technology: aspnetcore-tutorials
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Fixed in this commit


fix: Change kms policy statement resource to NOT resource

what

Changed IAM execution role Policy KMS permission statement from a resource to a NotResource

why

This change complies with the suggested policy for an AWS owned key.
Experienced that task logs were not generated in Cloudwatch and DAGs fail in Airflow with an error such as: 'Could not read remote logs from log_group: XXXXXXX'

references

AWS Recommended Policy for AWS owned key: https://docs.aws.amazon.com/mwaa/latest/userguide/mwaa-create-role.html#mwaa-create-role-aocmk

/test all


Webservice aint running

The webservice starts at localhost:5555 but the only think I can see is : -
Hello, World!
This is a sample web service written in Python using Flask module.

Where do I even go from here ?

Hi @rsingh2083
That really depends on what do you want achieve :)
The message you see is perfectly fine and it's like an index page for the web service.
Now depending on what kind of "routes" you have coded you can access different methods using those routes, i.e.:
http://localhost:5555/your/custom/route_name

This address would give you access to the piece of code below:
@app.route('/your/custom/route_name', methods=['POST'])
def your_method():
    file_upload = request.files['file']
    if file_upload:
        temp_file_path=os.path.join('./Temp',str(uuid.uuid4())+'.jpg')
        file_upload.save(temp_file_path)
        app.logger.debug('File is saved as %s', temp_file_path)
        ...
        ...

I strongly advise going through some sort of Flask tutorial for more details on how it works.


How to use 2 gpus?

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.90                 Driver Version: 384.90                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K40m          On   | 00000000:04:00.0 Off |                  Off |
| N/A   68C    P0   132W / 235W |   4267MiB / 12205MiB |     86%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K40m          On   | 00000000:84:00.0 Off |                    0 |
| N/A   42C    P0    61W / 235W |     80MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    143163      C   python                                      4254MiB |
|    1    143163      C   python                                        69MiB |
+-----------------------------------------------------------------------------+

On my server, there are two gpus, when  I run code, it seems that only one k40 is running

Hi!
Really this is default intention of the TensorFlow. If you want to use multi GPU training, the code itself should be adjusted like this one. Unfortunately, there was no reason to implement this when I was developing this model. If you want you may create fork and add all required modification and later even send pull request back/


Back button

Would fix #15

@leonkunert Hi, thank you for your PR. I will review it by this weekend and publish it.


Spring cloud loadbalancer with multiple different discovery client.

Hello everyone!  I will create multiple discovery client for different eureka server cluster in my project.
For each discovery client I want to have a client-side-base load-balancer.
Does spring cloud loadbalancer support this scene or  any good idea to do with spring cloud loadbalancer?
Thanks a lot!

I think I have get the way to resolve this problem.


Prevent accidentally getting dependencies with cabal

This make sure all dependencies are provided by nix and cabal can't won't download any, preventing one from forgetting to re-launch the shell after adding a not-currently-available package.
For instance, when doing cabal build all inside nix-shell -A shells.ghcjs
I have:

[x] Based work on latest develop branch
[ ] Followed the contribution guide
[ ] Looked for lint in my changes with hlint . (lint found code you did not write can be left alone)
[ ] Run the test suite: $(nix-build -A selftest --no-out-link)
[ ] Updated the changelog
[ ] (Optional) Run CI tests locally: nix-build release.nix -A build.x86_64-linux --no-out-link (or x86_64-darwin on macOS)

I have concerns regarding how this will play out with haskell.nix branch which is expected to become default somewhere in the future. haskell.nix uses cabal.project and *.cabal files to get the build plan, so setting active-repositories might interfere with that (though it needs to be checked).


STREAM support possible?

I'm just loving the new HA 90.x stream component support for cameras.    In all your great work in reverse engineering the Skybell API, did you ever run across a way to get a live stream that would be compatible?
Being able to detect front door motion and sending the stream to a Google hub in the back bedroom room would be so cool.     Of course the Skybell mobile app has the ability to live stream, but it's probably locked down, so my expectations are really low.  Just had to ask.
Thanks!

In all my poking around I didn't come across any ability to pick up or start a stream.

You can find details of how to start a live stream from a SkyBell HD at Protocol-HTTPS. The video is provided H.264 encoded transported by SRTP as described at Protocol-SRTP.
It looks like Home Assistant's stream component supports H.264 (only) over RTP. It isn't obvious whether SRTP is supported, but it would always be possible to use FFmpeg to proxy the video stream. I do something similar in homebridge-skybell to make the stream compatible with HomeKit.

Thanks guys for the responses - especially the reverse-engineered protocol info.  Wow, great work on that detail @thoukydides !
I haven't yet found the documentation on how to implement the new HA stream support, other than looking at camera source that starts off statements such as  homeassistant.components.camera import SUPPORT_STREAM.     Apparently the HA streaming (and audio) hooks occur in the HA ffmpeg component code itself, if support_stream is enabled by the parent camera.
I'm not familiar with what is involved with using ffmpeg to proxy the Skybell video stream.  I'll take a closer look at your homebridge-skybell source to see how that's done.      But since Nick (who is a lot more familiar with Skybell) has already poked around the code, I doubt I'll find any straightforward way to implement it.

Sorry to bring up this old issue. I have revamped the HA integration to get the latest activity video playing. The streamer that HA uses does not support POST so no live calls. Still scratching my head on how to get push updates. I could not get enough info from @thoukydides work. He no longer has a working device so all can ask him is how he went about REing the API. I've been trying to MITM the mobile app with no success.


UI version indicator doesn't show in latest fides

As per the test instructions here the version indicator doesn't show in the admin UI when run via nox -s test_env:

We moved this to Management > About Fides instead 👍


dataset

Hello author, I would like to ask how to make MAT file in the data set. I am new to graph nerve and do not understand the production of data set

Hi, https://github.com/muhanzhang/pytorch_DGCNN/tree/master/data contains instructions to convert MAT to txt. You can directly work with txt files. You may download new files from https://ls11-www.cs.tu-dortmund.de/staff/morris/graphkerneldatasets#file_format.


Show address transaction count for EOAs

Closes #744
Adds an address's transaction count next to the page transaction count. It's only shown for EOAs.

There's possibly other text that would fit here, such as using "X outgoing transactions" instead.
Maybe even "X total outgoing transactions"...

I feel like this info should be in the previous section, below or at the right of the balance box.
it doesn't mix well with the pagination stuff, it feels in the wrong place.

Okay. I don't think it deserves its own line below Balance - and we would have to stop it from page jumping if we put it below anyway. It's not easy to come up with other places where it would fit... the closest I'm thinking of is some kind of nonce-like indicator at the subtitle line. What do you think about that? Imagine a green nonce box (with the up arrow) to the right of the address label.
The reason I had it next to the pagination is that the transaction count gives you a lower bound on the total number of transactions in the table.

put it on the right? make it half and half panel

put it on the right? make it half and half panel

Exactly half?

Or thirds?

half looks better?

that somehow still feels a little odd, but the feature is good. I think we'll eventually make changes in the layout in the future.


Support Ctrl-O in insert mode

Hi,
In insert mode, Ctrl-O allows us to switch to normal mode for one command and immediately switches back to insert mode.
This is a godsend for navigating in insert mode. This doesn't seem to work in obsidian (on macos at least. The quick switcher is bound to command-O, doesn't look like Ctrl-O is bound to anything else).
Is there perhaps some way to implement this behaviour in Obsidian with some kind of remap hack? Forgive my ignorance but I'm not a big vim user 😅
(also please see this issue: It might be that this feature isn't replicated in codemirror at all)

The CodeMirror vim project has implemented this now. Do you have any idea how long that takes to make its way into an Obsidian release?

Obsidian release v1.1.15 includes all the latest upstream changes without this patch :(, maybe the next public build will include it.

The CodeMirror vim project has implemented this now. Do you have any idea how long that takes to make its way into an Obsidian release?

The CodeMirror vim project has implemented this now. Do you have any idea how long that takes to make its way into an Obsidian release?

Obsidian release v1.1.15 includes all the latest upstream changes without this patch :(, maybe the next public build will include it.


[Snyk] Security upgrade Microsoft.AspNet.WebApi.Client from 5.2.9 to 6.0.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `nuget` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

Src/SmtpLw/SmtpLw.csproj

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

589/1000  Why? Has a fix available, CVSS 7.5
Denial of Service (DoS) SNYK-DOTNET-SYSTEMNETHTTP-60045
Yes
No Known Exploit

589/1000  Why? Has a fix available, CVSS 7.5
Improper Certificate Validation SNYK-DOTNET-SYSTEMNETHTTP-60046
Yes
No Known Exploit

579/1000  Why? Has a fix available, CVSS 7.3
Privilege Escalation SNYK-DOTNET-SYSTEMNETHTTP-60047
Yes
No Known Exploit

479/1000  Why? Has a fix available, CVSS 5.3
Authentication Bypass SNYK-DOTNET-SYSTEMNETHTTP-60048
Yes
No Known Exploit

589/1000  Why? Has a fix available, CVSS 7.5
Information Exposure SNYK-DOTNET-SYSTEMNETHTTP-72439
Yes
No Known Exploit

589/1000  Why? Has a fix available, CVSS 7.5
Regular Expression Denial of Service (ReDoS) SNYK-DOTNET-SYSTEMTEXTREGULAREXPRESSIONS-174708
Yes
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.
Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Privilege Escalation
🦉 Authentication Bypass
🦉 Regular Expression Denial of Service (ReDoS)

:white_check_mark: Build SmtpLw 2.0.161 completed (commit https://github.com/guibranco/SmtpLw/commit/73be736b46 by @snyk-bot)


plans for :has() selector

any plans on implementing the level 4 :has() selector, which would effectively function as a parent selector, or is this not possible?
spec: https://drafts.csswg.org/selectors-4/#relational

Dup of #205


fixed message attachment defect

Description
Our samples failed at
attachment = MessageAttachment(file_id=file.id, tools=CodeInterpreterTool().definitions)

And
attachment = MessageAttachment(file_id=file.id, tools=FileSearchTool().definitions)

It is because tools keyword expects List["MessageAttachmentToolDefinition"] where MessageAttachmentToolDefinitionis Union of FIleSearchToolDefinition and CodeInterpreterToolDefinition.
To fix it, I did 2 things:

change CodeInterpreterTool().definitions and FileSearchTool().definitions to return List[CodeInterpreterToolDefinition] and List[FIleSearchToolDefinition] instead of List[ToolDefinition].   But to do this, I need to change for all tools to return their specific types of tool definitions.
List doesn't support covariant types.   With that said, with tool: List[Union[FIleSearchToolDefinition, CodeInterpreterToolDefinition]], you can't assign it List[CodeInterpreterToolDefinition] or List[FIleSearchToolDefinition] .   So I overload constructor.

If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

This is to main branch?

Is it functional failure in
attachment = MessageAttachment(file_id=file.id, tools=CodeInterpreterTool().definitions)
or static analyser failure?

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-ai-projects

Is it functional failure in
attachment = MessageAttachment(file_id=file.id, tools=CodeInterpreterTool().definitions)
or static analyser failure?

It faile

Is it functional failure in
attachment = MessageAttachment(file_id=file.id, tools=CodeInterpreterTool().definitions)
or static analyser failure?

It is a pylance error that cannot be resolved by casting.

This is to main branch?

Yes.   In sync channel, I mentioned that I will wait for the decision to merge to main or beta 3.0.1


Some tools have issues with unicode property escapes in regex

It seems that the following:
https://github.com/readium/ts-toolkit/blob/711740799f670e661bad29e0fe023488c2e4477e/shared/src/util/tokenizer/TextTokenizer.ts#L119-L124
Specifically,
/[\p{L}\p{N}]+/u

make tools such as babel throw an error:
Unknown property: L at matchProperty

Could find something similar for RI and Emoji, with babel + NextJS involved.

@JayPanoz I actually had this uncommitted and ready to push in the guided-navigation branch, because I encountered it as well trying to integrate with the Next.JS playground. Here's the fix:
const trimmedMatcher = new RegExp("[\\p{L}\\p{N}]+", "u");

// Unicode-aware of checking if there's anything that can be spoken in a string
// "Spoken" in this case means at least one unicode letter or unicode number character
export const speakableToken = (token: string): string | null => {
    const trimmedToken = token.trimEnd();
    if(trimmedToken.length === 0) return null;
    if(trimmedToken.match(trimmedMatcher) === null) return null;
    return trimmedToken;
}

Feel free to add it

@chocolatkey Great thanks. Will integrate in the vite-build then, and resolve the issue from the commit.


controller: Set expansion error when the info is not empty

Longhorn/longhorn#1674

@Mergifyio backport v1.4.x


swift build -c release -Xswiftc -static-stdlib -> same-type requirement makes generic parameter 'Element' non-generic

Error while compiling:
/Users/luismanuelramirezvargas/Projects/Marathon/Packages/Releases-1.0.6/Sources/Array+Version.swift:10:38: error: same-type requirement makes generic parameter 'Element' non-generic public extension Array where Element == Version {                                      ^ /Users/luismanuelramirezvargas/Projects/Marathon/Packages/Releases-1.0.6/Sources/Array+Version.swift:12:34: error: reference to generic type 'Array' requires arguments in <...>     func withoutPreReleases() -> Array {                                  ^                                       <Any> <unknown>:0: note: generic type 'Array' declared here <unknown>:0: error: build had 1 command failures
Full stacktrace:
$ swift build -c release -Xswiftc -static-stdlib Cloning https://github.com/JohnSundell/Files.git HEAD is now at 284d85d Merge pull request #11 from yageek/feature/currentFolder Resolved version: 1.8.0 Cloning https://github.com/JohnSundell/Unbox.git HEAD is now at ba42e01 Merge pull request #164 from hartbit/set-unboxable-collection Resolved version: 2.4.0 Cloning https://github.com/JohnSundell/Wrap.git HEAD is now at d6b953b Merge pull request #35 from darthpelo/master Resolved version: 2.1.1 Cloning https://github.com/JohnSundell/ShellOut.git HEAD is now at 38cd16d README: Update for new series of commands + path API Resolved version: 1.1.0 Cloning https://github.com/JohnSundell/Require.git HEAD is now at 44d5266 Remove whitespace & improve code formatting Resolved version: 1.0.2 Cloning https://github.com/JohnSundell/Releases.git HEAD is now at 787caab Use HTTPS instead of SSH for cloning dependencies Resolved version: 1.0.6 Compile Swift Module 'Require' (1 sources) Compile Swift Module 'ShellOut' (1 sources) Compile Swift Module 'Wrap' (1 sources) Compile Swift Module 'Unbox' (1 sources) Compile Swift Module 'Files' (1 sources) Compile Swift Module 'Releases' (4 sources) /Users/user/Projects/Marathon/Packages/Releases-1.0.6/Sources/Array+Version.swift:10:38: error: same-type requirement makes generic parameter 'Element' non-generic public extension Array where Element == Version {                                      ^ /Users/user/Projects/Marathon/Packages/Releases-1.0.6/Sources/Array+Version.swift:12:34: error: reference to generic type 'Array' requires arguments in <...>     func withoutPreReleases() -> Array {                                  ^                                       <Any> <unknown>:0: note: generic type 'Array' declared here <unknown>:0: error: build had 1 command failures error: exit(1): /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build-tool -f /Users/user/Projects/Marathon/.build/release.yaml

Hi @LuisMDeveloper 👋 Marathon requires Swift 3.1, so please update to Xcode 8.3 and try again 👍

Closing this for now. @LuisMDeveloper let me know if you need any other assistance 🙂


feat: integrate python-runtime into pipelines

Fixes: #1491
Changes

changed python-runtime base image from python:3.9 to python:3.9-alpine to make the image size smaller
added python-runtime to all pipelines
added env variable pointing to the image into klt-operator Deployment
added helm value for setting up the image + updated the docs

Image available in ghrc: https://github.com/keptn/lifecycle-toolkit/pkgs/container/python-runtime

let's wait the python runtime merge, since there are things that may change, for instance we introduce an env variable to keep track of the python runtime image used in the toolkit


Update dependencies - take 2

@facundominguez
It turns out that haskell_binary for //package-a:bench-b wants to have main_file attribute filled... Do you have idea what could cause this requirement to appear?
Traceback (most recent call last):
	File "/var/lib/buildkite-agent-agent/.cache/bazel/_bazel_buildkite-agent-agent/e8a5da19fef7f5fb3305a1799d9fb92e/external/rules_haskell/haskell/private/haskell_impl.bzl", line 98, column 39, in haskell_binary_impl
		return _haskell_binary_common_impl(ctx, is_test = False)
	File "/var/lib/buildkite-agent-agent/.cache/bazel/_bazel_buildkite-agent-agent/e8a5da19fef7f5fb3305a1799d9fb92e/external/rules_haskell/haskell/private/haskell_impl.bzl", line 208, column 40, in _haskell_binary_common_impl
		module_map = determine_module_names(srcs_files, not main_as_haskell_module, ctx.attr.main_function, ctx.file.main_file)
	File "/var/lib/buildkite-agent-agent/.cache/bazel/_bazel_buildkite-agent-agent/e8a5da19fef7f5fb3305a1799d9fb92e/external/rules_haskell/haskell/private/path_utils.bzl", line 137, column 21, in determine_module_names
		fail("""\

Error in fail: No source file defining the main module 'Main'.
You may need to set the 'main_file' attribute.

Analyzing: 18 targets (19 packages loaded, 2295 targets configured)
ERROR: Analysis of target '//package-a:bench-b' failed; build aborted: Analysis of target '//package-a:bench-b' failed

Indeed, setting the main_file attribute fixed this issue locally on my setup.

You're right... :bench-a target build without any objections and that's probably because the file is named Main.hs. No need to add main_file attribute in that case.
So in order to proceed with this PR I propose:

manually add main_file for :bench-b target to fix the build,
add a feature request issue for handling main_is cabal's attribute,
create and issue under rules_haskell about misleading error msg: Error in fail: No source file defining the main module 'Main'.

Best regards,
Karol


Add dynamic positional URL parameters

As it stands, if a user wants to access an endpoint that has a dynamic parameter, they must create a new endpoint. This enhancement would enable dynamic positional URL parameters via the use of keyword arguments on endpoint generation.

Implemented in #12


stop() or cancel() methods wont work

Hi,
Everything works flawlessly in the plugin except when I want it to stop listening, it won't.
stop() or cancel() doesn't do anything.
Please look into it.

On what platform ?

Just tried the example app with  2.0.0 version, and everything is working as expected


Make config adjustable after initialization

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report
[ ] Performance issue
[X] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

Library version

Library version: 0.1.0

## Current behavior
The MSAL config is initialized directly with the module and clientId of the API needs to be supplied directly in consentScopes and protectedResourceMap. However, we have a multi-tenant app where the API clientId for each tenant is stored in azure table storage. Based on the URL the tenant parameters are first retrieved, before a call to MSAL will be made. In ADAL this was no problem, because the context could be initialized at a later point, but with MSAL we have to import the module in the app.module. Then there is no chance to update the config parameters anymore before the first login call is made.

## Expected behavior
It would be nice if the parameters for all the config items, or the config itself would be adjustable after initialization of the module. Then we can adjust things like protectedResourceMap, consentScopes with the specific tenant parameters

@negberts  I just wanted to clarify that the only mandatory parameter that you need to pass at the time of initialization is clientId. You can pass the consentScope directly to login api . I agree that there is no way you can set the protectedResourceMap later.

@nehaagrawal correct! But if I want to use the authentication guard, I will not call the login method directly… So consentscopes would be nice to expose as well :-) And I was talking about the clientId of the API thats is being called, not the clientId of the application.

@negberts Does API here means the resource you want to access? I am little unclear about what you meant. The clientId of the application cannot be changed post MSAL JS initialization as of today. But for any accessToken() call, I believe you can call them separately as long as the service can give the token.
Can you send some sample code on what you mean? May be that will help for us to analyze if this is a feasible requirement.

@sameerag I want to be able to adjust consentscopes and protectedresourcemap after initialization of the module, because at the time of module initialization I do not know these parameters yet.

@negberts You should be able to.
Consentscopes:

module initialization is only for the scopes you need consent to at login, but each acquireToken() call that follows can be send newer scopes. There is a dynamic consent screen that will be displayed if you are unable to consent it prior.

Only in the case where you do not want interaction, you have an option to pre-decide all the scopes you want the tokens for and send them in login request instead of the following accessToken requests.

more details on how scopes work is listed in our scopes-readme

protectedresourcemaps:
Did you get a chance to try out our new msal 1.0.0 release? This is not yet supported for angular (which is planned for this summer - roadmap, but we have a 'response' object today supported in the core, which has the scopes that are consented until up the accessToken request you make: more details at msal apichanges
If you want a specific behavior for "protectedresourcemaps" we can consider including it as we are gathering the requirements while writing the angular wrapper since this is very angular specific. Please let us know.

No, I did not know about a 1.0.0 release until now :) But as I am using Angular it means I have to wait until summer right? I will give it a try then. Can we keep this one open until then?

@negberts Okay. We are tracking all angular issues with a tag and will close them as and when the release is made with the ask in the corresponding tickets.

Tagging this with #786.


Eliminate Data Tab

I am not sure whether or not there is still value in having the Data View. In fact, because it automatically queries my data source it ends up taking longer to getting to the querying I would actually need to do. Query View seems to serve all of the functionality Data has and more. You could even add an option to automatically execute the default query in Settings, preferably automatic=False by default)

because it automatically queries my data source it ends up taking longer to getting to the querying I would actually need to do.

I'm not sure I understand what you mean with this. Can you clarify?
Currently the data view queries the file directly, without storing it in memory. Every action, changing page size, going to a page does a new query on the file directly. It doesnt have a query editor obstructing the table.
However, the query view queries the file, and then stores the query result into memory, on which can be paginated, sorted (TBD) and searched (TBD). If i wanted to go to the last page i would need to query that, or do select * from data and click the last page button, whereas in data view id just need click last page.
I'm open to merging these two views as they're quite similar. But other users might have different use cases as well. I know that the data view currently is used much more than the query view.

For example, I am opening a 100 MB parquet file on a Network Share. When I open, the query for the Data view automatically is executing and downloading all columns and however many roles is my default page size from my file. Because this is a big file on a network file, there is a lot of time for it to download that information. And the Data view itself won’t allow me to do anything helpful to the file, it is just causing. A delay before my analysis basically.
Just because Data is the “most used”, the telematics may be misleading. A user might open the file and see that the Data tab has no useful functionality (like exporting) and not know the Query tab has the options they need. This may mean that they uninstall the plugin and try another parquet plugin. The data tab is just the default tab in the plugin and might actually be choosing users to go with alternative plugins.

I see, thanks for clearing that up. You would like the query view as the default view? And remove the data view. Also provide the option/config to automatically query the data or not.

Yes, query could be default view and you could make it automatic run True by default with a config option to set it to False within the settings for edge cases like mine. If you’re worried about screen real estate, you could add collapse/expand query functionality.

I'll make this my next priority. I will keep the data tab, however.
This MR should contain:

put query view default
add automatic run query view boolean configuration
add fetching data data view in separate thread in background

I'll make this my next priority. I will keep the data tab, however. This MR should contain:

put query view default
add automatic run query view boolean configuration
add fetching data data view in separate thread in background

This sounds like a good solution.


backend sqlite3 integer mapping changed from dt_integer to dt_long_long

Rationale discussed in #719

Sorry if I'm missing something, but isn't the compatibility breakage really bad here: i.e. do I understand correctly that this means that any existing code using into(int_var) will stop working and that there will be no way to fix it other than changing the variable type to long long (whether you need it or not)?
Also, from reading the code, it looks like int64 support should already work with SQLite if the column had been declared as "bigint" or "int8" when creating it, isn't it the case?

Yes I can use bigint except.
1.for auto increment PK, only allowed is INTEGER which is my case.
2.reading an INTEGER type column with value greater than 32bit (e.g. epochmillis).
And also, I can realize what i am doing is wrong for the old code. So, I think, I should adjust the backend construct with option to change the default datatype without breaking the compatability.

If the connection param is set as follows, we will get int64_t read in sqlite3.
params.set_option("soci_sqlite3_integer_type", "dt_long_long");

@vadz apparetnly @mohansella has been nagging about merging this PR. What's the final conclusion?

Sorry, I don't even know what patches should be reviewed: there is #719, #720 and #721 (this one), should they all be applied together/sequentially or are some of them included in the other one(s)?
I'm travelling right now but I could review this next week when I get back.

@vadz , sorry for the confusion. both the changes were merged into #720


Feature: Expose cpu_stats.throttling_data

Information is contained here: curl -o stats.json "${ECS_CONTAINER_METADATA_URI_V4}/task/stats"
        "cpu_stats": {
            "cpu_usage": {
                "total_usage":<PHONE_NUMBER>,
                "percpu_usage": [
                    676748195,
                    989670875
                ],
                "usage_in_kernelmode": 170000000,
                "usage_in_usermode":<PHONE_NUMBER>
            },
            "system_cpu_usage":<PHONE_NUMBER>000,
            "online_cpus": 2,
            "throttling_data": {
                "periods": 0,
                "throttled_periods": 0,
                "throttled_time": 0
            }
        },

Hi @jseiser,
I've started to work on metrics set update in my PR: https://github.com/prometheus-community/ecs_exporter/pull/46/files.
Welcome to participate in that.

Has anyone seen a situation where these are non-zero? It should notionally be possible if any CPU limits are set on the task or the container, but even with such things in place I do not see anything moving. My expectation is that periods would, well, periodically increase if those conditions are met, even if no throttling is taking place.
I can try to experiment with creating a while (true) {} container to see if I can make the stats move when throttling is actually happening.

I ran tasks with both ecs_exporter and an alpine sidecar running ["/bin/sh", "-c", "yes > /dev/null"] (i.e. chewing up a lot of CPU) on Fargate and EC2.  They both had less than 1 vCPU allocated, Fargate at the task level and EC2 at the container level.  The CPU-seconds metrics for both were definitely increasing slower than real time passed, indicating that throttling was occurring.
But the throttling container stats remained at 0.  I'm not sure why, but regardless I think this is a dead end without action from AWS.


Switched package.json reference from azer's npm "img" module to azer's "img" git repo

Not sure if you want to accept this or not. Just one of the many 'azer' casualties. The img module is now being maintained by the dubious "kazmer" and has been bumped to version 3. It did not seem to work.
I am not sure how sound of a strategy this is, but I just updated the package.json to use azer's git repo.
Seems his last update was a year ago. Funny enough, one of them was a pull request of yours.
https://github.com/npm-dom/img

Ugh what a nightmare. I will write a new module

We can replace with this:
https://github.com/mattdesl/load-img

Nice. I would have re-published myself but I wasn't sure what route you wanted to go.

latest patch should fix this stuff


Compound-V loading text is inside the UI

As said in the title the UI gets split and destroyed because of the loading status. i would clear the entire output buffer before rendering the ui

As said in the title the UI gets split and destroyed because of the loading status. i would clear the entire output buffer before rendering the ui

I'm not sure I understand. The UI doesn't get rendered until the Compound-V Loading indicator finishes.
Are you talking about the [status] Running text that's displayed at the bottom? That gets called every loop, and I left it there intentionally.

My main terminal is Terminal Preview by Microsoft, which displays everything nicely for me.
I just tested with cmd.exe, the status spinner doesn't show up, so I might remove that entirely, and the [status] Running text flickers, as it gets rendered every loop.
Wanted to leave some indication that the program is still running, but this is turning a bit ugly.
Do you have any thoughts on a alternative placement? Or removing it entirely?

As said in the title the UI gets split and destroyed because of the loading status. i would clear the entire output buffer before rendering the ui

I'm not sure I understand. The UI doesn't get rendered until the Compound-V Loading indicator finishes.
Are you talking about the [status] Running text that's displayed at the bottom? That gets called every loop, and I left it there intentionally.

My main terminal is Terminal Preview by Microsoft, which displays everything nicely for me.
I just tested with cmd.exe, the status spinner doesn't show up, so I might remove that entirely, and the [status] Running text flickers, as it gets rendered every loop.
Wanted to leave some indication that the program is still running, but this is turning a bit ugly.
Do you have any thoughts on a alternative placement? Or removing it entirely?

I think you can completely leave it. What i meant previously was that the text "Loading Compound-V" is placed pretty much inside of the menu which destroys the overlay for my terminal (cmd.exe)


Working example with fileflow

Hi Laura,
This is not an issue, but could you add a example to the docs, how exactly to use fileflow?
This would be extremely helpful, especially to understand major differences or advantages of airflow+fileflow compared to Luigi.
Watched your talk, and the question came up, thank you!

@ThetomekK definitely, will do asap :+1:

FYI @ThetomekK I'm working on it over at https://github.com/industrydive/fileflow/pull/9/files but need to pretty it up a bit. But if you wanted to see the example before I check the sphinx rendering you can see it there.


Change all submodules to be our forked version to have better LTS

Feature/Issue
Updated
Implementation/Solution
Dependencies (if any?)

[ ]

Screenshots/GIF (if applicable)
Tests

[ ] Have you reviewed your own code for quality?
[ ] Did you the Sandbox game project and get the expected results?
[ ] Did you run the FlingTest suite and ensure that there are no regressions?

Merging this to main because the only issue is a Sol2 compilation issue in 2019, which is not an integration blocker as it is not from this PR anyway. It's happening in main


Confidence Net in the model

Hey, Thanks for your great work.
Just wondering what is the notation of Confidence subnet (C-Net)  in your repository? I can't find any C-Net in your codes.
I'm pretty much sure that there were R-Net and C-Net in your codes (maybe the initial version)! But now it's just R_Net and Perceptual_Net in networks.py.

Hi, sorry that the current code does not include the C-Net part.  Actually, we have not release the C-Net part in previous versions.

Thanks for your reply. So, it means you do not use Confidence Net in your model? or you've used Confidence Net in your model but haven't released it's code?!

Gotcha! Thanks for your clarification!

Hi, the "average shape" means taking the average of identity coefficients and use the averaged coeff to construct the shape.


Remove dependency from https

No sense in https, please replace all occurrences to http,
thanks, Alex!

Its not a dependency.  Some users use this report from cloud/google-drive/dropbox etc and some libraries do not work correctly without the https.

i have opposite problem - https is restricted:) So the reports are not shown correctly....
Maybe better to put the artifacts to project?

:-1:  I would like to host the reports in a https context, and thus would want it to use https...perhaps a setting where the user could set http or https would be a bettery way to go


fixed the title of the fleet overview page

Just a little fix in the title of the Fleet overview page.

Thank you!


Exception loading FFTW wisdom

Via Joe Hunkeler:
BUG REPORT:

If pyfftw is installed it causes an instantaneous traceback in
webbpsfgui. If I uninstall pyfftw, the GUI starts up just fine and works
as expected. So uh -- I was looking a hotdog on a plate in my kitchen
when I created the environment below, so forgive me...

(hotdog)makeitso:~ jhunk$ webbpsfgui
Traceback (most recent call last):
   File "/Users/jhunk/miniconda/envs/hotdog/bin/webbpsfgui", line 4, in
<module>
__import__('pkg_resources').run_script('webbpsf==0.3.4.dev473',
'webbpsfgui')
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/setuptools-18.1-py3.4.egg/pkg_resources/__init__.py",
line 735, in run_script
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/setuptools-18.1-py3.4.egg/pkg_resources/__init__.py",
line 1652, in run_script
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/webbpsf-0.3.4.dev473-py3.4.egg/EGG-INFO/scripts/webbpsfgui",
line 5, in <module>
     import webbpsf
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/webbpsf-0.3.4.dev473-py3.4.egg/webbpsf/__init__.py",
line 75, in <module>
     from poppy import (display_PSF, display_PSF_difference, display_EE,
display_profiles, radial_profile,
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/poppy/__init__.py",
line 94, in <module>
     utils.fftw_load_wisdom()
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/poppy/utils.py",
line 1354, in fftw_load_wisdom
     wisdom = [lines[i].replace(r'\n', '\n') for i in [4,5,6]]
   File
"/Users/jhunk/miniconda/envs/hotdog/lib/python3.4/site-packages/poppy/utils.py",
line 1354, in <listcomp>
     wisdom = [lines[i].replace(r'\n', '\n') for i in [4,5,6]]
IndexError: list index out of range

Now that I've looked at (and replaced) the FFTW wisdom load/save code, it seems like that Joe Hunkeler had a somehow corrupted copy of the wisdom file (possibly corrupted by POPPY itself)... anyway, now it uses JSON to (de-)serialize, so exception messages should at least point to the file format in the future when something goes wrong.


Unable to enter text in eq-author 'create' page

Expected behaviour
Must allow entering text.
Actual behaviour
Does not allow to enter text.
Steps to reproduce the behaviour
Launch eq-author
click [Create Survey]
Try entering any text in 'Title' field
Technical information
Browser
Chrome
Operating System
Screenshot

@yoganandkunche Are you happy to close this defect now?

Works as expected.

As discussed will close this with the PR 👍


Some calls to Event::Event ctor require a very particular load sequence from .data section

The ctor at 800a0c40 is sometimes called with a pattern like this:
https://github.com/zeldaret/ss/blob/22a5cadd578840f592b41a6d341e8cf7afb7ebbe/src/REL/d/t/d_t_ship_window.cpp#L82-L88
Every call has its own copy of these statics even within the same file, and the volatile seems to be required to load them in the right order. This looks like an obvious hack but we haven't found a good way to match it...

Another thing to note:
0x00100001 is a common parameter to creating an event.
This usually is an immediate load, but in the cases of & ~f1 both become data. f1 is also consistently 0x00000001 in these cases

#107 has an instance where it doesnt follow this


WIP - feat: Ingredients Page

Related issues

https://github.com/openfoodfacts/smooth-app/issues/512
https://github.com/openfoodfacts/smooth-app/issues/107

My attempt at the ingredients input page.  I'm struggling to put together a full PR, so I'll plan to ask a bunch of questions about the mocks, process, etc. and point people to this PR.
How it looks currently

Mocks I've been using
From Figma: https://www.figma.com/file/WXBmNSPweKqXpoSNRSQWDc/Product-Addition---Nutrition?node-id=0%3A1
And from the issues:

Changes from this PR

I've added an "Edit Ingredients" button that navigates to the new EditIngredientsPae inside of the Ingredients AttributeListExpandable.  I'm sure this isn't right, but it kind of looked like this mock?  What is the correct way to access the EditIngredientsPage?  (Code is hacky here because I'm assuming it needs to be changed)

Mock
Mine

Clicking on that "Edit Ingredients" button goes to the EditIngredientsPage.  I made it look similar to the Figma nutrition editing mock.  Is that right?  Pictured at the very top of this PR description.

That's it!  Is there anywhere else that's supposed to link to the EditIngredientsPage or anything else I'm missing in this flow?  Below I have many other specific questions for me to move forward.
Questions

Process

What's the process for getting text localized, or introducing new strings?

Design/product

Tapping the ingredients image at the top of the product page goes to a photo view of the ingredients.  Should that remain, or somehow be combined with this "edit ingredients" flow?  Am I missing any part of the flow/navigation for editing ingredients?

What is the light grey text that is seen below the ingredients TextField in the mock?

On my EditIngredientsPage pictured farther above, I used BoxFit.cover for the image fit, which results in wide images like the one pictured being cut off.  Is that ok, or should I use BoxFit.contain or some other layout?

Engineering

Running the app locally, if I upload a product image via one of the ImageUploadCards, it doesn't persist.  Navigating out of the product and back again, the new product image will be gone.  Is that a bug or is that how the development server works?
How do I edit a product?  Specifically, I need to modify the ingredients.  Do I call OpenFoodAPIClient.saveProduct?  If so, which of the many ingredient-related fields in Product do I actually need to set?

Related issues: #512 #107
My attempt at the ingredients input page. I'm struggling to put together a full PR, so I'll plan to ask a bunch of questions about the mocks, process, etc. and point people to this PR.
How it looks currently:

Mocks I've been using
From Figma: https://www.figma.com/file/WXBmNSPweKqXpoSNRSQWDc/Product-Addition---Nutrition?node-id=0%3A1 And from the issues:
Changes from this PR

I've added an "Edit Ingredients" button that navigates to the new EditIngredientsPae inside of the Ingredients AttributeListExpandable.  I'm sure this isn't right, but it kind of looked like this mock?  What is the correct way to access the EditIngredientsPage?  (Code is hacky here because I'm assuming it needs to be changed)

Mock	Mine

2. Clicking on that "Edit Ingredients" button goes to the EditIngredientsPage.  I made it look similar to the Figma nutrition editing mock.  Is that right?  Pictured at the very top of this PR description.
That's it! Is there anywhere else that's supposed to link to the EditIngredientsPage or anything else I'm missing in this flow? Below I have many other specific questions for me to move forward.
Questions

Process

What's the process for getting text localized, or introducing new strings?

I think you just  update this file with a new string.
packages/smooth_app/lib/l10n/app_en.arb
See example: #647

Design/product

Working with PM/UX to get this straightened out by this week.

Tapping the ingredients image at the top of the product page goes to a photo view of the ingredients.  Should that remain, or somehow be combined with this "edit ingredients" flow?  Am I missing any part of the flow/navigation for editing ingredients?

What is the light grey text that is seen below the ingredients TextField in the mock?

On my EditIngredientsPage pictured farther above, I used BoxFit.cover for the image fit, which results in wide images like the one pictured being cut off.  Is that ok, or should I use BoxFit.contain or some other layout?

Engineering

Running the app locally, if I upload a product image via one of the ImageUploadCards, it doesn't persist.  Navigating out of the product and back again, the new product image will be gone.  Is that a bug or is that how the development server works?

Oh I'm not sure, I think it might actually be a bug, I tried it using the smoothie app on my phone and the behavior was as you described.
I created #709

How do I edit a product?  Specifically, I need to modify the ingredients.  Do I call OpenFoodAPIClient.saveProduct?  If so, which of the many ingredient-related fields in Product do I actually need to set?

Stephane's response
Product product = Product(         barcode: '4250752200784',         lang: OpenFoodFactsLanguage.GERMAN,         ingredientsText:             'Johanneskraut, Maisöl, Phospholipide (Sojabohnen, Ponceau 4R)',       );       Status status = await OpenFoodAPIClient.saveProduct(         TestConstants.TEST_USER,         product,       );

@M123-dev  I think this is ready to be reviewed and get merged.  I put everything behind a dev mode flag so it shouldn't break anything.  It just hides a button in the ingredient knowledge panel that looks like this:

I may be missing some coding style things, or using the appropriate themes or constants in places, let me know what I can clean up.

@g123k @monsieurtanuki Thanks for the review!  I think I've responded to everything.

Thanks for all the help on this PR! 🎉

Thanks @justinmc I've just triggered the playstore alpha build 🎉

Thanks @justinmc, looking foreward to testing it out


Research task: Create a microbenchmark setup to test the efficiency of WebSockets vs HTTP2/3 + SSE

In order to get an idea how best to proceed with #233, it would be good to have ballpark numbers of the performance characteristics of WebSockets vs HTTP requests + Server-Sent Events for our needs.
Setup
We can get this data - completely decoupled from the internals of Jazz - by creating some synthetic microbenchmarks.

We need a client & server setup that can handle WebSockets, HTTP 2 & 3 and Server-Sent Events.
The client should be in TypeScript, using only Browser-native APIs (WebSocket, fetch, EventSource)
For servers, I would like to evaluate

A) a plain Node.JS server (or higher-level wrapper of that)
B) uWebSockets.js, for both web sockets and HTTP3 requests
C) plain Node.JS server behind a Caddy proxy that handles HTTP3 & SSL
all of them should be single threaded, on a single node

Simulation details
Original data use that needs to be simulated
Currently, Jazz uses WebSockets to sync CoValue state between the client and syncing/persistence server.
The communication typically consists of three scenarios:

Loading a CoValue

The client doesn't have a CoValue's edit history or only has it partially and requests the rest from the sync server.
The sync server responds with the missing parts of the CoValue edit history

Creating a CoValue

The client creates a new CoValue locally and wants to upload it completely to the sync server

Subscribing to and mutating a CoValue

A client is interested in realtime changes to a CoValue and subscribes to it
The server sends individual changes to the CoValue to the client as they happen
The client sends individual changes to the CoValue to the server when the user modifies the CoValue locally

Websockets vs Requests & SSE
Currently, 1., 2. and 3. happen over WebSockets, with 1 package per request/response/incoming update
For using Requests and SSE instead, we would use Requests & Responses for 1. and 2., while for 3. we listen to incoming updates with Server-Sent Events and publish outgoing updates as a Request with no expected Response.
Simulation spec
There are roughly two classes of CoValues: structured CoValues (thousands of <50 byte edits) and binary-data CoValues (few edits that are each 100kB).
Since we are only interested in the data transmission performance, we can model the scenarios using packets containing random data:

Loading a CoValue

structured: A 100 byte request from the client and a 5kB response from the server
binary: A 100 byte request from the client and a 50MB response from the server, streamed in 100kB chunks

Creating a CoValue

structured: A 10kB request from the client and a 10 byte response from the server
binary: A 50MB request from the client, streamed in 100kB chunks and a 10 byte response from the server

Subscribing to and mutating a CoValue

structured: 50 byte incoming SSE messages/WebSocket packets, mutations are 50 byte outgoing messages as a request/WebSocket packet

assume one client creating a mutation that is published to 10 other clients

binary: 100kB incoming SSE messages/WebSocket packets, mutations are 100kB outgoing messages as a request/WebSocket packet

No extra HTTP headers should be set (other than what browser set by default, and these should be minimised if possible)
Target metrics
The main variables we are interested in are

Loading a CoValue

a) How many CoValues can be "simulation loaded" at once on a client? Do we get head-of-line blocking effects? (you can do structured/binary separately)
b) How many clients can "simulation load" a CoValue at the same time per server?

Creating a CoValue

c) How many CoValues can be "simulation created" at once on a client? Do we get head-of-line blocking effects? (you can do structured/binary separately)
d) How many clients can "simulation create" a CoValue at the same time per server?

Subscribing to and mutating a CoValue

e) How many updates per second can we push from a client?
f) How many updates per second can a server handle (receive and broadcast to interested clients)?
g) What data throughput can we achieve for binary CoValue updates?
h) What's the latency like

Variables
It would be good to get results for the metrics above assuming
Different network conditions

I Ideal network conditions
II 4G speeds
III 3G speeds
IV connections with high packet loss, including so bad that we need to reconnect WebSockets

Different protocols

WS: WebSockets only (with reconnect on timeout)
H2: HTTP2 + SSE
H3: HTTP3 + SSE

You don't need to actually deploy a server anywhere if you can simulate these conditions locally, just make sure to note down your hardware specs and use exactly one thread/core for the server
Dimensions summary
So in total we have the following dimensions:

Server tech: A, B and C
Target metrics: a), b), c), d), e), f), g), h)
Network condition: I, II, III, IV
Protocols: WS, H2, H3

Deliverable

Create a completely new project in a new folder in this monorepo called 'experiments'
Create an independent pnpm workspace in there
Use whatever test-harness / benchmarking libraries / node setups / browser automation you think are best
Make sure the setup is replicable / can be run by anyone
Post results from your machine in a README in the same folder

I realise this spec is a lot, so feel free to ask lots of clarifying questions before & after accepting the task!

/bounty $2000

Hi @aeplay
I see your project just joined Algora, so welcome!
Different projects pick different styles of working so I'm curious, how do you want attempts at this bounty to shake out:

first-come first served i.e. the first dev to ask nicely gets assigned to work on the issue;
battle royale i.e. let multiple devs attempt and you'll evaluate PRs as they roll in?

The risk with style 1. is that the assigned dev might take too long to show progress (if they are inexperienced or experienced but busy with a day job).
The risk with style 2. is since there is only 1 bounty reward, anyone willing to work on this risks getting blind-sided by other devs who open PRs to claim the bounty.

Hey @ayewo good question! This is what I'm most curious about in this bounty model as well.
 There is no super urgent deadline for it, so I would be happy to let the first serious contender iterate on it with my input.

Excellent! In that case, I'd like to /attempt #301 this.

@ayewo yes! Let's gooo
Please ask for clarifications here, I'm in GMT+1 and mostly available during normal work hours, but also during other times on my phone for quick answers

@aeplay I'm also in GMT+1 :)  and just joined your Discord.
I take it you prefer clarifications happen here in the open, right?

yes please, don't worry about making this issue noisy, that's what it's for

Roger that. Would appreciate it if you can assign the issue to me, otherwise there will be drive-by attempts from other devs feigning ignorance of our conversation above.

done, thanks for walking me through this

Hi @aeplay, I'm interested in the project's client and ci/cd workflow related part, if its okay can we split the project @ayewo?

Hey @DhairyaMajmudar I appreciate your offer but would like to keep this focused on one person attempting it. Thank you!

Hey @DhairyaMajmudar I appreciate your offer but would like to keep this focused on one person attempting it. Thank you!

That's fine!

And @ayewo just clarifying: there is no CI/CD aspect to this project - it's all meant to be run manually.

Hint: Next, time You May want to Keep Applications a Bit Longer open so You Can Evaluate a few applicants, it doesn't have to be, first come first serve or Battle Royale.

Hint: Next, time You May want to Keep Applications a Bit Longer open so You Can Evaluate a few applicants, it doesn't have to be, first come first serve or Battle Royale.

Makes sense, but this time I wanted to move quickly and @ayewo seemed eager and capable so I just went with him

I'd like to share some progress on the research I've done so far and ask a few questions.
I looked into the HTTP protocol versions supported by servers A, B, and C and it seems that only Caddy supports all three versions of the HTTP protocol natively (i.e. HTTP/1.1, HTTP/2 and HTTP/3).
HTTP Versions Supported by Web Servers

#
Server
HTTP/1.1
HTTP/2
HTTP/3

A
Node.js v22.6.0
✅
✅
❌

B
uWebSockets.js v20.47.0
✅
❌
⚠ (experimental, development paused)

C
Caddy v2.8.4
✅
✅
✅ (and HTTP/2 over cleartext (H2C))

Node.js doesn't yet suppport HTTP/3 natively but I came across a 3rd party repo (https://github.com/endel/webtransport-nodejs) that claims to offer HTTP/3 support but I didn't look too closely.
Since you also want to test against 3 different protocols:

WS: WebSockets only (with reconnect on timeout)
H2: HTTP2 + SSE
H3: HTTP3 + SSE

I tried to map servers A, B, C to the 3 protocols to see what is possible:
Web Server to Web Protocol Mapping

#
Server
Layer 7
Layer 6
Layer 4
Port
Supported

A1
Node.js
HTTP/1.1 + WebSocket (WS)
TLSv1.3 (Optional)
TCP
3001
✔

A2

HTTP/2 + Server-Sent Events (SSE)
TLSv1.3 (Optional)
TCP
3002
✔

A3

HTTP/3 + SSE
TLSv1.3 (Mandatory)
UDP (QUIC)
3003
❌

B1
uWebSockets.js
HTTP/1.1 + WS
TLSv1.3 (Optional)
TCP
4001
✔

B2

HTTP/2 + SSE
TLSv1.3 (Optional)
TCP
4002
❌

B3

HTTP/3 + SSE
TLSv1.3 (Mandatory)
UDP (QUIC)
4003
❌

C1
Caddy
HTTP/1.1 + WS
TLSv1.3 (Optional)
TCP
5001
✔

C2

HTTP/2 + SSE
TLSv1.3 (Optional)
TCP
5002
✔

C3

HTTP/3 + SSE
TLSv1.3 (Mandatory)
UDP (QUIC)
5003
✔

Questions

Is my understanding of the server to protocol mapping correct?

In the Simulation spec section, you wrote:

Subscribing to and mutating a CoValue

structured: 50 byte incoming SSE messages/WebSocket packets, mutations are 50 byte outgoing messages as a request/WebSocket packet

assume one client creating a mutation that is published to 10 other clients

For the actual test, does this imply that after each server is started, 10 clients will be spawned that will subscribe to a CoValue, then 1 client will mutate the CoValue triggering a notification by the server to those 10 clients?

Hey @ayewo, thanks for sharing your research results in such a well-structured format.

It matches what I was aware of. For uWebsockets.js please can you try the experimental HTTP3 support and let me know how it goes?

Your understanding is correct, and just to be clear, I am not expecting you to do anything with Jazz/actual CoValues, we are just simulating their traffic patterns by sending (client -> server) and then broadcasting (server -> 10 clients) random data

Some more clarifications:

In all cases I'd like you to contrast using only WebSockets (for outgoing requests and their responses, and incoming notifications) vs HTTP (for outgoing requests and their responses) + SSE (incoming notifications)
uWebsockets should support HTTP2, right?
With Caddy, I'm only really interested in using it as a HTTP3-handling reverse proxy in front of Node.JS

So the full mapping would look like this
Web Server to Web Protocol Mapping

#
Server
Layer 7
Layer 6
Layer 4
Port
Supported

A1
Node.js
WebSockets only
TLSv1.3 (Optional)
TCP
3001
✔

A2

HTTP/1 + Server-Sent Events (SSE)
TLSv1.3 (Optional)
TCP
3002
✔

A3

HTTP/2 + Server-Sent Events (SSE)
TLSv1.3 (Optional)
TCP
3003
✔

A4

HTTP/3 + SSE
TLSv1.3 (Mandatory)
UDP (QUIC)
3004
❌

B1
uWebSockets.js
WebSockets only
TLSv1.3 (Optional)
TCP
4001
✔

B2

HTTP/1 + SSE
TLSv1.3 (Optional)
TCP
4002
✔

B3

HTTP/2 + SSE
TLSv1.3 (Optional)
TCP
4003
✔

B4

HTTP/3 + SSE
TLSv1.3 (Mandatory)
UDP (QUIC)
4004
⚠ (try)

C1
Caddy (in front of Node.JS)
HTTP/3 + SSE
TLSv1.3 (Optional)
TCP
5001
✔

Thanks for the confirmation.

OK. I make a note to investigate HTTP/3 support in uWebsockets.js.
Understood.

Please note that I have updated the 2nd table to remove the port numbers. I imagine each of the servers A,B & C will be started as a standalone process so they could simply listen on the same port i.e. localhost:3000 instead of listening on individual ports localhost:3001, localhost:4001 etc that I originally used.

Yes, this was in fact one of my follow-up questions. I imagine that the set-up in A1 is essentially your baseline i.e. what you are currently using today. The others A2-A3, B1-C3 and C1-C3 are what the synthetic benchmark would be uncovering, correct?
From my research, it seems that don't support HTTP/2 at all, only HTTP/1.1.
Understood. It was super clear in your original description that Caddy would serve as a reverse proxy.

yeah makes sense re ports - we can run the different cases in succession
Just double checked re uWebSockets and HTTP2 - you're right, that's surprising. Remove that case then, but try HTTP1 + SSE, please

Yes, this was in fact one of my follow-up questions. I imagine that the set-up in A1 is essentially your baseline i.e. what you are currently using today. The others A2-A3, B1-C3 and C1-C3 are what the synthetic benchmark would be uncovering, correct?

This is exactly the case, correct

Another question: I want to assume all protocol combinations will use TLS in the benchmarks?
TLS is optional in HTTP/1.1 and HTTP/2 (h2c) but in HTTP/3 it will not work over plaintext which is why it is the only web protocol where TLS use is mandatory.

yes please assume and use TLS for everything (local certs are ok), because one thing I am interested in is how long it takes to bootstrap a connection - which is most noticed on interrupted connections. I'm expecting Websockets + TLS to be the longest and HTTP3 + SSE to be the fastest in this regard.

Got it.

More questions.
The Simulation spec talks about simulating the transfer of structured and binary data. But looking at the main differences (source) between WebSockets and SSE in the table below:

WebSockets
Server-Sent Events

Two-way message transmission
One-way message transmission (server to client)

Supports binary and UTF-8 data transmission
Supports UTF-8 data transmission only

Supports a large number of connections per browser
Supports a limited number of connections per browser (six)

SSE only supports UTF-8 data transmission. I guess for SSE, this implies the use of base64 to encode and decode binary each way?

What about the 50MB limit? It is the final payload size prior to being base64 encoded?

For the client that will interact with the sync server using browser-native APIs (WebSocket, fetch, EventSource), is using a (headless) Chrome instance from playwright sufficient? Or you want the browser client to be configurable? In other words, the tester gets to use only Chrome or they can pick from any of the browsers supported by playwright i.e. Chrome, Edge, Safari (WebKit) or Firefox, as long as those browser-native APIs are properly supported?

Use base64 encoding everywhere
50mb prior to encoding
Just chrome is fine

Use base64 encoding everywhere

50mb prior to encoding

Re: 1 & 2
Can you relax this so that base64 encoding is not necessary for loading/creating binary CoValues.
In other words, base64 encoding will only used for delivering subscription events over a WebSocket or SSE?
It's much easier to split a 50MB binary file, as is, and stream it in 100KB chunks in either direction (server->client and client->server) than to do so with base64 encoding added to the mix.

Hey @aeplay
Brief status update:
Right now, I’ve working all 3 use cases for text and binary CoValues over:

HTTP/1 + SSE (Node.js)
HTTP/2 + SSE (Node.js)

Still trying to finish the WebSocket implementation and hoping I can re-use most of the code for SSE browser client to build the browser client that will interact with the WebSocket server.
(PS: I’ve been really poor at sharing updates on my progress because I have been dealing with regular interruptions. So sorry about that.)

Hey @ayewo no worries, thanks for your update - looking forward to it!

Hey @ayewo any updates on this? :)

@aeplay
Still working on it :).
I should open a PR tomorrow, God willing.

I should open a PR tomorrow or Wednesday, God willing.

Turns out my estimate was off by a few days as there are parts of the benchmarks’ plumbing that are not yet finished. Sorry about that.
When I started, I must have interpreted the Deliverable section as saying that a PR should be only be opened when the code is close to done. But re-reading it now, I realize I should have simply asked for your preference:

are you are fine with opening a draft PR for WIP code,
or open a PR when the code close to being done?

Right now, I have most things working. The only requirement I haven't touched at all are simulating the various Network conditions: I, II, III, IV.

That sounds wonderful, I would love to see a WIP draft PR. Thank you!

I'd opened a draft PR and included basic instructions on how to set it up locally for testing at the end.

Any news @ayewo ?

Hey @aeplay
Sorry I haven't been able to share any updates yet.
I've been AFK for a few weeks because I traveled. I should be returning home this weekend, God willing.

Hey @aeplay
I now have all web servers working—6 of them—and it is clear I really underestimated the amount of plumbing required to have everything working, end-to-end.
As requested, I investigated using HTTP/3 with the uWebSocket project and it turned out to be unworkable, as I mentioned in my preliminary findings, because support was experimental and development had been paused.
Right now, their experimental HTTP/3 implementation exposed via uWS.H3App on Node.js seg faults on macOS, so could only use HTTP/1.1 in the benchmarks. (They never implemented HTTP/2.)
(Sorry for the long hiatus between updates.)


Fix navbar on every page

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Hello @thestarsahil, I would like to work on this issue under GSSOC'24. I have successfully figured out the solution on my local machine.


loading phylogenies works now ...

…although the mapping of taxa to societies isn't ideal yet.
This addresses #362

Note that this PR works with data from phylorefactor branch of D-PLACE/dplace-data.

Codecov Report

Merging #424 into master will decrease coverage by -0.61%.
The diff coverage is 54.76%.

@@            Coverage Diff             @@
##           master     #424      +/-   ##
==========================================
- Coverage   91.76%   91.16%   -0.61%
==========================================
  Files          30       30
  Lines        1944     1947       +3
==========================================
- Hits         1784     1775       -9
- Misses        160      172      +12

Impacted Files
Coverage Δ

dplace_app/load/sources.py
97.43% <ø> (ø)
:white_check_mark:

dplace_app/load.py
0% <0%> (ø)
:white_check_mark:

dplace_app/tests/test_load.py
100% <100%> (ø)
:white_check_mark:

dplace_app/load/tree.py
77.58% <58.06%> (-9.29%)
:x:

dplace_app/load/util.py
96.29% <75%> (+3.43%)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc9fa2a...ab35e96. Read the comment docs.

Note also that although source information is now available for the phylogenies in the database this isn't displayed by the web app yet. Fixing this would require jumping into the angular mess.

Don't merge yet; I'm in the process of reverting to parsing the label mappings from xdid_socid_links.csv, which will restore the current look of the phylogeny trees.


Docs: update wording for rest of Users section

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[ ] Have you successfully run brew style with your changes locally?
[ ] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

The third instalment of a rework of the docs, rounding out the remainder of the Users section.

Review period will end on 2022-02-11 at 00:32:48 UTC.

Review period ended.

The Markdown suggestions were provided by mdl which I added in #12505. I've also been using some of these additional Vale styles for additional guidance. For everything else, it's a lot of multi-file searching with regexes to see if a consensus emerges.

I plan to do both once I've gone through the rest of the docs. There's hope that by then this markdownlint fix will have made it into a new release.


Bug: Crew Retrieval View Usable Combos crashes for Solar Sails Sisko

Datacore crashes every time I try to see Usable Combos for Solar Sails Sisko.  It works for all other crew I've tried, and all the other Siskos I have, but dies on Solar Sails Sisko, in both Chrome and Edge.
Steps
Upload fresh player.json file (my dbid=738885465088000)
Open Player tools > Crew Retrieval
Enter Sisko in the Search field
Click on View (Usable Combos) for Solar Sails Sisko
Nothing happens, until browser eventually gives the "Page Unresponsive" error
I then tried clicking on the "Wait" button and leaving it for a long time, eventually it gave the crash message ("Aw, Snap!" in Chrome).

That's a new one. It looks like there's over 100 different unique combos for him, I suspect that's causing some performance issues we will need to resolve.


Safekeeper <-> pageserver connection shutdown

Recently we've implemented logic to shutdown connection between safekeeper and pageserver when compute is down. Right after that we discovered that such behavior can lead to a distributed livelock:

Compute went down
Safekeeper immediately closed connection to the pageserver, but some WAL left unreplicated
Compute starts and asks a basebackup@latest_lsn from the pageserver
Pageserver never saw latest_lsn so it waits for it. But safekeeper has no intention to replicate that WAL before compute will start a new connection.

So far we are addressing that issue by letting safekeeper finish its replication before shutting down. But I can see at least two other cases that are not covered with this approach:

During that residual replication safekeeper restarts (e.g. due to deploy or some random reason). Then it still would have some unreplicated data.
After that residual replication is done pageserver restarts/crashes and it's last_received_lsn decreases. So now we are again in the same situation -- even that some time ago everything was fine.

cc @arssher @lubennikovaav @petuhovskiy

I assumed persisting (latest) pageserver address in safekeeper to deal with 1. That doesn't cover 2 though, hm.

A couple of more options:

Wait on safekeeper side for actual checkpoint happening on pageserver before putting timeline offline, i.e. disk_consistent_lsn must reach commit_lsn. Even if there isn't much data to checkpoint, after reasonable amount of time this should happen.
(less attractive) When during compute creation basebackup errors out with 'timed out waiting for WAL', console can ping safekeepers 'please retry push to pageserver', or just restart compute creation -- such ping can be included in sync-safekeepers anyway. And note that if it is included, then 2 becomes a problem only if pageserver is restarted between sync-safekeepers and basebackup -- the probability of this is quite low.

For the record, it is clear --sync-safekeepers significantly makes probability of this issue really small, as its connection revives the one to pageserver. However, it doesn't eliminate it completely, e.g. pageserver restart after sync-safekeepers and before basebackup might still lose latest WAL.

@arssher is there any requirement from pageserver ? if not lets remove the pageserver label

This issue looks like it was created when we had callmemaybe (before etcd and broker), so it's no longer relevant. We may still have similar problems, but I think we can close this issue.

is there any requirement from pageserver ?

walreceiver is a part of pageserver, so it makes sense to use pageserver label when issue is related to the walreceiver code. Or do you have other suggestions?


style: prettier output svg

Description
Remove extra spaces and linebreaks in output svg file
Linked Issues
Additional context

Please revert the lock change


Unable to select any Cognito User Groups using the Mock endpoint for testing @auth directive

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v12.21.0
Amplify CLI Version
5.2.0
What operating system are you using?
Pop!OS 21.04 (Ubuntu)
Amplify Categories
auth, api
Amplify Commands
add, update
Describe the bug
The Dropdown List of available Auth Cognito User Groups in the GraphQL Mock endpoint -> "Update Auth" -> Auth Options modal only shows "No Results Found" when trying to select a group.
Note, I've asked about this in the AWS Discord and we could not figure out the issue.
I have set up 4 groups using the Amplify Auth CLI, I'd "Amplify Push"ed and tested the @auth groups in the appsync console successfully, so it all works fine in the cloud. However, in local Mock, I cannot test the API because the Auth Options do not display any of my cognito groups.
Expected behavior
I would see my Auth groups in the dropdown list in the mock endpoint Mock "Update Auth" -> Auth Options so that I can test group permissions in my API locally.
Reproduction steps

Create API, Groups Auth using the cli
Create Graphql Schema with @auth groups permissions
create Mock and try to select a group

GraphQL schema(s)

# Put schemas below this line
type Route
  @model
  @auth(
    rules: [
      { allow: groups, groups: ["StationsAdmin","Organizations","Admins"], operations: [read, create, update, delete] }
      { allow: groups, groups: ["Users"], operations: [read]  }
    ]
  )
  {
  id: ID!
  tenantId: String!
  name: String!
  template: Boolean!
  route: String! #need to validate-
  paused: Boolean!
  # Stops : [Stop] @connection(keyName: "byStop", fields:["id"])
}

enum StatusChangeType {
  ADDED
  STATIONED
  COMPLETED
  UNCOMPLETED
  UNSTATIONED
  REMOVED
}

Log output

# Put your logs below this line
[amplify-cli-2021-07-28.log](https://github.com/aws-amplify/amplify-cli/files/6901445/amplify-cli-2021-07-28.log)

Additional information
Workaround: comment out @auth directives when using Mock.

@johnpc is there possibly a way to just add groups in the Graphiql  Update Auth -> Additional FIelds  json like you can with custom attributes?
{
    "email_verified": true,
    "phone_number_verified": true,
    "cognito:roles": [],
    "token_use": "id",
    "phone_number": "+12062062016",
    "custom:tenantId": "bp"
}

@bmilesp in that model in the groups your can add your own entries for the groups you'd like to have added to the generated JWT token.

I mark it as a feature request as one thing we could enhance here is to parse the schema and pre-populate the groups dropdown with all the groups we found in the schema.


World Two - Design

Consists of eight levels.

Due to a redesign of the game, there will not be a division into multiple worlds. Therefore, this issue will be obsolete in favour of #19.


mouse acceleration, widgets, history, default apps, etc

Description
Add Apps to the list Remove-BloatwareAppsList
-instagram, whatsapp, tiktok, spotify, prime video etc.
option to remove the widget menu
Get-AppxPackage WebExperience | Remove-AppxPackage or winget uninstall --id 9MSSGKG348SP
option to enable Run dialog history (WIN + R)
option to disable mouse acceleration
optional features to enable hyper v and sandbox
tab or option to generate a script based on your selected tweaks and software
Reason
N/A
Examples
N/A

They are not installed, just links in the start menu. They download and install if you click them.

They are not "installed", but they are stubs that can (and should) be removed clicking on them automatically installs the full version of the app. The script removes a ton of other stubs... Maybe instead of a blacklist, it should be a whitelist? Is there any good way to detect the stubs?

tab or option to generate a script based on your selected tweaks and software
This part will take great time, I'll leave this to the future.


Rate limit feature for on_kubernetes_warning_event

We sometimes have GCP acting up and causing false positives in our alerts.Currently this is our configuration, but sometimes we still get false positives.
- triggers:
  - on_kubernetes_warning_event_create:
      include: [ "FailedGetPodsMetric", "FailedGetExternalMetric" ]
      exclude: [ "googleapi: Error 503", "googleapi: Error 429", "No recommendation" ]

Some time ago, @Avi-Robusta built a custom image to test a few additional options to the trigger:

rate_limit
min_count
delay_s

But I don't see those in the documentation, so they probably never made it into a release.It would be great if those could be included and e.g. have the warning event only trigger if the Kubernetes Warning event has been firing for a certain length of time, or has fired a certain amount of times.

Hi @otherguy,
We have the rate_limit param included today.
The branch adding the other changes is found here but it's been delayed in merging for now. We plan to get back to it, but it will take a little time due to backlog on our end.

@pavangudiwada can you update the docs for on_kubernetes_warning_event_create (and any related triggers) to add rate_limit?

@aantn @pavangudiwada any update on this? 😃 We're getting a lot of false positives with GKE prometheus. They go away after a few seconds/minutes and we get pointlessly alerted. I've excluded this alert from our pages, but it would be great to have OpsGenie pages for real alerts.

@otherguy reaching out to you about this on Slack.

I was wondering if there is any news about that :)


Library Dependencies not frozen

Hi there. I am not sure how much of an issue this actually is, but I recently upgraded Typescript to version 5 and received the following error while trying to run stylelint with this syntax library:
WARNING: You are currently running a version of TypeScript which is not officially supported by @typescript-eslint/typescript-estree.

You may find that it works just fine, or you may not.

SUPPORTED TYPESCRIPT VERSIONS: >=3.3.1 <5.0.0

YOUR TYPESCRIPT VERSION: 5.0.4

Please only submit bug reports when using the officially supported version.

I noticed that this project uses @typescript-eslint/typescript-estree as a dependency. I installed postcss-styled-syntax a couple of months ago, so the dependency and sub-dependencies are frozen as of that time. I was able to fix this issue by removing postcss-styled-syntax and re-adding it to my project. This causes the sub-dependency of @typescript-eslint/typescript-estree to be frozen as of the most recent version of today.
Should the dependency libraries be locked down to a specific version in postcss-styled-syntax? When someone consumes this library, there is no way to know what version of a dependency that they are using and it may be incompatible.
(i.e. "@typescript-eslint/typescript-estree": "5.47.0", vs "@typescript-eslint/typescript-estree": "^5.47.0",)
The upside is a predictable state where people know exactly what version of libraries are being downloaded when consuming this package. The downside, of course, is that this library would need to be updated when major changes happen, such as the Typescript 5.x update.

There is no need to freeze dependencies versions in package.json. The whole Node.js ecosystem is built around Semantic Versioning. Most packages do not have versions of dependencies locked.


python2Packages.protobuf: disable broken pythonImportsCheck

Motivation for this change
I'm hoping someone can explain this to me :thinking:
Without this PR python2Packages.protobuf fails on the pythonImportsCheck, yet if I disable pythonImportsCheck and then manually import the checks (google.protobuf and google.protobuf.internal._api_implementation) inside python (built from python2.withPackages (p: with p; [ protobuf ]) on this branch) everything imports correctly :man_shrugging:
Any explanation is greatly appreciated :bow:
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Assumption is that the native extensions are only installed in $out, so unless PYTHONPATH is set, it's not going to be aware of them.

Your statement makes it sound like this PR is the "correct" way to "fix" this problem. Correct?
* "fix" implying pythonImportsCheck simply doesn't work here, so we can't use it.
Thanks @jonringer!

You can do something like:
postBuild = ''
  export PYTHONPATH=$out/${python.sitePackges}:$PYTHONPATH
'';

and that should work

Sorry @jonringer I couldn't seem to get that to work :disappointed: but I did see something which is similar to your idea in nixpkgs:
https://github.com/NixOS/nixpkgs/blob/efbc723c5eca9f686c75b9239f28b8ef1dadef82/pkgs/development/python-modules/sip/4.x.nix#L31-L32
It doesn't look nearly as nice as what you have, though... Any ideas or suggestions?

python2Packages.protobuf broken:
error: mox-0.7.8 not supported for interpreter python2.7

I'm free of python 2!


Require continuous and factor variables in config file

Workflow will run with these variables omitted, but present a problem downstream.

There could be cases where only factor or continuous variables are provided.

Duplicate of #10


HOW TO RUN

Hello Shin,
I have installed all the libraries you have suggested and I changed the rate but when I run spl_meter.py an error stating that init driver cannot run. please help me to solve the error

Hi,
Please provide all the steps to reproduce the error.
Also does it work in any case?

Hi Seyoung:
Thanks for posting this solution.
I have a Raspberry Pi 3 Model B+ and I have plugged in an External USB Headset into it.
I'm close to getting your project working but I can't get the script to run:
./spl_meter.py
However, if I run this script, I at least see the current Decibel value being displayed:
./spl_meter_text.py
I will post more details soon but I just wanted to reach out briefly.
What would be very useful is a video on how to deploy this solution on a Raspberry Pi 3 (or Raspberry Pi 3 Model B+) unit from scratch.
Now that I at least have something working, I will document the steps I think I need to follow, flash my raspberry pi OS and document all the steps I followed.
Note:
I am including two screenshots:

A screenshot of the error I get when I try to run ./spl_meter.py from a console on Raspberry pi
A screenshot on a Windows Desktop displaying a few console sessions into the Raspberry Pi
That screenshot shows spl_meter_text.py running at least partially displaying the current decibel level.


[performance suggestions?]Parallelism btw trees and replace linear fit to SGD with batch?

It seems that the any 2 tree models in a forest can be trained in parallel, is there a way to do njobs=-1 in the parameter or wrap the entire thing into a with block passing in with joblib multiprocessing njob=-1?
Is it possible to replace linear fit with SGD fit for large scale data? Should we? (in terms of speed and model equivalence)
Also, is it possible to call gpu to solve linear each time(either the traditional way or the gradient based optimizers?)
I am thinking of this type of model, if applied on tabular data , can have tracable error sensitivity( because derivative or linear slopes are known, and jumps are finite). Maybe one thing to try is to use these model on a wide range biostats tabular datasets (some of them are very small(<2k obs, < 50 vars), but have good local correlations and need good interpretations). So I am planning to use it at scale.

Thanks, but I made it work with 3(CUDA) by rewriting linear class with coef_ and intercept_ and score attributes using pytorch

But I guess: 1. for small amount of data, this is indeed very slow
2. maybe I should publish this "TorchLinearRegression" to another package?


[#151808174] Upgrade UAA to v52

What
In v50 there was a memory leak issue which was fixed in v51. We decided to
upgrade to v52 instead as they exposed new JVM metrics through statsd therefore
we can monitor memory usage in DataDog.
This is a critical release as the memory leak can cause another downtime in production soon.
How to review

sanity check
deploy the changes
all tests should pass

Who can review
not me or @paroxp

Do you think we should be concerned that the previous attempt to add metrics introduced this memory leak and that the new release with them enabled has only been out 3 days (ie. not many people will be using it in anger)?
(not a loaded question: I'm unsure myself)

@dcarley it seems they didn't enable the metrics filters in v52 yet, but we eventually agreed with @paroxp that we should only go with v51.

FYI, I have tested the pipeline run with the v51.

Tests passed 👍