lxd client: UNIQUE constraint failed: certificates.fingerprint

Ubuntu 18.04, lxd-client 3.0.3 (no lxd daemon)
Trying to add a few remote LXD servers. Some worked, one failed:
$ lxc remote add lx24 <IP_ADDRESS>
Certificate fingerprint: 26.........df
ok (y/n)? y
Admin password for lx24:
Error: UNIQUE constraint failed: certificates.fingerprint

$ lxc remote add lx24 <IP_ADDRESS>
Admin password for lx24:
Error: Certificate already in trust store

$ lxc list lx24:
Error: The remote "lx24" doesn't exist

I saw in https://github.com/lxc/lxd/issues/4976 that 'lxd sql global' could work, but I don't have the lxd command installed. Any way to recover from this?

What didn't work too:
rm ~/.config/lxc/servercerts/lx24.crt
lxc remote add lx24 <IP_ADDRESS>
Certificate fingerprint: .......
ok (y/n)? y
Admin password for lx24:
Error: Certificate already in trust store

What worked, but I had to re-add all hosts:
rm -rf ~/.config/lxc

Can you connect to the server in question and show lxd sql global "SELECT * FROM certificates;"?
It'd also have been nice to have the client.crt from your client but given you've deleted your config directory, this is now gone. Hopefully we can find something wrong in the DB, if not, we'll have to close this until someone else hits the issue and hasn't deleted their config.

There are 5 certs on that server, 2 of them for my client's IP address. If there is a way to share the output from that command privately to you, I'd be happy to do it.

I also happen to have the old client certificate (actually moved it to /tmp/ instead of deleting).

The certificate in old client.crt is the same as the one in 4th row of the table. New client cert is now in row 5.

Regarding the  "UNIQUE constraint failed":
root@lx24:~# lxd sql global "SELECT COUNT(1) FROM certificates;"
+----------+
| COUNT(1) |
+----------+
| 5        |
+----------+
root@lx24:~# lxd sql global "SELECT COUNT(DISTINCT fingerprint) FROM certificates;"
+-----------------------------+
| COUNT(DISTINCT fingerprint) |
+-----------------------------+
| 5                           |
+-----------------------------+

I think it was just an error during insert, the database looks okay to me. I don't think it's desired behavior that the client fails if server already has the certificate - is it really a problem? All the client cares about is that its certificate is in server's trust store: if it's already there, we're good, the server is already in desired state.

Right but what's odd is that the certificate already being in the trust store should have allowed your client to connect without needing to send it to the server, so there's some odd disconnect there.
The server telling the client that the certificate is already in the trust store is fine, though it shouldn't tell you this way. And once it is in the trust store, the client should just have gone straight to making requests rather than attempting to add its certificate.

Hmm, could it be that the first time it failed somewhere between saveCert() and d.clientCerts[shared.CertFingerprint(cert)] = *cert ?
https://github.com/lxc/lxd/blob/7a9a4e24f0eba557c913e735527999aed841fee7/lxd/certificates.go#L187
There are 3 return statements between saving the cert to the database and saving it in the internal cache. I guess even a network error when notifying other cluster nodes could cause inconsistency between database and the copy in memory.

Perhaps moving the code around a little bit would fix this problem:
https://github.com/lxc/lxd/compare/master...gjedeer:master

well, that's papering over the issue though, you should have hit the "Already in trust store" case, but somehow the trust list wasn't up to date in your case which is a bit odd.
Are you using clustering?

I can see how in a cluster case we could hit this issue and in that case, effectively if a node was temporarily offline while the cert was added, then came back into the cluster without fully restarting.
I suspect we could indeed avoid this kind of issue by effectively checking if the cert is in the trust store, then if it's not, check if it's in the database already, if it is, re-sync the trust store and return pretending we just added it. This would avoid client confusion and would resolve the problem.
For the case where db/memory are up to date, I do want to keep returning an error if a cert is already trusted.

No I'm not using clustering. Double-checked my seed file:
cluster: null

The server in question was under some load when the first remote add failed, I'm not sure how this could have affected the problem. Maybe certSave() failed with some kind of timeout for the first time, but stored the row eventually, and the next time the UNIQUE constraint failed error happened?

I suspect we could indeed avoid this kind of issue by effectively checking if the cert is in the trust store, then if it's not, check if it's in the database already, if it is, re-sync the trust store and return pretending we just added it. This would avoid client confusion and would resolve the problem.

Hmm, wouldn't this allow some kind of DOS attack? Attacker could spam the server with bogus certificates forcing it to SELECT * FROM certificates every time.
Unless somehow this would only happen after verifying the trust password.

We'd want to validate the trust password first I think, yeah

That was fast. Thank you!


Feat: migrate queue to application commands

things still to do / figure out

[ ] moving queue state to DB to make them continue-able;
[ ] probably in addition to the above, move buttonInteraction listeners to global interaction listeners
[ ] give "extend queue" command functionality
[ ] muting/voice channel permissions

fixes #276

@all-contributors please add @crmbs for  userTesting

@all-contributors please add @7crmbs2 for userTesting

should be ready for testing now


Feature request: iOS option to view all articles using built-in Safari View Controller

Currently testing the new iOS app. Everything has been working well so far.
I was wondering: maybe it's a good idea to add the option to open articles in a Safari View Controller. Not only for the "Open in Safari" button but also as a permanent option. Some sites don't play nicely when added in an RSS reader and directly opening an article with Safari View Controller would 'fix' this.

I find it easier to manage my web browsing in safari - tabs can be opened on other devices if I didn't finish reading something, etc.

In 6.1 there’s a setting to control whether or not links open in Safari vs. in NetNewsWire (in a Safari view controller). I hope that does the trick for you! If not, we can re-open this ticket.


176 | Second Highest Salary | SQL

Title -
Second Highest Salary
What will change -
Type of Issue -

[x] Adding New Code

Programming Language

[x] SQL

Self Check

Ask for issue assignment and wait to get assigned to it before making Pull Request.
Add your file in the proper folder
Clean Code and Documentation for better readability
Add Title and Description of the program in the file

:star2: Star it :fork_and_knife:Fork it :handshake: Contribute to it!
Discord server ✉ - https://discord.gg/CXksGdn
Happy Coding,

I would like to work on this issue. Please assign it to me.

@pstreemaze I assigned you a different issue. I'll hand this to @SudhiMohan for now.

Hi  @JOKUE2002  , Thanks for assigning the issue to me. I have created a pull request #1198  with the solution. Please check it out when you get time.
Also thanks @pstreemaze for creating the issue.

Please assign me this issue?


Draggable component fly away on linux with svelte

Describe the bug
Hello,
When I make a draggable box with svelte it flies away when I try to move it. It perfectly works on my web browser, but when I make a small move when trying to move the box on linux, it go straight down right very fast, no matter the direction of my mouse.
Reproduction
On Linux, make a draggable component following this resources

	<slot></slot>

<svelte:window on:mouseup={onMouseUp} on:mousemove={onMouseMove} />

Welcome to SvelteKit

Visit kit.svelte.dev to read the documentation

Welcome to SvelteKit

<Position />
<Box>

Box

</Box>

As you can see I tried an alternative way with a rust function to get the mouse position my the crates I've been using don't work properly.
Expected behavior
The box should be draggable like in the browser behavior
Full tauri info output
[✔] Environment
    - OS: EndeavourOS Rolling Release X64
    ✔ webkit2gtk-4.0: 2.44.1
    ✔ rsvg2: 2.58.0
    ✔ rustc: 1.77.1 (7cf61ebde 2024-03-27)
    ✔ cargo: 1.77.1 (e52e36006 2024-03-26)
    ✔ rustup: 1.27.0 (2024-03-12)
    ✔ Rust toolchain: stable-x86_64-unknown-linux-gnu (environment override by RUSTUP_TOOLCHAIN)
    - node: 21.7.2
    - yarn: 1.22.22
    - npm: 10.5.2
    - bun: 1.0.30

[-] Packages
    - tauri [RUST]: 1.6.1
    - tauri-build [RUST]: 1.5.1
    - wry [RUST]: 0.24.7
    - tao [RUST]: 0.16.8
    - tauri-cli [RUST]: 1.5.11
    - @tauri-apps/api [NPM]: 1.5.3
    - @tauri-apps/cli [NPM]: 1.5.11

[-] App
    - build-type: bundle
    - CSP: unset
    - distDir: ../build
    - devPath: http://localhost:5173/
    - framework: Svelte
    - bundler: Vite

Stack trace
No response
Additional context
Also, often when I start my the application, the box isn't a box but a simple header. If I update it for instance by changing the thickness of the box, it will become a box.
If you have alternatives, I'm happy with it

Hmm. This works on Windows, but not Linux. The movementX and movementY appear to be equivalent to screenX and screenY in my testing in Manjaro Linux (not sure if it's a coincidence that we're both using Arch distros). This explains why the position shoots off. It's not giving you the movement distance you expect. To work around the problem, you could keep track of the last reported position of clientY, pageY, offsetY, or screenY along with the corresponding X value and use the difference to adjust the left and top variables of your moveable box accordingly.

Also, this is a duplicate issue: https://github.com/tauri-apps/tauri/issues/5063

Thank you indeed I didn't find this issue


Section about the IPFS API and hands-on exercise

Problem Description
Purpose
Part of improving the curriculum of deep students is providing hands-on exercises.
Proposed Solution

Introduce the IPFS API and the Go/JS development ecosystem
Create a hands-on exercise where they get to implement a simple application by using the Go client. Go is the primary implementation language of IPFS.

Where Can we find out more about this topic?

https://docs.ipfs.io/reference/
https://github.com/ipfs/go-ipfs-api

Acceptance Criteria

A section describing the IPFS API and the Go/JS development ecosystem
A hands-on exercise that student can complete by using the IPFS Go client

Done: http://localhost:1313/curriculum/ipfs/ipfs-api/


Allow to open launch buttons in new window

What  does this PR do?
It adds the boolean option open_launch_buttons_link_in_new_window in the theme. When enabled, the buttons in the screenshot below have the target="_blank" option which opens the link in a new window instead of replacing the page.

How was this PR tested?
I built the documentation with and without the option enabled and ran pre-commit conditions checks
Why do you need it?
We embed our books inside some html iframe in order to easily switch between books on the same website. It works great except for some links which will refuse to open inside an iframe.
I tried to change it with some Javascript but it's no use because of security restriction, thus I propose this new theme option.

Thanks very much for this improvement! I think that the implementation looks good, however I have one question about whether we can simplify this a bit:
I was thinking about this UX, and want to propose that instead of making this a theme option and increasing our config surface area, we instead just change the default behavior of launch buttons, so that they always launch in a new window.
I think this is a reasonable change to make, because launch buttons are inherently taking you outside of the book itself into a new page or application. I find that many sites use this kind of "exit" action to spawn new tabs/windows.
What do you thinK?

Hey ! Yes totally, I just updated the files :)

Hi @choldgraf :) Could you re-run the CI scripts ? Some of them were cancelled while others went through


Android codename lookup (this will be painful)

TBD

[ ]  to lookup by name https://github.com/androidtrackers/certified-android-devices/blob/master/by_name.json

[ ]  also need to add reverse (codename -> device)

actually half of this is done

[x]  to lookup by name https://github.com/androidtrackers/certified-android-devices/blob/master/by_name.json

[ ]  also need to add reverse (codename -> device)


Develop upstream sync 200805

Conflict file list

tensorflow/compiler/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc
tensorflow/core/kernels/BUILD
tensorflow/core/kernels/reduction_gpu_kernels.cu.h
tensorflow/python/eager/def_function_xla_jit_test.py
tensorflow/python/kernel_tests/BUILD
tensorflow/python/kernel_tests/array_ops/batch_gather_op_test.py

Can one of the admins verify this patch?

jenkins retest rocm
jenkins retest rocm-xla
jenkins retest rocm-cpp

jenkins retest rocm please

jenkins retest rocm-xla please

jenkins retest rocm-cpp please

jenkins retest rocm-cpp please


Feature_Request: Add navbar on "Rate Us?" Page

Is your feature request related to a problem? Please describe.
There is no navbar on "Rate Us?" Page
Describe the solution you'd like
I want to add navbar on "Rate Us?" Page under JWoC 2024
Describe alternatives you've considered

Demo Video/Screenshot (Mandatory )
Before:

After:

Are you contributing under any open-source program ?
I am contributing under JWoC 2024
Please assign me this issue

@Sanglap-Halder I think there is no need to make a navbar in the rate us page as a return to home page is already present there .
Sorry this issue can't be assigned.

Ok


Investigate a dashboard tool to monitor app performance improvement/degradation after deploys

Placeholder to remind us to get a dashboard that can track performance improvements or degradations when reassure testing is live

@rinej started working on this today - he'll investigate the ways we can utilise json outputs from Reassure to showcase historical data and how we can pipe this through to eg. https://grafana.com/

@mountiny I think we can assign @rinej to this one 🙂

Grafana would be ideal as we use that already!

@mountiny Hello, today I started investigating the approach, I plan to use grafana

Hello mountiny,
We created an example approach using the JSON REST API Grafana plugin -> https://grafana.com/grafana/plugins/marcusolsson-json-datasource/
Here is the example dashboard:

In that scenario, we created a simple node server that exposed the endpoints with JSON data, which Grafana consumed. It has some limitations; we will have to maintain a separate instance of the server to deliver data.
We are exploring the other two approaches:

Storing just the JSON file on the S3 bucket and updating it when the user runs tests.
Storing the SQLite database file in the S3 bucket and updating it when the user runs tests.

This will not require any additional server, just, for example, an S3 bucket to store the data. I will let you know when we have some insights about the approaches.
Meanwhile, do you have any preferences over what needs to be displayed and shown on the graph? I can send you the example output file from Reassure; just let me know! :)

Thank you @rinej!

Storing just the JSON file on the S3 bucket and updating it when the user runs tests.

I thin kthis sounds good to me, we already have solutions where we save files to S3 bucket from the GH actions so it shoudl also be reasonable easy to achieve

Meanwhile, do you have any preferences over what needs to be displayed and shown on the graph?

I was imagining that we would output the results of main branch over time, when the results get worse we can see based on timestamp what PR caused it, but we could just keep data about the results on main for various test scenarios and see how they evolved overtime

Thanks mountiny you for your answers!
We created some POC using JSON rest API grafana plugin
JSON api plugin: https://grafana.com/grafana/plugins/marcusolsson-json-datasource/
Plugin needs one endpoint which returns the data
Data used for the POC response (3 test cases run on different dates on different branches):
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":18.244273030757904, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":24.91762231886387, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should press add emoji button","type":"render", "meanDuration":12.9931875705718993, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":13.244273030757904, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":16.91762231886387, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should press add emoji button","type":"render", "meanDuration":2.9931875705718993, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":14.244273030757904, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":20.91762231886387, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" },
{"name":"should press add emoji button","type":"render", "meanDuration":5.9931875705718993, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" }

Example dashboards:

A - table view - we can choose from a dropdown which test data we want to display
B - Tests grouped by name on a timeline - we can preview additional information about each point
C - Table view for all test data
D - Tests grouped by name on one graph
I think dashboards A and B will be the most useful, so we can see on the timeline when performance has dropped
🎯 What needs to be done:

[ ]  Add github action to run reassure after merge to main
[ ]  Upload output.json file to S3 bucket

It can be done either in reassure lib or in the script in Expensify repo

[ ]  Create endpoint which gets JSONs files from S3, combines it and returns data in the above format
[ ]  Create dashboard in Grafana which consumes the endpoint

❓Questions to mountiny?

[ ]  Do you have some infrastructure so we can create and maintain the simple REST get endpoint? The logic in the endpoint:
Get all json performance files from S3 bucket (or any other storage place)
Combine and return the final json file which grafana will consume

@rinej Looks very promising, thanks! I will discuss with out infra team who could help us with the implementation as I do not have access to edit S3 or add this plugin to Graphana if we do not have it yet

Thread we were discussing this is here

@mountiny I don't have access to the above link or the link is broken, could you give me the access?

Sorry cannot give you access to that one. It was Thanksgiving week so @johnmlee101 will be back this week and he is going to be able to help us with this I am sure

@mountiny @rinej
https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts
@justinpersaud brought up using artifacts if we don't care about the data lasting for too long. We have a max 90 day policy.
As for the REST endpoint if we can easily find a way to list all artifacts this way that would be ideal

@johnmlee101 @mountiny
you mentioned that you already use Grafana for some metrics, what kind of data source do you use?
Maybe adding integration to the existing flow would be more efficient, instead of creating something new (like S3)?
Do you use grafana cloud service or standalone app?

@rinej we do use S3 already, but I think in this care @johnmlee101 is suggesting to use Github artifacts to store the results json for 90 days which is fine. Then we can show those in Grafana just fine too.

@johnmlee101
With that approach, we still need to create the endpoint that retrieves all the artifacts from GitHub and prepares the data for Grafana. Do you have some infrastructure that we can use?
Probably, we can use that API to get artifacts: GitHub Artifacts API.
@mountiny, do you currently use S3 to connect to Grafana, or do you use other plugins for Grafana? I am asking because if you already have some data storage for Grafana, maybe we can use it for the dashboard as well.

I am not sure what we use for Grafana now, I would err on @johnmlee101 opinion here as I am not that familiar with this. Sorry for not being of better help

Thank for the feedback @mountiny!
@johnmlee101 are you able to check what currently Expensify is using to connect to Grafana?

What could also be easier is allowing for authenticating GH Actions to hit our graphite endpoint somehow to upload this specific data without the need for formatting JSON. We can do it all from the workflow

That would be great, so we can reuse existing architecture instead of creating new.
Do you know if we can store the json data in graphine? So we don't need to save it as a artifacts on GH?
So the flow would be:  run reassure test in CI -> upload output to graphine -> connect graphine with grafana to display graphs, what do you think?
Do you have some example account in graphine so we can investigate how the data is stored there?

Hmm, does it have to be uploaded as JSON? I wonder if we can interact directly with graphite. https://graphiteapp.org/quick-start-guides/feeding-metrics.html this is how metrics are uploaded

Thanks, I will create some proposal with using Graphite

Hello @johnmlee101,
I created some POC and have some questions:

When using graphite, we assume we will use plaintext protocol to upload reassure data to graphite db -> https://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol
Do you know how can we authenticate that request using nc? Do we have some examples of how do you make authorized calls to the graphite db?

Do you have any suggestion on the structure on the metric keys in the graphite? As graphite stores data as value/key pairs, we might consider the structure like that:
reassure.{branchname}.{testcasename}: value

Also what do we want to track as a metric for our case? I would go with meanDuration- It is calculated by adding up all the values in perf time and then dividing the sum by the number of values.

Here is example data from reassure current.perf:
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":18.244273030757904, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":24.91762231886387, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should press add emoji button","type":"render", "meanDuration":12.9931875705718993, "creationDate":"2023-11-17T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"some-branch" },
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":13.244273030757904, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":16.91762231886387, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should press add emoji button","type":"render", "meanDuration":2.9931875705718993, "creationDate":"2023-11-15T12:45:04.146Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"branch123" },
{"name":"should render Composer with text input interactions","type":"render", "meanDuration":14.244273030757904, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" },
{"name":"should press add attachemnt button","type":"render", "meanDuration":20.91762231886387, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" },
{"name":"should press add emoji button","type":"render", "meanDuration":5.9931875705718993, "creationDate":"2023-11-16T14:49:26.957Z", "commitHash":"137a4c761ee686bd6e094ec025b0dcc3f5f454e1", "branch":"test-branch" }

We created a simple POC using local graphite, in that we can display data from graphite, for the single test cases in the timeseries graph:

So the flow for that approach would be:

Open the current.perf file

Grep it or jq or sed or awk to get data we need (prepare key/value pairs, e.g. reassure.branch1.shouldPressAddEmojiButton: 20.9176)

We will make call using nc to graphite

example call using nc:

echo -e "reassure.branch1.shouldPressAddEmojiButton 20.9176 $TIMESTAMP\nreassure.branch1.shouldPressAddAttachementButton 5.92 $TIMESTAMP | nc {GRAPHITE_SERVER} {GRAPHITE_PORT}

how can we authorize that call? ( I tested on local so no authorization was necessary, but I assume your graphite db requires some kind of auth)

Display the data in Grafana using Graphite plugin

Thanks for answering the questions!

Hello @johnmlee101, have you got a chance to have a look on the proposal and the questions?

Waiting for @johnmlee101 to pick this up in new year

Hey @johnmlee101 - looks like the team is awaiting your input here. Is there a chance you'll be available for this in the next few days to help push this forward? :)

@johnmlee101 I know you have been busy with some other stuff, would you prefer to reassign this issue? we are very eager to get this dashboard going

John is deployer this week I think - hopefully we can pick this up next week

Hey @johnmlee101 are you able to pick this back up next week, or should we reassign?

Bump @johnmlee101, do you have an idea of when you can pick this back up?

Okay if we pin the runner to a certain IP we can whitelist it for this case 👍

Hello @mountiny @johnmlee101, thanks for the answers! Here is our update:
We created example Dashboard with chart for two values: meanDuration and meanCount. Where each line is a single TestCase, when hovering over the point we can which branch it is:

Here is the JSON file for the dashboard:
GraphiteMetricDashboard.json
For the POC we used Graphite in the docker container and local Grafana, installed using homebrew:

https://graphite.readthedocs.io/en/latest/install.html
https://grafana.com/docs/grafana/latest/setup-grafana/installation/mac/

Structuring the data in Graphite
Here is the proposed solution for structuring the data:
{BUCKET_NAME}.reassure.{METRIC_NAME}.{BRANCH}.{TEST_NAME}

BUCKET_NAME - main global bucket
METRIC_NAME - name of the tracking metric (e.g. meanDuration and meanCount)
BRANCH - name of the branch
TEST_NAME - name of the specific test
Here is the example path to the data:
"bucket1.reassure.meanCount.branch9.[SearchPage]-should-interact-when-text-input-changes"

Feeding the data
To feed the data as recommended we use graphite plaintext protocol:
https://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol
Example bash script used for POC:
#!/bin/bash

BRANCH="branch4"
TEST_NAME1="[SearchPage]-should-interact-when-text-input-changes"
TEST_NAME2="[SearchPage]-should-render-options-list"
TEST_NAME3="[SearchPage]-should-search-in-options-list"

# Metric data
METRIC_PATH1="bucket1.reassure.meanDuration.${BRANCH}.${TEST_NAME1}"
METRIC_VALUE1=55.6
METRIC_PATH2="bucket1.reassure.meanDuration.${BRANCH}.${TEST_NAME2}"
METRIC_VALUE2=68.92
METRIC_PATH3="bucket1.reassure..meanDuration.${BRANCH}.${TEST_NAME3}"
METRIC_VALUE3=46.84

METRIC_PATH4="bucket1.reassure.meanCount.${BRANCH}.${TEST_NAME1}"
METRIC_VALUE4=4
METRIC_PATH5="bucket1.reassure.meanCount.${BRANCH}.${TEST_NAME2}"
METRIC_VALUE5=6
METRIC_PATH6="bucket1.reassure.meanCount.${BRANCH}.${TEST_NAME3}"
METRIC_VALUE6=4

echo -e "$METRIC_PATH1 $METRIC_VALUE1 $TIMESTAMP\n$METRIC_PATH2 $METRIC_VALUE2 $TIMESTAMP\n$METRIC_PATH3 $METRIC_VALUE3 $TIMESTAMP
         $METRIC_PATH4 $METRIC_VALUE4 $TIMESTAMP\n$METRIC_PATH5 $METRIC_VALUE5 $TIMESTAMP\n$METRIC_PATH6 $METRIC_VALUE6 $TIMESTAMP" | nc localhost 2003

In the real version the final string will be taken from reassure output file in the script we add in the github action
Grafana Integration
To integrate Grafana, we utilized the Graphite plugin, which can be found at https://grafana.com/grafana/plugins/graphite/.
This plugin facilitates the connection to Graphite in the Docker container
To display the data grouped by test name, we used the following query:
groupByNode(bucket1.reassure.meanDuration.*.*, 4)

We grouped the data by the 4th node, in our case it is the TEST_NAME node.
One of the challenges was how to display not only the test name but also the branch from which the test was run.
To achieve this, we can add another query and group it by the branch name. In our case, the branch name is node number 3.
groupByNode(bucket1.reassure.meanDuration.*.*, 3)

That solution is not perfect because it adds additional data to the chart. It may be worth investigating it further in the real Grafan environment. But the metadata about test name, branch, metric and values will be stored in the Graphite.
Here is a short video demonstrating how to display the branch on the chart:
https://github.com/Expensify/App/assets/2650904/723c9370-88c2-48d9-aa03-87fc19b5594f
Next steps we are working on:

Preparing script which gets all data from json reassure file
Integrating the script in CI in GH actions and send data to Graphite

@mountiny @johnmlee101 I am adding some proposal for parsing and sending reassure data to graphite:
In order to parse reassure output.json and send it to graphite server we need to create:

the function which will parse the file and create the graphite string -> getGraphiteString.js
new github action which will make call to graphite server with the parsed data -> send-reassure-to-graphite

Reassure output.json parser
getGraphiteString.js
const core = require("@actions/core");
const fs = require("fs");

const run = () => {
  // Prefix path to the graphite metric
  const GRAPHITE_PATH = "bucket1.reassure";

  const regressionOutput = JSON.parse(
    fs.readFileSync(".reassure/output.json", "utf8")
  );

  const commitHash = regressionOutput.metadata.current.commitHash;
  const creationDate = regressionOutput.metadata.current.creationDate;
  const timestampInMili = new Date(creationDate).getTime();
  // Graphite accepts timestamp in seconds
  const timestamp = Math.floor(timestampInMili / 1000);

  // We need to combine all tests from the 4 buckets
  const reassureTests = [
    ...regressionOutput.meaningless,
    ...regressionOutput.significant,
    ...regressionOutput.countChanged,
    ...regressionOutput.added,
  ];

  // Map through every test and create string for meanDuration and meanCount
  const graphiteString = reassureTests
    .map((test) => {
      const current = test.current;
      // Graphite doesn't accept metrics name with space, we replace spaces with "-"
      const formattedName = current.name.split(" ").join("-");

      const meanDurationString = `${GRAPHITE_PATH}.meanDuration.hash-${commitHash}.${formattedName} ${current.meanDuration} ${timestamp}`;
      const meanCountString = `${GRAPHITE_PATH}.meanCount.hash-${commitHash}.${formattedName} ${current.meanCount} ${timestamp}`;

      return `${meanDurationString}\n${meanCountString}`;
    })
    .join("\n");

  // Set generated graphite string to the github variable
  core.setOutput("graphiteString", graphiteString);
};

if (require.main === module) {
  run();
}

module.exports = run;

Github action
send-reassure-to-graphite
name: send-reassure-to-graphite
on:
  push:
    branches: [main]
jobs:
  testJob:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: "20"
      # In final version for setting up node we will use the created action from the repo:
      # - name: Setup NodeJS
      #   uses: ./.github/actions/composite/setupNode

      - name: Install dependencies
        run: npm install

      - name: Run JavaScript file
        id: saveGraphiteString
        run: node ./.github/actions/javascript/getGraphiteString.js

      - name: Send graphite data
        run: echo -e "${{ steps.saveGraphiteString.outputs.graphiteString }}" | nc $GRAFHITE_SERVER $GRAFHITE_PORT

In the GH action send-reassure-to-graphite, we call the JS function getGraphiteString . The function is responsible for parsing output.json and setting the graphiteString variable with the all necessary data.
In the next step we make the nc protocol call to the graphite server.
The $GRAFHITE_SERVER $GRAFHITE_PORT  will need to be set in the github env variables.
@mountiny In that approach we are saving data by commitHash, we are investigating the way to save by PR number. I think we might take it in the github action
Some concerns:

We are not sure about nc graphite call bandwidth, when we would have many more reassure test some data might be lost

cc
@adhorodyski

@mountiny @johnmlee101
Here is the final version of scripts, as you requested with that we will have access to PR number so we can save it in the graphite string. Also we had to decide when to run the script, ideally just once after the PR is merged to master.
Please let us know what you think, I am going to start working on draft PR.
Assumptions:

Currently we trigger reassure tests and generate output.json on separate branches every time we create a PR or modify it.
We want to be able to access the PR number within the action so it needs to be triggered in pull_request event

Under these assumptions, we propose that instead of creating a new GH action to be triggered on the main, we will modify the current reassurePerformanceTests . We need to make sure that the steps responsible for making call to graphite are triggered only after merging to the main.
Pros of this approach:

We have the certainty that we have access to output.json, because we are in the same job all the time
We have access to prNumber, which means we can save this data in graphite
We don't have to create a new action, just modify the current reassurePerformanceTests.yml.

Here are the the steps we will add to the end of reassurePerformanceTest.yaml
- name: Run JavaScript file
        # run only when merged to master
        if: github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main'
        id: saveGraphiteString
        run: node ./.github/actions/javascript/getGraphiteString.js

      - name: Send graphite data
        # run only when merged to master
        if: github.event.pull_request.merged == true && github.event.pull_request.base.ref == 'main'
        run: echo -e "${{ steps.saveGraphiteString.outputs.graphiteString }}" | nc $GRAFHITE_SERVER $GRAFHITE_PORT

Here is the modified getGraphiteString.js:
const core = require("@actions/core");
const github = require('@actions/github');
const fs = require("fs");

const run = () => {
  // Prefix path to the graphite metric
  const GRAPHITE_PATH = "bucket1.reassure";

  const regressionOutput = JSON.parse(
    fs.readFileSync(".reassure/output.json", "utf8")
  );

  const creationDate = regressionOutput.metadata.current.creationDate;
  const timestampInMili = new Date(creationDate).getTime();
  // Graphite accepts timestamp in seconds
  const timestamp = Math.floor(timestampInMili / 1000);

  // get PR number from the github context
  const prNumber = github.context.payload.pull_request.number;

  // We need to combine all tests from the 4 buckets
  const reassureTests = [
    ...regressionOutput.meaningless,
    ...regressionOutput.significant,
    ...regressionOutput.countChanged,
    ...regressionOutput.added,
  ];

  // Map through every test and create string for meanDuration and meanCount
  const graphiteString = reassureTests
    .map((test) => {
      const current = test.current;
      // Graphite doesn't accept metrics name with space, we replace spaces with "-"
      const formattedName = current.name.split(" ").join("-");

      const meanDurationString = `${GRAPHITE_PATH}.meanDuration.PR-${prNumber}.${formattedName} ${current.meanDuration} ${timestamp}`;
      const meanCountString = `${GRAPHITE_PATH}.meanCount.PR-${prNumber}.${formattedName} ${current.meanCount} ${timestamp}`;

      return `${meanDurationString}\n${meanCountString}`;
    })
    .join("\n");

  // Set generated graphite string to the github variable
  core.setOutput("graphiteString", graphiteString);
};

if (require.main === module) {
  run();
}

module.exports = run;

so now instead of using commitHash or branch name, we save the PR name in the graphite path.

Hey @johnmlee101 @mountiny just a reminder to check out the above when you get a chance!

@johnmlee101 @mountiny
I created the PR with the GH action which parses the output.json from reassure tests and make call to graphite server. Please have a look - https://github.com/Expensify/App/pull/36480
Also we will need to set the environmental variables for $GRAPHITE_SERVER $GRAPHITE_PORT.
Do you have some suggestions how can we test the flow before merging to main?

thank you! posting in slack too

Latest question for @johnmlee101 is here: https://expensify.slack.com/archives/C05LX9D6E07/p1708601197525589?thread_ts=1707855242.234049&cid=C05LX9D6E07

Grabbing the latest from Slack:

added the fix, the checks passes. Thanks @john!
So I think when all backend changes will be there we can merge the test PR and see if it works :) -> https://github.com/Expensify/App/pull/37019

This was a new feature/tool, no regression test needed here.

Correct! I think this is done but just checked the dashboard and cant see anything in it

asking if we moved it somewhere else

It just takes very long to load for anything more than a day

No regression test needed, this was a new feature! Expert contrib so no payment due on this one.


Fixes for issue  #248

Purpose

Fixes for issue # #248

Goals

Describe the solutions that this feature/fix will introduce to resolve the problems described above

Approach

Describe how you are implementing the solutions. Include an animated GIF or screenshot if the change affects the UI (email<EMAIL_ADDRESS>to review all UI text). Include a link to a Markdown file or Google doc if the feature write-up is too long to paste here.

User stories

Scenario23

Release note

Brief description of the new feature or bug fix as it will appear in the release notes

Documentation

Link(s) to product documentation that addresses the changes of this PR. If no doc impact, enter “N/A” plus brief explanation of why there’s no doc impact

Training

Link to the PR for changes to the training content in https://github.com/wso2/WSO2-Training, if applicable

Certification

Type “Sent” when you have provided new/updated certification questions, plus four answers for each question (correct answer highlighted in bold), based on this change. Certification questions/answers should be sent to<EMAIL_ADDRESS>and NOT pasted in this PR. If there is no impact on certification exams, type “N/A” and explain why.

Marketing

Link to drafts of marketing content that will describe and promote this feature, including product page changes, technical articles, blog posts, videos, etc., if applicable

Automation tests

Unit tests

Code coverage information

Integration tests

Details about the test cases and coverage

Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes/no
Ran FindSecurityBugs plugin and verified report? yes/no
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes/no

Samples

Provide high-level details about the samples related to this feature

Related PRs

List any other related PRs

Migrations (if applicable)

Describe migration steps and platforms on which migration has been tested

Test environment

List all JDK versions, operating systems, databases, and browser/versions on which this feature/fix was tested

Learning

Describe the research phase and any blog posts, patterns, libraries, or add-ons you used to solve the problem.

The flow is I call authorize endpoint and the system generates an authorize endpoint
At this point I will click on approve always and submit so both the scripts 5 and 7 will pass since a record is on the database but if we run the second time the consent ui does not exist
However, if I call approve once ex the id id="approveCb" the approval didnt seem to happen


aci-materials-journal citation style: incorrect formatting of pages info

Describe the bug
When citing an item having pages range/ start page - end page, e.g. journal article, aci-materials-journal CSL style assumes quoting pages in a shortened format while as structured within the version 3.0.0-alpha.2 it appears in full.
Please fill in the following sections and provide as much information as possible. It will help us to reproduce the issue. Leave sections blank or remove them if they don't apply.
Which version of citeproc-java are you using?
1.0.1
Expected behavior
Expected citation structure us as below:
1.Hisakata, R., Nishida, S., and Johnston, A. “An Adaptable Metric Shapes Perceptual Space,” Current biology, V. 26, No. 14, 2016, pp. 1911–5.

Actual behavior
1.Hisakata, R., Nishida, S., and Johnston, A. “An Adaptable Metric Shapes Perceptual Space,” Current biology, V. 26, No. 14, 2016, pp. 1911–1915. - please note that pages info appears in full instead of being in a shortened format.

CSL input
CSL api object fields: {"id":"-GEN-181h5kgumb","type":"article-journal","language":"eng","author":[{"family":"Hisakata","given":"Rumi"},{"family":"Nishida","given":"Shin’ya"},{"family":"Johnston","given":"Alan"}],"issued":{"date-parts":[[2016]]},"container-title":"Current biology","DOI":"10.1016/j.cub.2016.05.047","edition":"","ISBN":"","ISSN":"0960-9822","issue":"14","keyword":"Adaptation, Physiological ; Form Perception ; Humans ; Motion Perception ; Photic Stimulation ; Report ; Retina - physiology","note":"","number-of-pages":"","page":"1911-1915","page-first":"1911-1915","PMID":"27426520","publisher":"Elsevier Ltd","publisher-place":"England","section":"","title":"An Adaptable Metric Shapes Perceptual Space","URL":"","volume":"26"}
Used style and locale
aci-materials-journal
Rendered output/Screenshots
If applicable, add the output that citeproc-java produced and/or screenshots to help explain your problem.
Example code/unit test
Provide example code or a unit test to reproduce your issue. You may also insert a link to your repository containing a minimal reproducer here.

Duplicate of #148


Link ABAP SDK for Azure

Our generalizes ABAP SDK offerings and customers would profit from discoverability if cross-referenced here. Please add a callout that other Azure services can be consumed using that: https://github.com/microsoft/ABAP-SDK-for-Azure

Moving this to internal discussion -  so that we can find the right place in documentation for the call out.
The Documentation is currently in progress at : AI SDK Documentation.


Joining the group

How to join the study group?
Are there any pre-requisites?
What topics have already been covered?
In what languages do you provide the content in?

It's mostly Chinese here, English zk srudy can go to discord


[ExtensibilityEnhancement] Page "BC O365 Sales Quote": CurrencyFormat protected

Same as #18437 here.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 42225.


Fix for adding multiple controllers and swtiches

For adding multiple controllers and switches with custom mapping, the implementation suggested in the examples is not  straight forward. The implementation here is mapping the swithes to controllers in start.

Could you provide an example of how this API would be used?

#!/usr/bin/python

"""
Create a network where different switches are connected to
different controllers, by creating a custom Switch() subclass.
"""

from mininet.net import Mininet
from mininet.node import OVSSwitch, Controller, RemoteController
from mininet.topolib import TreeTopo
from mininet.log import setLogLevel
from mininet.cli import CLI

setLogLevel( 'info' )

# Two local and one "external" controller (which is actually c0)
# Ignore the warning message that the remote isn't (yet) running
c0 = Controller( 'c0', port=6633 )
c1 = Controller( 'c1', port=6634 )
c2 = RemoteController( 'c2', ip='<IP_ADDRESS>', port=6633 )

cmap = { 's1': c0, 's2': c1, 's3': c2 }
topo = TreeTopo( depth=2, fanout=2 )
net = Mininet( topo=topo )
for c in [ c0, c1 ]:
    net.addController(c)
net.build()
net.start(cmap)
CLI( net )
net.stop()


Modify default port numbers on local-metadata env

Currently, the MySQL and ES containers are running with ports 3306 & 9200, which are the default port numbers for these services.
I think it would be a good idea to add a prefix to all of the exposed local ports, by that avoid confusion and potential conflicts between other local services.
For example, the exposed ports would be 53306 & 59200 for the services mentioned above.

Moving this out to Milestone-2

Given we have environment variables. This should be straightforward now.

@akash-jain-10 we should document the environment variables and allow users to override them


cmq: lots of various fixes

added new top-level shortcut commands to list extents (x), list destinations (d), cg-extents (cgx), etc
a new "cmq fix fixdestuuid" command that updates the (newly added) 'destination_uuid' column for each row of 'consumer_groups' table, with the 'consumer_group.destination_uuid' value.
various minor fixes in 'cmq gc'

use "set" package instead of 'map[string]string'
removed redundant variables, maps, etc.

enhanced the 'cmqWriter' interface to make it easier to do "json" output, etc
implemented 'listStoreExtents' and 'listInputHostExtents' commands
moved top-level metadata args handling to 'meta.go'
the delete and undo writers (used by 'cmq gc') creates only one "_delete.cql" and "_undo.cql" each, instead of one per table
added support to recognize 'status' by numeric values (0 -> open, etc)
support routines for ANSI control sequences

Can you explain what this diff does?

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Coverage decreased (-0.3%) to 66.582% when pulling 63e6033635c37ffc5891576b39d0488ef8619294 on cmq-gc into 9e0668a6037ad200576b4798e128177324d099ba on master.


On Chant Detail page, allow sorting of concordances table by clicking on column headers

As Anna suggested in #414, it might be useful for people to be able to sort the table of concordances that appears on chant detail pages in a similar way to how people can sort results on the chant search page by clicking on the column headers. OldCantus doesn't do this, so I'm creating a new "enhancement" issue with this suggestion.

I realized that there are a bunch of different views where we might want to implement this, so #426 is now a list of tasks, and not just an issue related to a single view. I added chant-detail > concordances to the list, so I'm closing this issue.


add make_first/last_sibling methods

slab_tree doesn't seem to have an interface for node removal yet, so I'm continuing to use id_tree for now since it seems to be more complete, but I did add these convenience methods for moving a child to the front or back of the child list.

slab_tree doesn't seem to have an interface for node removal yet [...]

  I do believe you're correct that there's no remove on the Tree level, but you can remove_first / remove_last if you have a NodeMut of the parent node.  I understand that may not suit your needs, but I just wanted to make sure you're aware.

Yep, only issue with those is it leaves children with invalid parent reference. (https://github.com/iwburns/slab-tree/issues/14)


Fix the potentail negative sleep time in HelixServerStarter

Codecov Report

Merging #4277 into master will increase coverage by 0.11%.
The diff coverage is 5%.

@@             Coverage Diff              @@
##             master    #4277      +/-   ##
============================================
+ Coverage     67.21%   67.32%   +0.11%
  Complexity       20       20
============================================
  Files          1039     1039
  Lines         51535    51537       +2
  Branches       7228     7230       +2
============================================
+ Hits          34638    34697      +59
+ Misses        14534    14475      -59
- Partials       2363     2365       +2

Impacted Files
Coverage Δ
Complexity Δ

...pinot/server/starter/helix/HelixServerStarter.java
44.59% <5%> (-0.32%)
0 <0> (ø)

...core/operator/dociditerators/AndDocIdIterator.java
51.72% <0%> (-31.04%)
0% <0%> (ø)

...egation/function/customobject/MinMaxRangePair.java
75.86% <0%> (-24.14%)
0% <0%> (ø)

...pinot/core/operator/docidsets/OrBlockDocIdSet.java
84.9% <0%> (-13.21%)
0% <0%> (ø)

...impl/dictionary/DoubleOnHeapMutableDictionary.java
69.56% <0%> (-6.53%)
0% <0%> (ø)

...mpl/dictionary/DoubleOffHeapMutableDictionary.java
78.57% <0%> (-3.58%)
0% <0%> (ø)

...e/operator/dociditerators/SVScanDocIdIterator.java
67.85% <0%> (-3.58%)
0% <0%> (ø)

...e/operator/dociditerators/BitmapDocIdIterator.java
60.71% <0%> (-3.58%)
0% <0%> (ø)

.../impl/dictionary/BaseOffHeapMutableDictionary.java
83.92% <0%> (-2.98%)
0% <0%> (ø)

...g/apache/pinot/common/metrics/AbstractMetrics.java
76% <0%> (-1.34%)
0% <0%> (ø)

... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ccab9a1...def7837. Read the comment docs.


Customise switch_data name

Hi @gravesti , is it possible to add a clause in initiators() that allows us to customise the file name of "switch_data.csv"? Having it defaulted to this name makes parallelised simulations using this function difficult because there's only one switch_data.csv for multiple parallel jobs requiring their own switch_data.csv. Thanks!

Hi @juliettelimozin, yes I can look at this, but i wonder if it is already possible using the data_dir argument to initiators(). You would give each simulation it's own folder. Otherwise I'm worried that the other temporary files will be colliding if all the simulations are running in the same place.

Yes, exactly. In your iterations you could do something like
for (s in 1:10) {
  sim_dir <- paste0("my_dir/simulations/s_", s)
  if (dir.exists(sim_dir)) unlink(sim_dir, recursive = TRUE) # delete directory if it exists, to remove old results
  dir.create(sim_dir)
 # ...
  initiators( ..., data_dir = sim_dir)
 # ...
}


Feature: Store configuration in  a ScriptableObject

It would be very convenient to have the Configuration for the SDK in a dedicated object that is stored outside of the Package for portability.
During Editor- Time, the package could scan for a dedicated SO that contains all required Fields from TezosConfig.cs.
One could also add a page to the ProjectSettings to edit these.
Ripped this from one of my own projects and adapted it a bit, this is the Editor side:
            const string name = "TezosConfig";
            string[] guids = AssetDatabase.FindAssets(name);

            List<string> results = new();

            foreach (string guid in guids)
            {
                string entry = AssetDatabase.GUIDToAssetPath(guid);
                if (entry.EndsWith(name + ".asset"))
                {
                    results.Add(entry);
                }
            }

            if (results.Count == 0)
            {
                _config = ScriptableObject.CreateInstance<TezosConfigSO>();
                AssetDatabase.CreateAsset(_config, "Assets/Resources/TezosConfig.asset");
                AssetDatabase.SaveAssets();
                AssetDatabase.Refresh();
            }
            else
            {
                _config = AssetDatabase.LoadAssetAtPath<Config>(results[0]);
            }

And then at runtime:
          const string configName = "TezosConfig";
           _config = Instantiate(Resources.Load(configName) as TezosConfig);

           if (_config == null)
           {
               Debug.Error("Configuration has not been found, create TezosConfig from Context menu in the Assets/Resources folder to be loaded at runtime");
           }

This should be addressed in 3.0.0 with TezosConfigSO and DataProviderConfigSO


adds bestemployeever.com

This PR is:

[X] Adding a new domain
[ ] Updating existing domain size
[ ] Changing domain name
[ ] Removing existing domain from list
[ ] Website code changes (512kb.club site)
[ ] Other not listed

[ ] I used the uncompressed size of the site
[X] I have included a link to the Cloudflare report
[X] This site is not an ultra minimal site
[X] The following information is filled identical to the data file

I confirm that I have read the FAQ section, particularly the two red items around minimal pages and inappropriate content and I attest that this site is neither of these things.

[X] Check to confirm

- domain: bestemployeeever.com
  url: https://bestemployeeever.com/
  size: 198.36
  last_checked: 2024-08-03

Cloudflare Report: https://radar.cloudflare.com/scan/c9241314-bc8e-4712-993d-03615660a9c5/summary

Thanks!


Revert "[release-0.11] Fixing nil pointer error"

Reverts aws/eks-anywhere#3489

/approve
/lgtm
/cherrypick release-0.11


Layout issue with default logo

In the list of links - if a link has no logo (the default), the link text aligns with the start of the logos above/below, rather than aligning with their text.

For now, bodge this with a transparent PNG logo.  Do it properly later

Fixed by #342


fix dphyper support evaluate join that has one side condition (#33702)

Proposed changes
#33702

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall


"error FS3160: Problem reading assembly" when referencing Microsoft.AspNetCore.Http.Abstractions package

FSC has an issue reading an assembly that I believe it shouldn't have a problem with.
Repro steps
Provide the steps required to reproduce the problem

dotnet new console -n Repro -lang F#
cd Repro
dotnet add package Microsoft.AspNetCore.Http.Abstractions
dotnet run

Expected behavior
Should build without issue.
Actual behavior
I get this error:
FSC : error FS3160: Problem reading assembly '/home/nat/.nuget/packages/microsoft.aspnetcore.http.abstractions/2.2.0/lib/netstandard2.0/Microsoft.AspNetCore.Http.Abstractions.dll': Index was outside the bounds of the array. [/home/nat/Projects/fsharp-asm-repro/Repro/Repro.fsproj]

Known workarounds
Current version to cause this issue is 2.2.0.  Pegging to 2.1.1 solves the issue.  See nuget.org page here:
https://www.nuget.org/packages/Microsoft.AspNetCore.Http.Abstractions/
Related information
This occurs using SDK 2.1.503 and 2.2.102.
This package is a transitive dependency of basically every other AspNetCore package so if others are able to reproduce this is going to cause headaches in a big way.  Not sure whether it's a problem of package or FSC.
.NET Core SDK (reflecting any global.json):
 Version:   2.2.102
 Commit:    96ff75a873

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /usr/share/dotnet/sdk/2.2.102/

Host (useful for support):
  Version: 2.2.1
  Commit:  878dd11e62

.NET Core SDKs installed:
  2.1.105 [/usr/share/dotnet/sdk]
  2.1.503 [/usr/share/dotnet/sdk]
  2.2.102 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

@NatElkins I haven't had a chance to try and reproduce this yet. Have you seen this with later SDKs?

Ah...I'm not sure.  I switched to using macOS, haven't used Ubuntu for a few months now.

Using the latest 2.2.x on Ubuntu I can't reproduce. I'll close the issue. Not sure what would have caused it before...

@cartermp Just to confirm, when doing the repro, did you peg to version 2.2.0 (which is not the latest version)?

I'll try and repro this today again on a Ubuntu machine.

This is with version 2.2.0 of the package. It's the latest version now.


Use conditional numpy dependencies for Python 2/3

As https://github.com/nmslib/nmslib/pull/443 to develop branch

Hi @jjjamie many thanks for the pull request and sorry for the delay.
Do I understand it correctly that it affects only building/installing NMSLIB from sources, i.e., binary prebuilt library won't be affected?

It's the other way round - for building/installing from sources, setup.py got run in the target environment (with a version of Python that matched where it would be run) and picked the correct version constraints. For the pre-built library, it made pypi's requires_dist constraints specify <1.17 globally (rather than just for Python 2)
 "requires_dist": [
      "pybind11 (>=2.2.3)",
      "psutil",
      "numpy (<1.17,>=1.10.0)"
    ],

(I was installing prebuilt binaries partly because I can't always rely on builds running where there's a C++ compiler and partly because building for Windows is tricky due to Windows C++ header problems: https://github.com/nmslib/nmslib/issues/399)


option for how many results to return

tower job list returns a long list of jobs, please create an option to modify how many jobs should be listed, for example tower job list -n 5.

If it helps, we do have the --all-pages flag, which will return a list with all records.
This issue comes down to adding a --page-size flag for the list commands.

thanks for the workaround, which works nicely with tail, but the flag --page-size will be more helpful on the long run.

This will go out in 3.1.8 soon


Subtitles language not available

I got the plug-in working properly (works quite well, way to go!) but I have an issue with the available subtitles languages.
I am Greek and Netflix was only recently made available in my country.
On my phone/tablet I can watch most of the movies/TV shows with Greek subtitles, but when I try to watch the same content in Kodi using this plugin, I only get English and Turkish (!) as available options.
Kodi and Netflix language settings are both set to English (but so is the relevant settings on my phone, so I don't think this is the cause of the problem).
I saw somewhere in the Netflix forums that it is advised to switch the Netflix language preference to one's native language in order to always get the this language available for subtitles, but Greek is not available as an option in Netflix languages setting.
Strangely, I have exactly the same problem in the Netflix app on my Samsung TV (quite old one, not Android)  so it might not be related to the plug-in.
Any help/ideas would be appreciated.

Read this: https://github.com/asciidisco/plugin.video.netflix/issues/95

I don't think this has anything to do with my problem, since Greek is not image-based and Kodi already has system wide support for greek.

@nemphys I haven't found an episode with Greek subtitles to test with, but I found Russian subtitles & they´re image bases (which makes me think that this may be the case with other languages containing cyrillic chars as well).
If you could do me a favour & start an episode with greek subtitles enabled in a web browser & have look at the developer tools -> network panel -> filter by "img" while it´s running?!

Jesus, you're right. For some reason, Netflix does serve greek subtitles as images (can't imagine why).
Does that mean I am out of luck?
Has there been any progress on making image subtitles work?
PS. Sorry for my initial insistence, but I still find it hard to believe that they would use images for subtitles.

No need to feel sorry, and I fear that you´re indeed out of luck for the time being.
This needs to be adressed in inputstream.adaptive as well as Kodi core.
To be honest, I´m not sure if there´s a chance that this will be implemented some time soon. If somebody would volunteer to get this into both codebases, then maybe...
Another option might be, to sniff what the official Netflix app is doing, maybe it requests subtitles in a more digestible format.
Sorry for the bad news...
Hope it´s okay to close this & to have further discussions here: https://github.com/asciidisco/plugin.video.netflix/issues/95


Mobile cards - Term Inconsistency - CP & BTP

Hello,
Tutorial link: https://developers.sap.com/tutorials/cp-mobile-cards-welcome.html
I have observed that there are inconsistencies in the tutorial with respect to the terms that we use in the tutorial.
The Cockpit has not been renamed to BTP yet, and as we are asking to go to BTP, but we actually navigate to CP which might create confusion. Also, still at some places in  the tutorial we are calling SAP CP only.
in this tutorial, Step 2.1: In your web browser, open the SAP BTP trial cockpit -> but we land at SAP Cloud Platform.
Still the BTP application itself says Cloud platform till now.
For example:
Here, the link of this screenshot navigates to page which says register on SAP BTP, and we are saying Cloud Platform here

Also, in tutorial 2, Step 2.1:

We have mentioned CP, so bit confusing again for the end users.
Please update the  tutorial accordingly.
We need to see how to mention it as the App is still not updated with latest rebranding.
Labels to be added:
Landscape Labels: all
Sprint Label: SCPTest-2102
Scenario Label: SCPTest-mcard
Prio: High-prio
Thanks,
Shubham Agarwal.

Hi @i353590,
Thank you for raising the issue.
@prilk Can we please update our tutorials to ensure the rebranding is consistent with the new name? Thank you.

@prilk Can you confirm that this was done?
In the meantime, closing the issue.


Drop unused config files

Modulus Pull Request
Description
Minor: drop the unused config.yaml and regression.json in the StormCast example
Checklist

[x] I am familiar with the Contributing Guidelines.
[x] New or existing tests cover these changes.
[x] The documentation is up to date with these changes.
[x] The CHANGELOG.md is up to date with these changes.
[ ] An issue is linked to this pull request.

Dependencies

/blossom-ci

/blossom-ci

/blossom-ci

/blossom-ci


[PyTorch] fused attention and cu_seqlens

Hi team,
we are currently adapting our training environment to use the fused attention functions. In one of our training setups, we work with batch size one and concaternate multiple documents along the sequence dimension (sbhd format). We set cu_seqlens_q and cu_seqlens_kv so that these documents cannot attend on each other. This is actually not a padding use case, because we always fill up the whole sequence and there is no packing and unpacking with pack_tensors() and unpack_tensors() required. With the flash attention backend this worked perfectly fine and produces the results that we intended. With the fused attention functions we get device side assertions for this input. Here is a small sample code:
import os
import torch

from transformer_engine.pytorch.attention import DotProductAttention, _attention_backends

seqlen, batch_size, heads, kv_channels = 2048, 1, 16, 64

q, k, v = [torch.randn(seqlen, batch_size, heads, kv_channels, dtype=torch.float16, device="cuda", requires_grad=True) for _ in range(3)]

cu_seqlens_q = cu_seqlens_kv = torch.tensor([0, 300, 1100, 2048], device="cuda", dtype=torch.int32)

attention_kernel = DotProductAttention(heads, kv_channels)

os.environ["NVTE_FUSED_ATTN"] = "0"
os.environ["NVTE_FLASH_ATTN"] = "1"
output_flash = attention_kernel(q, k, v, cu_seqlens_q=cu_seqlens_q, cu_seqlens_kv=cu_seqlens_kv)

os.environ["NVTE_FUSED_ATTN"] = "1"
os.environ["NVTE_FLASH_ATTN"] = "0"
_attention_backends["backend_selection_requires_update"] = True
output_fused = attention_kernel(q, k, v, cu_seqlens_q=cu_seqlens_q, cu_seqlens_kv=cu_seqlens_kv)

torch.testing.assert_close(output_fused, output_flash, atol=1e-2, rtol=1e-2)

Was the use case we have been working with ever intended? Or is there just some assertion missing that forbids to use cu_seqlens without setting a padding mode?

Hey Hey,
oh, okay, then my use case was simply not correct 🙈
thank you so much for your explanation and for extending the documentation. That was exactly the information that I was looking for!


Example for lowlevel details.

@teoliphant I'm busy with the low-level details and have a short question:
In some cases the low-level details only have one sane representation (example char):
https://github.com/blaze/libndtypes2/pull/4/files#diff-59b88cb255b2a9aa8e6cd54c8df58e6dR855
In other cases there are several equally sane representations (example string):
https://github.com/blaze/libndtypes2/pull/4/files#diff-b3009a0a6c65cc6dbccc4e83ed8272b4R484
So either one of these gets to use the precious keyword string, or we introduce dynd_string and cstring or string gets attributes:
string[size=16, align=8].
I'm a bit uncomfortable having low-level details specified by the user, but perhaps it is the best way.


HBASE-22684 The log rolling request maybe canceled immediately in Log…

…Roller due to a race

Seems the pre commit for PR is broken. Uploaded a patch on the jira to trigger pre commit build.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

0
reexec
154
Docker mode activated.

_ Prechecks _

+1
hbaseanti
0
Patch does not have any anti-patterns.

+1
@author
0
The patch does not contain any @author tags.

+1
test4tests
0
The patch appears to include 2 new or modified test files.

_ master Compile Tests _

+1
mvninstall
258
master passed

+1
compile
50
master passed

+1
checkstyle
70
master passed

+1
shadedjars
273
branch has no errors when building our shaded downstream artifacts.

+1
findbugs
210
master passed

+1
javadoc
31
master passed

_ Patch Compile Tests _

+1
mvninstall
243
the patch passed

+1
compile
54
the patch passed

+1
javac
54
the patch passed

+1
checkstyle
70
the patch passed

+1
whitespace
0
The patch has no whitespace issues.

+1
shadedjars
270
patch has no errors when building our shaded downstream artifacts.

+1
hadoopcheck
739
Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.

+1
findbugs
214
the patch passed

+1
javadoc
30
the patch passed

_ Other Tests _

-1
unit
14427
hbase-server in the patch failed.

+1
asflicense
35
The patch does not generate ASF License warnings.

17454

Reason
Tests

Failed junit tests
hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory

Subsystem
Report/Notes

Docker
Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-378/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/378

Optional Tests
dupname  asflicense  javac  javadoc  unit  findbugs  shadedjars  hadoopcheck  hbaseanti  checkstyle  compile

uname
Linux 41e63d6d014e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux

Build tool
maven

Personality
/testptch/patchprocess/precommit/personality/provided.sh

git revision
master / 74731c2a46

maven
version: Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z)

Default Java
1.8.0_181

findbugs
v3.1.11

unit
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-378/1/artifact/out/patch-unit-hbase-server.txt

Test Results
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-378/1/testReport/

Max. process+thread count
5406 (vs. ulimit of 10000)

modules
C: hbase-server U: hbase-server

Console output
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-378/1/console

Powered by
Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.


Add new domain : isetjb.rnu.tn

Thanks to add this domain isetjb.rnu.tn
You can visit out web site : https://www.isetjb.rnu.tn
(You added already https://isetn.rnu.tn/, ... : Other Institutes from different town in Tunisia)

@isetjbdev I have to close this request as a duplicate. The domain is already in the trusted list. You can start requesting the licenses immediately. Thank you.


Proper error response graffiti

When updating graffiti, return proper response if failing authentication. Now its unclear if request failed because of missing node flag or incorrect auth

I tried to update my graffiti, but it was grayed out :(


Remove Vale language linting

This commit removes Vale from govuk-developer-docs. Hopefully
we can reintroduce it one day in a more useful way.
Vale was added in November 2017 with little explanation, but with
good intentions:
https://github.com/alphagov/govuk-developer-docs/commit/b3dcb10783b238de3fbc0acc808da2647ee851f7
It was made part of the CI build in February 2018:
https://github.com/alphagov/govuk-developer-docs/commit/fd67db213786389798fc73d58f695f6c17f0ac42#diff-58231b16fdee45a03a4ee3cf94a9f2c3R42
But looking at the configs, it isn't possible to raise an error;
all this does is raise warnings, which no developer is likely to
see on CI.
For example, the 'Words' config has a level of 'error' (added in
the 2017 commit), which we've explicitly overridden in the .vale
file to only raise a warning.
It was subsequently removed in November 2019:
https://github.com/alphagov/govuk-developer-docs/commit/e39eab147116ef810c232d5fd274112261b46341#diff-58231b16fdee45a03a4ee3cf94a9f2c3
Note the commit message:

The only thing that the default library Jenkinsfile doesn't do is
vale linting. We're using a really
old
version of Vale anyway (the current is 1.7.1), so I'm not sure of its
value any more. If we want it, we can migrate it to be run on
Concourse, or make it a pre-commit hook, or somesuch.

The pre-commit linting feature is an opt-in-only feature, requiring
devs to read the README (!) and install some additional dependencies.
It's not clear how many devs do this, if any; moreover, it isn't
compatible with the common workflow of editing docs directly in
the GitHub UI. There's little to no value to keeping this as an
opt-in, local-only thing.
I experimented with what it might take to run Vale on CI:
https://github.com/alphagov/govuk-developer-docs/pull/2702
Unfortunately there's no Ruby gem for running Vale, so we couldn't
build it into the standard Rake default task. The best solution
I could see was to use the dedicated GitHub Action for it, which
worked after a bit of experimentation. But the same point around
it only giving warnings, not errors, limits its usefulness.
(Moreover, I could not see any warnings in the CI output, even
when I tried to deliberately trigger them. Only raising errors
appeared to do anything).
Tangentially related Trello card: https://trello.com/c/XSgGyudd/181-fix-url-markdown-parsing
Tangentially related PR (linting markdown): https://github.com/alphagov/govuk-developer-docs/pull/2703

In Homebrew we implemented it like this. But the difference there is that we do actually define a whole bunch of rules and have a prose style guide.
We don't here, and our rules here have languished somewhat. It was a good intention, but your conclusion that this can be removed is sensible, given it's currently providing no real value.

I've just discovered a [style guide in the dev docs](https://docs.publishing.service.gov.uk/manual/docs-style-guide.html, though it hasn't had a meaningful content change since October 2017, and I don't know how rigidly we want to stick to it. We could probably run Vale via a GitHub Action and have it tackle this much smaller subset of rules, which hopefully wouldn't involve too much doc fixing.
I might have a play with that today. 🤔

I didn't find time to get around to it...!
@issyl0 shall we approve this and look at adding Vale linting later?


Adopt CadlRanch model-visibility spec

Fixes https://github.com/microsoft/typespec/issues/4001

API change check
API changes are not detected in this pull request.


TypedDict support

Feature Request
It would be great if pydantic supported TypedDict.

OS: MacOS
Python version import sys; print(sys.version): 3.7.4
Pydantic version import pydantic; print(pydantic.VERSION): 0.32.1

from pydantic import BaseModel
from mypy_extensions import TypedDict

class Data(TypedDict):
    a: int

class User(BaseModel):
    data: Data

if __name__ == '__main__':
    external_data = {
        'data': {
            'a': 'invalid',
        }
    }
    # should raise exception
    user = User(**external_data)

This is very close to dataclasses or pydantic's own models. What is the usecase where TypedDict would be preferable to dataclasses or models?

My primary use case is integrating pydantic into existing source code (which uses marshmallow currently) where plain dictionaries are used. Changing all the source code to using ModelBase instead of plain dicts is infeasible. I can obviously define a pydantic's model and then call .dict(), but it will most definitely incur a significant performance penalty (the project processes event stream and does hundreds or even thousands of validations per second).

I see.
Most of the overheads of parsing data are not in calling .dict(), it's in the actual parsing and (to a lesser extent) building the model of what what the data should be like.
As long as you're creating the models once and then just calling .dict() on them, the performance won't be very different from if we implemented support for TypedDict.
In raw models will be slightly faster, with dataclasses we create a hidden model to do the actual validation, we would probably have to do the same for TypedDict. So you would have normal pydantic performance + some overheads.
By the way dict(model) will be slightly faster than model.dict() since it doesn't have to worry about all the exclude/include logic.

Given integration requirements with non-pydantic packages/apis (where you have to load to / dump from a pydantic model), I could also imagine TypedDict having some static type-checking benefits over subclasses of BaseModel (at least, without the as-yet-unreleased mypy plugin), since replacing with BaseModel would drop the static checking of keyword arguments.

I looked into this a little, and it looks like it may be difficult to determine at runtime whether a given type is actually a TypedDict. So far, the best check I can find is:
def is_typed_dict_type(type_: AnyType) -> bool:
    return lenient_issubclass(type_, dict) and getattr(type_, '__annotations__', None)

@roganov If this is of critical importance to you, I think the same approach used to get validation for Literal types might work here. In particular, you'd need to write and incorporate TypedDict analogs of make_literal_validator and is_literal_type in the appropriate places. I don't currently have the time to implement this myself, but would review a pull request for it. (Though I would also understand if @samuelcolvin wanted to veto in favor of limiting scope creep of supported typing_extensions types.)
If you want to try implementing it yourself, here's a start, though I expect it may require some tweaks before it fully integrates into the field building process:
from typing import Any, Callable

from typing_extensions import TypedDict

from pydantic import BaseModel, AnyType

class A(TypedDict):
    x: int

def make_typed_dict_validator(type_: Any) -> Callable[[Any], Any]:
    class TypedDictModel(BaseModel):
        __annotations__ = type_.__annotations__

    def typed_dict_validator(v: Any) -> Any:
        return TypedDictModel(**v).dict()

    return typed_dict_validator

def is_typed_dict_type(type_: AnyType) -> bool:
    return issubclass(type_, dict) and getattr(type_, '__annotations__', None)

assert not is_typed_dict_type(BaseModel)
assert not is_typed_dict_type(dict)
assert not is_typed_dict_type(TypedDict)
assert is_typed_dict_type(A)

validator = make_typed_dict_validator(A)

print(validator({"x": 1}))
# {'x': 1}
print(validator({"x": "x"}))
"""
pydantic.error_wrappers.ValidationError: 1 validation error for TypedDictModel
x
  value is not a valid integer (type=type_error.integer)
"""

Alternatively, you may be able to come up with a way to use code like the above to produce a custom type that you can use to validate your TypedDicts.

I can obviously define a pydantic's model and then call .dict(), but it will most definitely incur a significant performance penalty (the project processes event stream and does hundreds or even thousands of validations per second).

The nature of pydantic validation is that it is done via parsing -- I don't think that you'll be able to use pydantic to validate a TypedDict without essentially having it parse the dict as a model.
That said, thanks to cythonization, slots, and other performance-oriented design choices, using pydantic may not be much slower than hand-crafted checks anyway.

Thanks @dmontagu I'll try implementing this hopefully next week.

This feature is now present in Python 3.8 (https://docs.python.org/3/library/typing.html#typing.TypedDict)

Sometimes python amazes me.

NamedTuple
dataclass
TypedDict

It doesn't really fit with the zen of python:

There should be one-- and preferably only one --obvious way to do it.

Hi guys, it is a very demanded feature :)
Do you have any plans for its implementation?

Happy to accept a PR to implement it.
I don't think in the short term I'll be building it myself.

Sorry for spamming, but maybe for someone, it would be useful as a quick solution (like in my case).
Let say we have the next data structure:
from typing import Dict, List, Optional, TypedDict
from uuid import UUID

class SessionUser(TypedDict):
    id: int
    name: str
    uuid: UUID
    email: str
    username: str

class SessionToken(TypedDict):
    user: SessionUser
    session_id: SessionID

then it could be parsed as:
from typing import _TypedDictMeta as TypedDictMeta

types: dict = {}

def parse_dict(typed_dict: TypedDictMeta) -> Type[BaseModel]:
    annotations = {}
    for name, field in typed_dict.__annotations__.items():
        if isinstance(field, TypedDictMeta):
            annotations[name] = (parse_dict(field), ...)
        else:
            default_value = getattr(typed_dict, name, ...)
            annotations[name] = (field, default_value)

    return create_model(typed_dict.__name__, **annotations)

def as_typed_dict(
        json_dict: Dict[str, Any],
        typed_dict: TypedDictMeta,
) -> Dict[str, Any]:
    model = types.get(typed_dict)
    if not model:
        model = types[typed_dict] = parse_dict(typed_dict)

    return model(**json_dict).dict()

as_typed_dict({...}, SessionToken)

It would be super cool if one could get a TypedDict from a pydantic class.
Use case: I have code which uses dictionaries a lot. At some point I will convert them to the Pydantic class, but it would help for refactoring to have an in-between state where I use a TypedDict derived from a pydantic class.

I also have a similar use case. I use celery for managing tasks. The biggest challenge with that is serialisation between tasks (since pickle isn't feasible) - to keep everything JSON serialisable I like to use TypedDict but often I'd like to convert those into a pydantic model instead. Custom classes is advantageous over dict in many cases obviously.
My challenge is to due this in a "static type safe" way as well as performant. I feel like there is a lot to gain from pydantic for this use case but I have cracked the code completely yet I feel.

Adding basic support for TypedDict is easy and doesn't cost much. Seeing the large number of up votes, I reckon it can be added. I opened a PR for this. Feedback more than welcome


Don't require lvl in buffs editing

In fact, buffs' parameters do not depend on levels.  This PR provides additional convenience overloads.

You cannot specify levels of a buff and it only uses it's level 1's parameters

Are you sure?
Otherwise looks good 👍

Are you sure?

Yea, I'm only setting 1'st level's fields and it works for my spells on any level.
And here it's like it looks in the World Editor:
https://imgur.com/OtZUiBA
Nothing about the levels.

Okay thanks 👍


x/crypto/ssh: support OpenSSH encoded ECDSA keys in ParsePrivateKey

What version of Go are you using (go version)?

go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/fransebas/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/fransebas/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3p/21s_plx55xq8zv2_hvnj93dc0000gn/T/go-build752561798=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I'm using the default key that my raspberry pi uses when I connect to it using ssh and it's ecdsa-sha2-nistp256 type (I could share the key if necessary, it's a dummy machine ). But I get an error when using it. I don't know if I should be using another method to parse the key or if it's not supported (I assumed it was because of constants you have declared).

// These constants represent the algorithm names for key types supported by this
// package.
const (
	KeyAlgoRSA        = "ssh-rsa"
	KeyAlgoDSA        = "ssh-dss"
	KeyAlgoECDSA256   = "ecdsa-sha2-nistp256"
	KeyAlgoSKECDSA256 =<EMAIL_ADDRESS>	KeyAlgoECDSA384   = "ecdsa-sha2-nistp384"
	KeyAlgoECDSA521   = "ecdsa-sha2-nistp521"
	KeyAlgoED25519    = "ssh-ed25519"
	KeyAlgoSKED25519  =<EMAIL_ADDRESS>)

This is the code that I use:

private, err := ssh.ParsePrivateKey(privateBytes)
if err != nil {
	log.Fatal("Failed to parse private key: ", err)
}

I debbuged the code to see where it ends and it's on x/crypto/keys.go

// Implemented based on the documentation at
// https://github.com/openssh/openssh-portable/blob/master/PROTOCOL.key
func parseOpenSSHPrivateKey(key []byte) (crypto.PrivateKey, error) {
	const magic = "openssh-key-v1\x00"
	if len(key) < len(magic) || string(key[:len(magic)]) != magic {
		return nil, errors.New("ssh: invalid openssh private key format")
	}
	remaining := key[len(magic):]

	var w struct {
		CipherName   string
		KdfName      string
		KdfOpts      string
		NumKeys      uint32
		PubKey       []byte
		PrivKeyBlock []byte
	}

	if err := Unmarshal(remaining, &w); err != nil {
		return nil, err
	}

	if w.KdfName != "none" || w.CipherName != "none" {
		return nil, errors.New("ssh: cannot decode encrypted private keys")
	}

	pk1 := struct {
		Check1  uint32
		Check2  uint32
		Keytype string
		Rest    []byte `ssh:"rest"`
	}{}

	if err := Unmarshal(w.PrivKeyBlock, &pk1); err != nil {
		return nil, err
	}

	if pk1.Check1 != pk1.Check2 {
		return nil, errors.New("ssh: checkint mismatch")
	}

	// we only handle ed25519 and rsa keys currently
	switch pk1.Keytype {
	case KeyAlgoRSA:
		// https://github.com/openssh/openssh-portable/blob/master/sshkey.c#L2760-L2773
		key := struct {
			N       *big.Int
			E       *big.Int
			D       *big.Int
			Iqmp    *big.Int
			P       *big.Int
			Q       *big.Int
			Comment string
			Pad     []byte `ssh:"rest"`
		}{}

		if err := Unmarshal(pk1.Rest, &key); err != nil {
			return nil, err
		}

		for i, b := range key.Pad {
			if int(b) != i+1 {
				return nil, errors.New("ssh: padding not as expected")
			}
		}

		pk := &rsa.PrivateKey{
			PublicKey: rsa.PublicKey{
				N: key.N,
				E: int(key.E.Int64()),
			},
			D:      key.D,
			Primes: []*big.Int{key.P, key.Q},
		}

		if err := pk.Validate(); err != nil {
			return nil, err
		}

		pk.Precompute()

		return pk, nil
	case KeyAlgoED25519:
		key := struct {
			Pub     []byte
			Priv    []byte
			Comment string
			Pad     []byte `ssh:"rest"`
		}{}

		if err := Unmarshal(pk1.Rest, &key); err != nil {
			return nil, err
		}

		if len(key.Priv) != ed25519.PrivateKeySize {
			return nil, errors.New("ssh: private key unexpected length")
		}

		for i, b := range key.Pad {
			if int(b) != i+1 {
				return nil, errors.New("ssh: padding not as expected")
			}
		}

		pk := ed25519.PrivateKey(make([]byte, ed25519.PrivateKeySize))
		copy(pk, key.Priv)
		return &pk, nil
	default:
		return nil, errors.New("ssh: unhandled key type")
	}
}

If it's not supported, what do you recommend? Cand I use another library to parse the key (which I think it's not possible). Or I can help you incorporate that type 😁 if it's not too messy (I worked for a company implementing some Bitcoin BIPs that used the ECDSA so I have some experience with this).
What did you expect to see?
What did you see instead?
ssh: unhandled key type

Indeed, we probably should support that. I think it wasn't an issue in the past because OpenSSH used the PKIX format until recently, which we do support.