tensorflow.proto and respective changes

TL;DR
proto files and generated ones for TensorFlow Flyte plugin
Type

[ ] Bug Fix
[ ] Feature
[x] Plugin

Are all requirements met?

[ ] Code completed
[ ] Smoke tested
[x] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
N/A
Follow-up issue
N/A

/cc @kumare3

@pingsutw Thank you for the PR, just a couple comments. Otherwise this looks good. Ping me and we can merge it soon

@kumare3 Thanks for the review.
I just updated my PR.
update include:

rename protobuf
ps -> ps_replicas,
chief -> chief_replicas
add a comment for ps

Lgtm, qq; do the workers etc need mem/cpu?

I think it should be added, but I found mem/cpu also don't include in PyTorch proto.
In order to make it consistency, I didn't add them. Maybe we could add it later if I run into some problems in other PRs.
https://github.com/lyft/flyteidl/blob/master/protos/flyteidl/plugins/pytorch.proto

hey @pingsutw mind incrementing the version here and here as part of this change?

@katrogan Thanks for the reminder.
The new version is 0.17.35?

I've updated the version.


Example unsigned token that bypasses verification

you need to specify the algorithm in the options hash whenever you call JWT.decode to ensure that an attacker cannot bypass the algorithm verification step

Does anyone have an example of such a token?
I thought our system was exposed but I've been unable to write a failing unit test. For example:
malicious_token = JWT.encode({hi: 1}, nil, 'none')
# => "eyJhbGciOiJub25lIn0.eyJoaSI6MX0."

JWT.decode malicious_token, 'secret'
# => JWT::DecodeError: Not enough or too many segments

I cannot get JWT.decode(t, secret) to accept an unsigned token.

@brianlow eyJhbGciOiJub25lIn0.eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ. is an example unsigned token (taken from https://tools.ietf.org/html/rfc7519#section-6.1)
For your decode call, you probably want to call something like JWT.decode(t, secret, true, { algorithms: %w(list of algorithms you are ok with)})
I believe that you've hit a bug with the encoder... In the case that the algorithm is none the encoder is returning a token that is not syntactically valid (it is missing the header declaring the algorithm). This bug was introduced by a refactor in the encoder in https://github.com/jwt/ruby-jwt/commit/43a08a40593c1e63b6e9684ef57e8b5ed02a0b25

Actually, I take it all back... there isn't a bug with the encoder... (I blame not enough coffee yet).
The "issue" here is that the decode call is trying to verify a signature on the token and a token with algorithm none has no signature to verify. The error message just isn't clear...
By default, JWT.decode sets the optional parameter verify to true, which is what you want in most cases. This will cause any token without a signature (e.g. with algorithm none) to fail decoding.
A "nicer" behaviour would be to raise an JWT::IncorrectAlgorithm error in this case (where verify is true - still check the claims - and the algorithm none is not specifically allowed)

That sound reasonable.
So it feels like using JWT.decode(token, password) is not vulnerable to attack by setting the token algorithm to none because it will raise an exception (though perhaps not an ideal one).
Is that fair?

Correct that it is not vulnerable to an attack whereby the attack uses the 'none' algorithm.
It may be vulnerable to other attacks. To reduce the attack surface, you should specify which algorithms you will allow the library to use when decoding. Not doing so may leave your application vulnerable to attacks such as:

providing a token signed using an algorithm that has weaknesses making it appear valid
using an asymmetric public key as a symmetric key to sign using a symmetric algorithm

If you are issuing the tokens, you should know what algorithm(s) you're using to sign the token. If you're not issuing the token, the issuer should be able to tell you what algorithms they use.
Hence advice to use: JWT.decode(token, password, true, { algorithms: %w(list of algorithms you are ok with)}) instead of just token and password.

@danleyden Thanks for the explanation!

سحب

Cash

سحب

ر

RDJ-D

سحب


Question. Is there an ESM version.

I would like to use it with TypeScript. Is there ESM version (or a way some other way to use it with TS)

There is currently no ESM version.
Does the following work for you?
import sqlite from "node-sqlite3-wasm";
const { Database } = sqlite;

const db = new Database();

Yes it works. It works with .mjs and .ts files.
Actualy I have a problem with Deno.
There is simple example:
// test.ts
import sqlite from "npm:node-sqlite3-wasm";

const { Database } = sqlite;

 function dbInsertRow(dbFile: string, table: string) {
  const db = new Database(dbFile);
  let query = 'CREATE TABLE IF NOT EXISTS test (v1 TEXT,v2 TEXT)';
  db.run(query);
  query = 'INSERT INTO test  (v1,v2)   VALUES (?,?);';
  db.run(query, [1,2]);
  db.close();
}

dbInsertRow('test.db','test')

It creates test.db  and table  'test' within it. But hen it trows an error:
error: Uncaught SQLite3Error: file is not a database
The program is run by:
deno run -A test.ts

node-sqlite3-wasm is a Node.js package not a Deno package, so I wouldn't expect full compatability.
But did you try this already<EMAIL_ADDRESS>?

Yes.
Here is a code:
Import { createRequire } from<EMAIL_ADDRESS>const require = createRequire(import.meta.url);
const sqlite=require("node-sqlite3-wasm");

const { Database } = sqlite;

 function dbInsertRow(dbFile: string, table: string) {
  const db = new Database(dbFile);
  let query = 'CREATE TABLE IF NOT EXISTS test (v1 TEXT,v2 TEXT)';
  db.run(query);
  query = 'INSERT INTO test  (v1,v2)   VALUES (?,?);';
  db.run(query, [1,2]);
  db.close();
}

dbInsertRow('test.db','test')

With the same result.
I will contact Deno team about this.
Thanks.


docs(Form): add an example of accessible Form.Select

As discussed in #3149, this PR adds an example of how to accomplish an accessible select inside a form.

Released in<EMAIL_ADDRESS>


Support for sphinx style docstrings

Is your feature request related to a problem? Please describe.
In sphinx, a special comment-based format is available for creating docstrings for attributes:
#: a docstring for a
a = 1

#: a docstring for b
#: that spans multiple lines
b

c    #: a docstring on the same line

Describe the solution you'd like
What do you think of attempting to parse these docstrings? It seems like currently griffe looks at the next AST node, to see if it can add a docstring for attributes (code here).
I wonder if we could extend it to do the following:

if next node is a string, use that (current approach). otherwise,
if de-dented line above starts with #:, parse docstring from that line upwards (until no more #:).
otherwise, if next node is not on the same line, look on the same line as node for #:

I'm happy to take a crack at it / incorporate any feedback!
Describe alternatives you've considered
Changing docstrings to use the """ format:
a
"""a docstring for a"""

b
"""
a docstring for b
that spans multiple lines
"""

This is nice, in that it keeps things in the python AST. The downside is it requires older libraries to update their approach, and can read a bit funky.

Unfortunately, this will be hard to implement, and IMO is not worth it, for the following reasons:

we indeed use ast to parse source code
ast does not keep comments
so it means we'd have to also tokenize the code with tokenize, greatly reducing perfs (at least when enabled)
even with tokens, it will be hard to identify on which object to attach the docstrings found in comments
these comments are a Sphinx-thing, not a standard (see https://peps.python.org/pep-0257/#what-is-a-docstring, which points anyway to the (rejected) https://peps.python.org/pep-0258/#attribute-docstrings)

Even if someone was willing to work on this and send a PR, I myself am not willing to review such a big feature, even less maintain it :confused:

Thanks for your quick response, and background on the issue. It's clear you've put a lot of thought into this scenario!
I had assumed LinesCollection holds the lines for the source code, which would allow fast checks / extraction (e.g. does this line start with #:?). Is it used in a different way?
I was hoping that if..

the AST nodes tell us where they start and stop (e.g. line number)
LinesCollection holds lines of source code

Simple extraction of these comments might be relatively simple. But I'm not too familiar with everything that's available at the time docstrings are extracted by the agent, so totally understand if it is a much bigger lift.

Oh. OH.
This is a moment where I realize again that external points of view are so valuable.
You're completely right, the lines collection could make this way easier. I didn't even think of that.
However getting inline comments would still be not trivial without a tokenizer:
do(thing, "  #: this is not a comment!")

...but we can start with a simple approach that solves 99% of the cases.

Ahhh, teamwork!
I think that--if do(...) is the node we're interested in, then if we used the three rules in my initial post (which I edited a bit for clarity; thoroughness..!), we wouldn't extract a comment from this code:
do(thing, "  #: this is not a comment!")

Since the do(...) node will tell you it ends at ) in the source.
The rightward search rule would look to the right of that node...
do(thing, "  #: this is not a comment!") #: this is a comment

More cases
Upward search (early termination)
a = 1  #: I am a comment
b = 2

When searching for docs for b, we wouldn't look upward because the previous node is on the line above.
Similarly, it wouldn't search upward for b here...
#: some comment for a?
a = 1; b = 2

Rightward search (multiline node)
b = int(
    "1"
) #: some comment

Rightward search (multiple nodes per line)
a = 1; b = 2;     #: a comment for b

My example was not good indeed.
Maybe this one is better:
a = "  #: not a comment"

And you're right again, we can probably use the ending column info from the node to know where to look at for a comment :+1:

TL;DR:

OP's line link seems outdated due to refactoring
Could this be implemented in phases to ship faster?
Is OP still interested in this?

1. OP's Link is Outdated
TL;DR: The relevant lines now seem to be here
There have been a lot of changes to visitor.py since this ticket was created. The link in OP currently points to a docstring in an augmented assignment (+=, etc) handler.
2. Could we implement this in phases to ship faster?
TL;DR: Could we start by supporting multi-line only?

Partial Sphinx docstring support would be better than nothing
Documenting limitations could compensate for imperfect support

Why Multi-Line?

It'd make a single line above an attribute the simplest case (n=1)
Above-the-attribute is the only style I've seen recently in public code bases

N=2, but there seem to be 200+ usages in each:

Library GitHub Page
TL;DR

pyglet
Pure Python ctypes multimedia & windowing

Arcade
Game framework built on pyglet

The other use cases could be added afterward if there's demand / data to support them.
3. Is OP still interested?
TL;DR: @machow Are you still interested in this ticket?
If not, I want to do the following before promising anything:

Read the code more thoroughly
Wait for other Arcade contributors to respond to my ping about this ticket in Discord

Bonus: Arcade's Use Case
TL;DR: Fix tech debt in our API doc generation
Arcade was once much smaller. At the time, using regex to parse code and generate Sphinx autodoc directives worked okay. As of our 3.0 dev previews, it's causing problems.

This is now supported thanks to the Griffe Sphinx extension, available to sponsors :slightly_smiling_face: See https://mkdocstrings.github.io/griffe-sphinx/!


Add AWS VPC CNINode, SecurityGroupPolicy CRDs

Add CRD definitions from AWS amazon-vpc-resource-controller-k8s controller

https://github.com/aws/amazon-vpc-resource-controller-k8s/tree/e0fa4681f2c00d062d465f79003c9881ae6f85c6/config/crd/bases


Cannot go to definition

Issue Type: Bug
When I try to go to the definition of a react component from the import nothing happens. If I try to go there from my actual usage of the component it only will go up to the import. However, I have no errors displayed that would indicate that the import was not working.
I tried refactoring the import path to be relative instead of drilling down from src/ and then I was able to jump to the definition. Using the full path does not seem to cause any build errors but just makes vscode not be able to follow the links. Is there a solution to this? I do not like being forced to use relative links as they are not as easy to read and it can cause issues when you need to refactor your project structure.
VS Code version: Code 1.33.1 (51b0b28134d51361cf996d2f0a1c698247aeabd8, 2019-04-11T08:22:55.268Z)
OS version: Darwin x64 18.5.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz (8 x 2700)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (0.02GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (5)

Extension
Author (truncated)
Version

vscode-eslint
dba
1.8.2

gitlens
eam
9.6.3

vsliveshare
ms-
1.0.67

indent-rainbow
ode
7.4.0

vscode-import-cost
wix
2.12.0

Hello !
Since I've upgraded to 1.33, the same problem appears with C++ source code as well (using Microsoft add-on). I guess this is a regression in the Visual Studio Code framework where IntelliSense plugs in more than in the language server itself.
Reproduction scenario
I've noticed that it is more likely to happen when I type in wrong code (ex: containing syntax errors) and triggers the auto-completion. After this, even when the syntax gets fixed (proven by a successful compilation with cl.exe), the Go to definition contextual entry does not work any more.
Possible cause
What I suspect is a misalignment between the clicked-on token and the token being analysed by the contextual menu. I am thinking this way because I noticed that while some token definitions cannot be retrieved (or the wrong file source gets opened, like the HPP instead of the CPP) the definition of tokens before the faulty one gets resolved correctly.
Thanks for your support,
Chucky

@Swazimodo Can you please share more information about your project and the file you are trying to run go to definition in, or the project itself if possible?

It is a react project and it will not f12 to the definition of other components in the project. It will however find the definition of imported components out of the node_modules folder.
This would not allow you to go to the definition automatically:
import {thing} from 'src/js/widgets/employees/container';
These components would allow you to jump to the definition:
import {Route, Switch, Redirect} from 'react-router-dom';
import {thing} from '../../widgets/employees/container';
I'm not exactly sure what type of information your looking for so please let me know if I can give you more details. This is a closed source repo so I cannot give you the project in question. However, this is a problem that I have seen on previous projects but they have always just used relative references to avoid the issue.

Does your project have a jsconfig.json file. If not, try creating one and setting the baseUrl option:
{
    "compilerOptions": {
        "module": "commonjs",
        "target": "es2016",
        "jsx": "preserve",
        "baseUrl": "."
    },
    "exclude": [
        "node_modules",
        "**/node_modules/*"
    ],
    "include": [
        "./src/**/*.js"
    ]
}

baseUrl tells vscode how to resolve non-relative paths

This solved the problem once I added support for .jsx. Thanks for pointing me in the right direction!
```

"include": [
"./src//*.js",
"./src//*.jsx"
]


Material Theme isn't working on vscode 1.16

Hello guys,
After update my vscode from 1.15 to 1.16, this extension is not changing vscode window interface.
I've tried to change accent color and color variant, nothing was changed, still on default vscode theme.
Also, I unistalled vscode entirely (yes I love this extension) to try to solve this problem.
I don't know how to provide more details, the only clue is, I've tried other themes and still in the same problem.

I just uodated the readme. You need first to acrivate the theme like other themes.

Il giorno 11 set 2017, alle ore 20:31, Jan Cássio<EMAIL_ADDRESS>ha scritto:
Hello guys,
After update my vscode from 1.15 to 1.16, this extension is not changing vscode window interface.
I've tried to change accent color and color variant, nothing was changed, still on default vscode theme.
Also, I unistalled vscode entirely (yes I love this extension) to try to solve this problem.
I don't know how to provide more details, the only clue is, I've tried other themes and still in the same problem.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


3 New Links

I added 3 new links but I suck at putting things in alphabetical order

no worries, alphabetical order is not needed will get sorted correctly automatically

nvm remove the third line completly.
discord-free.com is already existent


Kitti-based Code

Hi, thanks for your great work!
Do you have any plan to release kitti-based code?

Thanks for your patience! The code is available!


Fix join multiple

WIP PR to fix #10. I think once I've wrapped my head around the JoinMultiple impl, I've a feeling it's some trivial fix that's needed. @danieldg if you've just 15 mins and can take quick look, there is a good chance you'll be able to spot the issue much quicker than me.

21f1ec7 seems to fix the simple testcase at least. However, the test case only contains readily available streams. I'll test against my zbus work as well..

Yeah, as far as I could test this, it works. We can always change/improve things if needed, later.


Protractor waitforAngular() waits indefinitely and timesout

Even though the application is angular, when using the protractor function waitForAngular() it waits indefintely.
First i am getting the error "A Jasmine spec timed out. Resetting the WebDriver Control Flow." and then It waits for allScriptsTimeout and then stops. I could see that the element is available in the page, but it is just idle does not do anything. What is the reason, how to fix this?
below is the code:
    element(by.css('[ng-click="gencn(tokForm)"]')).click();
    browser.waitForAngular();

    var paymentuipageelement = element(by.id('method-title'));
    expect(paymentuipageelement.getText()).toEqual('Payment Methods');

Hi there,
Please follow the Issue Template when opening new issues. Otherwise it's unclear what the issue is and how we can actually help.
Please update your issue with the information from the template; providing a small reproducible example will also help us help you. Thanks! 😄

Please see http://www.protractortest.org/#/timeouts for general information on timeouts. Because of the mousemove behavior, I would guess that your page is waiting for a mouse event and then cancelling a $timeout. As Nick said, it's impossible for us to say anything else without a reproducible example. I'm going to close this to keep our issue tracker clean - please open up a new issue if you can create a reproducible example we can run. Thanks!

In case of the problem still there,
I got the same situation while I am using protractor with the angular page. Whenever I load a page and use the browser.waitForAngular() the page used to load forever. In our case fusion charts is the culprit. Since fusion charts are not the angular components whenever waiting for the angular page to load because of the fusion charts it always timed out. The best way to debug this problem is to comment out different sections one at a time and test and see the where you are getting the error.

Not sure if this will be helpful to anyone else, but I solved this problem by wrapping browser.get calls like so:
browser.waitForAngularEnabled(true)
await browser.get('http://localhost:8100')
browser.waitForAngularEnabled(false)

@edenworky no idea why your solution works but thanks! Been looking for a solution for hours.

Why ist there still a workaround as @edenworky suggest needed, the issue is closed, and no one explains why ?

I think the best option is to look at the http://www.protractortest.org/#/timeouts
In particular, your issue is more than likely some async code that runs inside Zone that is long-lived. The section of code dealing with running these async requests outside of Anguar.
this.ngZone.runOutsideAngular(() => {
setTimeout(() => {
// Changes here will not propagate into your view.
this.ngZone.run(() => {
// Run inside the ngZone to trigger change detection.
});
}, REALLY_LONG_DELAY);
});

I have added the defaultTimeoutInterval: 100000, now i am only getting "Timed out waiting for Protractor to synchronize with the page".
But surprisingly, the script is executing fine if i move the mouse over the ui (just a slight move is enough). I am not getting any timeout error and working good. But if i do not move my mouse, then it shows the above error. What is the reason for this? How to fix this?

R=
I think that in this case is possible that the button or the workflow is not visible unless you do the scroll, something happened to me and I could solve it using: await browser.actions().mouseMove(the-element).perform(); I hope it helps you, in case you don't test a specific element you could also use: await browser.executeScript('window.scrollTo(500,500)');

I have commonJS dependencies. If I don't call browser.waitForAngular(false); tests do not execute. It just waits ALWAYS.
If I set browser.waitForAngular(false);, some tests do not properly wait for the results of HTTP requests. I try to make it wait by using await and async statements but again SOMETIMES it doesn't wait. I couldn't really understand. It would be great if anybody could solve this issue.

I just found the reason. I'm using cytoscape.js. It causes lots of unnecessary change detections. So simply I used runOutsideAngular
this.ngZone.runOutsideAngular(() => {
      this.cy = cytoscape({ /* options here */);
});

I just found the reason. I'm using cytoscape.js. It causes lots of unnecessary change detections. So simply I used runOutsideAngular
this.ngZone.runOutsideAngular(() => {
      this.cy = cytoscape({ /* options here */);
});


Is there a way to see existing Assistants in ChatGPT?

I was hoping that by adding my API key I would be able to see list of existing Assistants, do you know if this is possible?

Currently it only shows the assistants created from this app, but we can look at providing an option for the same

Hi @Anil-matcha. I forked the project to have a play around. I got a list outputting by...
Creating an API route to get the list
const list = await openai.beta.assistants.list();
return NextResponse.json(list.data);

Then fetching that API route on the home page
const listResponse = await fetch("/api/openai/list");
const list = await listResponse.json();

@Anil-matcha Please go ahead and create a pull request and I will merge the changes


Update remaining Predicate tests to use the #Predicate macro

This updates the last of the predicate tests to use #Predicate instead of manual construction and removes the windows restrictions for usage of the macro

@swift-ci please test

Looks like the usage of the macro triggered a compiler crash. That crash should be resolved by https://github.com/apple/swift-syntax/pull/2312 (since similar crashes were successfully resolved by a similar change for SwiftPM plugins) so I'll hang onto this until a snapshot with the swift-syntax fix lands

We've determined that the Linux CI failure is indeed due to this swift-syntax issue, and is affecting the main branch already (i.e. it is not directly caused by this PR). Seeing that macOS passed, I'm merging this to update our tests with the known linux failure already happening on main.


Training Error

I am trying to train on NIGHTS dataset, since exact versions are not mentioned in requirements.txt I am using the latest available version.
It would be helpful to know what exact versions you are using for training.
Traceback (most recent call last):
  File "/mnt/raid0/nitin/dreamsim/training/train.py", line 290, in <module>
    run(args, device)
  File "/mnt/raid0/nitin/dreamsim/training/train.py", line 276, in run
    model = LightningPerceptualModel(device=device, train_data_len=len(train_dataset), **vars(args))
  File "/mnt/raid0/nitin/dreamsim/training/train.py", line 102, in __init__
    self.__prep_lora_model()
  File "/mnt/raid0/nitin/dreamsim/training/train.py", line 213, in __prep_lora_model
    self.perceptual_model = get_peft_model(self.perceptual_model, config)
  File "/usr/local/lib/python3.10/dist-packages/peft/mapping.py", line 136, in get_peft_model
    model_config = model.config.to_dict()
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1728, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'PerceptualModel' object has no attribute 'config'

Which dreamsim version are you using ?
Please check whether you are giving  correct config file path in the argument.
You should have this code at line 212 : config = LoraConfig(
r=self.lora_r,
lora_alpha=self.lora_alpha,
lora_dropout=self.lora_dropout,
bias='none',
target_modules=['qkv']
)

I found it to be dreamsim==0.1.3 probably from a previous installation.
Just found out that pypi has dreamsim==0.2.1 will try again.

Thanks, I got the training working! Hopefully if everything works out, I will be able to put a tutorial on YT regarding training with custom dataset.

Great to hear that . Best of luck.


[netdata] cloud daemon

Hello,
the netdata plugin is compiled and configured without cloud ability and thus is not claimable.
Is this just a configuration adjustment or are there packages missing to enable cloud connection?

I'd also be very interested to add my OPNsense box to my netdata cloud monitoring dashboard, preferably just by pasting my claim token and war room UUIDs into the configuration page.
See: https://learn.netdata.cloud/docs/agent/claim

I also tried to claim netdata into the cloud:
This agent was built with --disable-cloud and cannot be claimed
Is there any progress?

please re-open this issue, as no maintainer addressed it yet.

Not how it works. Free for all, reopen upon contribution.


Win 3.6 whel links.

Still need to sync with the website.

Oh, so they added 3.6 support on appveyor already, nice!


perf: Make ResetBatch accumulate a minimum batch size

ResetBatch is an optimization that is used for genesis, for not accumulating huge batches for node writes.
While a performance win, right now it goes to far, with having batch writes of one. Having some minimum size greatly reduces time here.
Here is benchmarks for doing this with a large osmosis import genesis file, where I batched on batch size, for every 512 keys. (In this PR its in 64kb chunks, a little bit annoying for me to test with our osmosis fork due to version differences. I can do it if people really insist, but this feels like a pretty clear speedup, with a sufficiently close test)
Here is pictures of the pprof time diffs (with triggering based on key amount, rather than bytesize as in this PR).
As this is genesis only logic, and it directionally is a significant speedup, I suggest merging it, and leaving to future work to figure out how to tune / configure this batch size.
Before (153 seconds):

After (27 seconds -- its only the contribution from SaveBranch thats in the same comparison):

This is on top of significant GC pressure being reduced. (I don't have precise numbers for how much of a reduction this change caused, I'd have to re-benchmark)
Note that there is a commit done after this, so theres no concern with a final remnant batch not getting committed: https://github.com/cosmos/iavl/blob/master/mutable_tree.go#L712-L728

@mergifyio backport release/v0.21.x


Receiving CPUAccelerator error when clearly specifying core count

I was trying to train a model from the QM9 checkpoint with the following dataset: 'epoch=649-val_loss=0.0003-test_loss=0.0059.ckpt'. I also used the following hyperparameters:
y_weight: 1.0
force_files: null
neg_dy_weight: 0.0
gradient_clipping: 40
inference_batch_size: 128
load_model: null
log_dir: /home/rgujral/torchmd/tutorial/logs/
lr: 0.0001
lr_factor: 0.8
lr_min: 1.0e-07
lr_patience: 15
lr_warmup_steps: 1000
max_num_neighbors: 64
max_z: 128
model: tensornet
ngpus: -1
num_epochs: 3000
num_layers: 3
num_nodes: 1
num_rbf: 64
num_workers: 6
output_model: Scalar
precision: 32
prior_model: Atomref
rbf_type: expnorm
redirect: false
reduce_op: add
save_interval: 10
seed: 1
splits: null
standardize: false
test_interval: 20
test_size: null
train_size: 110000
trainable_rbf: false
val_size: 10000
weight_decay: 0.0
box_vecs: null
charge: false
spin: false
When creating a batch script and allocating 20 cpu cores and 0 gpu cores, the script fails shortly after executing and leaves the following error messages in an error output folder:
torchmd-train 11
sys.exit(main())
train.py 203 main
trainer = pl.Trainer(
argparse.py 70 insert_env_defaults
return fn(self, **kwargs)
trainer.py 400 init
self._accelerator_connector = _AcceleratorConnector(
accelerator_connector.py 146 init
self._set_parallel_devices_and_init_accelerator()
accelerator_connector.py 376 _set_parallel_devices_and_init_accelerator
self._devices_flag = accelerator_cls.parse_devices(self._devices_flag)
cpu.py 53 parse_devices
return _parse_cpu_cores(devices)
cpu.py 94 _parse_cpu_cores
raise TypeError("devices selected with CPUAccelerator should be an int > 0.")
TypeError:
devices selected with CPUAccelerator should be an int > 0.

Facing the same issue here. Help!

Pass ngpus=20  (or whatever number of CPUs you want to use) when training.
Explanation
The "ngpus" parameter is passed directly to the Lightning Trainer object:
https://github.com/torchmd/torchmd-net/blob/6c42c8bfe467f2934ea6f319ce8697763404dafa/torchmdnet/scripts/train.py#L203-L217
The accelerator is set as "auto" (which favors GPUs I reckon).
The name "ngpus" is thus a bad one, it should be "devices" or smth like that.
For whatever reason the GPU mode in the Lightning Trainer allows to pass "-1" meaning "use all available", but the CPU mode requires an actual positive, non-zero value.
Feel free to reopen if this does not solve your issue.


Wormhole and API server instances replicas

Hi,
What I leaned by reading the code

Wormhole clients (with guttle) are in charge of creating a tunnel between api server instance(s) and nodes, allowing the apiserver to contact pod and services through their PodCidr IP and ServiceCidr IP.
Wormhole clients are deployed on klusters nodes, and are using the kluster url to connect to the wormhole server.
kluster url resolve on a load-balancer (an ingress). Therefore, when wormhole clients initiate the connection to the server, it connects to one of the pod of the service resolved by the ingress, and create a tunnel for this one apiserver pod. Tunnel for other apiserver pods will not be created.
With the actual implementation of the wormhole, it is not possible to deploy more than one api server pod in a kluster.
Request to serviceCidr from API Server are going through a random tunnel.

First of all, can you, please, confirm what I wrote?
What I think about it
If that is the case, you should agree that using only one instance of apiserver is unpractical.
With several instances, we could lower the unavailability time in case of node failure, or during rolling upgrades.
In my opinion, the main issue with the actual design is that the connection is instanciated from the node, which is unpractical since it is very difficult for the node to know about all apiserver instances.
On the opposite, it is really easy for apiserver instances to know about nodes.
What I suggest
I have implemented a proof-of-concept of a solution to this problem, see #299.
It seems that Guttle server and client has been designed to work in a symetric way.
Meaning that, like the guttle server, the guttle client can listen for incoming traffic and redirect it into the tunnel, sending headers with the original destination IP and port. Like the guttle client, the guttle server can take a proxy function in parameter and use this function to dial connection on its tunnel side.
TLDR:
I inverted the client/server concept and instanciation. Now one Wormhole server is deployed on each node, and multiple wormhole clients are instanciated in the apiserver pod (one for each node)
Here are quick schemas of how things work at present, and how things work in what I implemented:
Actual wormhole design :

Actual wormhole details :

Wormhole design proposition :

Node access to API server service is not represented here.
Wormhole proposition details :

What next
I identified one issue with this proposition:

A Guttle client is instantiated for each node (by each api server). WormHole clients can not route serviceCidr IP with Guttle route: Guttle routes concept are configured on server side.
We have to implements a way to randomized the request on nodes. Since Guttle is close to be symetrical, I suggest that we try to make all features available for both client and server. it would suit nicely to the use case.

Note:

I did not configure the wormhole to allow node to contact apiserver with its service IP. This could be done using hijjack-address on the node and a route pointing on the api-server load-balancer IP..
For the sake of easy testing and POC'ing I did not plug back the code and templates responsible for instanciating the wormhole in the api-server and on nodes (helm and ignitions templates)

I did not push further this work, since I wanted to know what you would think of it. Is this relevant to kubernikus? Do you think it deserves to be pushed further?
Thanks

You reverse engineered all this? Hat off! I'm flabbergasted... 😮
I'll leave this for @databus23 to answer but in general, I think, you analysis of the code is spot on.

Accidental close. /reopen

Thanks for the answer !

This is the root why we need the Wormhole in the first place. In our case the nodes are sealed off in a private network - unaccessible from anywhere else. The Apiserver is running outside and can't directly talk to the nodes. Hence the nodes need to open the tunnel.

Humf, I did not know about this.
On one hand it makes sense, on the other hand it looks a bit unpractical. Out of curiosity, what is the actual reason behind the private network ?
Also, I think that the wormhole is necessary even with public accessible nodes, since apiserver in pods would still not be able to dial pods with PodCidr IPs or services with ServiceCidr IPS, am I right ?
Also, do you confirm that this matter is the only thing restraining the number of apiserver replicas ? Or did I miss something else ?
Thanks

On one hand it makes sense, on the other hand it looks a bit unpractical. Out of curiosity, what is the actual reason behind the private network ?

Each customer is running their own private network in Openstack. The purpose is isolation and security. They can create a Kubernikus cluster in their network. The apiserver is running in a regional control-plane. The node's are running in the customer's private network. By default the private network and with that the nodes are not accessible from the outside. It is possible to expose individual nodes via floating ips and fence them with security groups, but we didn't want to have this as a requirement. Ideally, you only want to expose specific services.
We're running all customer apiservers in a dedicated Kubernetes cluster. The cluster itself is unaccessible from the outside. The apiservers on the other side are exposed via ingress and SNI.

Also, I think that the wormhole is necessary even with public accessible nodes, since apiserver in pods would still not be able to dial pods with PodCidr IPs or services with ServiceCidr IPS, am I right ?

Yes, that is right. Whether the apiserver actually does this is another question. I suppose the logs and exec calls are proxied through the Kubelet.

Also, do you confirm that this matter is the only thing restraining the number of apiserver replicas ? Or did I miss something else ?

I'm not 100% positive we actually do have a restriction on the number of replica's per Apiserver. I think, we could have multiple apiserver pods including multiple wormhole servers. Since they are hidden behind an ingress and service the connection would just load-balance between the pods. We need @databus23 to comment on this one.

Using a private network does seem justified and secure. Thanks for the explanation. The floating IP trick is nice.

Yes, that is right. Whether the apiserver actually does this is another question. I suppose the logs and exec calls are proxied through the Kubelet.

I'm pretty sure that logs, exec, and port-forward will work out the box, since, as you pointed out, it is proxied through the Kubelet. However, the apiserver proxy feature will not work, because in this case the apiserver makes request directly on PodCidr IPS and ServiceCidr IPS.

I'm not 100% positive we actually do have a restriction on the number of replica's per Apiserver. I think, we could have multiple apiserver pods including multiple wormhole servers. Since they are hidden behind an ingress and service the connection would just load-balance between the pods. We need @databus23 to comment on this one.

As you say, Node -> Apiserver flow will work with the actual design with multiple API instances. However Apiserver -> Node instanciated flow (apiserver proxy for example) will only work with one api server instance, the one resolved by the ingress/service at wormhole connection time. I also not 100% sure of this, so let's wait for @databus23 to answer/confirm :)


z_loss_weight 默认是0，给出的finetune示例也是0.所以实际没有用到z loss吗？

z-loss was adopted in our training.  But it is necessary so we turn it off in the opensource code.

z-loss was adopted in our training. But it is necessary so we turn it off in the opensource code.

hi @mmmans , do you mean it's unnecessary  in finetune stage?

z-loss was adopted in our training. But it is necessary so we turn it off in the opensource code.

hi @mmmans , do you mean it's unnecessary at finetune stage?

Yes,  not necessary. depends on your setting.

oh, I see. thx a lot

@mmmans I have added thousands of new tokens and made finetuning of full parameters. Do I need to set z_loss_weight?

@mmmans I have added thousands of new tokens and made finetuning of full parameters. Do I need to set z_loss_weight?

depends on your own setting actually.  if your training does not exhibit the training instability problem, there is no need to set z_loss

@mmmans thx~

@mmmans  loss = 6.x  does not converge,should set z_loss_weight?


[SE-0143] Amend to explicitly allow conditional conformances to Hashable

Following discussions on this forum thread, amend SE-0143 to explicitly include the adoption of conditional conformances for Hashable by the following types:

Optional
Array
ArraySlice
ContiguousArray
Dictionary
Range
ClosedRange

This involves a little bit more work than Equatable: while we could make use of existing implementations of == for Equatable, none of these types currently implement hashValue.
The above list intentionally does not include types whose Hashable implementation is even remotely controversial (such as Slice, AnyCollection or AnySequence), or that don't already implement Equatable (such as CollectionOfOne or DictionaryLiteral).
A draft implementation is available in https://github.com/apple/swift/pull/14527. (It is being updated in parallel to this PR; a version that's in line with this amendment will be available soon.)

/cc @airspeedswift, @DougGregor
(The alternative of course is to submit these conformances in a separate proposal.)

Taking this amendment after core team discussion.


fullscreen

Is your feature request related to a problem? Please describe
No response
Describe the solution you'd like
please add a open in fullscreen option
Describe alternatives you've considered
No response
Please specify the version of Windows Subsystem for Android
2209.40000.26.0

there is already a Full-screen option, press F11.

open always in fullscreen option

This is currently supported, as @AhmedZero has commented. You can see all of our supported inputs and keyboard shortcuts here!

#133


Fix widget example code layout on Site

Fix layout of code in examples for widgets.

Aligns code in valid YAML format
Indents  code block to match the line item

From:

To:

Deploy preview for netlify-cms-www ready!
Built with commit b056535dc1af49ad8a351df614df581032099834
https://deploy-preview-1606--netlify-cms-www.netlify.com

☹ This is an issue with Prettier vs Remark here: if I run Prettier it reverses every change you made here. Do you think it is worth just turning Prettier off for MD files, or trying to get a patch upstream to one of them?

I am not sure this was 100% Prettier causing all the problems, but we shouldn't be using prettier on content files. We will never get it right no matter how hard we try. There are too many nuances and will just cause more problems than it is worth.

We will never get it right no matter how hard we try.

Not sure I completely agree with that, but it's definitely not worth it right now. Removing.

Deploy preview for cms-demo ready!
Built with commit b5a9afc93ba2434adb043ff93ea5a2c48c253dcf
https://deploy-preview-1606--cms-demo.netlify.com

I guess the reason I saw it as less of an issue is that you can just run Prettier when you're done. It's not forcing you to write the MD a certain way, it just changes it when you are done.

There were some other bugs Prettier was causing, so I created https://github.com/netlify/netlify-cms/pull/1612 to just revert all the Prettier changes on docs.


Add Tangle Sidebar to Faucet

Summary of changes
Provide a detailed description of proposed changes.

Create Tangle Sidebar component
Add new Tangle Sidebar to the Faucet website

Proposed area of change
Put an x in the boxes that apply.

[ ] apps/bridge-dapp
[x] apps/hubble-stats
[ ] apps/stats-dapp
[ ] apps/webbsite
[x] apps/faucet
[x] apps/tangle-website
[ ] libs/webb-ui-components

Reference issue to close (if applicable)
Specify any issues that can be closed from these changes (e.g. Closes #233).

Closes #1767
Closes #1708

Screen Recording
If possible provide a screen recording of proposed change.
https://github.com/webb-tools/webb-dapp/assets/69841784/29080ded-9f66-4afe-b198-afc552fedbb2

Code Checklist
Please be sure to add .stories documentation if any additions are made to libs/webb-ui-components.

[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Flickering issue is created here


Split dropdowns in collapsed navbar

Hello, this is my first issue, I hope I haven't missed anything out.

components: Dropdown
bootstrap version: 4.1.1

It would make sense to be able to put split dropdowns (a's and buttons) into the navbar, but they both behave strangely (compared to the simple  dropdown) when the navbar is collapsed.
Here: http://jsbin.com/winoxajabi/edit?html,output
If confirmed I would like to propose a PR (I already fixed the issue in my local app). My solution will be to look for the parent of the parent if the MENU selector isn't found in its parent, thing that will happend with the btn-group because the dropdown-menu is (and I think it should be) outside said group.

I’m not fully understand what result do you expect, but one thing I can say for sure is that
 can contain only
 as a child element.

Yes, I'm sorry about the
s, I have updated the jsbin.

Expected behavior (Submenu under the parent):

Unexpected:

I have the same problem. If you move the .dropdown-menu below the .btn-group after setting up the dropdown (e.g. in developer console), it works as expected. I think one solution was to add a config option menu , which optionally specifies a remote .dropdown-menu.

Hi @Jandoria  and @sedrubal did you ever find a way to make this work without JavaScript? I thought it could be solved with some popper config as suggested by @sedrubal but the documentation says:

Popper isn’t used to position dropdowns in navbars though as dynamic positioning isn’t required.

I have found a way to get this working after inspiration from this stack overflow:
https://stackoverflow.com/a/66615795/284841
By adding d-table to a
 with btn-group that wraps the two  tags and the
 and adding d-inline to the two  tags it starts to work with bootstrap 5. However it is not perfect as the drop down menu is not full width when collapsed and the clickable area for the down caret is so small it is difficult to hit on a mobile device.
I ended up using d-table and w-100 on the
 and then on the first   I did not use d-inline, instead I used float-start pt-0 pe2 and on the second  I did not use d-inline instead I used w-100 pt-100.
For bootstrap 4 you would need float-left. This works in bootstrap 4 (change float-left back to float-start for bootstrap 5):

  Navbar working
  <button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent2" aria-controls="navbarSupportedContent2" aria-expanded="false" aria-label="Toggle navigation">

  </button>

          Split Dropdown

            Toggle Dropdown

Action

Separated link


feat: line viewer with additional information

Summary of PR
Setup nested routes so that clicking a line opens a "Line Viewer" page that shows additional Information. Similar to mockup.  Also updates dependencies for frontend.

🚧 WIP 🚧
Frontend

[x]  Fix: Currently focused line is the only one that opens Line Viewer. Using cursor one can open any line.
[X] Fix: Prev/Next button in Line Viewer
[x] Add 3 dots menu for opening issue on GitHub
[x] Style Line Viewer "Content" similar to mockup.

Backend

[x] Get translation source info with line data

[ ] Integrate Google translations

Time spent on PR
5.5 hours
Reviewers
@Harjot1Singh @bhajneet

Please push without translations if need be. Need to work on better releasing "rolling updates".

We'll do translations in a separate PR


Run tests against ScubaGear for WCAG 2.2 comformance

💡 Summary
Go through the report overview and product reports, then determine for each WCAG 2.2 guidelines if ScubaGear conforms to level A and AA success criteria. This is listed under the software requirements for 508 compliance.
https://www.access-board.gov/ict/#E207-software
https://www.access-board.gov/ict/#chapter-5-software
From E207.2 WCAG Comformance:
"User interface components, as well as the content of platforms and applications, shall conform to Level A and Level AA Success Criteria and Conformance Requirements in WCAG 2.2 (incorporated by reference, see 702.10.1)."
WCAG 2.0 guidelines: https://www.w3.org/WAI/WCAG22/quickref/?versions=2.2
Motivation and context
This would be useful because we are looking to make HTML reporting improvements in an upcoming release. #820
#93 is an accessibility improvement for how tables are read to better accommodate screen readers, which WCAG 2.2 will also cover.
Implementation notes
There are many tools  to aid with testing accessibility, but the majority of the WCAG 2.2 testing will have to be a case by case process for each guideline.

For example, Guideline 1.1 - Text Alternatives:
a. Go through any non-text content in the reports. There should be a text alternative that serves the equivalent purpose. We already have alt attributes for images, but for screen readers it will be helpful to include aria-labels.
It will help to create a checklist of sorts to mark which guidelines we've covered. Some of them such as Guideline 1.2 - Time-based media doesn't apply to ScubaGear.

Acceptance criteria
How do we know when this work is done?

[ ] Create a mapping of all WCAG 2.0 guidelines, ensure that each one has been tested against.

Guideline
Success Criteria
Description
Application in Report
Pass/Fail

1.1.1 Non-Text Content
A
All non-text content that is presented to the user has a text alternative that serves the equivalent purpose
CISA logo in ; add aria-label="Return to the report summary"
TBD


After executing CodeAction cursor jumps to incorrect split/pane

When there are multiple splits open with rust source files and when code action is opened and some action is picked, then cursor doesn't correctly jump back to last position, but it jumps back to last cursor position in top left split.
To fix, cursor should be placed on the split that opened the code action at the last position the cursor was located at before code action window opened.

I'm seeing this too - quite disruptive for workflows that keep a file browser split open.


SD option

Is it possible to save all that backups on disk/sd inside Mikrotik instead of FTP?

Of course, comment out or delete lines: 33 to 44 and 5 to 8

Project update.
You can now use tmpfs to store temporary files (requires ROS 7.7+) or store on external drives (need to remove lines 68 to 72).


Respect PROPERTY_USAGE_NEVER_DUPLICATE with Make Sub-Resources Unique

Make Sub-Resources Unique option in the inspector would list Script as property to duplicate.

I might do a follow-up to improve this dialog; there are some other changes I wanted to do.


Speed up boolean encoding/decoding

From our benchmarks we can see that we are consistently slower than everyone else when serializing/deserializing boolean values. We should fix that.
orjson is using an unsafe block to create a reference to a boolean:
https://github.com/ijl/orjson/blob/03d55e99a953ce93cedc05f03e4b63b0bcbbcc7a/src/decode.rs#L81-L96
This avoids additional allocations.
For comparison, this is our code at the moment:
https://github.com/mre/hyperjson/blob/ded13b4100638aa32fe19dc477f5cfe3e704893c/src/lib.rs#L475-L480
I wonder if we could achieve comparable performance without using unsafe.
@konstin, any idea? Maybe there was a recent development in pyo3 that we could leverage here?

It's weird that this slower than what orjson does. That's the implementation of ToObject:
unsafe {
    PyObject::from_borrowed_ptr(
        py,
        if *self {
            ffi::Py_True()
        } else {
            ffi::Py_False()
        },
    )
}

And that's the implementation of from_borrowed_ptr:
debug_assert!(
    !ptr.is_null() && ffi::Py_REFCNT(ptr) > 0,
    format!("REFCNT: {:?} - {:?}", ptr, ffi::Py_REFCNT(ptr))
);
ffi::Py_INCREF(ptr);
PyObject(NonNull::new_unchecked(ptr))

So in theory this should compile down to the same as orjson. You could try using PyBool::new and see if that makes a difference, but otherwise I don't know enough about inspecting assembly to debug that.


Allow raw cell output to be rendered

When printing to console, it would be useful to have that output understood as just raw text, so that whatever is transforming can process and render it. For example, if you were converting to latex later, you could do something like this:
```{code-cell}
---
tags: [raw-cell]
---

print("""
\begin{center}
\begin{tabular}{ c c c }
 cell1 & cell2 & cell3 \\
 cell4 & cell5 & cell6 \\
 cell7 & cell8 & cell9
\end{tabular}
\end{center}
    """)
```

and have that printing to the console show up as a table in your latex output. This is something that's currently implemented in pandoc and Rmarkdown as well.
Would that be possible?

Following up on this, can glue do something like this? If I save the latex string to a glue variable, will I be able to render the latex in the document?

Following up on this, can glue do something like this? If I save the latex string to a glue variable, will I be able to render the latex in the document?

heya I think this is a duplicate of #117?
I think it should be quite possible, I'm just updating myst-nb at present so I'll take this into account (it will be a few weeks away though probbly)

heya I think this is a duplicate of #117?
I think it should be quite possible, I'm just updating myst-nb at present so I'll take this into account (it will be a few weeks away though probbly)

I'm gonna close this because I think that it is a dupe of #117, but I am not positive so @amichuda please re-open if we are wrong!

I'm gonna close this because I think that it is a dupe of #117, but I am not positive so @amichuda please re-open if we are wrong!


npm WARN addRemoteGit ERRORS in sails 0.12.3

Sails version: 0.12.3
Node version: 5.11.0
NPM version: 3.8.6
Operating system: OSX 10.11.4

npm WARN addRemoteGit Error: not found: git
npm WARN addRemoteGit     at F (/usr/lib/node_modules/npm/node_modules/which/which.js:63:19)
npm WARN addRemoteGit     at E (/usr/lib/node_modules/npm/node_modules/which/which.js:72:29)
npm WARN addRemoteGit     at /usr/lib/node_modules/npm/node_modules/which/which.js:81:16
npm WARN addRemoteGit     at /usr/lib/node_modules/npm/node_modules/which/node_modules/isexe/index.js:44:5
npm WARN addRemoteGit     at /usr/lib/node_modules/npm/node_modules/which/node_modules/isexe/mode.js:8:5
npm WARN addRemoteGit     at Object.oncomplete (fs.js:108:15)
npm WARN addRemoteGit  git+https://github.com/rynomad/jsbn.git resetting remote /home/deploy/.npm/_git-remotes/git-https-github-com-rynomad-jsbn-git-e72d386d because of error: { [Error: not found: git] code: 'ENOGIT' }
npm ERR! git clone --template=/home/deploy/.npm/_git-remotes/_templates --mirror https://github.com/rynomad/jsbn.git /home/deploy/.npm/_git-remotes/git-https-github-com-rynomad-jsbn-git-e72d386d: undefined
npm ERR! git clone --template=/home/deploy/.npm/_git-remotes/_templates --mirror https://github.com/rynomad/jsbn.git /home/deploy/.npm/_git-remotes/git-https-github-com-rynomad-jsbn-git-e72d386d: undefined
npm WARN deprecated<EMAIL_ADDRESS><EMAIL_ADDRESS>is no longer maintained. Upgrade to<EMAIL_ADDRESS>npm WARN deprecated<EMAIL_ADDRESS>graceful-fs version 3 and before will fail on newer node releases. Please update to<EMAIL_ADDRESS>as soon as possible.
npm WARN deprecated<EMAIL_ADDRESS>graceful-fs version 3 and before will fail on newer node releases. Please update to<EMAIL_ADDRESS>as soon as possible.
npm WARN deprecated<EMAIL_ADDRESS>DEPRECATED. See readme: https://github.com/gruntjs/grunt-lib-contrib
npm WARN deprecated<EMAIL_ADDRESS><EMAIL_ADDRESS>is no longer maintained. Upgrade to<EMAIL_ADDRESS>npm WARN deprecated<EMAIL_ADDRESS>'native-or-bluebird' is deprecated. Please use 'any-promise' instead.
npm WARN deprecated<EMAIL_ADDRESS>Grunt needs your help! See https://github.com/gruntjs/grunt/issues/1403.

@jjgarciadev this seems to be coming from a dependency you have installed - https://github.com/rynomad/jsbn.git. It's list of modules that depend on it can be found here.


Add auto merge

@conda-forge-admin, please add bot automerge

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/sty-feedstock#2.


Unescape from HTML and escape to JSON

The HTML code I wanna convert is formatted as JSON.
My JSON string contains HTML entities inside properties values, and especially ", which is the encoded character for ".
So, I need to escape the previously converted characters in JSON, which means adding a backslash (\) to them.
Thanks

Can you give an example?
Use JSON.stringify() to generate JSON (and take care of escaping quotes in strings).

take care of escaping quotes in strings

That's exactly what I want to automate :)

Ok, I guess your issue is resolved then? :)

No because I didn't found out how to automate string quote escaping

I'm confused. I just told you that JSON.stringify does that for you.

JSON.parse(JSON.stringify(val)) returns val

It does indeed!
Again, can you give an example of what you're trying to do? Provide example input, desired output, and code you've tried so far.

Here is a JSFiddle example : https://jsfiddle.net/KaKi87/56pbLtuf/

How is the content inserted into the <pre>? Are you doing that on the server side?

No, this is an example.
The original is a real JSON response, fetched using NodeJS request.
But, this fiddle have the same behavior as in my script. The point is that I can't auto-escape quotes inside JSON strings.

The original is a real JSON response, fetched using NodeJS request.

If it's a proper JSON response, then JSON.parse would succeed, making it different from the demo you shared.
I'm still struggling to understand what you mean.

Okay then try to parse this : https://trigedasleng.net/api/api?action=search&query=dei de

Works for me:
(async () => {
  const response = await fetch('https://trigedasleng.net/api/api?action=search&query=dei%20de');
  const data = await response.json();
  console.log('Result:', data);
})();
// Result: {...}

Okay but did you decoded the HTML entities ?

To decode HTML entities in a string, use he.decode(string).
(async () => {
  const response = await fetch('https://trigedasleng.net/api/api?action=search&query=dei%20de');
  const data = await response.json();
  data.words = data.words.map((word) => {
    word.translation = he.decode(word.translation);
    return word;
  });
  console.log('Result:', data);
})();
// Result: {...}

So, I must decode property by property ??

Yeah. There's no way he or any other library can know which values need decoding and which do not.

All I need is to automatically escape the " character

It seems to me that you could use the string.replace method to replace any instance of " with " in your server strings (if that is truly "all you need"). The he library is used to create and remove HTML entities, not to use \ to escape characters in strings.


[UNDERTOW-1144] URL session tracking - sessionid is appended if there is old sessionid in the URL

Issue: https://issues.jboss.org/browse/UNDERTOW-1144

This falls over if there are path parameters in addition to jsessionid, or if any of the other additional parameters contains jsessionid as part of the name or the value of the parameter


fix(curriculum): allow screen readers to properly see emote values for rpg step 138 and 148

Checklist:

[X] I have read and followed the contribution guidelines.
[X] I have read and followed the how to open a pull request guide.
[X] My pull request targets the main branch of freeCodeCamp.
[X] I have tested these changes either locally on my machine, or GitPod.

Closes #52775

I'd love to be the person that gets the emote previews to actually render in the preview rather than be in pure text form. But I can't find that specific update function.

Hey @a2937 !
It looks like a decision was made here
https://github.com/freeCodeCamp/freeCodeCamp/issues/52775#issuecomment-1887701377
If you could apply those changes, then it will be ready for review 👍

!Screenshot](https://github.com/freeCodeCamp/freeCodeCamp/assets/9373245/95ca0a10-e8cf-4a6c-86d5-679bc52de159)
I believe things seem to be in order.

Wait. Were any other challenges added in this block or something?

Hey @jdwilkin4 , I forgot to update this branch before I started the major changes. I would like to adjust things so that they line up properly.


Gabby/notification view

Added Notifcation View and controller.

Change controller locations to /Controllers and commit again. thanks!

Controllers location fixed

All that needs to be is done is run the code and be sure the functions in the Notifications Controller work. To test just go to url/notifcation/testfunction

Also a lot of functionality is commented out due to hard coded data but the pages should return something at all times, just look at the return values of each function.


Replace publish_release.sh with a Just recipe

Problem

The script ingestion_server/publish_release.sh performs one, quite simple task. It should not live on as a standalone, undocumented script.
Description

It can be

changed into a Just recipe
generalised to also build and tag images for other services

Alternatives

In the bigger picture, it can be integrated into our CI + CD workflow (see PRs #424 and #437) or be completely replaced by it and deleted.
Additional context

Such undocumented scripts are detrimental to DX as people can forget why they exist.
Implementation

[x] 🙋 I would be interested in implementing this feature.

Hey, @dhruvkb. I would like to fix this issue, but in all honesty, I have no idea how to start. Any pointers that might help?
This is my first try at open source😄

@PrabhavPandey this ticket isn't ready for work yet (my mistake for the incorrect labelling). The PRs mentioned above need to be merged before this can be fixed.
Would you kindly pick an alternative ticket?

@PrabhavPandey this ticket isn't ready for work yet (my mistake for the incorrect labelling). The PRs mentioned above need to be merged before this can be fixed.
Would you kindly pick an alternative ticket?

Sure. Thank you!

@PrabhavPandey the blocking PR has been merged, so you can go ahead and delete the file, including all references to it.

@dhruvkb I have opened a pull request for the task assigned


Use project.json for .csproj package references via kpm list

This gets rid of our silly <PackageReference> elements and makes project.json the source of truth for both csproj and kproj-based solutions. It also enables us to use some of the newer contracts like System.IO.FileSystem from the csproj-based solution.
Unfortunately, there is a bug in .NET Framework 4.6 that prevents the tests from running in the csproj-based solution. We probably want to wait until the fix is released publicly before merging.

This depends on aspnet/Universe#167.

:+1: This is an awesome fix.

@bricelam, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, DNFBOT;

Closing for now. Will re-open when unblocked

New approach: Same thing, but net45 projects instead of .NET Core 5


List Icons Visibility

Its hard to distinguish between the dropdowns arrows and bullet points from a quick glance, I think it would be better visibility if the icons were filled in or perhaps have a darker outline.
Heres an example of what I see:

I agree. I didn't even notice the icons because of my screen brightness, and it took me a long time to realize that I could expand rows.

Yep that's appalling, will fix shortly. Thanks for reporting!

Fixed

Nice! 👍


chore: Open Discussion on Github

Description
I think it will be better to open the Discussion tab in Github as there could be a lot of topics that should go into the Discussion tab before even making it into an Issue or PR.

Hi @divyanshub024 👋
Thanks for opening an issue!
For now we've decided to stick to using issues even for discussions/questions. We'll re-evaluate using discussions in the future but for now feel free to continue opening issues for questions 👍


This library has a dependency on Android

This is not mentioned in the repo description anywhere, but it makes this library unsuitable for non-Android projects and tests, unless you want to use Robolectric.

I guess you are correct, But do you have any ideas to improve it? What is the main issue?

The main issue is that the interceptor constructor has a direct dependency on the Android context. An alternative would be to provide the Interceptor with a FileSource or something similar. Which could have an Android and a non-android implementation.

True. I'll see what I can do in my spare time

@maartenvang You can start using version 2.0 and please close this issue if the problem is solved.

That was quick. Nice work! 🍻


chore: upgrade config

Upgrade go.mod and config to version 1

closing in favor of https://github.com/ignite/modules/pull/74


[SPARK-48016][SQL][3.5] Fix a bug in try_divide function when with decimals

What changes were proposed in this pull request?

Currently, the following query will throw DIVIDE_BY_ZERO error instead of returning null
SELECT try_divide(1, decimal(0));

This is caused by the rule DecimalPrecision:
case b @ BinaryOperator(left, right) if left.dataType != right.dataType =>
  (left, right) match {
 ...
    case (l: Literal, r) if r.dataType.isInstanceOf[DecimalType] &&
        l.dataType.isInstanceOf[IntegralType] &&
        literalPickMinimumPrecision =>
      b.makeCopy(Array(Cast(l, DataTypeUtils.fromLiteral(l)), r))

The result of the above makeCopy will contain ANSI as the evalMode, instead of TRY.
This PR is to fix this bug by replacing the makeCopy method calls with withNewChildren
Why are the changes needed?

Bug fix in try_* functions.
Does this PR introduce any user-facing change?

Yes, it fixes a long-standing bug in the try_divide function.
How was this patch tested?

New UT
Was this patch authored or co-authored using generative AI tooling?

No

@dongjoon-hyun Thanks for pointing out the mistaken changes! It is now reverted in https://github.com/apache/spark/pull/46323/commits/0a43e3ab982dcb21bfaf9594db0233a354f12869

Let's make it sure that all tests pass on this PR builder.

CI passed. Merging to branch-3.5

Thank you!


add multiple "engines"

Right now, when you invoke Output::compute, we always use the naive differential-dataflow implementation to do that computation:
https://github.com/rust-lang-nursery/borrow-check/blob/8ef88dbe968db92dd9dfa0c0296c3c4a57608e0d/src/output/mod.rs#L33-L35
I would like to pass in an enum and select from multiple options. In particular, the current should be retained, with the title "naive". Then I imagine we can add more optimized variants. This would probably build on #2 to allow for a command-line option that selects which algorithm to use.

This issue is specifically just to add an enum with one variant:
enum Algorithm {
  Naive,
}

and perhaps the command-line support to select it. Other issues would add new variants to that enum.


migrate controller poolcoordinatorcert

What type of PR is this?
/kind enhancement
What this PR does / why we need it:

migrate poolcoordinator-cert-manager into yurt-manager framework
fix syntax in helm charts

Which issue(s) this PR fixes:

Fixes # https://github.com/openyurtio/openyurt/issues/1067
Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

@luc99hen Maybe you can use injector to prepare kubernetes.Interface, and you reference the injector in this pr: https://github.com/openyurtio/openyurt/pull/1334

/lgtm

/approve


Splitting is hard

How do you even beat one level? :P Video guide?

I was wondering if somehow the game speed is tied to the frame rate, as I've observed it running more slowly on other people's computers, which makes it a lot easier.

Yeah, the framerate right now on some games is tied to the monitor's refresh rate. There's a branch on matter.js' GitHub called timing-improvements which should help/solve this problem. Definitely looking to push that build out asap.
As for the splitting, as long as you have one unit remaining, you should get points.

Ah, thanks for the explanation. I'm not all that good at splitting, but I think I should at least be capable of passing a single level, if this gets worked out, haha.

Once the timing branch gets pushed I'm probably gonna reset all the scores since some people's are way higher because of it. I'll look to soften the splitting game too. Thanks for the feedback.


gRPC Http-Json transcoding throws 500 when supported serialization format does not include JSON.

When I enable transcoding it Http and use it such as,
.enableHttpJsonTranscoding(true)
.supportedSerializationFormats(GrpcSerializationFormats.PROTO)

it returns
Status: 500
Description: Internal Server Error

With little to no information. Taking a closer look, I realized in the debugger my content type was updated as application/grpc+json without asking me. Even though I used Content-type: application/json in my curl request. Such as
curl 'localhost:9027/sampleEndpoint' -H "Content-type: application/json; charset=utf-8"

Thus underlying issue became:
 A gRPC response must have the grpc-status header. response: DefaultAggregatedHttpResponse{headers=[:status=415, content-type=text/plain; charset=utf-8, content-length=39], content={39B, hex=4d697373696e67206f7220696e76616c}}

The solution was to add
.supportedSerializationFormats(GrpcSerializationFormats.JSON, GrpcSerializationFormats.PROTO)

And it was really tough for us to debug this. I think the response should be something more obvious , if not possible at least we should warn users during startup that transcoding won't work.
I still don't fully understand how my content-type header changed magically under the hood.

And it was really tough for us to debug this. I think the response should be something more obvious , if not possible at least we should warn users during startup that transcoding won't work.

I think adding validation in GrpcServiceBuilder is a good idea 👍  We can probably throw an ISE at GrpcServiceBuilder#build.

And it was really tough for us to debug this. I think the response should be something more obvious , if not possible at least we should warn users during startup that transcoding won't work.

I think adding validation in GrpcServiceBuilder is a good idea 👍 We can probably throw an ISE at GrpcServiceBuilder#build.

I think ISE makes sense overall but it can be a breaking change for some people. I am not sure what people would think if their service starts to throw exceptions during startup after a version upgrade 😓 That's why I suggested a warning and maybe we can completely disable the transcoding if JSON is not enabled so it just throws like 404 and stuff.
What are your thoughts?

I am not sure what people would think if their service starts to throw exceptions during startup after a version upgrade 😓

Although it can be a somewhat breaking change, I don't think this is very hard to migrate and throwing an exception is clearer. I still prefer throwing an exception, but any other thoughts are welcome. @trustin @minwoox @ikhoon

Agreed. Since it is a kind of defect, it is a bug patch and not a breaking change.


Add message reaction

Added message reaction command (!react [message id] [emoji])
Cleaned up README.md

Hi Shell

Working on new update that invalidates current code. New PR coming soon.


Refactor(web-react): Use Spacing hook in Grid component #DS-1425

Description

Additional context

Issue reference

coverage: 78.249% (-0.8%) from 79.082%
when pulling fac9af81f09340af72750928da6b707a47021cdb on refactor/grid-spacing-hook
into f159ab01bc3c4bd705c43ce0461c1a4911ba9bd5 on main.

Nitpicking: I noticed that internal Box component has undefined class. I was afraid that it is my fault but it looks like there are not properly chaining the classes (or maybe there are not properly set styles at all). Could you please fix it? 🙏🏻

Let's not mix two non-related tasks. #1571


chore: do not run tests in draft pr state

Changes introduced in this PR:

Prevents running tests in draft state to avoid exhausting infura api keys

This is in response to #215
Closes #215

@drewstone let me know if this sufficiently addresses #215, as you can see the tests do not run in the draft state.

This works, let's remind everyone to open drafts.


[AMORO-2012]Remove useless dependencies for project

Why are the changes needed?

As #2012 said, this PR try to remove useless dependencies of all modules.
Brief change log

Remove useless dependencies.
Manage dependency version with properties.
Manage dependencies with dependencyManagement block in parent pom.

How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before making a pull request

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable)

LGTM


NIFI-3901: Address Provenance Rest Api issues

NIFI-3901:

Addressing response code of POST /provenance/lineage.
Ensuring cluster node details are set accordingly.

reviewing

Setup a cluster using docker & compose and verified POST as well as those cases where node details were not set in clustered content.
Will get this merged.


[test](draft, for test only) enable info log level on catalog lock

Proposed changes

run buildall


added linker fragment syntax obj:sym[+] (scheme) to support retain sy… (IDFGH-2011)

…mbol

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@123swk123 Can we close this PR per comment https://github.com/espressif/esp-idf/issues/4199#issuecomment-543125496?

Closing due to lack of response and availability of alternate solution per above comment. Please feel free to reopen if you still think this is useful in some context.


automatically add debug and enabled fields to extension settings

this is somewhat problematic: pydantic schemas cannot easily be modified anymore. Perhaps just pretend it's there and check against it?
Pydantic Models can accept extra data


Automatically generate changelog.

We need way to automatically generate changelog. Any ideas?

Hi! lk-geimfari,
There is a way to automatically generate changelog, using the following lib: https://pypi.python.org/pypi/gitchangelog
:]

@hi-hi-ray Hi! Looks really awesome! Thank you!


Codecomments

Hello,
those are various extra code comments

using inheritdoc for basic ops like tostring and dispose
fixed a few codecomments links

Thanks
Julien

thanks!


⚠ Bofrost.nl has degraded performance

In 3fa1580, Bofrost.nl (https://www.bofrost.nl) experienced degraded performance:

HTTP code: 200
Response time: 2111 ms

Resolved: Bofrost.nl performance has improved in a24ce55.


Endpoint routing selecting incorrect controller/action after 2.0 -> 2.2 upgrade

Describe the bug
I ran into a scenario while migrating an app from 2.0 to 2.2. Basically a route that used to select a controller under an area, now is selecting a controller in the root controllers folder.
Here is a repro:
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;

namespace WebApplication1
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }

        public IConfiguration Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env)
        {
            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "areaRoute",
                    template: "{area:exists}/{controller=Home}/{action=Index}");

                routes.MapRoute(
                    name: "accountsCatchAll",
                    template: "Accounts/{*path}",
                    defaults: new { area = "Accounts", controller = "Home", action = "Index" });

                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

//Imagine this is in /WebApplication1/Controllers/HomeController.cs
namespace WebApplication1.Controllers
{
    public class HomeController : Controller
    {
        public IActionResult Index() => Content("Home/Index");
    }
}

namespace WebApplication1.Areas.Accounts.Controllers
{
    //This is in /WebApplication1/Areas/Accounts/Controllers/HomeController.cs
    [Area("Accounts")]
    public class HomeController : Controller
    {
        public IActionResult Index() => Content("Accounts/Home/Index");
    }

    //This is in /WebApplication1/Areas/Accounts/Controllers/OtherController.cs
    [Area("Accounts")]
    public class OtherController : Controller
    {
        public IActionResult Index() => Content("Accounts/Other/Index");
    }
}

Before using Endpoing Routing the url /accounts/forgot-username would match the "accountsCatchAll" and it would execute the HomeController under the "Accounts" area. After switching to endpoint routing the controller being picked up is the HomeController in the root controller folder (non-area controllers folder).
The issue goes away when I disable endpoint routing via:
services.AddMvc(options =>options.EnableEndpointRouting = false);

cc: @JamesNK

This looks patch worthy @rynowak @JamesNK

Are you using the latest version of the 2.2 SDK?
You can find out with: dotnet—info

I'm not running latest, I'm going to get latest and try again.
Here is the output anyway:
.NET Core SDK (reflecting any global.json):
 Version:   2.2.104
 Commit:    73f036d4ac

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17134
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.2.104\

Host (useful for support):
  Version: 2.2.2
  Commit:  a4fd7b2c84

.NET Core SDKs installed:
  2.1.100 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.2.103 [C:\Program Files\dotnet\sdk]
  2.2.104 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Upgraded to latest (2.2.3), but the same behavior remains:
.NET Core SDK (reflecting any global.json):
 Version:   2.2.202
 Commit:    8a7ff6789d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17134
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.2.202\

Host (useful for support):
  Version: 2.2.3
  Commit:  6b8ad509b6

.NET Core SDKs installed:
  2.1.100 [C:\Program Files\dotnet\sdk]
  2.1.202 [C:\Program Files\dotnet\sdk]
  2.1.503 [C:\Program Files\dotnet\sdk]
  2.2.103 [C:\Program Files\dotnet\sdk]
  2.2.104 [C:\Program Files\dotnet\sdk]
  2.2.202 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

By the way, nothing special in the csproj:
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.2</TargetFramework>
    <AspNetCoreHostingModel>InProcess</AspNetCoreHostingModel>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" />
  </ItemGroup>

</Project>

@JamesNK did you already investigate this?

Not yet. I'll try to get to it today or tomorrow

Hi @epignosisx
I've investigated your issue and I have reproduced what you see: your scenario is broken in 2.2 and it is fixed in 2.2 when endpoint routing is turned off.
I also tested your scenario in 3.0 with endpoint routing and confirmed that it works in the latest source code.
Because the fix in 2.2 is quite simple - turn off endpoint routing - and the issue is already fixed in 3.0, I'm hesitant to patch it in 2.2.
Are you ok with leaving endpoint routing off until you upgrade to 3.0?

@JamesNK thanks for looking into this. I was able to work around it by removing the root controller (didn't actually need it).


Automated Package Upgrade

:tada: This PR is included in version 13.0.107 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


GPU Devices

Hi,
I'm using maniskill and am unclear on the intended use in terms of which devices I'm supposed to have data on. It's returning
obs, reward, terminated, truncated, info = self.env.step(action)
But terminated and truncated are on different GPUs. Is this intended? What's the intended workflow? I'd expect something like computing my network stuff on one GPU, and having the environments return on the same one.
It feels silly to write:
obs, info = self.env.reset(options={'env_idx': torch.nonzero(torch.logical_or(terminated, truncated.to(terminated.device))).squeeze()})

Also, is there a way to sample from an action space that gives a tensor on the same env?

This doesn't seem right, what does nvidia-smi say?
Generally in GPU sims everything is on one GPU per python process, we shouldn't have any code that allows for tensors to be on different devices.

Well I'm working on reproducing it in a simpler script, but relatedly this is producing an error:
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.120                Driver Version: 550.120        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        Off |   00000000:01:00.0 Off |                  Off |
|  0%   43C    P8             29W /  480W |    3379MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 4090        Off |   00000000:02:00.0  On |                  Off |
|  0%   38C    P3             52W /  480W |    8280MiB /  24564MiB |     10%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Here's a python snippet with a similar problem:
import gymnasium as gym
import mani_skill.envs

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

env=gym.make("PickSingleYCB-v1", num_envs=10)
obs = env.reset()

res = env.step(action)
print(env.unwrapped.device)
for item in res:
    if type(item) == torch.Tensor: print(item.device)

import mani_skill
print(mani_skill.__version__)
print(gym.__version__)

outputs:
cuda
cuda:1
cuda:1
cuda:1
cuda:1
3.0.0b14
0.29.1

Okay looks like this is a good reproduction:
import gymnasium as gym
import mani_skill.envs

import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

import torch
print(torch.cuda.device_count())
print(torch.cuda.get_device_name(0))

import torch
torch.cuda.set_device(0)  # Explicitly set GPU 0

env=gym.make("PickSingleYCB-v1", num_envs=10)
obs = env.reset()

res = env.step(action)
print(env.unwrapped.device)
for item in res:
    if type(item) == torch.Tensor: print(item.device)

import mani_skill
print(mani_skill.__version__)
print(gym.__version__)

outputs:
1
NVIDIA GeForce RTX 4090
cuda
cuda:1
cuda:1
cuda:1
cuda:0
3.0.0b14
0.29.1

Not immediately sure what the issue is. Could you try upgrading maniskill to the latest version and try again?
And I think one possible workaround is to instead run your script as so
export CUDA_VISIBLE_DEVICES=1 python script.py

It is the latest version, one of the version markers just isn't updated.
https://github.com/haosulab/ManiSkill/blob/main/mani_skill/init.py#L6C1-L6C25
Although I tried moving the os environment variable export to before the mani_skill import, and that seems to have fixed things. Broadly speaking, is multi-gpu supported at all?

You can run separate maniskill instances using different gpus in different processes. For some distributed training workloads one would need to investigate a bit more into pytorch distributed I think. It is the solution isaac lab uses for multi-gpu training although I am not sure how straight forward it is at the moment.

Thanks!


Toggle button to enable/disable margin and padding at the top

I only want to see the sidebar

already fixed, just waiting for the zen store to update

🥳🎉


Convert examples to literate code

Convert examples to be in literate format such as https://jupytext.readthedocs.io/en/latest/formats.html#the-light-format and commit rendered Ipynb notebooks.

Sounds good! I like the simplicity of the light format!

Gonna call this one done for now as we have some examples in the docs. I'll add more as we go.


messages printed in LaTeX format are not escaped

@rultor release, tag is 0.0.14

@rultor release, tag is 0.0.14

@yegor256 OK, I will release it now. Please check the progress here

@rultor release, tag is 0.0.14

@yegor256 Done! FYI, the full log is here (took me 3min)


External mining

This PR is part of the bountied work PegaSysEng/BountiedWork#20.
This PR introduces a new flag that disables CPU mining for ethash, and instead presents work via getWork or stratum for external miners to provide solutions.
It also introduces a JSON-RPC method so miners can interact over http with Besu.
It introduces support for Stratum through a dedicated Stratum TCP connection, supporting stratum+tcp at the moment, with a mechanism to add more versions of Stratum.
This PR was tested against ethminer 0.15 (works on Mac). With those changes, ethminer was able to mine blocks over JSON-RPC HTTP interactions or the stratum+tcp protocol.

@shemnon ready for another round of review.

OK thank you for the review. I have attended to some of the comments.

More work needed - Will continue later.

The acceptance test fails on a timeout, I don't reproduce locally. Please review?

For now ignore the CircleCI errors and pay attention to the Jenkins builds,  Those looked like they passed.

Added javadoc to interfaces and classes as instructed.

What CLI options are you using on besu and your miner?  I keep getting
ETH: 11/07/19-16:20:58 - New job from <IP_ADDRESS>:8008
ETH - Total Speed: 177.893 Mh/s, Total Shares: 36, Rejected: 22, Time: 00:01
ETH: GPU0 177.893 Mh/s
ETH: Share rejected (15 ms)!
ETH: 11/07/19-16:21:00 - SHARE FOUND - (GPU 0)
Miner detected that you use local pool or local stratum proxy.
This mode is not currently supported and will cause more stale shares.
ETH: 11/07/19-16:21:00 - SHARE FOUND - (GPU 0)
Miner detected that you use local pool or local stratum proxy.
This mode is not currently supported and will cause more stale shares.
ETH: 11/07/19-16:21:00 - SHARE FOUND - (GPU 0)
Miner detected that you use local pool or local stratum proxy.
This mode is not currently supported and will cause more stale shares.
ETH: Share rejected (31 ms)!
ETH: Share rejected (16 ms)!
ETH: Share rejected (32 ms)!
ETH: Too many rejects from pool, disconnect...
ETH: - connection in 20 sec...

Is there a way to turn down how many shares a miner sends?  i.e. just the ones that prove a block?  Or can Besu be set to accept and not track shares? (or maybe track for hashrate calculations?)

Figured out two interacting problems.  (a) I was using claymore and (b) the fixed difficulty in our dev.json is configured for CPU mining.  A low fixed difficulty leads to many rejected shares.
I'll get my two system test done tomorrow.  There is no action needed on your end currently.

All the same, I owe you a bit of documentation. I admit I didn’t test claymore, there is always more to test and support. Reports of bugs are welcome.

On Nov 7, 2019, at 21:39, Danno Ferrin<EMAIL_ADDRESS>wrote:

Figured out two interacting problems. (a) I was using claymore and (b) the fixed difficulty in our dev.json is configured for CPU mining. A low fixed difficulty leads to many rejected shares.
I'll get my two system test done tomorrow. There is no action needed on your end currently.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Stratum 1 mining only ever solves one block.  Does stratum1 expect the server to proactively send out new solutions?

Field test worked.  Clearly my VR rig has the better GPU.

Can you add in the documentation what test rig you had?  Device, mining software, etc. Including CLI arguments of the miner and of Besu?
This should include parameters that exercise Stratum1EthProxyProtocol and Stratum1Protocol.  Does one correspond to the getwork: scheme and the other one to stratum+tcp:?
These demos should mine multiple blocks, or the protocol classes that are not exercised should be disabled/removed from the StratumServer
Providing the above in a PR comment is fine.
Merge the PR to chain head as well.
The I will merge unless you have any objections.

Where is the best place to contribute the new documentation on mining? In the mean time I'll add some info here.
I ran besu with: ./besu --network=dev --miner-enabled --miner-stratum-enabled --miner-coinbase=0xfe3b557e8fb62b89f4916b721be55ceb828dbd73 --rpc-http-cors-origins="all" --host-whitelist="*" --rpc-ws-enabled --rpc-http-enabled --data-path=/tmp/tmpDatadir
I ran ethminer 0.15 (the last verifiable version to work on Mac, as I was travelling, and could not get to my GPU mining rig). I ran it in 3 ways:
JSON-RPC calls (the getwork approach):
./ethminer -v 9 -G -P http://localhost:8545
stratum+tcp:
./ethminer -v 9 -G -P stratum+tcp://localhost:8008
stratum1+tcp:
./ethminer -v 9 -G -P stratum1+tcp://localhost:8008
Note the -v 9 flag is really useful for debugging, as ethminer will then print each network exchange.

Stratum 1 mining only ever solves one block. Does stratum1 expect the server to proactively send out new work? If so perhaps we should disable it in this release, open a new jira ticket, and either leave the code out or leave javadoc explaining what's not working.
ethminer -P stratum1+tcp://<host>:8008

To be clear, getwork: and stratum+tcp: appear to work fine and are sufficient for the bounty.

Alright, and I saw that too. I really hoped it was my old version of ethminer playing tricks on me.
That particular protocol seems to not be particularly well implemented by ethminer, and specifically there is a play on request IDs there that I traced. Initially, each request ID matches a specific exchange of information for ethminer. When the handshake is complete and the miner is ready for work, it adds 40 to request IDs and behaves differently for request IDs over 40.  However that's not what happens for the first exchange of work and solutions, and then the miner never asks for work again - which it should, on a timer, apparently.
There is more to it there, but I would now require to run my mining rig on Linux to debug further so I can run with the latest and greatest.
Happy to open a bug with my findings.

Hello @atoulme,
I'm trying to add support to iPollo miners on top of this work and I'll have to refactor part of this. While doing so I tested the current server using ethminer as you pointed in a comment above, but I couldn't see what case would use the HTTP detection in o.h.b.e.s.StratumConnection. Do you remember why that's there? Also the same would apply for the o.h.b.e.s.GetWorkProtocol, do you remember why are you wrapping the response in HTTP format?
Besides that, I'm thinking on flattening the protocols because the iPollo needs a similar subscription mecanism like the one you used in o.h.b.e.s.Stratum1Protocol, but this miner expects eth_getWork responses sent continously from the client after a eth_submitLogin. I'll also try to rely on o.h.b.e.a.j.i.m.EthGetWork, o.h.b.e.a.j.i.m.EthSubmitWork and o.h.b.e.a.j.i.m.EthSubmitHashRate instances to handle responses so we hopefully could remove some code duplication.

This was a while ago... The GetWorkProtocol is just exposing the JSON-RPC eth_getWork and eth_submitWork on this port. Not sure about StratumConnection. The TCP server has to deal with HTTP requests for JSON-RPC requests and the TCP protocols. So to keep it working, you have to deal with both.
Feel free to change the code however you like :)


move report file generation to library

json report generation moved to leapp.utils.report;
txt report is generated as part of leapp upgrade/
leapp preupgrade command;
txt report generation in VerifyCheckActor will be
made obsolete and removed in leapp-repository PR as
well as actor's library;
remove leapp report command (might be reverted).

Can one of the admins verify this patch?

Just wondering, should we merge this before #544 or after?

@drehak #544 should definitely be merged first imho


SDK usage - Important - Extension point for Jetty configuration

We wanted to introduce custom exception mappers in our JAX-RS controller. Other foreseeable use cases include middleware for authentication, payload management etc.
We would need an extension point in EDC to provide custom configurations for the Jetty web server. In embedded mode, it appears Jetty does not scan the classpath for @Provider annotations.

@Provider instances are already supported and can be registered with WebService, .cf HttpApiKeyAuthContainerRequestFilter for an example.


Engine startup time

Add within the logs the startup time for booting up the engine.

This should be done when there is enough going on in the startup of the engine.


data: PranavTripathi

Changes proposed

Added my profile to linkfree

Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[ ] My change requires changes to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Note to reviewers

Please take a look and let me know for any changes

Hey @srkds I've changed the name of the file, lemme know of there's anything else to change :)


Power Spectral Density Capability

Hello,
Would you please tell me if it would be possible for JDSP to have a new feature to calculate the power spectral density of a random signal (I've been told that it is inappropriate to do FFT on random signals)?  If it is a trivial thing that the user can do by just writing some loop and doing some simple math on the FFT results instead, please let me know that this is the case as I'd like to be able to implement power spectral density capabilities in my workflows to handle a more diverse set of signals.  Thanks.
LM

@LobsterMan123
Sorry for the late response. I think it is fairly simple to compute the spectral density from the FFT of a signal. This Matlab link provides a good explanation of how this can be done.


Consider adding SequenceableCollection#indexOf:startingAt:

This method was previously defined in the "Dialect Abstraction Layer" package, now removed, and looking at other methods similar to this seems that #indexOf:startingAt: is missing. (See indexOfAnyOf:, indexOfSubCollection: families)
indexOf: target startingAt: start
	"Answer the <integer> index of the first element of the receiver which is
	equal to the <Object> argument, target, within the receiver. If the receiver
	does not contain any elements equal to target, answer 0."

	^self nextIndexOf: target from: start to: self size

Yes, I'm surprised that is missing.