Rewrite approx

Based on https://github.com/dmlc/xgboost/pull/7183
This PR rewrites the approx tree method to use codebase from hist for better performance and prepare for categorical data support.
The rewrite has many benefits:

Support for both max_leaves and max_depth.
Support for grow_policy.
Support for mono constraint.
Support for feature weights.
Support for easier bin configuration (max_bin).
Support for categorical data.
Faster performance for most of the datasets. (many times faster)
Support for prediction cache.
Significantly better performance for external memory.
Unites the code base between approx and hist.

TODOs:

[x] Figure out how to handle MatchThreadsToNodes.
[x] Handle different sizes of batches.
[x] Handle empty partition gracefully.
[x] Add tests.
[x] Distributed training.
[x] Tests for build policies and related parameters (max_leave, max_depth).
[x] Fix task initialization.

Close https://github.com/dmlc/xgboost/issues/7244 .

Codecov Report

Merging #7214 (626ecc2) into master (3a4f51f) will decrease coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #7214      +/-   ##
==========================================
- Coverage   82.67%   82.63%   -0.05%
==========================================
  Files          13       13
  Lines        4017     4019       +2
==========================================
  Hits         3321     3321
- Misses        696      698       +2

Impacted Files
Coverage Δ

python-package/xgboost/data.py
69.74% <0.00%> (-0.28%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a4f51f...626ecc2. Read the comment docs.

The PR is pretty much ready other than some cleanups.  I won't merge it until 1.5 is made.

@ShvetsKS I did some quick tests with external memory:
The usability of external memory:

Current approx on the master branch: Unusable.  It iterates the data for every node.  Running for 40 minutes now with 40 GB of data, still not reaching the finish line of the first iteration.
Rewritten approx: Better, the bottleneck is in regenerating the histogram index page.
Hist: Not implemented yet.  But I think that's the best place to have external memory since it needs only reading the histogram index.
GPU Hist: I have tested a prototype implementation before but copying from disk to cpu then to gpu seems to be too much overhead.

I have 64 GB of memory, so the kernel has done lots of caching for the data.  Are you interested in implementing it for Hist?  This PR has prepared most of the foundational work.

@RAMitchell @hcho3 @ShvetsKS Could you please give a high-level review on this rewrite?  I will split up the PR for more detailed reviews, just want to know your thoughts about the idea of rewriting.

Close https://github.com/dmlc/xgboost/issues/7244 .

Optimal partitioning for categorical features is also implemented.

Since all parameters are supported, we can start to drop the experimental tag on parameter validation once this PR is fully merged.

max_bin: 256
sketch_eps: 0.0078125

AUC
AUC (rewrite)
Accuracy
Accuracy (rewrite)
MAE
MAE (rewrite)
Time
Time (rewrite)

airline
0.8083021582698621
0.8083021582679778
0.6653586256318165
0.6653586256318165

7452.502085362998
1946.756329375

bosch
0.7032129451430552
0.7064355238462672
0.8514382259767688
0.8494741288278775

249.46239526499994
183.57548371099983

covtype

0.8589967556775643
0.8574993760918392

155.14399087498896
85.87701512000058

epsilon
0.9258295171219089
0.9259069769846513
0.82366
0.82496

667.7028755839856
2270.314828159986

higgs
0.8224143371680539
0.8223955953797959
0.7132495454545454
0.7133218181818182

1277.4466711200075
347.96909062200575

year

6.341546058654785
6.340557098388672
35.443501809000736
39.575479084975086

@RAMitchell @hcho3 Thanks for all the reviews, I'm so excited that this can be merged.


Fix device templates

fixes nokia device templates which need an ip_version filter

Codecov Report
Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (f66084c) 23.55% compared to head (8e2e5ae) 23.58%.

Files
Patch %
Lines

src/django_peerctl/templating.py
44.44%
5 Missing :warning:

src/django_peerctl/models/peerctl.py
50.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@               Coverage Diff                @@
##           prep-release     #102      +/-   ##
================================================
+ Coverage         23.55%   23.58%   +0.03%
================================================
  Files                84       84
  Lines              4819     4829      +10
================================================
+ Hits               1135     1139       +4
- Misses             3684     3690       +6

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[WIP] Adding generic TRT plugin support

This PR enables generic plugin support for TFTRT, allowing custom TRT plugins to be included TRT conversion process, increasing TFTRT coverage.

@aaroey I created this for you to review. Please let me know what you think

@samikama thank you , can you please add some test cases.

@samikama thank you. @aaroey can you please review this PR . ?

@samikama Could you write a unit test for this or a python test? You might be able to take advantage of the cast plugin from https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/tf2tensorrt/plugin/plugin_cast.cu.cc

Can one of the admins verify this patch?

@samikama could you help to resolve the conflicts?

@samikama Could you please address the reviewer comments and resolve conflicts? Thanks!

@aaroey I was working on sorting out merge conflict and I noticed there were some changes made on of TRT plugins are initialized. Current changes, especially passing logger to plugin initialization function causing ownership or duplication problems. In current schema, caller of Converter::Create() owns the logger and it is passed down to plugins. However plugins library will require initialization without presence of converter which in turn require another copy of logger. I believe passing the logger to converter then to the plugins is not a good choice. How should we solve this?
Thanks,
Sami

As per private discussions I am closing this. If anybody would require this functionality, you can contact me.


chore: move to paths relative to monorepo root for new monorepo support

Description
Brings the project in line with the improved monorepo support.
Tests
All the demo sites should deploy as usual with the updated configurations and all checks are green.
Relevant links (GitHub issues, etc.) or a picture of cute animal

Closes #2258
Introducing the Enhanced Monorepo Experience on Netlify

I need to update the base paths in the Netlify UI. For the time being, everything should fail.


Search Config List is not available

In the "Import customized search configuration settings to a (...)" sections, the document states "On the Search Config List page, verify that:(...)".
That list is not available via the Browser - the redirect to [tenant]-admin.sharepoint.com/Lists/SearchConfig/AllItems.aspx results in a "File Not Found." error.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7a6d5348-0b97-842d-cc4d-e50c04bd8bd4
Version Independent ID: 1a9d4b68-d4cb-ff62-cd2d-32c0e7e73298
Content: Export and import customized search configuration settings - SharePoint in Microsoft 365
Content Source: SharePoint/SharePointOnline/export-and-import-search-settings.md
Service: sharepoint-online
GitHub Login: @SerdarSoysal
Microsoft Alias: serdars

Hello @merlinbrohl-adesso
I could follow the process and the redirect to
https://tenant-admin.sharepoint.com/Lists/SearchConfig/AllItems.aspx
didn't result in 404 for me.
If it is still not working for you I recommend to open a support request.
Thank you!

Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.

Sorry, I wasn't describing my issue correctly. This issue is sepcific to importing the search config to a tenant, not at any other scope (section https://learn.microsoft.com/en-us/sharepoint/export-and-import-search-settings#import-customized-search-configuration-settings-to-a-tenant).
The List "SearchConfig" exists and I can query it using e.g. the REST API. But the AllItems.aspx View seems to be nonexistent in the tenant admin site.
I could reproduce this in four different tenants, that I have access to.


feat: drop_duplicates for arrays

Pull Request type

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?
Issue number #172
What is the new behavior?

Added the drop_duplicates methods for SpanTraitExt & ArrayTraitExt,
Added the corresponding unit tests.

Does this introduce a breaking change?

[ ] Yes
[X] No

Other information
For this little snippet:
let shortest: Span<T> = if (self.len() <= 1) || (ret.len() <= self.len()) {
    ret.span()
} else {
    self
};

Without the self.len() <= 1 member some of my tests were failing.
Tried multiple things but couldn't find any better even though I feel like there's a more optimized solution... Curious to have your inputs! 🙏

Maybe it is best to keep the rust name (unique) as dev might be more used to that one?


supports the latest ElasticSearch?

ElasticSearch removed type in version 7.x, currently sharpplug. ElasticSearch component, supports the latest ElasticSearch?

I will support version 7.0.

I will support version 7.0.

Great, I'll wait for this support for Elastic Search 7.X!

Does this client already support 7.x?

我会支持7.0版。

好的，我会等待对弹性搜索7.X的这种支持！

Does this client already support 7.x?

Does this client already support 7.x?

Focus branch https://github.com/liangshiw/SharpPlugs/tree/feature/ElasticSearch7.X
I will improve it the next week.

I am considering redesigning the entire framework, but before that, I will upgrade the adaptation of es first.

我正在考虑重新设计整个框架，但在此之前，我将首先升级es的改编。

That would be great.

done
https://www.nuget.org/packages/SharpPlug.ElasticSearch/1.2.3


react-scripts trying to access .steampath

Describe the bug
If some module resolution fails, react-scripts try to stat all files in user's home directory. if there are some broken link no home folder, development server crashes.
Did you try recovering your dependencies?

Yes.
Which terms did you search for in User Guide?

.steampath
Related, and explained comment: https://github.com/facebook/create-react-app/issues/9156#issuecomment-645512859
Environment

Environment Info:

  current version of create-react-app: 3.4.1
  running from /home/rolivegab/.npm/_npx/171872/lib/node_modules/create-react-app

  System:
    OS: Linux 5.8 Fedora 32 (Workstation Edition) 32 (Workstation Edition)
    CPU: (12) x64 AMD Ryzen 5 2600 Six-Core Processor
  Binaries:
    Node: 14.7.0 - ~/.nvm/versions/node/v14.7.0/bin/node
    Yarn: 1.22.5 - /usr/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v14.7.0/bin/npm
  Browsers:
    Chrome: 86.0.4240.75
    Firefox: 81.0.1
  npmPackages:
    react: ^16.14.0 => 16.14.0
    react-dom: ^16.14.0 => 16.14.0
    react-scripts: 3.4.3 => 3.4.3
  npmGlobalPackages:
    create-react-app: Not Found

Steps to reproduce

Since it depends on machine's having a broken link on home directory, you need to create a broken file there.
My .steampath file is just a broken link:

npm start

force a module resolution failed, or something
output:

Expected behavior

react-scripts should not stop development server because a broken link in home directory.

Issue still persists.

IMHO, these scripts shouldn't be doing anything in my home directory period unless its in a specific, well-named directory specific to react scripts. What is the justification for this anyway?

IMHO, these scripts shouldn't be doing anything in my home directory period unless its in a specific, well-named directory specific to react scripts. What is the justification for this anyway?

I did a bit of digging.
The bug is caused by chokidar library not policing its errors when trying to resolve symlinks.
https://github.com/paulmillr/chokidar/blob/master/lib/fsevents-handler.js#L377
I'll create a bug report there.
I don't know why the script is trying to read outside of its project directory, I haven't dug that far.

I did a bit of digging.
The bug is caused by chokidar library not policing its errors when trying to resolve symlinks.
https://github.com/paulmillr/chokidar/blob/master/lib/fsevents-handler.js#L377
I'll create a bug report there.
I don't know why the script is trying to read outside of its project directory, I haven't dug that far.

Today i looked closer at why my react-scripts server is randomly crashing during development (only happens after multiple hours or days) and came across the same issue as op:
Compiling...
/home/alex/work/bla/blabla/myproject/node_modules/react-scripts/scripts/start.js:19
  throw err;
  ^

[Error: ENOENT: no such file or directory, stat '/home/alex/.steampath'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'stat',
  path: '/home/alex/.steampath'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: `npx rimraf node_modules && npm i && react-scripts start`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/alex/.npm/_logs/2021-01-23T08_43_37_732Z-debug.log

It's easy to fix with a simple restart, but what i don't understand is:
WHY THE HECK IS IT TRYING TO ACCESS MY STEAM STUFF OR STAT-ING MY PRIVATE HOME DIR AT ALL?
Does Facebook really need to know what's in my steam lib or how many hours i've killed in Hatoful Boyfriend?

Today i looked closer at why my react-scripts server is randomly crashing during development (only happens after multiple hours or days) and came across the same issue as op:
Compiling...
/home/alex/work/bla/blabla/myproject/node_modules/react-scripts/scripts/start.js:19
  throw err;
  ^

[Error: ENOENT: no such file or directory, stat '/home/alex/.steampath'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'stat',
  path: '/home/alex/.steampath'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: `npx rimraf node_modules && npm i && react-scripts start`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/alex/.npm/_logs/2021-01-23T08_43_37_732Z-debug.log

It's easy to fix with a simple restart, but what i don't understand is:
WHY THE HECK IS IT TRYING TO ACCESS MY STEAM STUFF OR STAT-ING MY PRIVATE HOME DIR AT ALL?
Does Facebook really need to know what's in my steam lib or how many hours i've killed in Hatoful Boyfriend?

Having same issue.
Compiling...
/home/erfan/Documents/GitLab/SurgAccord/surgaccord-landing/node_modules/react-scripts/scripts/start.js:19
  throw err;
  ^

[Error: ENOENT: no such file or directory, stat '/home/erfan/.steampath'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'stat',
  path: '/home/erfan/.steampath'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Only happens when a module is not found. The main problem is that the developement crashes and I should npm start again. Which is quite annoying.
Really don't know why it is behaving like that...

The bug has almost an year now(this isnt the first time it has been reported #9156 ).... Maybe some additional info might help... In my case the dev server crashes completely if I use relative paths (have .env file with  NODE_PATH=src) for imports like this:
import ComponentA from "myComponentsFolder/ComponentA"
import { ComponentB } from "myComponentsFolder/ComponentB"

But ComponentA.js doesn't exist inside myComponentsFolder, same thing happens if the folder doesn't exist and same thing happens if A exists and B doesn't, the dev server should throw a "ComponentA is missing", not completely crash...
As a workaround to actually find what I'm missing I'm running npm run-script build, but that's at least a minute lost everytime I'm missing a component or haven't even defined it yet....

I think I know the cause ..
hopefully not staying for long..
On Sun, 2 May 2021 at 20:59 Andrei Tudor @.***> wrote:

The bug has almost an year now(this isnt the first time it has been
reported #9156 https://github.com/facebook/create-react-app/issues/9156
).... Maybe some additional info might help... In my case the dev server
crashes completely if I use relative paths (have .env file with
NODE_PATH=src) for imports like this:
import ComponentA from "myComponentsFolder/ComponentA"import { ComponentB } from "myComponentsFolder/ComponentB"
But ComponentA.js doesn't exist inside myComponentsFolder, same thing
happens if the folder doesn't exist and same thing happens if A exists and
B doesn't, the dev server should throw a "ComponentA is missing", not
completely crash...
As a workaround to actually find what I'm missing I'm running npm
run-script build, but that's at least a minute lost everytime I'm missing
a component or haven't even defined it yet....
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/facebook/create-react-app/issues/9825#issuecomment-830846722,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AS3AD2FSCTMWTYKDKLL4CQLTLWHGVANCNFSM4SUP3O2Q
.

@alisscco Maybe also add as a feature something like -v / --verbose to react-scripts so it doesn't clear the console?

I'm getting this while calling npm start, is there any workaround?

I'm getting this same error, but I have no missing package. If I delete the broken symlinks the errors stops appearing but this is very awkward

The bug has almost an year now(this isnt the first time it has been reported #9156 ).... Maybe some additional info might help... In my case the dev server crashes completely if I use relative paths (have .env file with NODE_PATH=src) for imports like this:
import ComponentA from "myComponentsFolder/ComponentA"
import { ComponentB } from "myComponentsFolder/ComponentB"

But ComponentA.js doesn't exist inside myComponentsFolder, same thing happens if the folder doesn't exist and same thing happens if A exists and B doesn't, the dev server should throw a "ComponentA is missing", not completely crash...
As a workaround to actually find what I'm missing I'm running npm run-script build, but that's at least a minute lost everytime I'm missing a component or haven't even defined it yet....
Edit: Oh yeah, in my case I don't even see the module missing, just the error like erfoon has...
Edit2: It also crashes from some npm packages downloaded but not all, I don't know which crash and which don't...

One can use the debugger to pause at the message point!
You can check that here: https://stackoverflow.com/a/68370804/7668448

Today i looked closer at why my react-scripts server is randomly crashing during development (only happens after multiple hours or days) and came across the same issue as op:
Compiling...
/home/alex/work/bla/blabla/myproject/node_modules/react-scripts/scripts/start.js:19
  throw err;
  ^

[Error: ENOENT: no such file or directory, stat '/home/alex/.steampath'] {
  errno: -2,
  code: 'ENOENT',
  syscall: 'stat',
  path: '/home/alex/.steampath'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>dev: `npx rimraf node_modules && npm i && react-scripts start`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/alex/.npm/_logs/2021-01-23T08_43_37_732Z-debug.log

It's easy to fix with a simple restart, but what i don't understand is:
WHY THE HECK IS IT TRYING TO ACCESS MY STEAM STUFF OR STAT-ING MY PRIVATE HOME DIR AT ALL?
Does Facebook really need to know what's in my steam lib or how many hours i've killed in Hatoful Boyfriend?

It's just checking the home directory! And iterating through all files! Why ?? I duno! Didn't Find why yet!
Otherwise here the same error but with a broken symlink on my name

Suggestion for others who land here, I ran
npm run start &> /tmp/output

so I could see which package was missing. &> may be a bash-specific construct—it redirects both stdout and stderr.

Suggestion for others who land here, I ran
npm run start &> /tmp/output

so I could see which package was missing. &> may be a bash-specific construct—it redirects both stdout and stderr.

it worked. Thanks

This problem still existed.

+1

+1


[Feature] Allow to specify custom drivers path (for security reasons)

We use playwright to serve PDF files in an ASP.NET Web Application (currently .NET 4.8 with IIS but soon .NET core with IIS).
For security reasons all files of the Web Application are on a separate drive at P:\Projects\ProjectName\Website. On that drive Users have no access, just the App-Pool user can access files of the appropriate Project itself. This way all applications are isolated from each other. There were security bulletins in the last years highly recommending that approach.
In that setup Playwright fails, because node can't open files in P:\Projects\ProjectName\.playwright\, unless the App-Pool user has Read-Permission on both P:\ and P:\Projects. Otherwise it throws PlaywrightException: Process exited (details below). But this breaks the security aspects of isolating each application completely.
It's probably an issue of node which can't handle permissions just granted on a deeper level than on the root of a drive. Probably it's not on the top priorities of node to ever fix that.
Suggestion:
It would be valuable if the location of the .playwright directory could be specified in this case. Currently the .playwright directory must always be in the Application's directory. If it could be at a custom location, security can be configured as needed for node.
This is related to #2241 but affects both .NET 4.8 and ASP.NET Core. I could imagine that other projects will also benefit, if that .playwright directory could be in a custom path, for various reasons. #2241 would already solve that need.
Exception details (captured in transport.LogReceived):
node:internal/fs/utils:345
    throw err;
    ^

Error: EPERM: operation not permitted, lstat 'P:\'
    at Object.realpathSync (node:fs:2461:5)
    at toRealPath (node:internal/modules/cjs/loader:394:13)
    at Function.Module._findPath (node:internal/modules/cjs/loader:550:22)
    at resolveMainPath (node:internal/modules/run_main:19:25)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:71:24)
    at node:internal/main/run_main_module:17:47 {
  errno: -4048,
  syscall: 'lstat',
  code: 'EPERM',
  path: 'P:\\'
}

I'm hitting the same issue here. Do you guys have some hints on how to overcome this ?
I'd like to still benefit from the lib updates.

I'm looking for solutions to the same thing. +1


Update mod_chart_example to create chart

Closes #5
Added code to create a basic line chart for the first metric. Adam please can you review and let me know if any changes are required?

@philgodfrey, I just checked the latest master and this branch will need to be merged as at the moment, it shows an example chart! When you resolve the conflict (to show correct R file), it should render correctly.


Handle url encoding in query param match

Hi, first of all, thanks for the great library. We found an issue with query_param. According to the documentation, it's not necessary to encode the query value.

Sets a query parameter that needs to be provided.
Attention!: The request query keys and values are implicitly allowed, but is not required to be urlencoded! The value you pass here, however, must be in plain text (i.e. not encoded)!

In order to match, we need to replace spaces with +. I'd expect that this library can handle it for me so I don't need to care about encoding internals.
Reference: https://url.spec.whatwg.org/#url-parsing (spaceAsPlus)
let m = server.mock(|when, then|{
    when.query_param("query", "Metallica is cool".replace(" ", "+"));
    then.status(200);
});

let req = hyper::Request::builder()
    .method(Method::GET)
    .uri(
        Url::parse_with_params(
            "http://example.com/",
            &[
                ("query", "Metallica is cool")
            ],
        )
        .unwrap()
        .to_string(),
    )
    .header("content-type", "application/json")
    .body(Body::empty())
    .unwrap();

hi @StarpTech , thanks fur submitting this issue. Interesting. httpmock uses this library call to convert incoming query strings into key/value pairs: https://github.com/algesten/qstring/blob/master/src/lib.rs#L210-L214 . I think it doesn't handle that situation right as it only treats %20 as a space but not +. This is because qsting uses 'percent_decode', which in turn only handles "percent" values as per https://url.spec.whatwg.org/#percent-decode.
It seems to be an known open issue in qstring: https://github.com/algesten/qstring/issues/3

Hi @alexliesenfeld thanks for the detailed response.  We could switch to https://github.com/servo/rust-url.

Should be fixed in release v0.6.6.


Update Meshery docs with listing of NSM adapter (alpha)

Preemptively, let's ensure that the NSM adapter effort is listed under Supported Service Meshes here - https://meshery.layer5.io/docs/overview#supported-service-meshes.

Updating to include the correct link - https://meshery.layer5.io/docs/#supported-service-meshes.
@Harshini-M you'll find the Meshery Docs under the /docs folder in the Meshery repo. The CONTRIBUTING.md in the Meshery repos gives instructions as to how to update the docs.


Use shellcheck

I found a few bugs just by using shellcheck: https://github.com/numtide/nixos-remote/pull/8/commits/c5bf76c63efdd1043da8fd93ac7a341ba5cb3745
I think this is must for this kind of projects.

Fixed in master

32ba35c4ee7eaba2bf08507d69133b972e2bccbe


Ubuntu Package name changed

python-software-properties was renamed to software-properties-common
See https://askubuntu.com/questions/422975/e-package-python-software-properties-has-no-installation-candidate

Coverage remained the same at 60.841% when pulling b642c7279d336b846e283811439d11eb73acacc6 on GaryCarneiro:master into 2497f3fd4eb3ca95519debca3b66f4daa415b1bb on Netflix:master.

Thanks @GaryCarneiro for the PR. Just merged it.

No problem at all @hosseinsh.


Can not see Output results

I can not see any output result only red and green background appears. I submit a screenshot please check and give me a suggestion.

Please use the issue template next time to provide all the relevant information.
Do you see any error in the console?
Can you reproduce it with the browser extensions disabled? (Try to open the page in Private/Incognito window).

The issue resurfaces from time to time, recently reported again on Gitter:

hi guys, I am doing tasks on codewars and before everything was fine, the tests passed, but now there is some bug and when trying to pass the test in the "Output" window, just a gray screen appears, maybe someone has already encountered a similar problem, thanks in advance http://prntscr.com/u1hxgq

But still, no details on it.

if it helps to anyone - I disabled the browser extension "ADGUARD" and the tests started to display

My solution:
I started to delete extensions one by one installed on my browser(chrome). Each time I refreshed the page on the codewars website. That way I was able to find malfunctioning extension that meses up the website. I reinstalled all extensions back, except the broken one. All good so far.

So what extension caused this? Got the same problem on linux, deleted all my extensions, and it didn't help at all. Works fine on my windows laptop btw...

So what extension caused this? Got the same problem on linux, deleted all my extensions, and it didn't help at all. Works fine on my windows laptop btw...

I'm using win 11, cannot tell anything about linux. I don't remember the app's exact name but it was "VPN something". Like "FreeVPN", "RangeVPN", "GimmeADollarVPN"...

Deleted Urban VPN extension and all got good. Thank you! I wonder what is the problem between VPN extensions and codewars


Multiple ManyToOne relations with the same column

Is your feature request related to a problem? Please describe.
I have the class "Base" which holds the primary key and a class "A" which is OneToOne related to the "Base" class with FK as PK. In addition there is a class "B" which has a "ManyToOne" relation to "Base". I want to define a  "OneToMany" relation from "A" to "B".
The "node" of "A" and "node" of "B" are each inherited by another class. So I can't change the relation in B to A instead of B and either way i want both ways (from B to A and B to Base) to work.
It's a rather special feature, so just close this requests if it's out of scope. I'm currently using separate finds to get the data i need, which works but is rather ugly in comparison of using relations.
export class A {
  [PrimaryKeyType]: number
  [PrimaryKeyProp]: 'node'
  @OneToOne({ primary: true, onDelete: 'cascade', onUpdateIntegrity: 'cascade' })
  node!: IdentifiedReference<Base>

  @OneToMany(() => B, b => b.node)
  bs!: B[]
}

export class B  {

  @PrimaryKey()
  id!: integer

  [PrimaryKeyType]: number
  [PrimaryKeyProp]: 'node'
  @ManyToOne({ onUpdateIntegrity: 'cascade', onDelete: 'cascade' })
  node!: IdentifiedReference<Base>
}

export class Base {
  @PrimaryKey()
  id!: integer
}

Throws as expected:
MetadataError: A.node has unknown 'inversedBy' reference: Base.bs

Describe the solution you'd like
Some kind of way to create a readonly "ManyToOne" relation variant which uses the same column name as another "ManyToOne" relation, which then can be mapped normally by a "OneToMany" relation in the class.
@ExtendManyToOne({ original: 'node' })
a!: IdentifiedReference<A>

Sounds a bit like polymorphic relations (#330)? Keep in mind that this would mean you cannot use native FKs (aka forget about native cascade delete, and be prepared to maintain inconsistent data).
Or maybe a use case for STI?
In general it is not possible to point to a base class from a relation, you can only target leafs or you need to use STI (where you can target the base class that defines the discriminator column. Base classes (mapped superclasses) do not live during runtime, their metadata are inlined to the leaf entities.
Btw your B entity has dedicated id PK, but your are telling the ORM that the PK is node.

You are right, it is kind of a polymorphic relation. STI does not solve our problem in a good way, MTI probably would. Native FKs would work at least in some ways over the "Base" class, as if I delete A then it deletes the correspondent Base which deletes B. Thanks.

Btw your B entity has dedicated id PK, but your are telling the ORM that the PK is node.
Edited :+1:


[Feature] Add Zoom to Fit Waypoints Shortcut

Closes #349

Was wondering if a toast should actually be shown if there are no waypoints to zoom to?


Setuptools and pip manage vendored namespace incompatibly

In Debian we've been having an issue for some time where version comparisons where one comes from pip and the other comes from setuptools are incompatible [1].  Today @dstufft  helped me look into it and determined that the two tools have incompatible approaches to vendoring that are the source of this problem.  Based on that insight (and very specific help, again from @dstufft ), I came up with a hack to work-around the problem for now [2].
It's certainly not ideal, but I think it demonstrates the problem.  To quote him from IRC earlier today:

what pip does is basically loop over all of our vendored modules, manually import them, and then munge sys.modules so that like, pip._vendor.packaging.version is the same object you would get back when you do import packaging.version.
setuptools does it differently, they install a meta importer, that at the import level does the aliasing that effectively means that for pip, there is only one packaging module imported both at packaging and pip._vendor.packaging, the pip._vendor.packaging is just a tiny shim, whereas for setuptools packaging and pkg_resourcex.extern.packaging are two fully unique objects
it's my opinion this is an upstream bug, either in pip or setuptools (or likely both) and we're going to need to coordinate to ensure that both of our debundling strategies can co-exist, because they currently can't

Thanks for considering,
Scott K
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=912379
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=912379;filename=pip.versions.patch;msg=39

FWIW, pip used to do it the same way that setuptools did it here, and we switched away from doing it like 5 years ago. Unfortunately I don't have a lot of memory of why that was done, but that was done in https://github.com/pypa/pip/pull/2925.
I have the vaguest memory of the problem basically being that in cases where our vendored libraries had import errors on their own, we would hide those import errors, thus making it more difficult for end users to diagnose a broken install, even i they weren't actually using the bundling.
The way the de-bundling logic works now, is that it's entirely a no-op for everyone using pip, HOWEVER, if a downstream (such as debian) does debundle, they're expected to carry a small patch that effectively just toggles a constant from False to True, and then the debundling logic kicks in, which makes it impossible for the debundling logic to cause issues for installs that weren't debundled.
Given pip used to have a mechanism similiar to setuptools and we moved away from it, I think it makes sense to port pip's logic over into setuptools here-- but I'm not personally 100% tied to that if the setuptools devs feel strongly.
Tagging @pypa/pip-committers because this is a cross cutting issue that only really becomes a problem with the interaction between our two projects.

I don't actually understand the problem here. Why do these vendoring mechanisms need to be compatible? Is there a simple minimal example that demonstrates a problem it could cause?

They need to be compatible because we're both using packaging.version.Version and when you try to compare a packaging.version.Version that comes from inside of pip with one that comes from inside of setuptools (well pkg_resources) it fails, because it thinks they're two unrelated classes that you're trying to compare.
It works inside of pip natively, because pip's bundling of setuptools rewrites the imports inside of setuptools to pull from pip._vendor.packaging instead of pkg_resources.extern.packaging.
However, when both pip and setuptools are running in a debundled state pip's debundling logic means that import pip._vendor.packaging is actually the same object as import packaging (and if you look at like, __class__ and stuff on the objects inside of it, you'll se the qual name makes no mention of pip). Whereas setuptools views them as two entirely distinct namespaces (and actually, if pip adopted setuptools style debundling support actually, we'd have 3 distinct namespaces, so that wouldn't even work..).
So effectively, there are 2 real options:

setuptools adopts a pip style debundling logic, which means that when debundled packaging.version.Version, pip._vendor.packaging.version.Version, and pkg_resources.extern.packaging.version.Version will all be the exact same object (and that object will be packaging.version.Version.
Setuptools declares what pip is doing as unsupported, and pip has to go and find anyplace we comingle pip._vendor.packaging.version.Version and pkg_resources.extern.packaging.version.Version (really it's pkg_resources.SetuptoolsVersion) and likely "casting" any setuptools version to our copy of packaging's Version by doing like. version = pip._vendor.packaging.version.parse(str(dist.parsed_version)), which seems wasteful to me?

Also you can repro in any place where both pip and setuptools are debundled (Debian unstable docker container is what I'm using) with:
import pip._vendor.packaging.version as pv
import pip._vendor.pkg_resources.extern.packaging.version as sv

pv.parse('1.0') >= sv.parse('1.0')

Which gives an error like:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: '>=' not supported between instances of 'Version' and 'Version'

Hm, my inclination here is to suggest that we not support this sort of "debundling" configuration, and leave it to Debian or whoever wants to "debundle" to get it working. We barely have the resources to support the standard configuration here, so I'm not super in favor of supporting unusual configurations with regards to dependencies. That said, that might be tinged with a bit of bias because I really dislike the way Debian modifies packages to separate out python and python-pip, python-venv, etc.
On the other hand, from what I can we are not vendoring our dependencies in an attempt to pin them to a specific version for compatibility purposes - we're vendoring them because it's too hard to solve the bootstrapping problem (I believe @jaraco has more insight into this), so I suppose it isn't the worst thing in the world for us to make it easier for people who want to solve that bootstrapping problem a different way.
With regards to compatibility issues, I think the answer here may depend on why pip and setuptools are vendoring in the first place. If the answer is just bootstrapping and not pinning, then it maybe shouldn't be a problem for them to hit the same actual module objects (though mucking around with sys.modules sounds much more likely to cause problems than the approach setuptools is taking IMO), but if any of the packages in this cluster are vendoring because they need to pin specific versions and there is a version conflict, the "incompatible" solution that setuptools uses will solve that problem, whereas the "compatible" solution won't.
One way to cut the Gordian knot in this situation might be to change packaging.Version's comparison operations to depend on a protocol rather than isinstance checks, in which case the two different versions of packaging.Version can be compared without an issue, but I assume that this was just offered as an exemplar of the wider problem and not the major item of concern here.

I think the answer here may depend on why pip and setuptools are vendoring in the first place.

I do not think that in either case we are vendoring to pin version, just to solve the bootstrapping problem yea. At least that's true in pip's case, and my memory of setuptools is that they did it for the same reason but my memory could be faulty.

though mucking around with sys.modules sounds much more likely to cause problems than the approach setuptools is taking IMO

Well, pip took a very similar mechanism at first and ultimately changed it due to problems (which affected people who weren't even using a debundled configuration), and since then the only problem I've seen come out of it has been in it's interaction with setuptools debundling.
Trying to dig through the old issues, I suspect that the problems pip ran into wouldn't effect setuptools in it's current state, but that's mostly an accident of what packages setuptools is bundling. The fundamental problem that pip ran into, is the setuptools style shim, is that it couldn't determine the difference between an ImportError because the package wasn't bundled anymore and an ImportError because someone's Python wasn't functional for us. The specific issue pip ran into, was someone had a Python built without ssl support, so trying to import ssl rose an ImportError for _ssl, when we attempted to import pip._vendor.requests, we got an ImportError, so we swallowed that ImportError (as setuptools does as well), and moved on to trying to import from the unbundled package, which also got an ImportError, so we just rose an ImportError saying that pip._vendor.requests couldn't be imported, with no additional information.
All of that is true with setuptools style debundling, and importantly this error was happening to people who were not debundling, all it took to trigger this edge case was one of the vendored libraries throwing an ImportError for something it expected to exist outside of itself (typically the stdlib). The pip style approach is very specific in that it does not ever swallow an ImportError or replace it with something other than the actual expected ImportError-- infact it entirely a no-op for people who are not patching pip to remove the bundled dependencies, because the entire debundle logic is hidden behind a conditional that always evaluates to True, so for it to ever have an effect on end users, they would have to modify the installed package.
More importantly, I don't think two instances of the setuptools style debundling can work together as suggested by this issue, because the fundamental problem here is that we're getting a version from inside setuptools and one from not inside setuptools and those being seen as two entirely different objects, having pip move to setuptools style debundling wouldn't change that situation at all, it'd still be two entirely different classes as far as Python is concerned.

if any of the packages in this cluster are vendoring because they need to pin specific versions and there is a version conflict, the "incompatible" solution that setuptools uses will solve that problem, whereas the "compatible" solution won't.

I don't think this is a major concern, downstream already have to ensure that a single version works for all of the packages they make available since Python's import system doesn't allow multiple versions like that (and afaik, pip is the only project that gets special treatment here that has a distro effectively re-bundle to where they conceivably could do that so it wouldn't affect setuptools anyways). I suspect that, worst case scenario, if say setuptools depended on a version of a dependency that was just fundamentally incompatible with the rest of the single set of versions they make available in the archive, they would either patch setuptools to support the version they do provide OR simply not unbundle that particular dependency.
I don't think that matters though, because AFAIK none of us are actually using bundling to pin to specific versions due to incompatibility but only to solve the bootstrap problem (and for pip specifically, to ensure you can't get your system in a broken state where you pip uninstall packaging or something).

One way to cut the Gordian knot in this situation might be to change packaging.Version's comparison operations to depend on a protocol rather than isinstance checks, in which case the two different versions of packaging.Version can be compared without an issue, but I assume that this was just offered as an exemplar of the wider problem and not the major item of concern here.

As far as I know, this is the only place that this currently affects, though the problem could certainly exist elsewhere that we just haven't uncovered yet. Personally, since this only affects debundling, I wouldn't change the API of packaging at all, if setuptools ultimately decides they'll keep their vendoring solution, I would just adjust pip to ensure that anytime we get a Version object from setuptools, that we translate it to a "real" packaging.version.Version object by roundtripping it through str(). That seems making things harder though for something that only really matters in one specific case.

To add some context, pip vendoring mechanism is here while setuptools' one is here.

I think the answer here may depend on why pip and setuptools are vendoring in the first place.

(and for pip specifically, to ensure you can't get your system in a broken state where you pip uninstall packaging or something).

From my point of view, this is the main reason pip vendors its dependencies: to prevent the user from
easily breaking pip and getting stuck.
What would prevent setuptools to switch to "pip-style" versionning ?

What would prevent setuptools to switch to "pip-style" vendoring ?

A gentle nudge on this.

I’m not opposed to setuptools adopting pips technique. The motivation for the meta importer was to make it trivially easy for a downstream packager to debundle (install vendored packages, delete _vendor dir).
Another requirement I have is that the tool chains work on all major platforms and doesn’t depend on bash or make or other Windows-hostile features. The vendoring package services this need nicely.
I would be inclined to accept a PR that adapts setuptools bundling you use pips technique.

I’m also highly interested in removing vendoring in setuptools. It feels like to me there should be at most one tool in the python ecosystem that requires specialized bootstrapping. I’ve seen some signs that downstream packagers may be able to move to PEP 517 and pip to build setuptools from source for package manager. I’d love for this to become the standard op procedure so that setuptools could drop vendored dependencies altogether.

This is a duplicate of #1383
The problem isn't due to pip + setuptools, it's a general setuptools problem as I demoed there.
I also have vague memories that some other project using pkg_resources.parse_version() and packaging.version.Version() was broken for this reason, but I can't remember where this vague memory comes from. Given there are projects that try to be helpful and make their modulename.__version__ attribute be a Version instance out of the box (especially when they already depend on or assume setuptools is installed, whether for entry point plugins or because historically setup.py entry points imported pkg_resources, or even more excitingly, if they try importing packaging and fall back on importing pkg_resources.extern.packaging), it really is quite unsafe IMO to assume you know which kind of Version() it is across project boundaries.
This is why my bug report referenced pip a lot in the report, but also took pains to point out the comparison to using packaging.version.Version as a third point of reference.

Thanks @eli-schwartz. I'll close it as duplicate, but I'll flag this issue as having a lot of useful context.
FWIW, Setuptools is now discouraging the use of pkg_resources in almost every case in favor of importlib.{metadata,resources}. The __version__ could (and should) be serviced by importlib.metadata, although I've been discouraging use of this pattern and instead advising packages to remove this attribute in favor of having the consumer just call importlib.metadata.version() directly. In either case, use of packaging becomes explicit downstream (importlib.metadata doesn't provide any Version objects).


the learning of rapid is inconsist when seeds in rapid is set to the same seeds

hi daochen,
When the seed in Rapid is set to 123, we run  5 experiments  on Minigrid-MultiRoom-N1-S10  and find  the training process and results are different for each group. Under the same parameters,  we only disable the rapid and run 5 experiments on Minigrid-MultiRoom-N1-S10. We find  the training process and results are same for each group.
Do we need to set other parameters during training rapid, in order to keep the same results when  the seed in Rapid is set to the same seed ?
Our environment is ubuntu18.04.5 lts，python 3.7.11， anaconda 4.10.1

@mao-xu Hi, I have just tried seed 123. It seems that the results are the same across runs in my side. I also used ubuntu. I would suggest manually seeding numpy, tf, and numpy module to see whether it works in your side.

@mao-xu Thanks for bringing this up. I also observe similar results when running it agin. But unfortunately, I do not have a clue why it happens after checking the code. It seems that there is only one process and everything should be seeded.


make unawaited a generic function

I just recently hit a snag using unawaited that I believe could have been avoided were it generic.
Thoughts? Reservations?
/cc @davidmorgan @bwilkerson

What was the issue and how does this work around it?

Hey @natebosch.  The basic issue has to do with type  inference where the type of Future  is seen as dynamic which doesn't agree with an expectation (and reality) of a more specific type required inside the wrapped unawaited expression.
I'll  ping you with the internal issue (David has  seen it) -- there's more  context there...

Thanks for the context. I think this fix makes sense - it means that adding unawaited around an expression is less likely to change the inference within that expression.
The reason this helps is that before the Future<void> parameter type would cause inference on the expression for that argument to want to pick Future<void>. It would push it "down" into the argument. But with a generic of <T> that can't be filled in elsewhere, won't cause <void> to be preferred.
Here is an example:
Future<E> genericIsImportant<E>(E arg) async {
  print(E);
  return arg;
}

void unawaitedOld(Future<void> f) {}

void unawaitedNew<T>(Future<T> f) {}

void main() {
  unawaitedOld(genericIsImportant(1));
  unawaitedNew(genericIsImportant(1));
}

In genericIsImportant with no context the E gets inferred base on the argument. But when there is a context it pushes the E to it's argument. So this program prints void pushed down from the Future<void> by unawaitedOld, and then int inferred up from the 1 since unawaitedNew doesn't stop that inference from happening.
@leafpetersen may be able to point out if I've made any mistakes.

Yes, that's right.

Sounds good--I'll run it through internal global presubmit just to be on the safe side...

Thanks!


is #ddev-nodisplay still a thing for add-ons?

Is there an existing issue for this?

[X] I have searched the existing issues

Output of ddev debug test
No response
Expected Behavior
I searched the code and found one test but I think all this stanza behavior was removed from ddev, not sure if by accident of on purpose. So if it was removed on purpose, probably worth removing it altogether and fixing https://github.com/ddev/ddev-addon-template/blob/main/install.yaml
project-wide search of #ddev-nodisplay
Searching 5610 files for "ddev-nodisplay"

~/Sites/work/ddev/cmd/ddev/cmd/testdata/TestCmdGetComplex/recipe/install.yaml:
    3  pre_install_actions:
    4  - |
    5:   #ddev-nodisplay
    6    echo "This action should not have any output"
    7  - |

~/Sites/work/ddev/docs/content/users/extend/additional-services.md:
  102  In any stanza of `pre_install_actions` and `post_install_actions` you can:
  103
  104: * Use `#ddev-nodisplay` on a line to suppress any output.
  105  * Use `#ddev-description:<some description of what stanza is doing>` to instruct DDEV to output a description of the action it's taking.
  106

2 matches across 2 files

Actual Behavior
n/a
Steps To Reproduce
No response
Anything else?
No response

It's used extensively in ddev/ddev-platformsh
https://github.com/ddev/ddev-platformsh/blob/main/install.yaml

I am just saying it doesn't appear that there's code on ddev that does anything with it.

See https://github.com/search?q=repo%3Addev%2Fddev %23ddev-nodisplay&type=code

I found some of it was added on https://github.com/ddev/ddev/commit/80a2907fd708618ff5a511ce2723ae17b3bb6669
But that logic is not currently anywhere.

You're right of course.
I see it was removed in

https://github.com/ddev/ddev/pull//4431

(git log -S 'ddev-generated')
I think it turned out that the output was rarely useful, but I would think I would note it in the PR.


Move Withdrawn event to Vault and add balanceAfter params

Description
The Vault interface is the more logical place to emit the Withdrawn and Deposited events because the Vault is always the starting and final destination for fungible tokens, and provides us with more metadata to emit in the event.
This PR updates the FungibleToken standard to emit Withdrawn from FungibleToken.Vault.withdraw() instead of FungibleToken.Provider.withdraw(). It also adds balanceAfter parameters to both events to show the balance of the Vault after the operation has occurred.

For contributor use:

[x] Targeted PR against v2-standard branch
[x] Code follows the standards mentioned here.
[x] Re-reviewed Files changed in the Github PR explorer
[x] Added appropriate labels

@sisyphusSmiling Bjarte and I talked for a little about this and he is totally on board with it. He was the one who suggested it actually! We figure that if provider implementations want to surface information about their token movements, then they should define custom events to do that. I think it is safer to have both Deposit and Withdraw events show information about token movements in and out of the same place.

Doesn't this miss some events? if I just implement  provider or receiver, they will not emit events it seems.

Previously events where never emitted when you only implemented some of thr interfaces.
It will miss them in this case, but things like TokenForwarding already have a good events.
So for me the trade off is worth it. The guidelines just need to say that if you do this you have to emit your own events.


CSI (Container Storage Interface) driver for Kubernetes and other container orchestrators

Tell us more about this new feature.
Any plans to implement a CSI driver(container storage interface) that would allow a kubernetes cluster to mount an s3 bucket as a Persistent Volume?

Thank you for the feedback. We've heard this ask from a few customers and we're looking into it, but nothing to share right now.
That being said, I encourage you to explore if existing CSI drivers such as the CSI for Amazon FSx for Lustre may work for your use case if you haven't already. An FSx for Lustre filesystem can be linked to an Amazon S3 bucket, as documented here: https://docs.aws.amazon.com/fsx/latest/LustreGuide/create-dra-linked-data-repo.html

@dannycjones Can you provide any updates now that Mountpoint for S3 has been released for GA?  https://aws.amazon.com/blogs/aws/mountpoint-for-amazon-s3-generally-available-and-ready-for-production-workloads/

I have no update to share right now.
Our goal has been to get Mountpoint into customers' hands as a production product, which we've shared this week with the release of v1.0.0. We're still looking into how we can best support customers using mountpoint-s3 with Kubernetes, but there is no information for me to share at this time.

Hi, I'm devloping/operating an in-house ML platform on the top of EKS clusters and now providing "CSI for Amazon FSx for Lustre" as a high-throughput cache for users to read their S3 data.
And now, we can see mountpoint-s3 as a more cost effective alternative. So, we would be very happy if we will hear that mountpoint-s3 have a plan to support kubernetes CSI in the near future!!
Moreover, normally FUSE(filesystem in user namespace) require privilege operation. But we, as a platform provider, we don't want to give any privilege permission to user pods even to side-car containers.
So, we would expect mountpoint-s3

supports kubernetes CSI
with an appropriate privilege isolation

similar to https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver
In gcs-fuse-csi-driver,

fuse process runs in a sidecar container in a user pod and user pod does NOT require any privilege
but, only csi driver daemonset pods require privilege
ref: this is achieved via https://copyconstruct.medium.com/file-descriptor-transfer-over-unix-domain-sockets-dcbbf5b3b6ec

Thanks @dannycjones for the update.
I know it's hard to get a precise ETA, but it'll help users to invest in other alternatives while waiting for the CSI driver to have a roughly estimated target. Should we expect something for the re:Invent ? This year ? Q1 2024 ? H1 2024, later ? etc.

Thanks @dannycjones for the update.
I know it's hard to get a precise ETA, but it'll help users to invest in other alternatives while waiting for the CSI driver to have a roughly estimated target. Should we expect something for the re:Invent ? This year ? Q1 2024 ? H1 2024, later ? etc.

I get that, and absolutely see the value in delivering this feature for users on Kubernetes. I can't share an estimated date, but it is something we're actively working on!

Thank you, @dannycjones The mountpoint-s3 CSI Driver is set to be an excellent addition for EKS csutomers, particularly those running Data and ML workloads on Amazon EKS. I'm eagerly anticipating this feature and plan to conduct some benchmarks to demonstrate its capabilities through the Data on EKS (DoEKS) blueprints. 👍🏼

This issue can be solved by using https://github.com/pfnet-research/meta-fuse-csi-plugin

Very excited to say we just launched the new Mountpoint for Amazon S3 CSI driver: https://github.com/awslabs/mountpoint-s3-csi-driver. It's also available as an EKS add-on for easy installation in an EKS-managed Kubernetes cluster. Thanks again for all your feedback on Mountpoint!


Merge gz-fuel-tools8 ➡  main

➡  Forward port
Port gz-fuel-tools8  ➡  main
Branch comparison: https://github.com/gazebosim/gz-fuel-tools/compare/main...gz-fuel-tools8
Note to maintainers: Remember to Merge with commit (not squash-merge or rebase)

Cancelling windows build. It was waiting for close to 3 hours in the queue and it'll take another couple hours to build.


flaky tests in tutorial CI

I see the tutorial CI time out multiple times. It takes about one hour to run.
======================================================================

FAIL: test_tutorials.test_onnx_fine_tuning_gluon
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/work/mxnet/tests/tutorials/test_tutorials.py", line 158, in test_onnx_fine_tuning_gluon
    assert _test_tutorial_nb('onnx/fine_tuning_gluon')
AssertionError:
-------------------- >> begin captured stdout << ---------------------
Cell execution timed out

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_tutorials.test_onnx_inference_on_onnx_model
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/work/mxnet/tests/tutorials/test_tutorials.py", line 161, in test_onnx_inference_on_onnx_model
    assert _test_tutorial_nb('onnx/inference_on_onnx_model')
AssertionError:
-------------------- >> begin captured stdout << ---------------------
Cell execution timed out

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_tutorials.test_python_predict_image
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/work/mxnet/tests/tutorials/test_tutorials.py", line 173, in test_python_predict_image
    assert _test_tutorial_nb('python/predict_image')
AssertionError:
-------------------- >> begin captured stdout << ---------------------
Cell execution timed out

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 33 tests in 3242.108s

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10843/1/pipeline/

@ThomasDelteil

@zheng-da do you see multiple time out in this log or have you experienced that multiple time?
The fact that test_tutorials.test_python_predict_image and test_tutorials.test_onnx_inference_on_onnx_model time out is really strange. It is running simple inference code on CPU. The only obvious common denominator in these tests is the fact that they are downloading a sizable model. A possible reason I can think of for this time out is if the underlying instance is running out of disk space / bandwidth usage is very high ? @marcoabreu could that be possible ?

I should say two times myself. Once in my own PR and once in someone else PR.

I also encountered a connection error for test_onnx_fine_tuning_gluon after rebasing my PR to master branch.
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10104/39/pipeline#step-1512-log-901

@TaoLv could be related, someone was working on implementing a retry logic on mx.test_utils.download to avoid failed HTTP connections issues but can't remember who.

If slaves run out of disk space, they'll be automatically disabled and no more jobs are being enqueued. None the less, your test should be able to catch that. The network bandwidth should not be a problem, EC2 has a quite nice connection and our slaves are connected using 10Gbit/s

@ThomasDelteil it was @KellenSunderland who worked on it. He had to close his PR due to breaking API

@ThomasDelteil and @marcoabreu  I should bring that PR back to life.  Classic move of get some feedback, don't have time to address it, put the PR in the backlog.  I think making it backwards compatible shouldn't be an issue.

Failed again here: http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10938/1/pipeline/

Thanks @anirudh2290 , I have been distracted by other projects, will try to prioritize moving to smaller models this week.

@zheng-da a fix has been put in to reduce the size of the models from 500MB to <50MB for the tutorials. That should prevent this bug from happening in the future. Please consider closing for now.


branch method does not respect existing node aliases

app_role.includes.branch { privileges } generates incorrect alias in the query (result_includes)
while explicit aliasing works properly: app_role.includes(:i).branch { privileges }
Additional information which could be helpful if relevant to your issue:
Code example (inline, gist, or repo)
[11] pry(main)> >> app_role.includes.branch { privileges }
AppRole#includes#privileges
MATCH (approle18)
WHERE (ID(approle18) = {ID_approle18})
MATCH (approle18)-[rel1:`include`]->(node3:`AppRole`)
MATCH (node3)-[rel2:`privilege`]->(result_privileges:`Privilege`)
RETURN result_includes | {:ID_approle18=>18}
HTTP REQUEST: 20ms POST http://localhost:7474/db/data/transaction/commit (1 bytes)```

### Runtime information:

Neo4j database version:
`neo4j` gem version:
`neo4j-core` gem version:

I made PR #1402 which is going through CI which should address this

If it passes in the morning I'll release it as a patch

Released as 8.1.4


FEQ-2431 / chore: unit tests for payment methods

unit test for payment methods
advertiserpaymentmethods
createadvertiserpaymentmethods
updateadvertiserpaymentmethods
deleteadvertiserpaymentmethods

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

10 of 10 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 85.448%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
5323

Relevant Lines:
5785

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

10 of 10 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 85.448%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
5323

Relevant Lines:
5785

💛  - Coveralls


add constants in one file

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Files
Coverage Δ

ros/api/modules/recommendations.py
100.00% <100.00%> (ø)

ros/lib/config.py
51.26% <ø> (-0.41%)
:arrow_down:

ros/lib/constants.py
100.00% <100.00%> (ø)

ros/api/v1/hosts.py
86.07% <80.00%> (-0.12%)
:arrow_down:

ros/processor/insights_engine_consumer.py
70.37% <66.66%> (-0.28%)
:arrow_down:

:loudspeaker: Thoughts on this report? Let us know!.

@r14chandra Just reviewed your PR after the update. Thanks for considering suggested changes. Looks good to me ✅

\retest

/retest

Thanks @srdeotarse @upadhyeammit  for the review!


serial abstraction layer (sal)

this PR introduces a "serial abstraction layer" called NewScaleSerial, which represents a platform-agnostic abstraction of the serial pipe to the USBInterface (M3-USB-3.1-EP). the NewScaleSerial class exposes a class method get_instances() which returns the instances of a serial port which match the VID and PID of the newscale device; on linux this is a pyserial "Serial" object, on windows it's a USBXpressDevice. instances of NewScaleSerial expose methods like get_port_name, get_serial_number, set_baudrate, set_timeout, write, and readLine which call the appropriate subroutines depending on the platform. a NewScaleSerial instance can be used to instantiate a USBInterface object, and then the rest of the library works in the same way.
examples/ have been updated, and i added back_and_forth_3d.py as a simple, repeatable (no random numbers) alternative to multistage_test.py. all examples have now been tested on both windows and linux.

License Stuff
Does usbxpress.py have a license associated with it? I can't find a good source for where this file came from apart from the answer to this forum question. If there is no license, then we should probably reference their "Master Software License Agreement."
Since the code is present as source code, it looks like this file might qualify as open source code, except that the file itself is not tagged as such. See:

“Silicon Labs Open Source Code” means Software created by Silicon Labs and which is (a) delivered to Licensee in Source Code format, (b) is identified as open source code and (c) states that use of the software is subject to the terms of this Agreement.

In our case, I don't know about (b) and (c).
Is there a general license for code that appears in the forums?

appreciate the review!

licenses on usbxpress.py

great point, i think this may actually fall under section 4.2.5 of the MSLA. i have added a copyright header and license reference as of 0dd801d, let me know what you think.

adding native usb functionality, rather than replacing the existing COM port functionality.

ok, while i don't think we should spend too much effort on supporting cases that New Scale doesn't support (like aftermarket PID modification), this is simple enough to keep that i agree with you. check out c4d61d9; i did something a little different than what you suggest but it achieves the same result: i just extended the NewScaleSerial.get_instances to include checks for the modified PID (0xea60), and then the rest is identical to the pyserial library as we use it in linux.

hiding the get_instances calls that the user doesn't need to do if they know the serial number.

i think i disagree with this suggestion, because asking "what new scale devices are connected right now" is a very common use case (i'm calling this in parallax to populate a drop-down menu with the serial numbers of available stages).

I can't speak to adding a license to code where there is no license in it already, but this might be the best we can do for now.

check out https://github.com/AllenNeuralDynamics/python-newscale/commit/c4d61d93d2d3b282e676d799e7a4cd30cf6f2c5d

This looks great. That way we don't break linux functionality that we got for free.

i think i disagree with this suggestion, because asking "what new scale devices are connected right now" is a very common use case (i'm calling this in parallax to populate a drop-down menu with the serial numbers of available stages).

Whoop I meant the following.
Let's keep what you have, i.e:
devices = NewScaleSerial.get_instances()

But when it comes to instantiating a device, let's support both of these use cases:
my_xyz_stage = M3USBInterface(devices[0])
# OR
my_xyz_stage = M3USBInterfaces(serial_num=46109)

That way you can support:

drop-down menu to list out devices (good for interactive prototyping)
a way of instantiating the device if you already know the serial number (good for permanent setups with a fixed configuration)

To do the latter, we just push the NewScaleSerial instantiation into the class USBInterface's __init__, rather than pass in an existing one as a parameter.
The only other thing I might suggest is considering making get_instances a standalone function to this module. Here's a similar example from another package. Up to you on this one though!

ok, i pushed a couple of changes: 0dd801d adds copyright and license, and c4d61d9 adds support for the provisioned devices.
my_xyz_stage = M3USBInterface(devices[0])
are you aware that M3USBInterface is not used anywhere else in the library? it seems to only add setting/getting of baud rate, which i am currently handling at the NewscaleSerial level. i feel like we should either (a) remove these methods/class entirely or (b) move them into USBSerialInterface as a passthrough to NewscaleSerial. what do you think?
finally, i took a stab at implementing instantiation given a known serial number, but wasn't able to come up with anything i was happy with. maybe you could try and propose something?


Feature Request: Media committed with Post / Collection Item

When running in publish_mode: editorial_workflow the current setup is to create a PR for the post / collection item but upload media directly to master ( or the branch specified in backend)
This means that images are automatically committed, while the collection item needs approval.
We would like to see, by default, when an image is uploaded to the media gallery it goes to the same PR as the collection item to be posted.
This way both the collection item and any related media are contained in a single PR and can be reviewed and approved.

Completely agree! I had not noticed this yet, but I'm days away from a use case where it would have come up.
One of the things I hate about CMS media galleries I've used in the past is the clutter and thumbnail duplication that happens when unused/abandoned images sit in the filepicker window!

I created a very naive implementation for github.
Not useful to merge in here but might be useful for anyone who needs a quick implementation
https://github.com/Encrypt-S/netlify-cms/commit/a86b5e87655dfcb60b3ae743219a614dca034157
https://www.npmjs.com/package/@encrypt-s/netlify-cms

Awesome - thanks for submitting this!
In our use case, we'd like to open editing to the general public and have configured our site so that only admins can merge to master. Currently, users logged into Netlify CMS can edit content but cannot upload images because they are not allowed to commit directly to master.

Currently, users logged into Netlify CMS can edit content but cannot upload images because they are not allowed to commit directly to master.

@rdebeasi That's a very good point, we'll definitely need to consider.
cc/ @Benaiah

Not sure how that's possible, aren't write permissions required for successful login?

Ah, protected branch, so the user does have write permissions, nevermind.
The closest solution is to use an external media library, a feature I expect to be merged in the near future. The first integration is Uploadcare and they have a great free tier.

@erquhart I think this is still worth discussing -- would you mind giving the reasons you decided to commit to master instead of the PR when you first worked on the media library?

The reason media commits to master is because we wanted assets to be reusable between entries rather than being tied to them. If you had an oft used image you had to have access to it somewhere physically so you could upload it again every time you needed it in an entry. What we now see is that there's a need for both approaches.


Fixed directory path, grammar, and formatting.

@erjohnso, please review.

lgtm, thanks @mbrukman!


Using Components decoupled from layout/styles

Hey there, I'm not entirely sure if I'm asking in the right spot, but I'm a bit lost on where exactly my question should go. I want to use the components within my markdown files, while also completely eliminating the page UI and layouts that docus.dev offers, but keeping the component styles. I want a completely custom layout. Is that something that can be done?

Hello,
You can use @nuxt/content, @nuxt-themes/typograpy and @nuxt-themes/elements to create your own Docus with the same elements and typographie style.
Take a look at the source code of Docus to understand how it's built!


Failed to start zigbee: RaspBee/Conbee with homeassistant OS (Supervised) on Raspberry 4 (2GB)

Description of problem:
Cannot start zigbee2mqtt with desden-elektornik's RaspBee on homeassistant OS (Superviced)
Zigbee2mqtt add-on version (if edge, please report commit hash):
Current version: 1.17.1-4

Your entire configuration from the frontend (with sensitive fields redacted):
data_path: /config/zigbee2mqtt
external_converters: []
devices: devices.yaml
groups: groups.yaml
homeassistant: true
permit_join: false
mqtt:
  base_topic: zigbee2mqtt
  server: 'mqtt://homeassistant'
  user: replaced
  password: replaced
serial:
  port: /dev/ttyAMA0
  adapter: deconz
advanced:
  log_level: warn
  pan_id: 6754
  channel: 11
  network_key:
    - 1
    - 3
    - 5
    - 7
    - 9
    - 11
    - 13
    - 15
    - 0
    - 2
    - 4
    - 6
    - 8
    - 10
    - 12
    - 13
  availability_blocklist: []
  availability_passlist: []
device_options: {}
blocklist: []
passlist: []
queue: {}
frontend:
  port: 8099
experimental: {}
socat:
  enabled: false
  master: 'pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777'
  slave: 'tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5'
  options: '-d -d'
  log: false
zigbee_herdsman_debug: true

Running platform (details from Suypervisor > System)
Core:
Version core-2021.2.3
Newest Version core-2021.2.3

Supervisor:
Version supervisor-2021.02.9
Newest Version supervisor-2021.02.9

Host:
IP Address <IP_ADDRESS>/24
Operating System Home Assistant OS 5.11

Your logs from Home Assistant Supervisor
[15:47:04] INFO: Zigbee Herdsman debug logging enabled
[15:47:05] INFO: Handing over control to Zigbee2mqtt Core ...
><EMAIL_ADDRESS>start /app
> node index.js
2021-02-18T14:47:09.131Z zigbee-herdsman:adapter Failed to validate path: 'Error: spawn udevadm ENOENT'
2021-02-18T14:47:09.139Z zigbee-herdsman:deconz:driver Set delay to 300
CREATED DECONZ ADAPTER
2021-02-18T14:47:09.144Z zigbee-herdsman:controller:log Starting with options '{"network":{"networkKeyDistribute":false,"networkKey":[1,3,5,7,9,11,13,15,0,2,4,6,8,10,12,13],"panID":6754,"extendedPanID":[221,221,221,221,221,221,221,221],"channelList":[11]},"serialPort":{"path":"/dev/ttyAMA0","adapter":"deconz"},"databasePath":"/config/zigbee2mqtt/database.db","databaseBackupPath":"/config/zigbee2mqtt/database.db.backup","backupPath":"/config/zigbee2mqtt/coordinator_backup.json","adapter":{"concurrent":null,"delay":null}}'
2021-02-18T14:47:09.146Z zigbee-herdsman:deconz:driver Opening with /dev/ttyAMA0
2021-02-18T14:47:09.156Z zigbee-herdsman:deconz:driver Serialport opened
2021-02-18T14:47:09.159Z zigbee-herdsman:controller:log Started with result 'resumed'
2021-02-18T14:47:09.159Z zigbee-herdsman:controller:log Injected database: true, adapter: true
2021-02-18T14:47:09.240Z zigbee-herdsman:deconz:driver send read parameter request from queue. seqNr: 1 paramId: 1
2021-02-18T14:47:12.144Z zigbee-herdsman:deconz:adapter checking coordinator simple descriptor
2021-02-18T14:47:12.151Z zigbee-herdsman:deconz:driver DATA_REQUEST - destAddr: 0x0 EP:0 SeqNr. 2 request id: 1
2021-02-18T14:47:20.139Z zigbee-herdsman:deconz:driver Timeout for request - CMD: 0xa seqNr: 1
Zigbee2MQTT:error 2021-02-18 15:47:20: Error while starting zigbee-herdsman
Zigbee2MQTT:error 2021-02-18 15:47:20: Failed to start zigbee
Zigbee2MQTT:error 2021-02-18 15:47:20: Exiting...
Zigbee2MQTT:error 2021-02-18 15:47:20: undefined
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>start: `node index.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-02-18T14_47_20_216Z-debug.log

Try to replace your port with
port: /dev/serial/by-id/usb-dresden_elektronik_ingenieurtechnik_GmbH_ConBee_II_DE2412023-if00
Verify your path with the log Supervisor -> Host -> Hardware

Below there is an extract from the hardware output. I think it shows the right thing. I don't have anything like "/dev/serial/by-id/..."
As the RaspBee is directly attached to the GPIO Pins, I wonder if its possible to configure UART somehow as I have to do with Raspberry PI OS as mentioned here deconz-docker-instructions. But I don't see how to do this with Home-Assistant OS 5.
Extract from hardware:
 - name: ttyAMA0
    sysfs: /sys/devices/platform/soc/fe201000.serial/tty/ttyAMA0
    dev_path: /dev/ttyAMA0
    subsystem: tty
    by_id: null
    attributes:
      DEVLINKS: /dev/serial1
      DEVNAME: /dev/ttyAMA0
      DEVPATH: /devices/platform/soc/fe201000.serial/tty/ttyAMA0
      MAJOR: '204'
      MINOR: '64'
      SUBSYSTEM: tty
      TAGS: ':systemd:'
      USEC_INITIALIZED: '2568292'

Well sometimes talking to others of the same domain just helps.
Before opening this issue I ran HA with an USB CC2531 Chip with antanna but due to poor performance I now try to replace the adapter under the hood with RaspBee which I own already. Nevertheless I use the same RPi with the same HA OS installation. That's the situation. As mentioned above simply removing the cc2531 USB Stick and plugin the RaspBee into GPIO didn't work.
Additionally I had to remove the microSD card, mount it in another machine with Linux. There, mount the first partition (FAT32) from the micoSD and edit the "config.txt". Add the line: dtoverlay=disable-bt in order to disable Bluetooth. Save + put it back into RPi and start. Now everything works as expected. Opposed to the referenced issue below (they have port: /dev/ttyS0), serial is still set to and it works:
serial:
  port: /dev/ttyAMA0
  adapter: deconz

This issue helped a lot https://github.com/home-assistant/operating-system/issues/1006.
Thank you guys for reading and helping.


Missing bindings of methods of base classes

It seems dartpy (or Chimera) doesn't generate bindings for methods of parent classes. For example, I couldn't call Joint::hasPositionLimit() or Joint::setPositionLimitEnforced() from a pointer to a concrete joint class (see this code).
I don't have the exact error message at this moment, but it was like "RevoluteJoint doesn't have hasPositionLimit method". Let me update this with the actual error message whenever available.

Resolved by #61


Can/Does this use the Chrome selenium-stealth add-in?

Hello,
The new Cloudflare turnstile makes it very hard to web scrape from Cloudflare sites.
I found this on stackoverflow which refers to a package which prevents cloudflare from detecting headless mode in the answer, and I wondered if you had implemented it.
Thanks!
B.

WTF MAN????   Please open the issue somewhere else where I'll IGNORE you ???

Thanks for the fulsome explanation!


I recommend to not use the word count example

This couples this lesson to the reproducibility lesson: https://github.com/coderefinery/reproducible-research/issues/206
I think the coupling is unnecessary and harms more than it helps.

Original motivation was to have continuity and let learners add documentation to a familiar example from an earlier lesson.
However, these are the downsides:

some learners only join for a particular lesson and this is then not familiar
word count example is very python-focused
it contains files that are not relevant to documentation

I believe we can make a strong point with a simpler example.

Good idea. Changes has been implemented accordingly in https://github.com/coderefinery/documentation/pull/258 & https://github.com/coderefinery/documentation/commit/8a3edf065a276a7cd857f539e634126fb689866e .


Feature request: Display JWT Header when decoding

Is your feature request related to a problem? Please describe.
When decoding JWTs only the payload is displayed. To view the header and the signature I have to use other tools.
Describe the solution you'd like
It would be great to see both the header and signatures block decoded in the CyberChef interface, or at least the header.
Describe alternatives you've considered
N/A
Additional context
N/A

for the header you can use From Base64


githubauth - support request signing by Cloud KMS

Currently GitHub requests are signed by a private key we store in an environment variable and pass into githubauth.App to sign the AppToken. This PR adds support for signing the AppToken with a private key stored in Cloud KMS instead, so we never have to handle the private key directly.

Are we actually able to import GitHub's private key into KMS? Cloud KMS only permit a submit of key format and algorithms.
I'm not convinced we want to make this API call to Cloud KMS in the hot path?

Cloud KMS does support 2048 bit RSA - PKCS#1 v1.5 padding - SHA256 Digest which I believe is the format of the GitHub private key.
Talked with Brad and Mike about this. They said you have concerns about a possible DDoS attack that causes us to hit KMS quota for minty on public repos, but Mike suggested only using KMS signing for alphabet enterprise orgs to mitigate that risk. We could use the enterprise_id from the GitHub token to secure the minty endpoint for only alphabet enterprise orgs. The relevant changes to the WIF pool and app configuration would be done in the minty repo. Does this sound reasonable?


Players still showing after disconnection from resource pack forcing

When joining from a Server without a forced resource pack to a server with forced resourcepack and rejecting the pack players still show in tab as online

Hi, could you open a ticket on discord please?


Support non-square matrix for lu decomposition

What do these changes do?

Try to let lu decomposition support non-square matrix.
An issue is that because we append zeros on axis 0 or 1, the SolveTriangular op which generated in tiling of lu could oftenly encounter  the error: LinAlgError: singular matrix: resolution failed at diagonal 0  which is quite annoying. Thus I try to use np.linalg.lstsq to address the problem, but the effort is enabled only for the SolveTriangular generated by lu.
Related issue number

Resolves #380

@NJUPole Please check if the logic in execution of solve_triangular is reasonable.


move (un)boxMessage- methods to separate struct

@patrickxb like this :-) Does this look good to you?


Fix datatype will appear in function.json even it is not set

Fix datatype will appear in function.json even it is not set, fix issue #466

Codecov Report

Merging #477 into develop will increase coverage by 0.02%.
The diff coverage is 66.66%.

@@             Coverage Diff             @@
##           develop     #477      +/-   ##
===========================================
+ Coverage    45.68%   45.71%   +0.02%
===========================================
  Files           69       69
  Lines         2377     2380       +3
  Branches       265      265
===========================================
+ Hits          1086     1088       +2
- Misses        1225     1226       +1
  Partials        66       66

Impacted Files
Coverage Δ

...azure/maven/function/bindings/CosmosDBBinding.java
88.04% <66.66%> (-0.73%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update db48d61...baea440. Read the comment docs.


Connect session with respect database

When a user is already registered o logged, the database is connected with the respetive user.

➤ Paul Barajas commented:
This issue is not part of the project.


URL Page Routing

Adding url-based routing to the CVManager

Created sub-paths for admin pages, sub-pages, and add/edit sub-sub-pages
Added 404 page for invalid routes
Embedded emails/IPs for admin editing in URLs

I did a bunch of testing, and the changes look good. Each of the edit pages are now protected from invalid values

I'm not seeing the<EMAIL_ADDRESS>user when I go to review users. Not sure if this is intended since I logged in with this user? Manually going to the edit page for this user also fails.

I am not able to reproduce this. I see the<EMAIL_ADDRESS>user in the users page... Thinking this may be an issue with the api not returning the right data?

This seems to work well for me. Why do we have a RSU Map page if it isn't used though?

Agreed, can we drop the RSU Map page?

@drewjj The RSU map page has been removed, and I fixed the final few bugs on the edit organization page. To my knowledge, that resolves all known issues!


CurseForge removal question

https://www.curseforge.com/minecraft/mc-mods/redstone-gates
Why mod was removed from CurseForge? Is it cancelled?

Project is now online again.


port of DIRT to Windows?

Has anyone attempted to port to Windows?  I am unfamiliar with  the build/linking questions on windows, since this libtensorflow_framework.so (DLL) seems to be missing.

where you able to run it on windows?

Has anyone attempted to port to Windows? I am unfamiliar with the build/linking questions on windows, since this libtensorflow_framework.so (DLL) seems to be missing.

hello，are you successful to run it on Windows？ I want to run it on Win10, but I don't konw how to do. Could you help me?


ripgrep: 0.8.0 -> 0.8.1

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build ripgrep

Success on x86_64-linux (full log)
Partial log (click to expand)
post-installation fixup
moving /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1/man to /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1/share/man
shrinking RPATHs of ELF executables and libraries in /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1
shrinking /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1/bin/rg
gzipping man pages under /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1/share/man/
strip is /nix/store/adidfx4pa7vmvby0gjqqmiwg2x49yr27-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1/bin
patching script interpreter paths in /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1
checking for references to /tmp/nix-build-ripgrep-0.8.1.drv-0 in /nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1...
/nix/store/3wa9rsfyjihylrqrayz8gpl607dx29z5-ripgrep-0.8.1

Failure on aarch64-linux (full log)
Partial log (click to expand)
cannot build derivation '/nix/store/d6adqr78k0r3dkbwdw1aw2dvmg0qpbv6-rust_serde_derive-1.0.18.drv': 6 dependencies couldn't be built
cannot build derivation '/nix/store/1srab01w45mf8sq84v490z7g2x393240-rust_crates-io-0.11.0.drv': 30 dependencies couldn't be built
cannot build derivation '/nix/store/ysgr8mc5x5jn23zlx8526zrywdhymqms-rust_docopt-0.8.1.drv': 19 dependencies couldn't be built
cannot build derivation '/nix/store/k45a8czj6hhkp3gdfzf9msn6wczy9z36-rust_git2-curl-0.7.0.drv': 20 dependencies couldn't be built
cannot build derivation '/nix/store/f94xyc6glqgzmy3pq9rjrcg2j6arjimd-rust_ignore-0.2.2.drv': 18 dependencies couldn't be built
cannot build derivation '/nix/store/31a3lv5912r9z23snbq8zry9cqn0jji4-rust_cargo-0.22.0.drv': 78 dependencies couldn't be built
cannot build derivation '/nix/store/l336s50mcqp3qabawxwdy4rsk5p6hmhc-rust_cargo-vendor-0.1.13.drv': 79 dependencies couldn't be built
cannot build derivation '/nix/store/jks849hh7g4lfqdx91d7wzrvah2pgcdl-ripgrep-0.8.1-vendor.drv': 2 dependencies couldn't be built
cannot build derivation '/nix/store/k48gzj6wp82q7z11mpa4dnrab2xazx0c-ripgrep-0.8.1.drv': 3 dependencies couldn't be built
error: build of '/nix/store/k48gzj6wp82q7z11mpa4dnrab2xazx0c-ripgrep-0.8.1.drv' failed


Difficulties Unzipping Downloads

Dear authors,
in contrast to another dataset that was released by you and provided through tum cloud (So2Sat), I face some diffculties downloading the SS4LEO-S12 dataset from https://dataserv.ub.tum.de/index.php/s/m1660427.001?path=%2Fssl4eo-s12.
After wget the download link for s2_l2a.tar.gz, I receive a very big file of 239GB, but neither unzip or tar work for this downloaded file. Could you double check if the file might be corrupted? Or, instead, could provide me with a working link from which I could download the dataset per wget?
Best,
Tom

Hi Tom, thanks for reporting. What was your error message when unzipping? The s2_l2a.tar.gz should be around 500GB (1TB extracted with 14M files). Could it be the downloading was interrupted? What about trying rsync?
Optionally try out our compressed 8-bit version at https://mediatum.ub.tum.de/1702379? This is only around 50GB and should be easier to try.

Hi, I actually now managed to download it without any data loss. It works now. But thanks for the quick answer, anyways!


preserve zoom level on details open

Related to #34
On details open, preserve the zoom level, to prevent the map from zooming back.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@tangollama Can you review this pull request, please?


Support for Markdown Tables

In general, confluencebuilder exports Markdown input which has been parsed by the recommonmark or the myst_parser extension just fine. Unfortunately, tables are not supported (recommonmark would support either plain HTML tables or the pipetables via sphinx_markdown_tables).
I understand that those extensions are not part of sphinx.ext and therefore not officially supported, but it seems to be possible to support, as all that recommonmark or myst_parser do is map to docUtils format in the end, and all other Markdown features I've used so far with both parsers are supported.
The following is an example for a Markdown table, showing which combinations of input and output work for tables:

reStructuredText
recommonmark (md)
MyST (md)

HTML
yes
yes
yes

Confluence
yes
no
no

(note: the following was only looked at using recommonmark)
Honestly, this extension (at least, from a development aspect) has never really looked at support with respect to Sphinx's existing support with Markdown. Any capabilities working inside the confluencebuilder extension now works by chance.
Taking a look at the table support via the sphinx_markdown_tables extension, it appears the process is as follows:

Hook onto a source-read event to look for detected tables (via a Markdown table extension).
Replace the contents of the Markdown table block with raw HTML.
Replace any processed source with the new one before running the source through a builder/translator.

This works for Sphinx's html/singlehtml builders since it looks like Sphinx will detect the HTML content in the Markdown file and wrap it in a raw node type flagged as html content. When html/singlehtml translators process these nodes, it will just dump them in the generated documents.
The reason why these tables are ignored by the confluencebuilder extension is that when using storage format translator (i.e. when attempting to translate a document set into Confluence storage format compatible output), the raw type identifier is not set to a confluence_storage value. Supported storage format markup is not completely compatible with HTML/HTML5/etc. markup, and this extension ensures the raw types do not mix by explicitly requiring Confluence-marked raw type identifiers to be set (which allows users to customize unique output between HTML and confluence builders, if desired).
There can be an agreement that it is annoying since the generated output partially works on a Confluence instance if the raw HTML types are permitted (with some implementation tweaks). One could hack a way to permit the inclusion of Markdown tables by using the following inside a conf.py file:
from docutils import nodes
from sphinxcontrib.confluencebuilder.translator.storage import ConfluenceStorageFormatTranslator

def override_raw_hook(self, node):
    if 'html' in node.get('format', '').split():
        node['format'] += ' confluence_storage'
        ConfluenceStorageFormatTranslator.visit_raw(self, node)
    raise nodes.SkipNode

def setup(app):
    app.add_node(nodes.raw,
        confluence=(override_raw_hook, None),
        override=True)

The above may work for some cases, but there will most likely be limitations (e.g. lack of alignment) and complications for users with some HTML-specific customizations that may not be desired to translated for a Confluence instance.
A more proper change, if it were to be ever integrated into this extension, would appear to be to somehow pre-process table blocks into table entries which are compatible for Sphinx and let builders/translators operate on node entries as they normally do. This may not even be specific to the confluencebuilder extension -- if there was a generic extension which managed converting a Markdown table extension block into docutils/Sphinx table nodes before processing through a translator, any non-standard builder/translator of Sphinx could take advantage of rendering these tables.

In this extension's v1.5 release, an undocumented configuration option confluence_adv_permit_raw_html was added to permit the automatic injection of raw HTML nodes onto a Confluence page. For example:
confluence_adv_permit_raw_html = True

Note that this is an advanced and limited supported option -- there is no guarantee that any injected HTML content will render properly on a published document, without getting into scenarios where Confluence may reject the document from being published. At this time, it is not recommended to rely on this option for long-term use.

I can confirm that it works with sphinx > 4, myst-parser==0.15.2 and sphinxcontrib-confluencebuilder==1.6.
Thanks!


coqPackages.VST: also build and install floyd.io_events

Motivation for this change
Fixes #131203
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Toplevel input, characters 0-35:
> Require Import VST.floyd.io_events.
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Cannot load ExtLib.Core.Any: no physical path bound to ExtLib.Core

Yeah. This depends on ITree.

Then it should be propagated,

Then it should be put in propagatedBuildInputs.

Really? It’s just for a single module, isn’t it?

Thanks, that works. Actually there are a few more ones that are not compiled:

ASTsize.v
measure.v
powerlater.v
printf.v
quickprogram.v
real_forward.v
rep.v

Can you please clarify what you need? Some of those files (or their dependencies) won’t build:

File "./floyd/measure.v", line 2, characters 23-24:
Error: The reference Z was not found in the current environment.

File "./veric/ghost.v", line 97, characters 9-17:
Error: The reference AV.valid was not found in the current environment.

I think that this is not a packaging issue: the proposed fix modifies the build system. You may want to contact upstream so that they can help you to get what you want.

Ah, ok. What I need at least is printf but I saw some other uncompiled files, which I thought were mistakenly left out from the Makefile.

I’ve added to this all the files that you mentioned and that do not cause a build failure.

I'll open an issue upstream as well.

Result of nixpkgs-review pr 131828 run on x86_64-linux 1

1 package built:

coqPackages.VST

Thanks!


Refresh from floating button updates the selected repository (not the one containing machete file)

uild-chain belongs to filter-service repo,
the opened machete file belongs to the other one (generator-service)

No longer relevant since there's no floating button/external system anymore.


The icons on the Cluster page are cut off on the right side

The icons on the Cluster page are cut off on the right side. In addition to fixing this small design detail, my proposal is to change all the icons from grey to their own corporate colour. If we do this, the user would be able to easily recognize all kinds of Clusters.
Image of the current demo:

Image of the Figma design:

See the design here:
https://www.figma.com/file/IVHnM9iyeFWpd11evtY8ux/Weave-GitOps?type=design&node-id=17914%3A199097&mode=design&t=Oy9FiwH9kVdhRT0X-1
You can find all types of cluster icons in this folder at 32px size:
https://drive.google.com/drive/folders/1vXJQ_U5TkM_Hw6wrQ5IBSm3yaC815-sj?usp=sharing

Hey Manuela!
Can you direct me to who you got the list of cluster types from? I need their help to get the exact type strings for all the new kinds. We're also missing the Kind icon for Docker clusters, and we have an icon for OtherOnprem - not sure what that case is addressing specifically

Hey!! I came from this page :  weave-gitops-enterprise/ui-cra/src/utils/icons. You sent it to me a while ago and I took a screenshot to work on Illustrastor. I marked with a rectangle the icons that I couldn't find. The OtherOnprem was in the unknown icons. Investigating now I realized that they are the local machines. It is right? Anyway I couldn't find the icon. Is there a standard? or Can we create one? I added to the folder two Docker icons (regular and simple)