[zuul] Configure prometheus endpoint

Needed for autoscaling testing downstream and for testing here in the future.

/retest

/hold until beta

recheck

/unhold
We are merging downstream CI related PRs

/lgtm

recheck


Update generateId

We're using generateId(id) for each enhancer (like extension, react-native-debugger), so I think make it on here will be better.

That's good to have, but I was thinking maybe to use uuid instead, as socketcluster-client is already using it. Not sure it's the case to have a better algorithm of generating an unique id as there shouldn't be so much instance ids to get overlapped, but who knows. What do you think?

I think we could include uuid right here instead of Math.random().toString(36).substr(2).

I'm updated it. But actually I'm prefer to show short id on instance list, maybe shortid will be better choice?

shortid looks good. Let's go with it if you want.

Ok, it's done! :)

Awesome! :)
Releasing 0.1.0.


Not able to install icons yarn add @material-ui/icons package

[x] The issue is present in the latest release.
[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

Expected Behavior 🤔

Steps to Reproduce 🕹

Steps:
1.yarn add @material-ui/icons, showing network errors

Context 🔦

Your Environment 🌎

`npx @material-ui/envinfo`
  Don't forget to mention which browser you used.
  Output from `npx @material-ui/envinfo` goes here.

@eps1lon would you rather keep this issue open so we can talk about future improvements or close as a duplicate?
With developers keep struggling on v4, and v5 adding a good +30% more icons, it seems that we need to do something about it.

With developers keep struggling on v4, and v5 adding a good +30% more icons, it seems that we need to do something about it.

Agreed. I do have some ideas that should hopefully get us down to the v4 install size. Even going so far as just shipping ES modules since we no longer support node 10 but that's a bit more prone to bugs.

[ ] minify published icons code
[ ] Replace import * as React from 'react' with import of what we need (e.g. just import { Fragment } from 'react';
[ ] just ship ES modules

But that requires an update to the icons itself which aren't checked in. So first we need to update the icons (probably ignore the new ones) and then vendor in material-ui-icons/material-icons.


Build page tweaks

The (i) info icon has a different font for Build Constraints vs 2-Set/4-Set.
"Use Excluded" and "Use Equipped" have inconsistent unchecked colour states.
Minimum Artifact level text box is not centre justified.

When checked, the checkbox for "Use Excluded" and "Use Equipped" justifies to the left unlike the Artifact 4-Set conditional.

It would be a good idea to add a title and (i) info icon above the "Default Artifact Set Effects" button to make the 4th column consistent with the others and allow for a deeper explation of the function. I suggest:
"Artifact Conditionals and Constraints (i)"
"Enable 4-Set Conditionals that you want the Optimizer to take into account when generating builds or force the Optimizer to use certain 2-Set or 4-Sets."

The info tooltip for Build Constraints is poor english. Suggest  changing to:
"Set certain requirements that generated builds must confirm to.
eg: Generated builds must have at least 140% Energy Recharge."

The (i) info icon has a different font for Build Constraints vs 2-Set/4-Set.
Fixed in #348
The rest remains untouched


FIB SEM run error - 11/3/2022

Pipeline run: http://<IP_ADDRESS>:8080/default/flow-run/156ae765-3a93-40e5-bf92-d6d2185441c1
The pipeline failed at the last stage -> "copy work_dir"
A card was created. Both the thumbnails and the Neuroglancer assets are there too, but the Neuroglancer does not look right.
See Hedwig UI:
https://hedwig2qa.niaid.nih.gov/neuroglancer/UmVjb3JkOjE1OTg1NA== ( Neuroglancer asset)
https://hedwig2qa.niaid.nih.gov/samples ( Large Thumbnail)
File Path:
\ai-fas12\RMLEMHedwigQA\Projects\RTB\efischer\hansenbry-2022-0112-NIAID-QA\SEM-Tomo-2022-0112-JanTest\Test1
\ai-fas12\RMLEMHedwigQA\Assets\RTB\efischer\hansenbry-2022-0112-NIAID-QA\SEM-Tomo-2022-0112-JanTest\Test1\set 1

This is a no space left on device error. Closing.


Modified Simplified Chinese topic data and fixed the interface contro…

Modified Simplified Chinese topic data and fixed the interface control confusion in Simplified Chinese Settings(修改简体中文专题数据，并修复简体中文下设置界面控件错乱问题)

Left: Current Version, Right: PR Version

Please refer to the screenshot and check your modified version again @mzj21. Btw thanks for the topic data update.

After confirmation, the interface was confused due to the font, so I closed it.


webui-aria2 raspberry stretch

I've installed aria2 on my raspberry with raspbian stretch, i'd like handle aria2 by web.
I tried https://ziahamza.github.io/webui-aria2/ , setting my raspberry IP , port and SSL.
but it is impossible establish the connection.
I tried on crome, firefox.
crome on rasperry
firefox on windows10
firefox on windows 7
starting to the point I've not have experience on raspberry an linux, i'd like receive cleare explanation.
thank you

To have something that can be remotely controlled with this project is an RPC server, which is explained in the aria 2 manual, which is also available with shell command man aria2c (if you have installed package man, which stands for "manual") or aria2c --help=rpc.
To run aria 2 RPC interface over SSL require your own domain and a certificate for it. https://dot.tk can help you with a free domain with a minor top domain like .tk or .ml. Let's encrypt could help with a free certificate (need to be renewed periodically, but there are scripts to automate that process). rpc-certificate option manual should manage to tell you how to enable SSL with aria 2 RPC server.
Otherwise, if you don't want to F*ck with the mess of SSL, you can just visit http://ziahamza.github.io/webui-aria2/ (without https) to go unsecured connection. You can even set up an Nginx or Apache server serving the WebUI from your RPi.

Most aria 2 RPC server tutorial is available in Chinese. I don't know why but maybe I should build something.


Fair use policy from Download box goes nowhere

bug. will fix

missed that one, sorry. should now be fixed. please feel free to reopen if not.


Inverse error sending as dafault

Describe the bug
The SendError function is being used backwards in the SLib.gd, the first argument is defined as the error message, and the second argument is the location. So the doc comments reports errors will say!
To Reproduce
Steps to reproduce the behavior:
1- Load or backup a non-existent file
2- Go to Debugger > Errors and see incorrect error
Expected behavior
If the file doesn't exist, it will send an error to the console like this: SLib.gd:57 @ SendError(): "Can't load from --file_location<--, file not exists!" From "LoadFile"
Details:

OS: Windows 10 x64
Godot Version: v4.2.1.stable.official
SLib Version: 1.0.2-stable

The bug was fixed for 1.0.3


comparison with sorty

In order to compare with sorty I make the following change in bench.c:
int max = 1 << 26;

and compile with:
g++ -O3 -fpermissive bench.c -o bench

It is been running for over 5 minutes with no output. an issue with the benchmark code?

I also added
int samples = 2;

and the output is
|  quadsort | 67108864 |  i32 | 6.934483 | 7.032585 |             |     random order |
|stablesort | 67108864 |  i32 | 7.314444 | 7.478966 |             |     random order |
|  wolfsort | 67108864 |  i32 | 4.174401 | 4.180588 |             |     random order |

with much faster results for partially sorted input. sorty attains 3.19 seconds with two goroutines.
Considering g++ -O3 vs go -B, g++ is considerably better at optimizing code but go toolchain is not bad either.
I think if you write a concurrent version of wolfsort, that should be even faster.
Cheers

Keep in mind that wolfsort performs optimal at 1,000 - 100,000 elements.
A concurrent version should be quite a bit faster. For the best overall results one would need to make quadsort concurrent as well. I don't think I'll attempt this myself anytime soon however.

Keep in mind that wolfsort performs optimal at 1,000 - 100,000 elements.

interesting that performance peeks in an interval. is it by design?

I guess you could call it by design. Quadsort is significantly faster than quicksort as long as the L1 cache isn't exhausted. This is primarily due to the swapping optimization which gives a 50% performance boost.
So the primarily goal of wolfsort is to create buckets that contain right around 250 elements. This is easiest in the 32K - 128K range, which might explain why wolfsort is faster at sorting 100K elements than 10K elements.


Test, move to master, and submit to PyPi

A first version of the sclblonnx tools is now available. It contains quite some tests and examples (see /test/ and /examples/, although things can always be improved.
However, this seems like a reasonable V0.1. Please check the setup, and if ok:

Add version info for ONNX, etc. to the readme (makes sure we have a version set for which things work).
Move deploy branch to master.
Build and submit to PyPi.

This is done.


multicluster pilot test flakes

Traffic Shifting: Sometimes 1 cluster is not included in LB of 100 callls + a traffic shifting rule

https://prow.istio.io/view/gs/istio-prow/logs/integ-pilot-multicluster-tests_istio_postsubmit/1326591759664287744

instanceip expects 503 but gets 200 (may just need a longer timeout)

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/28830/integ-pilot-multicluster-tests_istio/1327000170545549312

2020-11-12T21:38:07.653055Z	info	tf	=== BEGIN: Test: 'pilot[TestTraffic/instanceip/without_sidecar#03]' ===
    traffic.go:76: failed calling a->'http://b.echo-1-93881.svc.cluster.local:82/': call failed from a to http://b.echo-1-93881.svc.cluster.local:82 (using http): timeout while waiting (last error: 5 errors occurred:
        	* expected response code 503, got "200"

TestTraffic/gateway/external 503s

https://prow.istio.io/view/gs/istio-prow/logs/integ-pilot-multicluster-tests_istio_postsubmit/1326693003682648064

2020-11-12T01:35:03.672191Z	info	tf	=== BEGIN: Test: 'pilot_endpointslice[TestTraffic/gateway/external.external-2-21612.svc.cluster.local]' ===
    traffic.go:76: call failed from TestRunner to http://<IP_ADDRESS>:80 (using http): timeout while waiting (last error: 1 error occurred:
        	* response[0] Status Code: 503

#24581

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/28457/integ-pilot-multicluster-tests_istio/1337956884673466368

traffic.go:76: failed calling a->'http://b.echo-1-55619.svc.cluster.local:80/': call failed from a to http://b.echo-1-55619.svc.cluster.local:80 (using http): timeout while waiting (last error: expected 50 calls to "b", got 39)

The endpointslice test is still failing:
https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/29411/integ-pilot-multicluster-tests_istio/1338491547649314816
1/588 Tests Failed.
...
endpointslice: Failure
...
2020-12-14T14:49:21.655778Z	info	tf	=== FAILED: Setup: 'pilot_endpointslice' (build instance: timeout while waiting (last error: configmap istio-ca-root-cert was not found in namespace echo-1-93436: configmaps "istio-ca-root-cert" not found)) ===
...
FAIL	istio.io/istio/tests/integration/pilot/endpointslice	115.293s

Would this be the same cause as
pilot: TestTraffic
pilot: TestTraffic/services
pilot: TestTraffic/services/case_4_no_match#02
    traffic.go:73: failed calling a->'http://b-alt-4:12346/': call failed from a to http://b-alt-4:12346 (using http): timeout while waiting after 17 attempts (last error: no responses received) | pilot: TestTraffic/services/case_4_no_match#02 expand_less | 1m16s | traffic.go:73: failed calling a->'http://b-alt-4:12346/': call failed from a to http://b-alt-4:12346 (using http): timeout while waiting after 17 attempts (last error: no responses received)
pilot: TestTraffic/services/case_4_no_match#02 expand_less | 1m16s
traffic.go:73: failed calling a->'http://b-alt-4:12346/': call failed from a to http://b-alt-4:12346 (using http): timeout while waiting after 17 attempts (last error: no responses received)

https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/31114/integ-pilot-multicluster-tests_istio/1365368988074250240


WIP: Unify Ecto.ChangeError with Ecto.Type argument errors

Depends on #2798
Closes #2797

Just to double check: this PR was merged into jv-utc-datetime branch, is that correct?

Nevermind, I see you put it into master 👌


Extension causes high cpu load

Issue Type: Performance
Extension Name: vscode-icons
Extension Version: 10.0.0
OS Version: Linux x64 4.19.107-1-MANJARO
VSCode version: 1.43.0

:warning: Make sure to attach this file from your home-directory:
:warning:/home/death/vscode-icons-team.vscode-icons-unresponsive.cpuprofile.txt
Find more details here: https://github.com/microsoft/vscode/wiki/Explain-extension-causes-high-cpu-load

Refer to #2392


Move to platform independence

Here's a PR that I hope helps move druid in a direction where it has a place to put code for other platforms. Users should only need to call piet, druid, or druid-shell functions and shouldn't need to interface directly with winapi, directdraw or directwrite
It's possible this PR is too early or unwanted at this time, I'm ok with that, I thought this would help at least crystallize discussions.
With this structure we'd have a place and direction to add cairo and other OS shell support. Here's a WIP example: https://github.com/snoe/druid/tree/support-macos
The piet for 2d graphics crate I think/hope follows the discussion here:
https://xi.zulipchat.com/#narrow/stream/147926-druid/topic/2D.20abstraction.20layer
It's likely the public api for piet needs to be cleaned up and tightened instead of just exposing direct2d, but I wonder if that can come later as other platforms may force different apis.
I'm also a rust noob so if there are more idiomatic ways to do anything here I'd appreciate the feedback.

Apologies for the delay in reviewing.

No worries, my time can be pretty constrained as well.

have you tried making the corresponding patch to xi-editor?

Oh I thought I had but i guess I was confused between xi-editor and xi-win, you can see the changes for xi-win here: https://github.com/xi-editor/xi-win/compare/master...snoe:deplatform

For "author" in the metadata, I'm ok with that being you.

That feels a bit disingenuous to me since I just moved your code around, I just added myself to the AUTHORS file.

I'm slightly concerned about the order that changes happen

A big danger I see from exporting:* from directdraw is we don't know and can't control  what pieces others are using. Doing a search on github it seems that xi-editor and your synthesizer-io are the only users visible there.
An option that could mitigate some of this danger of exposing the directdraw apis, is I could lock down the pub exports to the set of vars used by known clients of piet.

Another possibility to explore is to get a minimum functioning piet library
A last possibility is to have a druid branch

Either of these could be good options only I'd worry about them not tracking the users/usages of druid and getting out of sync - since there are only a few maybe this concern isn't as important.

I think it will be necessary fairly soon to have a new repo for piet. That could be done now (I'm happy to create the repo in the xi-editor org), or as a separate step.

xi-editor org would definitely be the right place for a piet repo. I worry about publishing a standalone crate before the API is better resolved.
A thought I had was that druid could re-export piet and druid_shell so that it's users didn't need to care where the crate was located, but that may not be idiomatic rust.


Support synchronous "run" methods in actions

If they don't take a callback parameter, don't wait for them.

This can be done as part of making actions simpler by extracting a superclass


Fedora 28 support

Issue Title
Fedora 28 support
General
Can Fedora 28 support be added for dotnet core 2.1?
I tried using these instructions for Fedora 28 Server but they didn't exactly work.
Here's what I ended up doing
~$ sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
~$ wget -q https://packages.microsoft.com/config/fedora/27/prod.repo
~$ sudo mv prod.repo /etc/yum.repos.d/microsoft-prod.repo
~$ sudo dnf update
I replaced
~$ sudo dnf install libunwind libicu compat -openssl10
with
~$ sudo dnf install libicu compat -openssl10
because libunwind is no longer required but got this error
dnf install: error: unrecognized arguments: -openssl10
I managed to fix that by changing compat -openssl10 to compat-openssl10
And finally ~$ sudo dnf install dotnet-sdk-2.1.300-rc1-008673 (worked without any issues)
Here's my dotnet --info output after installing successfully?

.NET Core SDK (reflecting any global.json):
Version:   2.1.300-rc1-008673
Commit:    f5e3ddbe73

Runtime Environment:
OS Name:     fedora
OS Version:  28
OS Platform: Linux
RID:         fedora.28-x64
Base Path:   /usr/share/dotnet/sdk/2.1.300-rc1-008673/

Host (useful for support):
Version: 2.1.0-rc1
Commit:  eb9bc92051

.NET Core SDKs installed:
2.1.300-rc1-008673 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.1.0-rc1-final [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.0-rc1-final [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.1.0-rc1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download

@Saevar2000 - yep, that looks like a successful install. I'll remove the errant space in compat-openssl10 in the instructions.


feat(sumolambda): initial commit

Initial commit adding collector, java, nodejs and python lambda layers build possibility.

Moved from: https://github.com/SumoLogic-Labs/sumo-opentelemetry-lambda


Natron Integration

Brief description
OpenPype integration for open-source compositing software Natron
Description
This is a prototype/draft integration for OpenPype with Natron. It's very early stages with just some placeholder code to be implemented to get it working. Currently it's able to launch Natron and have the OpenPype menu entries added - however the Qt tools do not load as of yet.
Status:

[x] Implement application
[x] Implement OpenPype menu entries
[ ] Implement Qt tools
[ ] Implement loaders
[ ] Implement save/load/scene callbacks
[ ] Implement publishing logic

Description
Natron 2.5.0 is the first version on Python 3+ I believe and uses 3.10. It still uses PySide however and it seems there are some issues with the Qt implementation that make it fail for a bit. I'm getting odd errors like:
Failed to import qtpy.QtCore, make sure it is bundled with your Natron installation or reachable through the Python path. Note that Natron disables usage of site-packages).
Failed to load S:/openpype/OpenPype/openpype/hosts/natron/startup/init.py: S:\openpype\OpenPype\.venv\lib\site-packages\blessed\terminal.py:186: UserWarning: Failed to setupterm(kind='unknown'): Could not find terminal unknown

And also what I've reported here.
Since I'm not a Natron user myself I'll leave this PR as a draft for now to gauge interest in the OpenPype community whether it's worth spending some more time on a potential Natron integration. It might also be worth waiting around a bit for a new 2.5+ release of Natron which might already update to a newer Qt or resolve some of the issues of its Qt (if they are actual issues, they might come from OpenPype usage of Qt of course). It seems Qt5 progress is tracked here and release is targeted for Natron 2.6+.
Again, this is a very quick 'draft' layout for the code.
Testing notes:
It's hardly functional but you can start Natron through OpenPype and maybe debug yourself.

Unzip latest Natron to C:/
Run Natron from OpenPype

Natron 2.5.0 is the first version on Python 3+ I believe and uses 3.10. It still uses PySide however and it seems there are some issues with the Qt implementation that make it fail for a bit. I'm getting odd errors like:

Develop is still using qtpy which should support PySide. It did work with Qt.py?

@BigRoy As much as this would be great to merge in. At this point I have to suggest re-creating it as AYON addon instead of commiting directly to OP repository. Even if it's absolutely minimal, it can make it's way as community addon to AYON market and live it's own life.
Closing in favour of separate repo for this work.


add angular apac logo

Checklist for logo submission

[ ] Logo file added to the logos folder
[ ] Created a new entry in metadata.json (only name is mandatory)
[ ] Logo is an SVG with vector paths (no embedded pixel image)
[ ] The logo fills the image and there is no white margin (please remove the white margin if there is any 🙂 )

Thanks for the submission! @queency-koh
Unfortunately, I can't open the file in any browser... looks like it's corrupted. From the sourcecode it looks like a PNG embedded into SVG. Do you have a "real" vectorized version of the logo?

Thanks for the submission! @queency-koh
Unfortunately, I can't open the file in any browser... looks like it's corrupted. From the sourcecode it looks like a PNG embedded into SVG. Do you have a "real" vectorized version of the logo?

I do not have a "real" vectorized version. I will try to recreate it.

Hi @maartentibau I have a new commit. I replaced the old logo with a vectorized version.

Thanks!! 😊


Reduce the number of Conv layers in ResNet18 and 34?

In the current ResNet model (examples/imagenet/models.py), there are three Conv layers in every ResidualBlock instance regardless of the number of layers. This is accurate for ResNet50 and above. ResNet18 and 34, however, are known to use two Conv layers per block instead. It would be fitting to bypass the 3rd Conv for models with less than 50 layers.

This should be fixed in PR #397


Request support site: Viceland

Viceland is a new channel from Vice. Website at https://www.viceland.com. Appears to use Uplynk, and may be encrypted, so it may not be possible.

Post an example URL.

For example, https://www.viceland.com/en_us/video/gaycation-japan/568fdf623cd5b1ad4d43efeb?playlist=3

I think this would be a cool addition.
It's currently rippable using Atresdownloader, Livestreamer or ffmpeg with the m3u8 so it shouldn't be hard to implement for "best open-source video downloader tool"

Videos seems to be geo-restricted to the US and requires login.

My bad: there are videos not geo restricted and that doesn't require login. It currently works by manually parsing m3u8-files to youtube-dl (fails with ffmpeg, btw; the streams are encrypted, it seems). As @pishposhmcgee mentions, the site uses upLynk, which also extractor/crackle.py does.

I would also like to see viceland support added, I have an active membership although the site can be very buggy at times.


fix: support grep for dry-run

Motivation/Description of the PR

support --grep for dry-run command

Type of change

[x] :bug: Bug fix

Checklist:

[x] Tests have been added
[x] Lint checking (Run npm run lint)
[x] Local tests are passed (Run npm test)

Thanks!


chore(userpace/falco): do not print error code in process_events.cpp

Signed-off-by: Aldo Lacuku<EMAIL_ADDRESS>

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

If contributing rules or changes to rules, please make sure to also uncomment one of the following line:

/kind rule-update

/kind rule-create

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area rules

/area tests

/area proposals

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: alacuku
To complete the pull request process, please assign jasondellaluce after the PR has been reviewed.
You can assign the PR to them by writing /assign @jasondellaluce in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@leogr: Closed this PR.

In response to this:

Closing and reopening to trigger the CI
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/reopen

@leogr: Reopened this PR.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


Copying NIF from cache and extracting to _build/dev/lib/midiex/priv/native/midiex-v0.6.1-nif-2.15-x86_64-pc-windows-msvc.dll

On Windows 11 using Elixir 1.17.2 and OTP 27.0.1 or Elixir 1.16.3 and OTP <IP_ADDRESS>, I get this error when I run this script and it crashes the script.
Mix.install([{:midiex, "~> 0.6.1"}])
Midiex.ports()

Copying NIF from cache and extracting to _build/dev/lib/midiex/priv/native/midiex-v0.6.1-nif-2.15-x86_64-pc-windows-msvc.dll

The crash seems related to https://github.com/philss/rustler_precompiled/issues/56

Thank you for reporting the issue @SocksHelp. I'm glad you were able to resolve the issue.
I'll take a look to see if I need to bump the versions up and generate a new release.


Updated Collated files

Coverage remained the same at 81.175% when pulling 84bb1bdcd5b2f46d5266e973ea5873e50f67b788 on inGall:v1.5_updated into 31d04375b7015c4590e2031f974499d288994b7b on CS2103AUG2017-T09-B1:master.


No error status and  detail message are returned when deploying a proxy revision to an apigee environment without active runtime pods

Describe the bug
No error status code and detail message are returned when deploying a proxy revision to an apigee environment without active runtime pods.
Here is a screenshot when trying to deploy latest available revision to such an environment:

Version
2.7.0
To Reproduce
The environment must not have active runtime pods. Try to deploy any revision to it.
Expected behavior
An error status code should be returned and also a detail message of the error

fixed and released in v2.7.1


Create ru_ru.json

Russian translate

seems fine to me. anything i said in my comment on the pull request you made for chromatic arsenal applies here, and will also apply to the other pull requests you made for my mods when i get to those.

Of course, I also hope for the return of Bandanas of Gigantism and a port to 1.20.1, although the mod is from another author, but with the Marks in the totality of your mods it was quite interesting to play.
All the best and thank you.


sessions.getSessionHistory filter param does not work?

Running this code:
const response = await plexAPI.sessions.getSessionHistory(
  'viewedAt:desc',
  undefined,
  { viewedAt: '>=1723511329' }
);

Sends this url: http://<MY_PLEX_SERVER>/status/sessions/history/all?sort=viewedAt%3Adesc
So as you can see, the filter is completely omitted. The documentation doesn't say anything outside of:

The lib's source code defines the Filter type as an empty object:

The only thing helping to try and figure this out is inspecting the Network tab when going in Plex's history page:

Updated to 0.23.0 and this still does not seem to work. Am I doing it wrong or is it a bug?

Hey @BobWithHair,
Can you confirm that the local package is up to date?
I can confirm the most recent regeneration contains the updated filter value:
export type GetSessionHistoryRequest = {
  /**
   * Sorts the results by the specified field followed by the direction (asc, desc)
   *
   * @remarks
   */
  sort?: string | undefined;
  /**
   * Filter results by those that are related to a specific users id
   *
   * @remarks
   */
  accountId?: number | undefined;
  /**
   * Filters content by field and direction/equality
   *
   * @remarks
   * (Unknown if viewedAt is the only supported column)
   */
  filter?: QueryParamFilter | undefined;
  /**
   * Filters the results based on the id of a valid library section
   *
   * @remarks
   */
  librarySectionID?: number | undefined;
};

Hey @LukeHagar! Sorry, haven't been working on my project in the last week or so. So, yes my package is up to date. Just re-updated it to the most recent 0.23.5
I did and still do see that GetSessionHistoryRequest type, but the QueryParamFilter type is only an empty object:
.
But could you comment on the syntax I used for the filter in my original post? Because it is not so simple to figure out what syntax is expected to make the filter work


[FEATURE]: Replace standard messages with custom messages

Feature Type
Changing existing functionality in Angler
Problem Description
The current implementation uses existing standard messages internally like the MultiDOFJointTrajectory message along with some others from MoveIt. These messages are generally fine; however, they can be extremely cumbersome to use as the target domain is distinct from the current domain.
Feature Description
Implement a set of custom messages specific to underwater vehicle manipulator systems.
Alternative Solutions
I am not fully convinced that implementing a set of custom messages is the right way to go about this, but I want to make an issue to remind myself to reconsider the solution in the future. This may look like integrating only MoveIt messages or some other solution. But, ultimately, a less cumbersome solution should be implemented.
Additional Context
No response

I don't think that we need any custom messages here. It's better to stick with the standard messages.


Go linter warnings fixed and code quality badge added

Go Report Card badge added
Multiple golint warnings fixed. Feel free to update new code comments if they don't make much sense.

Only two warnings left. I'm not sure what would be the right name for common labels that will avoid common in its name:
pkg/routing/reconciler/common/adapter.go:154:6: func name will be used as common.CommonObjectLabels by other packages, and that stutters; consider calling this ObjectLabels
pkg/sources/reconciler/common/adapter.go:237:6: func name will be used as common.CommonObjectLabels by other packages, and that stutters; consider calling this ObjectLabels

If we could make stylecheck require comments on exported functions/types perhaps that would be enough. But in general, this time I run aggressive golint and all these recommendations were not that pointless. Maybe if we enable it in CI and after this PR there will be 0 warnings, it would be easier for us to stick to this practice?

Yes definitely. I was not saying this is a bad idea, just that stylecheck and golint have the same purpose.
We could select what checks we want to enable/disable like this:
https://golangci-lint.run/usage/linters/#stylecheck


fix number of FORMAT arguments

Corrected two choices into three choices (CSV, PARQUET, DELTA).

Learn Build status updates of commit e3a7eab:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/synapse-analytics/sql/develop-openrowset.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


Fix issue with missing remediation

When the policy misses remediation, this causes a serious UI error. This PR will fix that issue

/test console-acm-212-on-pull-request

/test
Red Hat Konflux / console-mce-mce-27-on-pull-request

/retest


VNC+Retina rendering issue

I realize that native window support is not ready for prime-time yet but I am trying to understand the quirks of the current Morphic code.
I'm running a GToolkit-based image on Linux and viewing the display with TigerVNC on a Mac with Retina display and this is what I am seeing:

So the rendering is screwed up somehow such that the display is shrunk to half size on each axis. Meanwhile the mouse is still acting like the graphics take up the whole display.
Does anybody understand this phenomenon? And have I could make this render in a usable way even if not beautifully?

(This works fine on my desktop Mac with its non-Retina display also using TigerVNC there.)

Unfortunately, we do not have much of an answer for this. Does GT in native windows work better through VNC?

I'm starting to dig into this via the various dpi options available on X11/vncserver/vncclient. I'll report what I can work out.

I have not been able to successfully test native windows on Linux yet. It works on Mac for me but on Linux it has seemed to lock up the original window without opening a new one.
Do native windows work on Linux for other people?

This VNC problem does not seem to be specific to Pharo actually. I seem to have this when I run native Linux applications like xterm too. That 1/4 part of the screen that is rendered seems to be the whole X display.
I'll keep digging for a solution but I close this because I don't think the problem is directly related to GT or Pharo.

I opened an Issue on TigerVNC for this at https://github.com/TigerVNC/tigervnc/issues/805.

(This seems to have been due to a bug in the TigerVNC package for homebrew. The official binary release doesn't have the problem. Going to follow up on getting it fixed in homebrew.)

@lukego you said it happens in other applications too? Then the bug would rather be outside tigervnc? The homebrew “bug” is just linking to the latest fltk?


Add any missing channel capacity to reserve

To avoid confusion like in https://github.com/MutinyWallet/mutiny-web/issues/863 lets just add any remaining capacity to the reserve

needs rebase


Rewrite sosfilt to make a single loop over the input?

signal.sosfilt is currently implemented as a loop that calls signal.lfilter. The drawback of doing things this way is that a complete loop is made over the input data for each section.  I've got a branch (not currently on github) that creates a dedicated sosfilt function in C.  Using this I find a factor of 4 improvement for small inputs and a factor of 2 for larger inputs.  Right now, my code doesn't handle filtering along an axis of an nd array, but before I worked on that I thought I would see what people thought.   Some timings are below.  Note that the signal.sosfilt numbers include the improvements from #4628, which have already improved things.  With my new code, using the sos representation is as fast as using the ba representation.
In [1]: run test.py

In [2]: t, y = make_y(1000)

In [3]: sos = signal.butter(6, [0.1575, 0.1625], 'band', output='sos')

In [4]: y1 = signal.sosfilt(sos, y)

In [5]: y2 = signal.sigtools._sosfilt(sos, y)

In [6]: all(y1 == y2)
Out[6]: True

In [7]: timeit y1 = signal.sosfilt(sos, y)
10000 loops, best of 3: 79.7 µs per loop

In [8]: timeit y2 = signal.sigtools._sosfilt(sos, y)
100000 loops, best of 3: 19.8 µs per loop

In [9]: t, y = make_y(1000000)

In [10]: timeit y1 = signal.sosfilt(sos, y)
10 loops, best of 3: 37 ms per loop

In [11]: timeit y2 = signal.sigtools._sosfilt(sos, y)
10 loops, best of 3: 20 ms per loop

In [12]: sos = signal.butter(20, [0.1575, 0.1625], 'band', output='sos')

In [13]: timeit y1 = signal.sosfilt(sos, y)
10 loops, best of 3: 124 ms per loop

In [14]: timeit y2 = signal.sigtools._sosfilt(sos, y)
10 loops, best of 3: 64 ms per loop

In [15]: b,a = signal.butter(20, [0.1575, 0.1625], 'band', output='ba')

In [16]: timeit y3 = signal.lfilter(b, a, y)
10 loops, best of 3: 64.5 ms per loop

@ewmoore what do you think about using a Cython implementation rather than pure C? I think it might be easier to maintain if we can manage to get (near-)equivalent performance to the pure C code. Though perhaps to do that we may need to make the Cython code look a lot like C code anyway...?

I have no issues at all with moving to Cython.

@ewmoore can you push your changes to GitHub? I (or someone else) could take a stab at Cython-izing the C code.

Sorry, I don’t think I have that code anymore.

@ewmoore I first turned your code above into a benchmark:
Benchmark
class Sosfilt(Benchmark):
    param_names = ['n_samples', 'order']
    params = [
        [1000, 1000000],
        [6, 20]
    ]

    def setup(self, n_samples, order):
        self.sos = butter(order, [0.1575, 0.1625], 'band', output='sos')
        self.y = np.random.RandomState(0).randn(n_samples)

    def time_sosfilt_basic(self, n_samples, order):
        sosfilt(self.sos, self.y)

The results on master are:
              --                    order
              ----------- -------------------------
               n_samples       6            20
              =========== ============ ============
                  1000      44.3±1μs     132±2μs
                1000000    25.5±0.2ms   80.5±0.3ms
              =========== ============ ============

I then Cythonized the loop.
Cython code
cimport numpy as np
cimport cython

ctypedef fused DTYPE_t:
    float
    float complex
    double
    double complex
    long double
    long double complex

@cython.cdivision(True)
@cython.boundscheck(False)
@cython.wraparound(False)
def _sosfilt(DTYPE_t [:, ::1] sos,
             DTYPE_t [:, ::1] x,
             DTYPE_t [:, :, ::1] zi):
    # Modifies x and zi in place
    cdef Py_ssize_t n_signals = x.shape[0]
    cdef Py_ssize_t n_samples = x.shape[1]
    cdef Py_ssize_t n_sections = sos.shape[0]
    cdef DTYPE_t x_n
    cdef DTYPE_t b[3]
    cdef DTYPE_t a[2]
    with nogil:
        for s in range(n_sections):
            b[0] = sos[s, 0] / sos[s, 3]
            b[1] = sos[s, 1] / sos[s, 3]
            b[2] = sos[s, 2] / sos[s, 3]
            a[0] = sos[s, 4] / sos[s, 3]
            a[1] = sos[s, 5] / sos[s, 3]
            for i in range(n_signals):
                for n in range(n_samples):
                    x_n = x[i, n]  # make a temporary copy
                    # Use direct II transposed structure:
                    x[i, n] = b[0] * x_n + zi[i, s, 0]
                    zi[i, s, 0] = b[1] * x_n - a[0] * x[i, n] + zi[i, s, 1]
                    zi[i, s, 1] = b[2] * x_n - a[1] * x[i, n]

The results were less than promising:
              --                    order
              ----------- --------------------------
               n_samples        6            20
              =========== ============= ============
                  1000      59.9±0.3μs   125±0.4μs
                1000000    28.4±0.04ms   93.7±0.5ms
              =========== ============= ============

I then jitted it with Numba.
Numba code
import numba
@numba.jit(nopython=True, nogil=True, fastmath=True, cache=True)
def _sosfilt(sos, x, zi):
    n_signals = x.shape[0]
    n_samples = x.shape[1]
    n_sections = sos.shape[0]
    b = np.zeros(3, sos.dtype)
    a = np.zeros(2, sos.dtype)
    for s in range(n_sections):
        b[0] = sos[s, 0] / sos[s, 3]
        b[1] = sos[s, 1] / sos[s, 3]
        b[2] = sos[s, 2] / sos[s, 3]
        a[0] = sos[s, 4] / sos[s, 3]
        a[1] = sos[s, 5] / sos[s, 3]
        for i in range(n_signals):
            for n in range(n_samples):
                x_n = x[i, n]  # make a temporary copy
                # Use direct II transposed structure:
                x[i, n] = b[0] * x_n + zi[i, s, 0]
                zi[i, s, 0] = b[1] * x_n - a[0] * x[i, n] + zi[i, s, 1]
                zi[i, s, 1] = b[2] * x_n - a[1] * x[i, n]

This worked better, but still only ~20% faster for long signals:
              --                    order
              ----------- --------------------------
               n_samples       6             20
              =========== ============ =============
                  1000     50.4±0.2μs     97.6±1μs
                1000000    20.6±0.3ms   67.1±0.09ms
              =========== ============ =============

I also profiled the Cython version with snakeviz with an adaptation of the asv benchmark, and over 98% of the time was actually spent in the _sosfilt call, so the overhead does not seem to come from the reshape/moveaxis/ascontiguousarray gymnastics, at least for long arrays.
Did I miss something about how to implement the filter to get ~2x speed gains? lfilters source says it's implemented as direct II transposed so I followed that idea.

Also I realized I didn't post what the reshape/moveaxis stuff is about. It shouldn't matter but in case someone else wants to look at this or try any of this stuff it's on my branch here:
https://github.com/larsoner/scipy/blob/4133f5a89a56e43638f3dd3b57350dd3182f4076/scipy/signal/signaltools.py#L3426-L3439

I looked again but I still can’t find my old code.
The problem with your cython/numba versions is that you are still making n_sections loops over the signal. The key here is to make one pass over the signal, feeding the output of the first section directly into the second and so on.

That's just a reordering of the for loops, right? So the number of operations won't change?

That’s true, but it’s not the only consideration. I think this probably has to do with memory access patterns. If you keep going back to parts of the signal that have already been evicted from the processor cache, then you pay a time penalty for access.   By looping over the sections you ensure that you pay this for each section instead of just paying once.

I’ll see if I can find some time this week to try to recreate what I had before.

Just for comparison's sake, here is master again:
              --                    order
              ----------- -------------------------
               n_samples       6            20
              =========== ============ ============
                  1000      44.3±1μs     132±2μs
                1000000    25.5±0.2ms   80.5±0.3ms
              =========== ============ ============

If I change the Cython code to iterate in over samples (outer loop) and sections (inner loop):

@cython.cdivision(True)
@cython.boundscheck(False)
@cython.wraparound(False)
def _sosfilt(DTYPE_t [:, ::1] sos,
             DTYPE_t [:, ::1] x,
             DTYPE_t [:, :, ::1] zi):
    # Modifies x and zi in place
    cdef Py_ssize_t n_signals = x.shape[0]
    cdef Py_ssize_t n_samples = x.shape[1]
    cdef Py_ssize_t n_sections = sos.shape[0]
    cdef DTYPE_t x_n
    cdef DTYPE_t [:, ::1] b = np.empty((n_sections, 3), cython.typeof(sos[0, 0]))
    cdef DTYPE_t [:, ::1] a = np.empty((n_sections, 2), cython.typeof(sos[0, 0]))
    with nogil:
        for s in range(n_sections):
            b[s, 0] = sos[s, 0] / sos[s, 3]
            b[s, 1] = sos[s, 1] / sos[s, 3]
            b[s, 2] = sos[s, 2] / sos[s, 3]
            a[s, 0] = sos[s, 4] / sos[s, 3]
            a[s, 1] = sos[s, 5] / sos[s, 3]
        for i in range(n_signals):
            for n in range(n_samples):
                for s in range(n_sections):
                    x_n = x[i, n]  # make a temporary copy
                    # Use direct II transposed structure:
                    x[i, n] = b[s, 0] * x_n + zi[i, s, 0]
                    zi[i, s, 0] = b[s, 1] * x_n - a[s, 0] * x[i, n] + zi[i, s, 1]
                    zi[i, s, 1] = b[s, 2] * x_n - a[s, 1] * x[i, n]

It is indeed faster!
              --                     order
              ----------- ---------------------------
               n_samples        6             20
              =========== ============= =============
                  1000      51.7±0.2μs    94.1±0.4μs
                1000000    13.5±0.06ms   54.7±0.05ms
              =========== ============= =============

Then doing the same thing in Numba:

    b = sos[:, :3] / sos[:, 3:4]
    a = sos[:, 4:] / sos[:, 3:4]
    for i in range(n_signals):
        for n in range(n_samples):
            for s in range(n_sections):
                x_n = x[i, n]  # make a temporary copy
                # Use direct II transposed structure:
                x[i, n] = b[s, 0] * x_n + zi[i, s, 0]
                zi[i, s, 0] = b[s, 1] * x_n - a[s, 0] * x[i, n] + zi[i, s, 1]
                zi[i, s, 1] = b[s, 2] * x_n - a[s, 1] * x[i, n]

              --                    order
              ----------- --------------------------
               n_samples        6            20
              =========== ============= ============
                  1000       40.0±2μs    66.1±0.4μs
                1000000    9.96±0.03ms   35.8±0.3ms
              =========== ============= ============

I wasn't sure why Cython was slower than numba, so I tried various things. Turns out if I use fastmath=False (instead of True) in the numba.jit decorator, the speed difference is negligible.
So I'll open a PR for the Cython version, and at some point if we want the extra speed gains from fastmath=True (and the tradeoffs for using it are acceptable) we might consider using numba (?).

As a last ditch effort to get Cython to be as fast as numba I tried adding extra_compile_args=['-ffast-math'] to the add_extension call (and verified it was passed when compiling) and it didn't make up the speed difference.

@ewmoore instead of trying to find your code, it might be easier to try #10650. If it gives you roughly the same speedup as before, then things should be working.


DataFusion weekly project plan (Andrew Lamb) - Nov 20, 2023

The idea of this ticket is make my plans for DataFusion visible, largely for my own personal organizational needs, but also to:

Try some different ways to communicate / coordinate int he community
Help provide an interesting summary of what is happening in DataFusion this week

I request anyone else who has plans like this for DataFusion could try something like this (maybe tickets are the best place 🤔 )
Plans for this week personally:

[ ] https://github.com/apache/arrow-datafusion/issues/8227 (specifically, https://github.com/apache/arrow-datafusion/issues/8229 as a step towards https://github.com/apache/arrow-datafusion/issues/8078)
[ ] Edits to paper (improve language and get it under the page limit) https://github.com/apache/arrow-datafusion/issues/6782 with @ozankabak @Dandandan @yjshen and @JayjeetAtGithub  @sunchao and @viirya
[ ] sqlparser release: https://github.com/sqlparser-rs/sqlparser-rs/issues/1030

My plans for next week

[ ] https://github.com/apache/arrow-datafusion/issues/8130, in support of memory explosion in HashJoin outputs: https://github.com/apache/arrow-datafusion/issues/7848 with @korowa

Major projects I intend to help review and coordinate

[ ] Make function packages more modular (make all BuiltInScalarFunctions ScalarUDFs): https://github.com/apache/arrow-datafusion/issues/8045 with @2010YOUY01  (help https://github.com/apache/arrow-datafusion/pull/8258)
[ ] ARRAY function hardening:  https://github.com/apache/arrow-datafusion/issues/6980 / https://github.com/apache/arrow-datafusion/issues/7988with @jayzhan211 @Veeupup @edmondop and @xudong963
[ ] Review any new ticket / bug reports

Reviews planned:
Arrow-rs

[ ] https://github.com/apache/arrow-rs/pull/5100

DataFusion

[ ] https://github.com/apache/arrow-datafusion/pull/7581

Review Queue
DataFusion

[ ] https://github.com/apache/arrow-datafusion/pull/8289
[ ] https://github.com/apache/arrow-datafusion/pull/8281 (1k+ lines)
[ ] https://github.com/apache/arrow-datafusion/pull/7581
[ ] https://github.com/apache/arrow-datafusion/pull/8266#pullrequestreview-1738538700
[ ] https://github.com/apache/arrow-datafusion/pull/8272
[ ] https://github.com/apache/arrow-datafusion/pull/8271
[ ] https://github.com/apache/arrow-datafusion/pull/8268
[ ] https://github.com/apache/arrow-datafusion/pull/8251#issuecomment-1819149188
[ ] https://github.com/apache/arrow-datafusion/pull/8193#issuecomment-1817239349
[ ] https://github.com/apache/arrow-datafusion/pull/8269#issuecomment-1818963564

sqlparser-rs

[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1048
[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1032
[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1045
[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1036
[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1051
[x] https://github.com/sqlparser-rs/sqlparser-rs/pull/1044

DataFusion:

Next week's work: https://github.com/apache/arrow-datafusion/issues/8329


Docker build failure

docker build . -f Dockerfile -t vessel-detection-sentinels:$IMAGE_TAG

 => ERROR [ 4/11] RUN pip install --no-cache-dir -r /home/vessel_detection/requirements.txt

38.37 Collecting networkx>=2.2
38.38   Downloading https://download.pytorch.org/whl/networkx-3.2.1-py3-none-any.whl (1.6 MB)
39.14 ERROR: Package 'networkx' requires a different Python: 3.8.10 not in '>=3.9'

Thank you for filing this issue along with https://github.com/allenai/vessel-detection-sentinels/issues/10 and https://github.com/allenai/vessel-detection-sentinels/issues/9.
Opening all of those was timely as we need to upgrade the in production models that this repo creates for use in the Skylight platform. As you probably noted, the human annotations that we open released here is intended to be joined with the raw imagery (currently available for free from ESA), and Copernicus has since migrated to a newer/more modern architecture as well. In addition we need to upgrade to a more recent version of Python (3.10), and significantly streamline the setup/quick start (which will also make it more closely match another remote sensing repo that we have open sourced but uses more modern libraries and Python 3.10). In sum, this repo is due for an upgrade which is on our roadmap (within the next couple months). Please be a little patient with us and we will respond to all of the current issues in one go.


Reduce calls to GetRequiredService<> across the board

It's a bad pattern, and breaks the dependency graph.  We probably won't be able to remove all usages, but we should try to minimize the calls.

@davidfowl @lodejard how strongly do we care about this at this point?

I care about this a lot, mvc and ef are the worst offenders and we should take a look at why we need to do it so much

It's kind of an evergreen issue, something to watch out for in new code in general, and good to make a pass and see if there's more hanging fruit? Maybe it's part of a best practices kind of thing - calling GetService should be a last resort, something people should question if they see it in a PR, and should also be accompanied with a comment "why this spot in the code can't use DI"

It needs to be in the our coding guidelines. We can close this issue and formiulate guidance like @lodejard said.


Fixed wrong closing parenthesis in codecov.yml

What kind of change does this PR introduce? (Bug fix, feature, docs update, …)
Fixed wrong closing parenthesis in codecov.yml

Why was this change needed? (You can also link to an open issue here)
The codecov.yml file will not work as intended because the closing parenthesis are wrong.

Other information:

Hey @dartpain, please look into this, thank you.

Thank you!


Make Programme/ Speakers page

Rename programme page to more explicitly refer to 'Speakers'
Include JavaScript to load speakers from external JSON file

Registration details can be extracted as an excel file from the KCL e-store. We should aim to consume this and render speaker details on the web page automatically. Also relates to issue #28 (attendants page).


Add option to disable default commandBlocked responses

This PR adds option to selectively disable default responses for various commandBlocked events. Apologies for the unsightly code in client.js; ESLint was complaining about the line lengths during the tests.

There is very rarely any valid reason for removing these.

EventEmitter.emit returns true if the event had listeners (which means it was handled). why not use this behavior to disable default command blocked (and possibly command error) handling if the event was handled by the user? user could then override the default responses or disable them completely.
think
const handled = this.client.emit('commandBlocked', this, 'clientPermissions');
if (!handled) {
  return this.reply('...')
}

alternatively make a commandBlockedMessage (and commandErrorMessage) event

Yes, please.

#193 brings forth a better (IMO) method of implementing this.


Chapter 2 Analyze the Best Models and Their Errors

To analyze the relative importance of each attribute for making accurate predictions, the book use the next code (with its output):
>>> feature_importances = grid_search.best_estimator_.feature_importances_
>>> feature_importances
array([7.33442355e-02, 6.29090705e-02, 4.11437985e-02, 1.46726854e-02,
1.41064835e-02, 1.48742809e-02, 1.42575993e-02, 3.66158981e-01,
5.64191792e-02, 1.08792957e-01, 5.33510773e-02, 1.03114883e-02,
1.64780994e-01, 6.02803867e-05, 1.96041560e-03, 2.85647464e-03])

And to add their corresponding attribute names:
>>> extra_attribs = ["rooms_per_hhold", "pop_per_hhold", "bedrooms_per_room"]
>>> cat_encoder = full_pipeline.named_transformers_["cat"]
>>> cat_one_hot_attribs = list(cat_encoder.categories_[0])_

>>> attributes = num_attribs + extra_attribs + cat_one_hot_attribs
>>> sorted(zip(feature_importances, attributes), reverse=True)
[(0.3661589806181342, 'median_income'),
(0.1647809935615905, 'INLAND'),
(0.10879295677551573, 'pop_per_hhold'),
(0.07334423551601242, 'longitude'),
(0.0629090704826203, 'latitude'),
(0.05641917918195401, 'rooms_per_hhold'),
(0.05335107734767581, 'bedrooms_per_room'),
(0.041143798478729635, 'housing_median_age'),
(0.014874280890402767, 'population'),
(0.014672685420543237, 'total_rooms'),
(0.014257599323407807, 'households'),
(0.014106483453584102, 'total_bedrooms'),
(0.010311488326303787, '<1H OCEAN'),
(0.002856474637320158, 'NEAR OCEAN'),
(0.00196041559947807, 'NEAR BAY'),
(6.028038672736599e-05, 'ISLAND')]

My question is: Why do I have to add extra_attribs? or How do I know that I must add this attributes?
I add the output without add extra_attribs
>>> feature_importances=grid_search.best_estimator_.feature_importances_

>>> #extra_attribs = ["rooms_per_hhold", "pop_per_hhold", "bedrooms_per_room"]
>>> cat_encoder = full_pipeline.named_transformers_["cat"]
>>> cat_one_hot_attribs = list(cat_encoder.categories_[0])

>>> attributes = num_attribs + cat_one_hot_attribs #+ extra_attribs
>>> sorted(zip(feature_importances, attributes), reverse=True)

[(0.303268232301214, 'median_income'),
 (0.1730639450304893, 'NEAR OCEAN'),
 (0.10895862174634888, 'INLAND'),
 (0.0844196144263057, 'ISLAND'),
 (0.07557206707255014, 'longitude'),
 (0.06398786252477989, 'latitude'),
 (0.06315655490931624, '<1H OCEAN'),
 (0.04240720593117474, 'housing_median_age'),
 (0.01829282732311651, 'total_rooms'),
 (0.017560189966804522, 'population'),
 (0.01689244166020893, 'total_bedrooms'),
 (0.01668817806453196, 'households'),
 (0.008535150622100876, 'NEAR BAY')]

How do I know that is wrong? Because without extra_attribs I can not say apparently only one ocean_proximity category is really useful, so you could try dropping the others
Thanks for your time.

feature_importances is a list giving the importance of features in the same order as your input data presented them. In the example case in the book the full_pipeline was used. And the resulting Dataframe (I think it was called housing_num_tr) has a certain order of its columns. This holds true after converting it to numpy using DataFrameSelector().
Python zip does just connect the first entry of feature_importances with the first entry of attributes. Therefore, you have to:

know the order of both lists
make sure that they fit
if one list is shorter than the other, zip will just stop after reaching the last entry of the shorter list

In your example the entries belonging to num_attribs are correctly connected to its feature_importances. After that you started connected the the feature_importances of the created extra_attribs to the first three entries of cat_one_hot_attribs.

Cool my computer is okay

@

What best for logiciel rest frond page

Hey,
I did not understand this step also :(

Thanks for your question @Ilya-Curie , and thanks for the excellent answer @TobiLang , that's exactly right.
The feature_importances list contains the importance of the model's input features, in the same order as they were used to train the model, so if we want to know the names of the most importance features, we must find out what the names of every input feature that was used to train the model, in the right order. That's what attributes is. To train the model, we used housing_prepared, which was created by the full_pipeline, like this:
class CombinedAttributesAdder(...):
    [...]

num_pipeline = Pipeline([
        ('imputer', SimpleImputer(strategy="median")),
        ('attribs_adder', CombinedAttributesAdder()),
        ('std_scaler', StandardScaler()),
    ])

full_pipeline = ColumnTransformer([
        ("num", num_pipeline, num_attribs),
        ("cat", OneHotEncoder(), cat_attribs),
    ])

housing_prepared = full_pipeline.fit_transform(housing)

The full_pipeline starts by applying the num_pipeline to the numerical attributes: this includes all the original attributes except "ocean_proximity": specifically, it's these 8 attributes: "longitude", "latitude", "housing_median_age", "total_rooms", "total_bedrooms", "population", "households", "median_income". The numerical pipeline starts with an imputer, which fills in missing values, but it does not change the list of feature names. Then comes the CombinedAttributesAdder, which adds three extra attributes: "rooms_per_hhold", "pop_per_hhold", "bedrooms_per_room". This is where these extra attributes come from. Next comes the standard scaler, which scales the values but does not change the list of feature names.
Back to the full_pipeline: it also applies a OneHotEncoder to the categorical attributes: in this case, there's just one categorical attribute, the "ocean_proximity". After one-hot encoding, this attribute gets replaced with one attribute per category. We need to list them in the correct order, which is the same order as in the categories_ attribute of the OneHotEncoder.
So the final list of attributes is:

"longitude", "latitude", "housing_median_age", "total_rooms", "total_bedrooms", "population", "households", "median_income" (the original numerical attributes)
plus: "rooms_per_hhold", "pop_per_hhold", "bedrooms_per_room" (added by the CombinedAttributesAdder)
plus: "<1H OCEAN", "INLAND", "ISLAND", "NEAR BAY", "NEAR OCEAN"

Once we have a list of feature names in the right order, we can use zip(feature_names, attributes) to match each importance value in the first list, with the right feature name in the second list.
Hope this helps.


Feature: log thrown exception

It's pretty hard to get a clue where in your template an exception was thrown.
The usual way is to test/disable each and every custom method and wait if you got the error.
Or you wrap your output of a method (only if it's a string) with a try/catch and return the exception's message.
This is pretty cumbersome.
I enhanced the logging of the errors in the Parser to log the exception as a whole.
Comparision:
OLD:
ERROR: Error rendering template. Cannot get identifier 'getTypeName'. Exception has been thrown by the target of an invocation. Source path: C:\Project\SomeFile.cs

NEW:
ERROR: Error rendering template. Cannot get identifier 'getTypeName'. Exception has been thrown by the target of an invocation. Source path: C:\Project\SomeFile.cs
System.InvalidOperationException: Oh oh, something happend here. where is the callstack? ---> System.Exception: Inner exception text for demonstration
   --- End of inner exception stack trace ---
   at __Typewriter.Template.getInterfaceName(String name)
   at __Typewriter.Template.getTypeName(Class c)

Notice the callstack which really points me the way where the exception was thrown.
getInterfaceName and getTypeName do exist in my template.
I hope you will accept the PR because it makes it so much easier to find errors.
The modification of the logging only changes the behavior of the Output window, not the errors in the visual studio error list for now.
Should the same error be shown in the error list as well?

Nice one @SeriousM!
The TargetInvocationException is probably never be the real error so I guess it would be even better if the inner exception is displayed in the error list as well.

Ok, done. I also added a hint to look into the output window for stacktrace information.

Sorry for the spam. I accidentally deleted the branch but I was able to restore it and reopened the PR.

This looks awesome, thanks!

Thanks!
Am 30.03.2016 15:58 schrieb "Fredrik Hagnelius"<EMAIL_ADDRESS>
Merged #105 https://github.com/frhagn/Typewriter/pull/105.
—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
https://github.com/frhagn/Typewriter/pull/105#event-608302638

may I ask for the next release? I guess after this one I'm done more or less.


RHOAIENG-16403: chore(Makefile): remove the now-unnecessary trailing slash (\s*;?\s*\\$)

https://issues.redhat.com/browse/RHOAIENG-16403
Follow-up to

https://github.com/opendatahub-io/notebooks/pull/791

Description
We're able to remove the unnecessary continuation lines from the Makefile, now that the recipe body is running in a single shell.
How Has This Been Tested?

https://github.com/jiridanek/notebooks/actions/runs/12320423761

Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

/lgtm

/lgtm
/override ci/prow/images

/approve
i went over the diff again, still looks good to me, giving it approve


Match the version of consoleplugin to odf operator

@SanjalKatiyar Please review

@TimothyAsirJeyasing plz update title correctly...

/retest

/retest

once backport PRs are up (after this master PR is merged), add the BZ number to the title of both PRs and make sure both PRs are added to "Links" section of the BZ... after that we can move BZ to "POST" and need to ask for "qa_ack" on the BZ... once done we can merge the backport PRs and move BZ to MODIFIED...

fix the build tests...

/cherry-pick release-4.13

/cherry-pick release-4.13-compatibility

/retest

/approve

/approve

/approve

@TimothyAsirJeyasing it does not work for me.

@TimothyAsirJeyasing Please rebase.

@TimothyAsirJeyasing plz resolve the conflicts...

/lgtm

/test odf-console-e2e-aws

plz resolve the conflicts...

Manually its applying now to the branches, Could you please cherry pick one more time

/cherry-pick release-4.13

/cherry-pick release-4.13-compatibility


SVG element position issue

I am facing issue with incorrect position of items in SVG. I am attaching my sample with SVG, could you please help me with it. Thanks
SVGExample1.zip

Hi Santosh,
Your SVG file contains elements which are not yet supported by Macaw. There are at least two uncovered features:

mm units instead of pixels (easy to support)
nested SVG parts (hard to support)

We can quickly fix the first issue, but it will take time for the second one. Feel free to create a PR to support this, or you need to find a way to use SVG without these features for now.


Add internal models for unit tests

Instead of using fixed path to URDF models, it would be useful to rely on local models like ROMEO.

Solved with #5.


Add StencilSwiftKit

What did

Up Stencil version to 0.15.1
Added StencilSwiftKit

Motivation
Add new nodes & filters dedicated to Swift code generation.

Why do we need StencilSwiftKit?

Why do we need StencilSwiftKit?

StencilSwiftKit will add more flexibility in code generation.

StencilSwiftKit adds 3 new dependencies that isn‘t good.
Current templates written without StencilSwiftKit. Could you please show how StencilSwiftKit can simplify templates?

Do you plan to use StencilSwiftKit in FigmaExport?

Yes, we are using StencilSwiftKit now, from my branch.
We use custom template in the project.
And we have several color assets whose names should be the same. Without filters like replace, lowerFirstWord and more this is impossible to implement...
P.S. It would be cool to add custom names for templates.


cmd/config: limit tap info output

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.

[x] Have you successfully run brew typecheck with your changes locally?
[ ] Have you successfully run brew tests with your changes locally?

https://github.com/Homebrew/brew/pull/16385#pullrequestreview-1794162825
We really only need to see info that is surprising or out of the norm. For that reason, we can skip showing the tap remote and the tap branch if they are what we expect them to be.
$ brew config
HOMEBREW_VERSION: 4.2.0-55-g857ab13
ORIGIN: https://github.com/Homebrew/brew
HEAD: 857ab132a2942c6519e8016b63c1d458ca2e90bf
Last commit: 3 minutes ago
Core tap HEAD: 2d28e828ac78151a5076fe1060beb53dabdaa02f
Core tap last commit: 63 minutes ago
Core tap JSON: 22 Dec 18:22 UTC
Core cask tap HEAD: 30d910ada9cd36305e6110130a8d6ee2e74e61f9
Core cask tap last commit: 59 minutes ago
Core cask tap JSON: 22 Dec 18:22 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_AUTOREMOVE: set
HOMEBREW_CASK_OPTS: []
HOMEBREW_DEVELOPER: set
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.F4lxiBJvUl/org.macosforge.xquartz:0
HOMEBREW_EDITOR: code
HOMEBREW_MAKE_JOBS: 4
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.1.4 => /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/bin/ruby
CPU: quad-core 64-bit ivybridge
Clang: 12.0.0 build 1200
Git: 2.24.3 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.7-x86_64
CLT: <IP_ADDRESS>.29
Xcode: 12.4

Compare to the output in https://github.com/Homebrew/brew/pull/16385.

@apainintheneck thanks!

ORIGIN: https://github.com/Homebrew/brew

Could probably skip this too if we really wanted and it's the default but no big deal either way.


:package: fix: Repair wheel also on macOS

Needed to patch delocate :/.
Adding libpq as static needs also additional code on the CMake side because we need to retrieve the libpq.a, while find_package(PostgreSQL) just finds the dynamic library it seems...
It was more code than just embedding the dynamic library...

@0x0L can you check the artifacts and see if they work on your side?

Ok, I checked and it works on a vanilla system...


Performance of gallery item detail - gen art

when you open a generative art in gallery detail, i experience quite a big performance drop

https://canary.kodadot.xyz/ahp/gallery/77-507
is there anything we can do to optimize the preview?
https://github.com/kodadot/nft-gallery/assets/90852205/b4782b72-23cd-4883-8506-deaaa5ee4536
scrolling is somewhat okay, but once I click on anything its quite slow

might be related with #9210, could be an issue from generative code


Remove unnecessary columns

Remove unnecessary blanks and columnsRemove unnecessary blanks and columns.
The last two rows contained two meaningless columns.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Good catch. Thanks for doing this!


Lua Error that I don't get with Groupie Disabled

Happens on join group and zone in I think. Conflict with other add-on?
28x ...I\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0-21.lua:470: attempt to compare number with string
[string "@ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0-21.lua"]:470: in function <...I\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:451>
Locals:
self =  {
0 =
}
event = "PLAYER_LOGIN"
spec = "2"
(*temporary) =  defined =[C]:-1
(*temporary) =  {
0 =
}
(*temporary) = "ACTIVE_TALENT_GROUP_CHANGED"
(*temporary) = true
(*temporary) = 135773
(*temporary) = 54
(*temporary) = "Unholy"
(*temporary) = "attempt to compare number with string"
GetSpecialization =  defined @Outfitter\Outfitter.lua:7907
CanPlayerUseTalentSpecUI =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:89
lib =  {
registry =  {
}
mixin =  {
}
upgrades =  {
}
currentSpec = "2"
_EnhanceDatabase =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:252
OnProfileDeleted =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:239
IterateDatabases =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:443
EnhanceOptions =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:384
EnhanceDatabase =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:263
eventFrame =  {
}
options =  {
}
}
isRetail = false
upgrades =  {
= true
}
UpgradeDatabase =  defined @ElvUI\Libraries\Core\LibDualSpec-1.0\LibDualSpec-1.0.lua:218
registry =  {
=  {
}
}
AceConfigRegistry3 =  {
RegisterOptionsTable =  defined @AnnoyingPopupRemover\Libs\AceConfigRegistry-3.0\AceConfigRegistry-3.0.lua:315
validated =  {
}
tables =  {
}
callbacks =  {
}
RegisterCallback =  defined @AnnoyingPopupRemover\Libs\CallbackHandler-1.0\CallbackHandler-1.0.lua:90
GetOptionsTable =  defined @AnnoyingPopupRemover\Libs\AceConfigRegistry-3.0\AceConfigRegistry-3.0.lua:360
UnregisterCallback =  defined @AnnoyingPopupRemover\Libs\CallbackHandler-1.0\CallbackHandler-1.0.lua:153
IterateOptionsTables =  defined @AnnoyingPopupRemover\Libs\AceConfigRegistry-3.0\AceConfigRegistry-3.0.lua:346
NotifyChange =  defined @AnnoyingPopupRemover\Libs\AceConfigRegistry-3.0\AceConfigRegistry-3.0.lua:289
ValidateOptionsTable =  defined @AnnoyingPopupRemover\Libs\AceConfigRegistry-3.0\AceConfigRegistry-3.0.lua:276
UnregisterAllCallbacks =  defined @AnnoyingPopupRemover\Libs\CallbackHandler-1.0\CallbackHandler-1.0.lua:174
}

I think this is the other add-on. But it doesn't seem to happen as much with Groupie disabled. Meh. Put a pin in this. We'll keep an eye on it.

This is a conflict between libdualspec and outfitter


MINOR: Use full package name when classes referenced in documentation

The metric.reporters description in the documentation says to implement the MetricReporter class, but the actual class is MetricsReporter. MetricsReporter.java
The configurations documentation is also inconsistent as some references to classes do not have the full package name while others do.
@ijuma

I totally forgot about this. Any updates on this? @ijuma


Log Page “Powered by” footer points to non-existent elmah.googlecode.com

The link at the footer of the ELMAH.AXD page is still pointing at http://elmah.googlecode.com/, which gives a 404 Not Found if followed.

This has been addressed in the code base to become 2.0. At this point, there's no plan to make a 1.x patch release so I am going to close this but thanks for taking the time to report the issue nonetheless.


update application to 0.0.54

UpdateBot pushed helm dependency: application to: 0.0.54

UpdateBot commands:
updatebot push-version --kind helm application 0.0.54

/test this
/approve


fix: unit shows empty if it's undefined in the legend table datastreams

This PR is for ticket https://github.com/awslabs/iot-app-kit/issues/2660 and updated the condition to check the unit in the legend datastream name. shows empty If the property has a unit undefined
Verifying Changes
Before:

After:

Legal
This project is available under the Apache 2.0 License.

LGTM


How to disable zoom in and how to set fix zoom in Performance Line Chart?

I want to set a fixed zoom in the performance line chart. if that's not possible then I want to disable auto-zoom in that chart.

for disable, set controller's these value
scaleXEnabled: false,
scaleYEnabled: falsse,
pinchZoomEnabled: false,
for set a fixed zoom, use controller.painter.zoom method


hover color change problem

when we hover on Question Papers in nave bar it turn black , @amanazads assign it to me
https://github.com/user-attachments/assets/a0b64856-d31a-432e-8008-ff6a28caf8e3

assigned


Documentation and examples of retrying expired tokens.

Please see notes at https://github.com/CDL-Dryad/dryad-product-roadmap/issues/162 .
Token expiry is part of the OAuth2 spec and client code needs to handle token expiry and retry when it becomes unauthorized because of expiry. Added documentation examples of a way to retry in Ruby.

This looks great. Thanks!


Majority of content in readme removed.

Was looking at the repo and noticed that almost all of the readme was erased in commit  3eb99dbe8378a347dd42a1ff87c89327e989443f. Seems like a mistake.

We're working on the readme(s), in the meantime, you can find the "old" readme here


Added support for parallel executions. Also fixed a bug in enhanced semantic roles.

@kwalcock: can you please take a look at this PR? I tried to implement transparent support for parallel executions. You may start with the Metal.getInferenceModel() method, and double check the Layers.clone() method next.
Thank you!

Thank you for the use case.  I keep updating my example with your good ideas and that documentation is still catching up.  In a later version of fatdynet than what you probably have, one should be able to do
  def cloneBuilder(builder: RnnBuilder): RnnBuilder = {
    val newBuilder = builder.clone()

The newBuilder.newGraph is needed or else a runtime exception is thrown.  I'm still working on the theory as to why.  The code looks good.  I will try out Metal.main as soon as I can.

Thanks @kwalcock!
Some unit tests are failing here. Can you please look into this?

Being dependent on a locally published snapshot, it's not going to work on Jenkins.  I will test locally for now.  So far I've made just one change to the C++ code that would necessitate recompiling.  Things can't be removed from Maven Central, so I'm not real enthusiastic about sending it (fatdynet) there yet.

The tests do still pass locally.  If I understand correctly, the parallel ability is not yet tested.

Not yet. A test of the parallel execution would be awesome :)

In cases where clone calls copy without changing any of the values, like
override def clone(): GreedyForwardLayer = copy()

one could just return the original:
override def clone(): GreedyForwardLayer = this

It would save some memory.

Someone left helpful hints in the C++ code that explains how newGraph fits in:
// CURRENT STATE | ACTION              | NEXT STATE
// --------------+---------------------+-----------------
// CREATED       | new_graph           | GRAPH_READY
// GRAPH_READY   | start_new_sequence  | READING_INPUT
// READING_INPUT | add_input           | READING_INPUT
// READING_INPUT | start_new_seqeunce  | READING_INPUT
// READING_INPUT | new_graph           | GRAPH_READY

Thanks for catching the clone() issue. I pushed the fix.

ViterbiForwardLayer is another opportunity.  That's the only other I noticed.

another good catch. fixed and pushed. Thanks!

For ParallelProcessorExample example, I get
Serial: 57 seconds
Parallel: 17 seconds
on 8 threads.

Thanks!
What about FastNlpProcessor? And FastNlpProcessorWithSemanticRoles?
On Wed, Sep 16, 2020 at 3:25 PM Keith Alcock<EMAIL_ADDRESS>wrote:

For ParallelProcessorExample example, I get
Serial: 57 seconds
Parallel: 17 seconds
on 8 threads.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/422#issuecomment-693697760,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TXQ62KZGHBP53FSNE3SGE3MHANCNFSM4ROFLFOQ
.

Thanks for the reminder.  I was just curious about whether it was working or not.  Here are the times for the difference processors on a collection of just 10 documents.  Each time the serial and parallel version got the same answer, but output for the various processors always differed:

Processor
Threading
Time (s)

CluProcessor
Serial
57

Parallel
17

FastNLPProcessor
Serial
31

Parallal
13

FastNLProcessorWithSemanticRoles
Serial
52

Parallel
20

Thanks! I think the correct comparison is between CluProcessor and FastNLPProcessorWithSemanticRoles. Good to see the times are about the same.

So, ready to release fatdynet, and then merge in master for processors?

Btw, in that runtime analysis, did you remove the model loading time from the runtime?
I think CluProcessor spends more loading than FastNlp

River just stopped responding.

This ParallelProcessorExample annotates one sentence to prime the pump and starts timing after that.  However, the test above does not really show the advantage of the threaded dynet because the gains could come through parallelization of the other stages.  Here are times for the current master branch, before dynet was threaded.  Differences in the parallel times are mostly attributable to dynet speedup then.  One test goes from 18 to 13 then, both parallel.  The others have problems.
Eidos is using 8.1.3, so a problem may have been introduced recently.  I will redo with 8.1.3.

master branch
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
58
24
52

Parallel
crash
18
crash

parallel branch
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
57
31
52

Parallel
17
13
20

Thanks!

The crashes are caused by running the tests in IntelliJ.  The resources cannot be found in that environment.  That should be fixed.
The tests run in sbt and the times are below.  I do not know what changes are in each version, so this is single blind and I'm not sure whether they make sense.  Times are in seconds, copied directly from the output.

8.1.3-sbt
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
29.945
29.511
49.905

Parallel
30.416
10.066
36.987

master-sbt
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
61.912
20.727
51.098

Parallel
56.073
10.356
36.758

parallel-sbt
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
54.486
21.332
54.003

Parallel
19.742
10.277
18.391

Thanks @kwalcock! Not to give you additional work, but would it be possible to average these results across, say, 3 runs?
I was worries about the difference between the master CluProcessor and v81.3, but then I realized we added syntactic parsing in 8.2.0, which most likely explains the difference. The good news is that we have some ideas how to make it faster, so it's not too bad.

And, in any case, I think we might be ready to release?

Yes, I can average.  I wonder how much variation might be attributable to the set of files that was accidentally selected.  An average would be confirmation of a bias.  Would it be better to increase the number of files instead?
River crashed before Jenkins could run, so I'm trying to resubmit the job.

Yes, adding more text would be useful. Thanks!
On Thu, Sep 17, 2020 at 17:40 Keith Alcock<EMAIL_ADDRESS>wrote:

Yes, I can average. I wonder how much variation might be attributable to
the set of files that was accidentally selected. An average would be
confirmation of a bias. Would it be better to increase the number of files
instead?
River crashed before Jenkins could run, so I'm trying to resubmit the job.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/422#issuecomment-694578295,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TUXLT6CAYPL4ETB7ULSGKUBBANCNFSM4ROFLFOQ
.

A bug report issue is likely to be filed soon.  An exception is being thrown.

Thanks! The bug is in processors or fatdynet?

Processors, I think.  I'm still trying to figure out which file it was and reproduce it.

#424, but it doesn't seem to relate to parallelization.  Still fixing it might spare us from needing a point revision.

Here is a preview with just the parallel branch of processors and times in seconds:
Processing time

parallel-sbt
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
5035.115
1416.642
4929.366

Parallel
3000.88
537.953
2268.556

Startup time

parallel-sbt
CluProcessor
FastNLPProcessor
FastNLPProcessorWithSemanticRoles

Serial
112.743
25.742
98.258

Parallel
82.184
25.944
95.624

Thanks! How many threads did you use?

8 threads

Hmm... Very small speedup then...
On Mon, Sep 21, 2020 at 20:59 Keith Alcock<EMAIL_ADDRESS>wrote:

8 threads
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/422#issuecomment-696496146,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TVK237EBSKJCPVACSTSHAOL5ANCNFSM4ROFLFOQ
.

I wonder if there is another synchronized left over somewhere in the Scala/SWIG wrapper? I suspect there are no critical sections in the C++ code?

There are small ones sprinkled around in C++.  I was tempted to remove all global variables, but then passing them around would be a disaster.  There is a small number in the Java code as well, but also of very limited scope.
Of all the things that the processor does while annotating, how many were already parallel?

[ ] mkDocument
[ ] tagPartsOfSpeech
[ ] lemmatize
[ ] recognizeNamedEntities
[ ] parse
[ ] chunking
[ ] relationExtraction
[ ] srl
[ ] resolveCoreference
[ ] discourse

Does StanfordNLP have any limitations?  At some point, adding threads will not help and it will be better to start up multiple copies of Eidos.  On the servers that will be long before the number of processors is used up.

I checked all these methods, and none are synchronized. I do use one synchronized in ConstEmbeddingsGlove.apply(), but that one is called just once.
I have an idea on how to debug this issue, based on the observation that if we sum up the runtimes in CluProcessor methods during the parallel execution, the sum should not be higher than the seq times. If it is, it means the threads are blocking each other somewhere, and we have to dig deeper.
In CluProcessor, the key methods are:
tagSentence()
nerSentence()
parseSentence()
srlSentence()
Thanks!!

I uploaded some Flight Recorder information, but so far it has not been especially helpful to me: https://drive.google.com/file/d/1oviO6z2RgtjDPa5tRYFLOEDrMHVErSoD/view?usp=sharing
This is for a FastNLPProcessorWithSemanticRoles in the parallel branch with just 10 files of varying sizes, so it is only for a short time that 8 files can be processed in parallel.  After that, individual threads don't have anything to do and the efficiency goes down.  With bad luck, the last thread processes some really big file and the other 7 are idle for a long time.  The most recent timings were for 52 files in an attempt to avoid the phenomenon.  It would help for me to sort files by size and process from longest to shortest and I'll try that, but it's not what I was hoping for.  For testing I could also even do the exact same file in different threads to test the limits of parallelization.

Hmm, good point. Maybe we should try 8 files of the same size, so we know that the max speedup is up to 8? If these files are dominated by a big one, parallelism is limited...

FWIW I'm reading up on Scala parallelization at places like https://docs.scala-lang.org/overviews/parallel-collections/overview.html and https://docs.scala-lang.org/overviews/parallel-collections/performance.html#how-big-should-a-collection-be-to-go-parallel.  I've been using the easiest thing that works/compiles, nothing that was based on actual effectiveness.

I suspect you called .par on a list of files, which is Ok. But because of the wait at the end on the whole collection, all short jobs have to wait on the big one (if it exists).

stating the obvious here :)

I have all sorts of numbers, but have not found an explanation or workaround for this kind of limitation.  I've almost decided it's something like memory contention or garbage collection overhead.

Thanks!
Have you tried using the same number of threads, but an increasing amount of RAM available to DyNet?

No.  One thing that had occurred to me is that the memory in DyNet could be fragmenting so that malloc takes longer to find an appropriate space and free takes longer to combine regions.  I doubt that's the problem, though.  I'll see if more memory can bend the line up.

@kwalcock: In any case, should we merge this in master? I am not sure we should wait.

While observing execution related to that crash issue, I notice in the IntelliJ debugger a lot of threads in the MONITOR state.  Something isn't quite right and it seems to be a clue.

Good finding. So, that means we did miss a sync somewhere in the Java/Scala
code, no?
On Thu, Oct 1, 2020 at 17:36 Keith Alcock<EMAIL_ADDRESS>wrote:

While observing execution related to that crash issue, I notice in the
IntelliJ debugger a lot of threads in the MONITOR state. Something isn't
quite right and it seems to be a clue.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/422#issuecomment-702466695,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TVPQPPNOP5SWOWHZC3SIUN7RANCNFSM4ROFLFOQ
.

It looks like a false alarm.  It went away on the next run.  I think something was stale.  Still making sure.

I've been trying to run this in an Ubuntu18 virtual machine with 16mb of memory.  It will run serially but not in parallel.  The error message seems almost unrelated, but it must be some side effect of memory shortage.
[error] (run-main-4) java.lang.RuntimeException: ERROR: parameter glove.matrixResourceName must be defined!
[error] java.lang.RuntimeException: ERROR: parameter glove.matrixResourceName must be defined!

This doesn't look familiar, does it?

It's probably coming from here:
https://github.com/clulab/processors/blob/master/main/src/main/scala/org/clulab/dynet/ConstEmbeddingsGlove.scala#L64
But I have no idea why...

The test program I'm running in processors seems to max out at a 6x speedup or so (graph above).  I wanted to make sure that isn't some general limit based on something like the memory bus bandwidth in the servers.  This memory intensive C++ program was able to get 20x.  I'm now suspicious of the malloc that is part of the forward pass in dynet and trying to get a profiler to show some insight.

This is encouraging! Are you running the C and Scala tests on the same machine?

The two graphs above are both from Jenny.  I'm not able to do the C-level profiling there, though, and that's the reason for yesterday's failed attempt to get processors to run locally on Ubuntu 18.  It was probably just something I ate/installed.  Processors just started working on Ubuntu 20, though, so I'll see if the running dynet can explain itself.  It wasn't anything obvious to me at the Java level.

BTW the graph above showing the 20x speedup is for a C++ program unrelated to Dynet, and that was probably responsible for the confusion this morning.  I was about to compare it to speedup for a C++ Dynet program and may see something closer to the Java 6x speedup than to this 20x.  That would mean that the problem isn't with Java but is already in the C++ version, possibly in the synchronized memory allocation.  Details would be investigated initially with perf tools that aren't installed for the servers, but they are running locally now.

Got it. Thank you!
On Mon, Oct 12, 2020 at 12:27 PM Keith Alcock<EMAIL_ADDRESS>wrote:

BTW the graph above showing the 20x speedup is for a C++ program unrelated
to Dynet, and that was probably responsible for the confusion this morning.
I was about to compare it to speedup for a C++ Dynet program and may see
something closer to the Java 6x speedup than to this 20x. That would mean
that the problem isn't with Java but is already in the C++ version,
possibly in the synchronized memory allocation. Details would be
investigated initially with perf tools that aren't installed for the
servers, but they are running locally now.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/clulab/processors/pull/422#issuecomment-707249919,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI75TUUSFNQI6VNQWBAXDTSKM4ATANCNFSM4ROFLFOQ
.

This PR will be abandoned when it is finally replaced by updated parallel processing code.

@MihaiSurdeanu ,there are graphs for TorchScript multi-threaded performance at https://github.com/kwalcock/torchscript/issues/1.


forward pass dimenssion error

When I copied an example from the colab for graph node classification, I replace the dataset with my own inmemeory dataset.  It causes trouble during the forward pass  when the first self.conv1() in the forward() function is executing.  The error is  in the following

GCN(
(conv1): SGConv(5, 64, K=1)
(conv2): SGConv(64, 64, K=1)
(conv3): SGConv(64, 64, K=1)
(lin): Linear(in_features=64, out_features=2, bias=True)
)
Traceback (most recent call last):
**return out.scatter_add_(dim, index, src)**

RuntimeError: index 264 is out of bounds for dimension 0 with size 264
But I checked the dimension of the input x. It matches. So I am confused what happened.

import torch
from torch_geometric.datasets import TUDataset
from gcn_fmri import fMRIDataset

dataset = fMRIDataset(root='./')

print()
print(f'Dataset: {dataset}:')
print('====================')
print(f'Number of graphs: {len(dataset)}')
print(f'Number of features: {dataset.num_features}')
print(f'Number of classes: {dataset.num_classes}')

data = dataset[2]  # Get the first graph object.

print()
print(data)
print('=============================================================')

# Gather some statistics about the first graph.
print(f'Number of nodes: {data.num_nodes}')
print(f'Number of edges: {data.num_edges}')
print(f'Average node degree: {data.num_edges / data.num_nodes:.2f}')
print(f'Contains isolated nodes: {data.contains_isolated_nodes()}')
print(f'Contains self-loops: {data.contains_self_loops()}')
# print(f'Is undirected: {data.is_undirected()}')

# print the node attributes
print(f'node no: {data.num_nodes}')
print(f'node features no: {data.num_node_features}')
print(f'features no: {data.num_features}')
print(f'faces: {data.face}')

# print the edge attributes
print(f'Number of edges: {data.num_edges}')
print(f'edge list: {data.edge_index}')
print(f'edge atrributed: {data.edge_attr}')
print(f'edge features no: {data.num_edge_features}')

torch.manual_seed(12345)
dataset = dataset.shuffle()

train_dataset = dataset[:653]
test_dataset = dataset[653:]

print(f'Number of training graphs: {len(train_dataset)}')
print(f'Number of test graphs: {len(test_dataset)}')

from torch_geometric.data import DataLoader

train_loader = DataLoader(train_dataset, batch_size=1, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=1, shuffle=False)

for step, data in enumerate(train_loader):
    print(f'Step {step + 1}:')
    print('=======')
    print(f'Number of graphs in the current batch: {data.num_graphs}')
    print(data)
    print()

from torch.nn import Linear
import torch.nn.functional as F
from torch_geometric.nn import GCNConv
from torch_geometric.nn import global_mean_pool

class GCN(torch.nn.Module):
    def __init__(self, hidden_channels):
        super(GCN, self).__init__()
        torch.manual_seed(12345)
        self.conv1 = GCNConv(dataset.num_node_features, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, hidden_channels)
        self.conv3 = GCNConv(hidden_channels, hidden_channels)
        self.lin = Linear(hidden_channels, dataset.num_classes)

    def forward(self, x, edge_index, batch):
        # 1. Obtain node embeddings
        x = self.conv1(x, edge_index)
        x = x.relu()
        x = self.conv2(x, edge_index)
        x = x.relu()
        x = self.conv3(x, edge_index)

        # 2. Readout layer
        x = global_mean_pool(x, batch)  # [batch_size, hidden_channels]

        # 3. Apply a final classifier
        x = F.dropout(x, p=0.5, training=self.training)
        x = self.lin(x)

        return x

model = GCN(hidden_channels=64)
print(model)

optimizer = torch.optim.Adam(model.parameters(), lr=0.01)
criterion = torch.nn.CrossEntropyLoss()

def train():
    model.train()

    for data in train_loader:  # Iterate in batches over the training dataset.
         out = model(data.x, data.edge_index, data.batch)  # Perform a single forward pass.
         loss = criterion(out, data.y)  # Compute the loss.
         loss.backward()  # Derive gradients.
         optimizer.step()  # Update parameters based on gradients.
         optimizer.zero_grad()  # Clear gradients.

def test(loader):
     model.eval()

     correct = 0
     for data in loader:  # Iterate in batches over the training/test dataset.
         out = model(data.x, data.edge_index, data.batch)
         pred = out.argmax(dim=1)  # Use the class with highest probability.
         correct += int((pred == data.y).sum())  # Check against ground-truth labels.
     return correct / len(loader.dataset)  # Derive ratio of correct predictions.

for epoch in range(1, 201):
    train()
    train_acc = test(train_loader)
    test_acc = test(test_loader)
    print(f'Epoch: {epoch:03d}, Train Acc: {train_acc:.4f}, Test Acc: {test_acc:.4f}')

``

Please help. I have stalled on this one for a long time. I am not suere whether I created the wrong datatset or something else.

This problem has sovled.  The erros was caused by the index out of bound. Edge list index of Pytorch Geometric starts from 0 instead of 1.
Thanks, .


Support defmt?

Should we add support for defmt via a feature flag?

Closing - I think not. Sorry for the noise.

lol


Run defined command from arg (for use in keybinding)

Hi! AFAIU all commands to run must be defined in commands.commands, however I want to define them (for example a sequence) in keybindings.json.
For example:
{
    "key": "cmd+r",
    "command": "commands.executeSequence",
    "args": [
        {
            "sequence": ["editor.action.toggleMinimap", "workbench.action.toggleStatusbarVisibility"]
        }
    ]
},

Motivation: easier to use in shared keybindings

You already can:
{
    "key": "ctrl+shift+9",
    "command": "commands.run",
    "args": [
        "editor.action.toggleMinimap",
        "workbench.action.toggleSidebarVisibility",
    ],
},

Oh this command wasn't in manifest, so I didn't see it Feature Contributions of VSCode. Also I think it'd good to remove commands and settings from Readme since they're out of sync we have Feature Contributions tab in VSCode.
So, missing commands should be added. Also, I created framework, that allows to use onCommands activation event and shows TS errors if you use command that is not in manifest. I hope it would help you a lot, tell WDYT

Anyway, you answered my question so thank you.

@usernamehw it doesn't work for me. After I added your example to keybindings.json and pressing the keys, I've got error:  command 'commands.run' not found. Would you be so nice and tell me how to fix it, please? :)

Install extension
Ensure it gets activated

@usernamehw will this extension support web some day? I understand you have millions of features here that might be hard to adopt, but Im using only simple commands.run because vscode doesnt have support out of the box for running more than command for some mysterious reason. Can you also please add completions for commands inside command.run args (i can work on it)

@wdanilo It should work, extension activates on startup and additionally has activation event for this command: https://github.com/usernamehw/vscode-commands/blob/2e5e580098d1228df1aaeb2bed6e7907976ff66c/package.json#L44-L46
Like previous comment: you should check if extension installed and activated. Possible to see with workbench.action.showRuntimeExtensions Title Developer: Show Running Extensions
Maybe there are errors in devtools workbench.action.toggleDevTools title: Developer: Toggle Developer Tools?

@zardoy Contribution point is kind of bloated because VSCode doesn't support referencing in json schema for configuration:https://github.com/usernamehw/vscode-commands/blob/2e5e580098d1228df1aaeb2bed6e7907976ff66c/package.json#L61-L846 - Things are duplicated like 8 times. Do you want autocomplete only for properties that have autocomplete or for all? It won't work for things like extension contributed things, unlike commands.suggestCommands command.

@zardoy I'll try to make a web version next week. It's possible to do it without codespaces, right?

It's possible to do it without codespaces account, right?

Of course, simplest way: https://vscode.dev (I'm actually using it sometimes for repros, as opening new incognito tab is faster than installing/updating Insiders). I remember vscode folks were forcing vscode.dev and . GitHub keybinding for a like 8 months

@usernamehw hmm, it still does not work for me. What I've checked:

Developer: Show Running Extensions - I can see "Commands 1.4.3" running with caption "Activation: 6ms".
I restarted the IDE after installed the extension (full restart).
In the dev console I see only "𝗙𝘂𝗻𝗱𝗿𝗮𝗶𝘀𝗶𝗻𝗴 𝗔𝗻𝗻𝗼𝘂𝗻𝗰𝗲𝗺𝗲𝗻𝘁" but not errors.

Btw, I'm running M2 MacOS, but I hope it does not affect it.

@wdanilo Yeah, that's not the right, version is 1.6.0. Hashtag #notMyExtension...

Oh wow, this was so confusing. Damn. They should not allow for multiple extensions with the same name. Thank you!