Ambilight+Hue choppy transitions

I'm using zigbee2mqtt to control HA lights. When I enable Ambilight+Hue on my Philips TV with emulated Hue, I can see lot of not smooth color changes (instant changes). When I trace mqtt messages, I see that message is sended to lights 3-8 times per second with transition parameter from 0.03 to 0.08s as you can see on the screen from MQTT Explorer:

It looks that this is too small amount for transition time, and transition is instant... I think also that throttling doesn't work in Ambilight+Hue mode (8 messages per second is way too much for non Hue bulbs I think...)
diyHue is behaving differently - transition is longer and smooth, and there is less noticable color changes, so color is little laggy comparing to TV ambilight but it feels much smoother
EDIT:
Also I noticed unhandled message in Zigbee2Mqtt sended on toggling Ambilight+Hue off:
Publish 'set' 'effect' to 'Taśma LED RGBW - salon (prawa)' failed: 'Error: 'none' not allowed, choose between: blink,breathe,okay,channel_change,finish_effect,stop_effect'

I've tested with ZHA lights and ESPHome driven LED strips and that worked very smooth without throttling (so all packets are sent to the lights). In your case I think throttling is needed for smoother results.
Do you have access to the data directory ? There's a file in there called emulated_hue.json.
Each light has a config in that file and a setting called "entertainment_throttle".
The default is 0 (no throttling at all so send all packets to the light).
You can put a value in there in milliseconds. For example start with 500 and see what it brings and decrease until it is less attractive.
Remember to stop the emulator before you make changes to the file, otherwise they'll be overwritten.
Please let me know the results so we can set those as default for a zigbee2mqtt light

I've tested with ZHA lights and ESPHome driven LED strips and that worked very smooth without throttling (so all packets are sent to the lights). In your case I think throttling is needed for smoother results.
Do you have access to the data directory ? There's a file in there called emulated_hue.json.
Each light has a config in that file and a setting called "entertainment_throttle".
The default is 0 (no throttling at all so send all packets to the light).
You can put a value in there in milliseconds. For example start with 500 and see what it brings and decrease until it is less attractive.
Remember to stop the emulator before you make changes to the file, otherwise they'll be overwritten.
Please let me know the results so we can set those as default for a zigbee2mqtt light

I've set "entertainment_throttle": 500 to those 2 lights I'm using with Ambilight+Hue but that didn't make any difference... Offcourse I've resstarted addon and checked if my new values are still there ;)
Still I see mqtt messege each 0.07s to 0.2s with same transition values...
Oh - my non Android Philips TV doesn't use entertainment mode for Ambilight+Hue... It just keeping puting light set messages, for example I see lot of this in Emulated Hue log:
2021-01-01 23:58:52,214 DEBUG emulated_hue.api -- [<IP_ADDRESS>] PUT /api/550a2fa9c65448dca682/lights/17/state
2021-01-01 23:58:52,216 DEBUG emulated_hue.api -- {'xy': [0.26169338822364807, 0.20043809711933136], 'transitiontime': 10}

I've set "entertainment_throttle": 500 to those 2 lights I'm using with Ambilight+Hue but that didn't make any difference... Offcourse I've resstarted addon and checked if my new values are still there ;)
Still I see mqtt messege each 0.07s to 0.2s with same transition values...
Oh - my non Android Philips TV doesn't use entertainment mode for Ambilight+Hue... It just keeping puting light set messages, for example I see lot of this in Emulated Hue log:
2021-01-01 23:58:52,214 DEBUG emulated_hue.api -- [<IP_ADDRESS>] PUT /api/550a2fa9c65448dca682/lights/17/state
2021-01-01 23:58:52,216 DEBUG emulated_hue.api -- {'xy': [0.26169338822364807, 0.20043809711933136], 'transitiontime': 10}

Ah, that explains it. Your TV has the first version of Ambilight+Hue which doesn't utilize the entertainment mode.
It is really strange because your TV sends transitiontime of 10 (which is 10*100ms = 1 second) while you see a transitiontime 0f 0.3 in your zigbee2mqtt logs. So somewhere in the path that transitiontime got changed.
I'll have a quick look at zigbee2mqtt's internals if I can spot what's going on.

Ah, that explains it. Your TV has the first version of Ambilight+Hue which doesn't utilize the entertainment mode.
It is really strange because your TV sends transitiontime of 10 (which is 10*100ms = 1 second) while you see a transitiontime 0f 0.3 in your zigbee2mqtt logs. So somewhere in the path that transitiontime got changed.
I'll have a quick look at zigbee2mqtt's internals if I can spot what's going on.

That's independent examples, but I see correlation - if in emulated_hue is 'transitiontime': 10 then in mqtt message I see "transition": 0.1; if in emulated_hue is 'transitiontime': 7 then in mqtt message I see "transition": 0.07; and so on...

That's independent examples, but I see correlation - if in emulated_hue is 'transitiontime': 10 then in mqtt message I see "transition": 0.1; if in emulated_hue is 'transitiontime': 7 then in mqtt message I see "transition": 0.07; and so on...

OK, so that is correct. Maybe diyhue alters the messages and sends another transitiontime instead. We just forward the messages as-is to Home Assistant

OK, so that is correct. Maybe diyhue alters the messages and sends another transitiontime instead. We just forward the messages as-is to Home Assistant

diyHue divides transition values by 10:
            state['transition'] = value / 10
https://github.com/diyhue/diyHue/blob/dfc6c158e14a1789eb446f98e1186356103bff90/BridgeEmulator/protocols/mqtt.py#L69
So 'transitiontime': 10 will be "transition": 1 in mqtt message; 'transitiontime': 7 will be "transition": 0.7, and so on...

diyHue divides transition values by 10:
            state['transition'] = value / 10
https://github.com/diyhue/diyHue/blob/dfc6c158e14a1789eb446f98e1186356103bff90/BridgeEmulator/protocols/mqtt.py#L69
So 'transitiontime': 10 will be "transition": 1 in mqtt message; 'transitiontime': 7 will be "transition": 0.7, and so on...

we do too:
https://github.com/marcelveldt/hass_emulated_hue/blob/master/emulated_hue/api.py#L625

we do too:
https://github.com/marcelveldt/hass_emulated_hue/blob/master/emulated_hue/api.py#L625

Yes, but here is dividing by 100 not by 10:
transitiontime = request_data[const.HUE_ATTR_TRANSITION] / 100

Yes, but here is dividing by 100 not by 10:
transitiontime = request_data[const.HUE_ATTR_TRANSITION] / 100

Oops, your are correct. Can you adjust that on your dev setup to see if it solves the issue ?

Oops, your are correct. Can you adjust that on your dev setup to see if it solves the issue ?

I've checked in the change btw and will trigger a new docker build

I've checked in the change btw and will trigger a new docker build

Transition is now smooth after changing division to 10, so this fixes choppy color changes. But I have second request - can You please add throttling also for standard api? Not only for entertainment mode? I see that sometimes on dynamic scenes it can generate to 15-20 messages per second! That has negative impact on performance not only on emulated_hue but whole HA and other addons...

Transition is now smooth after changing division to 10, so this fixes choppy color changes. But I have second request - can You please add throttling also for standard api? Not only for entertainment mode? I see that sometimes on dynamic scenes it can generate to 15-20 messages per second! That has negative impact on performance not only on emulated_hue but whole HA and other addons...

Sure, I think that will be possible but it will require some tinkering.

Sure, I think that will be possible but it will require some tinkering.

Thanks in advice! I'm running on Raspberry Pi 3

Thanks in advice! I'm running on Raspberry Pi 3

Ah yes, that explains it I guess. I was testing with several ZHA and ESPHome lights in entertainment mode (20+ messages per second to each light) and I did not notice any delays or performance issues but HA is running on a NUC in my case.
I'll implement some basic throttling and let you know.

Ah yes, that explains it I guess. I was testing with several ZHA and ESPHome lights in entertainment mode (20+ messages per second to each light) and I did not notice any delays or performance issues but HA is running on a NUC in my case.
I'll implement some basic throttling and let you know.

As a side note, you may want to look into better hardware as the hue api is generally pretty resource intensive with large amounts of packets being sent. An old laptop or desktop may do wonders.

As a side note, you may want to look into better hardware as the hue api is generally pretty resource intensive with large amounts of packets being sent. An old laptop or desktop may do wonders.

I've changed topic to better describe to what this issue transformed currently :)

I've changed topic to better describe to what this issue transformed currently :)


:sparkles: Adds support for ENS resolution

⚠ Part of: https://github.com/Shopify/blockchain-components/issues/4
What is the context for these changes?

Adds support for ENS resolution.

❗ Note
This makes use of the provider functions which can potentially lead to rate limited providers. For the time being, we will allow the use of ENS resolution without non-public providers being present. However, we will output a console warning when the developers are utilizing only a public provider.

This expands on our tests and test utilities as well as providing ens forward and reverse resolution. That means we use the address provided at connection to look up an ENS name (if present), and we then utilize forward resolution to double check that the ENS name provided points back to the address we originally looked up for.
Demonstration

Video demonstration

https://user-images.githubusercontent.com/4250423/219260209-2a4ea2a9-f450-49af-b38b-6903da68659f.mov

How can this be tophatted?

I will provide a CodeSandbox sample after a /snapit command has been run.
CodeSandbox Demo
🎩 Checklist

[ ] Tested on mobile
[x] Tested on multiple browsers
[x] Includes unit tests
[ ] ~Tested for accessibility~
[ ] ~Updated relevant documentation for the changes (if necessary)~

/snapit

I haven't looked at the code, but I tested it, it works for normal domain and subdomain, but I noticed a delay, also visible in your video, where the address first shows up and then gets replaced by the ENS name.
Could we do one of the following:
a) Show a loading indicator next to the address.
b) Eager fetch the ENS during the signing step similarly to how it was done in the delegate cash?

Looks good to me. I don't have an ENS so I'm not sure how I could tophat.

Sent you a message about this 👀

Are you still planning on adding tests for useMiddleware? No rush, just wondering.

Yep! I'd like to do that at some point. I started adding some test infra (only a file or two) which would allow me to add those tests and then stashed it because the PR was growing too much. I'm also open to someone else picking that work up if they'd like!


Correct some spelling mistakes

Have changed 'AS' to 'ASR', 'pretraind' to 'pretrained'.

thanks!


zipped epub is not working on iOS

Zipped epub is not loaded on my iPhone5S (ios 8.1.3).
both browser (chrome and safari) is not working.
The indicator is animating forever.
additional info:

unzipped epub is loaded well.
ios simulator(iPhone6) shows same results
android(4.4.2) works well on both (zipped and unzipped)

I believe this is the same issue: https://github.com/readium/readium-js/issues/72#issuecomment-60201123

Adding two lines of code fixes this issue.
zip-ext.js -> HttpRangeReader -> init (add below line after request.open)
request.setRequestHeader("If-None-Match","webkit-no-cache");
zip-ext.js -> HttpRangeReader -> readArrayBuffer (add below line after request.open)
request.setRequestHeader("If-None-Match","webkit-no-cache");

Reopening, as seems different issue to I believe this is the same issue: https://github.com/readium/readium-js/issues/72#issuecomment-242793007

This should be fixed by PR https://github.com/readium/readium-js/pull/193 for the general case.
There should be no need to patch zip-ext.js with respect to fetching byte ranges (HTTP 1.1 partial requests).


Built Chromium 83 refuses to start

I have the following script to build UGC:
#!/bin/sh
UGC_TARGET="/path/to/chrome_install_directory/"
cd "$HOME"
mkdir kocsog_gugli 2>/dev/null
cd kocsog_gugli
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH="$PATH":"$HOME"/kocsog_gugli/depot_tools
export CPATH="$CPATH":/opt/gtk3/include/gtk-3.0/:/opt/gtk3/include/gtk-3.0/unix-print/
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH":/usr/lib/x86_64-linux-gnu/pkgconfig/:/opt/gtk3/lib/pkgconfig/
mkdir chromium 2>/dev/null
cd chromium/
fetch --nohooks chromium
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
cd src
git checkout 83.0.4103.116
gclient sync -D
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
git clone https://github.com/Eloston/ungoogled-chromium/
cd ungoogled-chromium/
git checkout 83.0.4103.116-1
cd ..
find ungoogled-chromium/patches/ -name '*.patch' -exec mv {} ./ \;
rm -rf ungoogled-chromium
rm event.patch icu.patch icu57.patch icu-new.patch fingerprinting-flags-client-rects-and-measuretext.patch flag-fingerprinting-canvas-image-data-noise.patch
for PATCH in *.patch;
do
	patch -p1 -f < "$PATCH";
done
rm *.patch
sed -i 's|client_->GetPrefs()->GetBoolean(prefs::kGoogleServicesConsentedToSync)|false|g' components/signin/internal/identity_manager/primary_account_manager.cc
sed -i 's|switches::kPDFPluginName|"DummyPDFPlugin"|g' chrome/browser/about_flags.cc
wget https://raw.githubusercontent.com/Eloston/ungoogled-chromium/83.0.4103.116-1/flags.gn
echo 'use_gnome_keyring=false' >> flags.gn
echo 'use_pulseaudio=false' >> flags.gn
echo 'is_official_build=false' >> flags.gn
gn gen out/Release "--args=`cat flags.gn | tr '\n' ' '`"
gn args --list out/Release/ > ninja.settings
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
autoninja -C out/Release chrome
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
cd out/Release
mkdir "$UGC_TARGET""/ungoogled-chromium-83-devuan2-gtk3-portable"
cp -R locales chrome *.pak *.info *.so *.bin *.dat "$UGC_TARGET""/ungoogled-chromium-83-devuan2-gtk3-portable/"
free > /dev/null && sync > /dev/null && echo 3 > /proc/sys/vm/drop_caches && free > /dev/null
cd "$HOME"
rm -rf kocsog_gugli/

The build works, but then (after i symlinked the gtk and gdk libs into it's dir), when i try to start the resulting binary:
LANGUAGE=en /opt/ungoogled-chromium-83-devuan2-gtk3-portable/chrome --no-sandbox --user-data-dir --ignore-certificate-errors

I get this error message:
[23881:23881:0320/230918.375058:INFO:content_main_runner_impl.cc(976)] Chrome is running in full browser mode.
Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
[23881:23881:0320/230919.239501:ERROR:browser_dm_token_storage_linux.cc(100)] Error: /etc/machine-id contains 0 characters (32 were expected).
[23881:23881:0320/230919.373735:FATAL:remote.h(252)] Check failed: pending_remote.is_valid().
#0 0x7f150bf42dff base::debug::CollectStackTrace()

Received signal 6
#0 0x7f150bf42dff base::debug::CollectStackTrace()
  r8:<PHONE_NUMBER>000000  r9: 00007ffc9de56be0 r10:<PHONE_NUMBER>000008 r11:<PHONE_NUMBER>000246
 r12: 00007ffc9de58e60 r13:<PHONE_NUMBER>000000 r14: 00007ffc9de58d08 r15: 00007f150c042f90
  di:<PHONE_NUMBER>000002  si: 00007ffc9de56be0  bp: 00007ffc9de56e20  bx:<PHONE_NUMBER>000006
  dx:<PHONE_NUMBER>000000  ax:<PHONE_NUMBER>000000  cx: 00007f14e01b5fff  sp: 00007ffc9de56c58
  ip: 00007f14e01b5fff efl:<PHONE_NUMBER>000246 cgf: 002b000000000033 erf:<PHONE_NUMBER>000000
 trp:<PHONE_NUMBER>000000 msk:<PHONE_NUMBER>000000 cr2:<PHONE_NUMBER>000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

[23914:23914:0320/230934.721688:INFO:child_thread_impl.cc(839)] ChildThreadImpl::EnsureConnected()
[23910:23910:0100/000000.255213:INFO:child_thread_impl.cc(839)] ChildThreadImpl::EnsureConnected()

If i create /etc/machine-id with 32 chars inside (what is this BTW?), then the message changes:
[24467:24467:0320/231158.128724:ERROR:file_util_posix.cc(895)] fallocate: Operation not supported (95)
[24467:24467:0320/231158.144703:INFO:content_main_runner_impl.cc(976)] Chrome is running in full browser mode.
Fontconfig warning: "/etc/fonts/fonts.conf", line 100: unknown element "blank"
[24467:24467:0320/231158.487073:FATAL:remote.h(252)] Check failed: pending_remote.is_valid().
#0 0x7f0c3aa3adff base::debug::CollectStackTrace()

Received signal 6
#0 0x7f0c3aa3adff base::debug::CollectStackTrace()
  r8:<PHONE_NUMBER>000000  r9: 00007ffc8df05d70 r10:<PHONE_NUMBER>000008 r11:<PHONE_NUMBER>000246
 r12: 00007ffc8df07ff0 r13:<PHONE_NUMBER>000000 r14: 00007ffc8df07e98 r15: 00007f0c3ab3af90
  di:<PHONE_NUMBER>000002  si: 00007ffc8df05d70  bp: 00007ffc8df05fb0  bx:<PHONE_NUMBER>000006
  dx:<PHONE_NUMBER>000000  ax:<PHONE_NUMBER>000000  cx: 00007f0c0ecadfff  sp: 00007ffc8df05de8
  ip: 00007f0c0ecadfff efl:<PHONE_NUMBER>000246 cgf: 002b000000000033 erf:<PHONE_NUMBER>000000
 trp:<PHONE_NUMBER>000000 msk:<PHONE_NUMBER>000000 cr2:<PHONE_NUMBER>000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

[24497:24497:0320/231213.919570:INFO:child_thread_impl.cc(839)] ChildThreadImpl::EnsureConnected()
[24495:24495:0100/000000.254126:INFO:child_thread_impl.cc(839)] ChildThreadImpl::EnsureConnected()

I've tried to search what could the problem of fallocate be, but to no avail.
My OS is Devuan 2 (ASCII), the used GTK3 is 3.22.29. The UGC i built is 83.0.4103.116. fallocate --version gives back this:
fallocate from util-linux 2.29.2

Also, any idea why the resulting package is nearly 6GB? My older UGC (73) is only ~300 MB. Are debugging symbols enabled by default? Or is it, because is_official_build is set to false?

83 is rather old to put it mildly. Why would you be willing to build it instead of 99?

Please consider updating your OS. I suppose Bit rot might be in play here.

This has nothing to do with the age of the OS. Debian Stretch is still supported and Chromium 83 compiles on it. Something is not correctly set, or some of the files needs to be patched.

By adding is_official_build=true instead of is_official_build=false and also adding symbol_level=0 to flags.gn and running the result like this:
LD_LIBRARY_PATH=/opt/gtk3/lib/ LANGUAGE=en /opt/ungoogled-chromium-83-devuan2-gtk3-portable/chrome --no-sandbox --user-data-dir --ignore-certificate-errors

Everything worked.


the "open with" option in windows broser doesn't work

When using Unity, or even the windows browser, i am unable to open a file in VSC, despite the default program for that specifid file type (.cs files) being VSC. I cannot open the files through double clicking either, i have to open VSC by itself and copy the text into the window which completely defeats the purpose of using it in the first place.

What happens when you double click it?

I’m not sure if this worked before, but if I open the files manually in the Microsoft file browser with double click it opens them fine in VSC, but not if I try to do that in Unity
From: João<EMAIL_ADDRESS>Sent: Tuesday, 28 April 2020 9:58 PM
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [microsoft/vscode] the "open with" option in windows broser doesn't work (#96036)
What happens when you double click the file?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/96036#issuecomment-620624810, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIJB5DVO6FLQ772I2DDM5A3RO3OHBANCNFSM4MP27LFA.

Oh. Please file an issue to Unity.


Feature/cce 399 mobile tabs

Added first mobile styling for Tabs without $modifiers, needed review

This needs further refactoring before merging.


Contract Request API

Contract Request:

[x] Create contracts data structure #2177
[ ] create contract request #2180
[x] Decline contract request by NHS #2248
[x] Update contract request by nhs #2190
[x] Approve contract request by NHS #2196
[ ] APPROVE Contract Request by MSP #2369
[ ] contract print form template #2322
[ ] Terminate Contract Request #2220

Sign Contract Request:

[ ] Sign Contract Request by NHS #2211
[ ] Sign Contract Request by MSP side #2212

Get Contract Request:

[x] Get contract request list by nhs employee #2183
[x] Get contract_request details #2184
[ ] get partially signed content #2257

Change Request:

[ ] Approve contract request by NHS via signed content #2370

[ ] Contract Request printout form improvement #2405

[ ] Upload documents (Contract Request)  parent of #2420

can we finally close this epic?

[ ] new contract_request for contract with is_suspended=true parent of #2686
[ ] change error msg parent of #2688


Add a guide to maintain nice pull requests to Contributing.md

Feedback welcome

LGTM

+1. Although it seems like you are also teaching people git rather than just giving pull request/commit guidelines (the first paragraph). No objection to a little git tutorial of course, but maybe separate into further subheadings? Like perhaps "Commit Guidelines" and "How to Rebase" or something?
 reads better?

Updated the wording and added a subsubheader

:+1:

I would add a link to something like http://davidwalsh.name/squash-commits-git so people can see beforehand what is about. I just google that reference, anyone you like should be fine or, if you prefer, an asciicast in crystal repo itself.
I am not sure if the first entry for example is the oldest or the youngest, clarifying that would raise fewer questions in the reader's mind.

:+1:

I think the first entry is the oldest iirc.
I think it's fine giving links (maybe to something official?). But at the same time, with external links, you never really have a guarantee of it always existing or being up to date, whereas being in the Contributing.md, this can be controlled by us.

Added a clarification about the ordering. I would abstain from an external guide, this way we have something we can update if we change our branching model or similar. Also most guides I've seen rebase ontop of a local master branch, which is often messed up or outdated.

I think the point is on the importance of keeping the commits clean, so I suggest highlight the purpose in the first paragraph, and illustrate details in the following two separate paragraphs. Something like:
===========
Maintain clean pull requests
All pull requests to this repository are encouraged to include clean, purposeful, and readable commits. It helps us understand your commits as well as pull requests in order to respond more efficiently.
Good commits
The commit history should consist of commits that:

Transform the codebase from one state into another one.
Motivated by something that should change.
A bugfix.
A new feature or some ground work to support a new feature, like changing an existing API or introducing a new isolated class that is later used in the same pull request.

For example:
- Add Enumerable#max? and Enumerable#min?
- Benchmark::IPS bugfix and better comparison alignment
- Removed the anti-feature that allowed invoking methods on a union class. Fix #1061

Bad commits
The following commit types are strongly discouraged:

Only showing development history
Review history

For example:
- Start work on X
- More work on X
- Finish X
- Fix comment A
- Fix comment B

Squashing commits
Remember that review fixes should be squashed into the commits that introduced them. If your change fits well into a single commit, simply keep editing it with git commit --amend. Partial staging and committing with git add -p and git commit -p respectively are also very useful. Another good tool is git stash to put changes aside while switching to another commit. But Git's most useful tool towards this goal is the interactive rebase.
===========
I take your paragraphs and slightly rephrase and re-format. The point is to put emphasis more on the importance of clean commits. This comment is a bit long, hope I make it clear enough :)

I'm afraid it's already quite long for my taste, so I'd like to keep it a bit denser.

Yeap definitely, it's just a matter of style.

Contributing guide becomes a longer and longer. Not sure that every newcomer will read it from the beginning to the end. Moreover, this is a general git-related information and should be as concise as possible here. I think it would be better to have it on the wiki in "Interactive rebase and clean commit history" (or something like that) article  and just to link it within Contribution guide. What do you think ?

I think with the subsection it's easy to skip intentionally but also harder to miss.

@jhass I'll soon merge the formatter branch into master. Well, I'll probably rebase it. I honestly don't know how to do interactive rebase, squash, etc., so I'll read your guide and see if I understand everything and if there's something missing. So from my side it's a big :+1 for this PR

I too lack some git know how on that. So it's welcome. But the tl;dr-factor is important; perhaps footnotes could be used, referring to the detailed technical explanations, placed at the end of the document?


Add URL to the ResourcesResults returned by resources.

Changes
Today, this contains commits and digests (for Git and Image Resources). These are great,
but need the addition of a URL to really verify or fetch the content again. This could be
fetched off the Resource itself, but that would require resolving the Resource (since it could
be passed by reference and change after the fact).
We decided not to dereference Resources into the Status because they are still alpha. This
gives us the requried information, without coupling the beta API to an alpha field.
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[X] Includes tests (if functionality changed/added)
[X] Includes docs (if user facing)
[X] Commit messages follow commit message best practices
[X] Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
The TaskRun.Status.ResourcesResult field now contains a URL for Git and Image PipelineResources, containing the URL of the resource data.

/kind feature

/test pull-tekton-pipeline-integration-tests

Just gave this one a rebase, it's ready for review.

@dlorenc how do you feel about the equivalent Tasks in the catalog? (i guess there's just the git Task in this case, maybe the kaniko Task kinda counts) - feels like we should include this output for those as well?
/lgtm


Compress old logfiles

At startup locate old logfiles and gzip them to save disk space.

https://github.com/R2Northstar/NorthstarLauncher/pull/73


⚠ Jenkins Plugins Artifactory has degraded performance

In 68cdaf0, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1154 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 11377bf after 29 minutes.


Add support for time v0.3 types

Add support for the time crate v0.3.

Convert time::Duration with Duration*Second.
Convert time::OffsetDateTime and time::PrimitiveDateTime with Timestamp*Seconds.
Turn Rfc2822 and Rfc3339 into conversion type, such that they can be used inside collections.
Add tests for the new functionality

Todo:

[x] Update Changelog

bors merge


Extra characters at the end of encrypted JWT tokens don't prevent them from being considered valid

Hey,
Yesterday, the following report was posted on the OpenIddict repository: https://github.com/openiddict/openiddict-core/issues/1254.
I was able to reproduce it on my machine and confirmed this was caused by IdentityModel.
It looks like IdentityModel doesn't reject a JWE payload if it contains extra characters after the authentication tag.
Here's a repro:
using System.Diagnostics;
using System.Security.Claims;
using System.Security.Cryptography;
using Microsoft.IdentityModel.JsonWebTokens;
using Microsoft.IdentityModel.Tokens;

namespace ConsoleApp13
{
    class Program
    {
        static void Main(string[] args)
        {
            var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType);
            var signingKey = new RsaSecurityKey(RSA.Create(2048))
            {
                KeyId = "id1"
            };

            var encryptionKey = new RsaSecurityKey(RSA.Create(2048))
            {
                KeyId = "id2"
            };

            var handler = new JsonWebTokenHandler();
            var token = handler.CreateToken(new SecurityTokenDescriptor
            {
                Audience = "audience",
                Issuer = "issuer",
                EncryptingCredentials = new EncryptingCredentials(encryptionKey, SecurityAlgorithms.RsaOAEP, SecurityAlgorithms.Aes256CbcHmacSha512),
                SigningCredentials = new SigningCredentials(signingKey, SecurityAlgorithms.RsaSha256Signature),
                Subject = identity
            });

            var parameters = new TokenValidationParameters
            {
                ValidAudience = "audience",
                ValidIssuer = "issuer",
                TokenDecryptionKey = encryptionKey,
                IssuerSigningKey = signingKey
            };

            var result = handler.ValidateToken(token + "A", parameters);
            Debug.Assert(!result.IsValid, "The token shouldn't be considered valid.");
        }
    }
}

@brentschmaltz @jennyf19 could you please take a look?

Kévin,
Nice to hear from you, ill have a look.
Sometimes there are insignificant bits. For example, for a JWS some of the last bits can be changed due to Base64Eencoding or Base64UrlEncoding.
What is significant is if ANY bit in the CypherText can be changed.
Brent.
From: Kévin Chalet @.>
Sent: Tuesday, May 4, 2021 12:27 PM
To: AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet @.>
Cc: BrentSchmaltz @.>; Mention @.>
Subject: [AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet] Extra characters at the end of encrypted JWT tokens don't prevent them from being considered valid (#1641)
Hey,
Yesterday, the following report was posted on the OpenIddict repository: openiddict/openiddict-core#1254https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopeniddict%2Fopeniddict-core%2Fissues%2F1254&data=04|01||a73ddfbaa47e4925eb8308d90f329de9|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637557532331923810|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=5AXZvIKW7Gg9rnekAsvpf26c4fbqjjZnqp1fnwjjd70%3D&reserved=0.
I was able to reproduce it on my machine and confirmed this was caused by IdentityModel.
It looks like IdentityModel doesn't reject a JWE payload if it contains extra characters after the authentication tag.
Here's a repro:
using System.Diagnostics;
using System.Security.Claims;
using System.Security.Cryptography;
using Microsoft.IdentityModel.JsonWebTokens;
using Microsoft.IdentityModel.Tokens;
namespace ConsoleApp13
{
class Program

{

    static void Main(string[] args)

    {

        var identity = new ClaimsIdentity(TokenValidationParameters.DefaultAuthenticationType);

        var signingKey = new RsaSecurityKey(RSA.Create(2048))

        {

            KeyId = "id1"

        };

        var encryptionKey = new RsaSecurityKey(RSA.Create(2048))

        {

            KeyId = "id2"

        };

        var handler = new JsonWebTokenHandler();

        var token = handler.CreateToken(new SecurityTokenDescriptor

        {

            Audience = "audience",

            Issuer = "issuer",

            EncryptingCredentials = new EncryptingCredentials(encryptionKey, SecurityAlgorithms.RsaOAEP, SecurityAlgorithms.Aes256CbcHmacSha512),

            SigningCredentials = new SigningCredentials(signingKey, SecurityAlgorithms.RsaSha256Signature),

            Subject = identity

        });

        var parameters = new TokenValidationParameters

        {

            ValidAudience = "audience",

            ValidIssuer = "issuer",

            TokenDecryptionKey = encryptionKey,

            IssuerSigningKey = signingKey

        };

        var result = handler.ValidateToken(token + "A", parameters);

        Debug.Assert(!result.IsValid, "The token shouldn't be considered valid.");

    }

}

}
@brentschmaltzhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbrentschmaltz&data=04|01||a73ddfbaa47e4925eb8308d90f329de9|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637557532331923810|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=FGSwSXE%2FM41IBReLrD%2Bt4eV7ABBU7PrBR4DYZl%2BcokI%3D&reserved=0 @jennyf19https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjennyf19&data=04|01||a73ddfbaa47e4925eb8308d90f329de9|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637557532331933804|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=upQoD5Koi90tZDc9cMn0WoU1TdR7AUbRda50hHyKHqM%3D&reserved=0 could you please take a look?

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzureAD%2Fazure-activedirectory-identitymodel-extensions-for-dotnet%2Fissues%2F1641&data=04|01||a73ddfbaa47e4925eb8308d90f329de9|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637557532331943802|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=agwqzl0eAmJBCm8rFdNtit9xYgs9v82K8NyHokNFlu8%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAYGQRNGP4GEAGC4ZZLWFH3TMBDA7ANCNFSM44DJZ56A&data=04|01||a73ddfbaa47e4925eb8308d90f329de9|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637557532331943802|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=oVzEx%2BqTgOCYAHncpRUZzSw%2Br4na1uop%2Fu9GAxgqazg%3D&reserved=0.

@brentschmaltz awesome, thanks!

Sometimes there are insignificant bits.

That was my feeling too, but in this particular case, the "extra last bits" are considered by IdentityModel as being part of the authentication tag (which is expected since it's technically part of it):

I suspect IdentityModel correctly base64decodes it but only uses a fixed-size array copy that ignores the extra bytes before calling the crypto primitives. Feeding the crypto primitives with the original tag or adding a check in IdentityModel to ensure the authentication tag length matches the expected length would certainly help fix that issue.

Someone else emailed me to tell me they were concerned about this behavior. Do you have any news?

@kevinchalet nothing yet, should get some time soon.

Duplicate of #2201


ci(eslint): add rule to encourage generic naming convention as ^([A-Z]|T[A-Z][A-Za-z]+)$

@TkDodo I saw your post  in X and @juliusmarminge's solution by @typescript-eslint/naming-convention with regex
I want to add it in this repository
I've been using this rule in various repositories for quite some time, and I really liked the fact that when defining a new generic, it encouraged me not to just name it T. It make coworker and interface user feel better DX.
@juliusmarminge Thanks for sharing your cool solution too.

could we try to error it and just rename the type parameters if necessary? This should be non breaking ...

@TkDodo I will resolve eslint errors soon. wait a minutes please.

could we try to error it and just rename the type parameters if necessary? This should be non breaking ...

@TkDodo I will resolve eslint errors soon. wait a minutes please.

I tried to resolve eslint errors, but there was too many cases that don't need this rule like below. 🥲
How about this rule as warning now? This rule with warning also will make us follow this rule too.
export type MaybeRef<T> = Ref<T> | T // isn't it T also good? I think warning is enough for this case

export type MaybeRefOrGetter<T> = MaybeRef<T> | (() => T)

...smillar cases

T alone should be valid according to the regex, no?

T alone should be valid according to the regex, no?

Ah~, I thought T should be banned before.
Then K, X, Y should be valid right?

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (3a127bf) 41.71% compared to head (6fa6c4f) 41.72%.
Report is 2 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6684   +/-   ##
=======================================
  Coverage   41.71%   41.72%
=======================================
  Files         179      179
  Lines        7007     7008    +1
  Branches     1416     1416
=======================================
+ Hits         2923     2924    +1
  Misses       3713     3713
  Partials      371      371

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@TkDodo sorry, My reply was late because I didn't know much about regex and was studying it. I update regex for this rule as ^([A-Z]|T[A-Z][A-Za-z]+)$
This rule will make contributors use only below.
okay
alone T, U, V, etc. is okay
TQueryKey, TQueryFnData is okay
not okay
KQueryData, Result, Depth, State, DefinedQueryObserver, Runner, Args, TaggedQueryKey, Type, Value, etc. is banned from now. these should be started with T

@TkDodo check this again please

@manudeli I would prefer it if T would be the only allowed single character type parameter. This would make sure that one function can have T as a generic, but as soon as you add a second, you can't make <T,U> - you need to give meaningful names.
Imo one-character generics are best if there is only one on a given type / function. With two or more, you always need to now what they stand for ...

@manudeli I would prefer it if T would be the only allowed single character type parameter. This would make sure that one function can have T as a generic, but as soon as you add a second, you can't make <T,U> - you need to give meaningful names.
Imo one-character generics are best if there is only one on a given type / function. With two or more, you always need to now what they stand for ...

Ahha, I understood your intention. then I'll change it as following you want

@TkDodo I update rule as ^(T|T[A-Z][A-Za-z]+)$ from ^([A-Z]|T[A-Z][A-Za-z]+)$ and I resolved error from this rule in some code. We can check change in 31140ac

can you please fix the conflicts ?

can you please fix the conflicts ?

@TkDodo Sure, I resolved conflict now


Compilation of script & how to run the main function of .mvir file

🐛 Bug
Running the following command in the /.../move_getting_started_examples directory with the script flag enabled produces an error:

Why is this happening? I have also tried using --script instead of -s.
Also, how can one run the main function of a .mvir file?
Expected Behavior
Compilation and verification of transaction script.
System information

rust --version: rustc 1.36.0-nightly (50a0defd5 2019-05-21)
MacOS Mojave Version 10.14.4

If you're using cargo to run one of the binaries in the project you'll need to delineated command line options that you want passed to the binary and not to cargo with --.  For example, try the following:
cargo run -p compiler -- create_account_script.mvir


ExampleGQLClient_Mutate_simple

Fixes https://github.com/cli/go-gh/issues/98
Adds a mutation for https://docs.github.com/en/graphql/reference/mutations#addstar with input and first params, and inline fragments on the response to demonstrate how these are used.
I tested this in a local func main() to confirm the format is correct and the request can succeed (output is vilmibm).
The node that global_id R_kgDOF_MgQQ references is left as an exercise for the user :smile:

Thank you!


Update and clean-up dependencies, remove npm CLI dependency in scripts

Here is a set of changes to update dependencies, in addition to the following changes:

Remove the need to have npm CLI installed to run the package scripts, by replacing npm with yarn in them.
Remove the pre-commit hook dependency on npm CLI by using husky instead of pre-commit.
Update the linting rules to @rubensworks/eslint-config version 2, and fix all the linter errors that appeared.
Simplify the TSConfig and ESLint include/exclude listings.

Any feedback is welcome! If needed, I can also split this PR further.

Could you bump the CI to Node 16? (and perhaps also include 20)

This is rebased to include the changes from #19 so the tests now pass in the CI

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 7 (100.0%)  changed or added relevant lines in 7 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
639

Relevant Lines:
639

💛  - Coveralls

Super, thanks!


Avoid incorrect "engine" label

Feature Request
Is your feature request related to a problem? Please describe:
Our cluster may need to contains two different kinds of stores - TiKV and TiFlash. In TiDB's GC, there are some store-level RPC requests: UnsafeDestroyRange, and Green GC stuff (PhysicalScanLock, RegisterLockObserver, etc). But only TiKV supports these RPCs, and when the cluster contains TiFlash nodes, TiDB need to know.
Currently, the only possible way to distinguish different type of stores is to check the "engine" label. TiFlash node will set "engine" label to "tiflash". It's necessary to ensure TiDB have a strict enough way to check the store's type, so I think we need to add a mechanism to ensure TiKV won't be mis-configured with "engine" -> "tiflash"
Describe the feature you'd like:
Ensure TiKV will never have the label "engine" with value "tiflash". Or further, ensure the value cannot be anything other than "tikv".
Describe alternatives you've considered:
It's also ok if there can be other mechanism for TiDB to distinguishi different kinds of stores.
Teachability, Documentation, Adoption, Migration Strategy:
No

@nolouch PTAL

To resolve GreenGC issue, the only way, for now, is to skip TiFlash node on TiDB side.
Also, I'd recommend TiKV checks labels before start and make sure they are not engine:tiflash.

cc @disksing

we may have other way to distinguish nodes with different engines. For now, I think Ensure TiKV will never have the label "engine" with value "tiflash". Or further, ensure the value cannot be anything other than "tikv" is a good idea.


Custom parsing

Thanks for the great library.
I am trying to import a fixed width flat file (from COBOL no less) and I need to format some of the fields manually. For instance, the dates are YYYYMMDD. The decimal numeric fields have no placeholders and have a trailing negative like this:   5546 = 55.46, 123450- = (-)1234.50. So I need to manually parse certain fields. The app is in core 2 and I cannot for the life of me figure out how to get a custom formatter working with your code.
.WithMember(o => o.na_dist_dat, set => set.WithLength(8)  )  -> "YYYYMMDD" to a datetime
Should I use .WithFormat and implement my own iFormat Provider?
Thanks

Hi,
Have you tried the WithMember(o => o-na_dist_dat, set => set.UseValueConverter<ValueConverterType>())
where your ValueConverterType inherits from FieldValueConverter<DateTime>

I will check into that. In the meantime I moved to FileHelpers (they also have a bunch of handlers for very annoying COBOL text export standards). I much prefer your library so I will see if I can make it work.

Thanks for your preference, I do too, but I'm biased 😄
Anyway, that's the good of open source... you have options


[SIP-5] Refactor MapBox

Break MapBox into smaller files
Enable renderTrigger for viewport changing
Pass explicit props rather than pass all that exists in payload.data
Detach setControlValue from the vis component
Replace React.createElement with regular jsx
Remove usage of refs which is deprecated.

@williaster @conglei @hughhhh @betodealmeida @mistercrunch

Codecov Report

Merging #5783 into master will decrease coverage by 0.03%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #5783      +/-   ##
==========================================
- Coverage   63.75%   63.71%   -0.04%
==========================================
  Files         364      365       +1
  Lines       23104    23118      +14
  Branches     2579     2575       -4
==========================================
  Hits        14729    14729
- Misses       8360     8374      +14
  Partials       15       15

Impacted Files
Coverage Δ

superset/assets/src/explore/controls.jsx
46.26% <ø> (ø)
:arrow_up:

...perset/assets/src/visualizations/MapBox/MapBox.jsx
0% <0%> (ø)

...c/visualizations/MapBox/ScatterPlotGlowOverlay.jsx
0% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f72cdc3...b43f5bf. Read the comment docs.


Orb damage range on import seems fixed.

The damage range on Orbs are always 407-450. Which takes the lowest (407) range from Ancient item and takes the highest range (450) from non-Ancient item. Actual Ancient Orbs in game have a much higher range of 407-600.

For legendary items:
damageOffhand: {min: 340, max: 370, min2: 380, max2: 450}
For ancient items:
damageOffhand: {min: 407, max: 485, min2: 495, max2: 600}
I assume that only occurs when importing characters?

You mean you're getting incorrect offhand damage when importing a character? That's a bug on battle.net side, and has been around for ages. Official armory has the same issue.
Or do you mean something changes about it?

Ah yes you're right, I didn't even check the official battle.net armory. Well I guess that's not something that can be fixed then.


KeyError when using SlimPruner in dependency_aware mode with ResNet

Describe the issue:
When I run SlimPruner on ResNet in dependency_aware mode, I am getting this error:
 File "/home/ubuntu/....", line 67, in init_pruner
    pruner.compress()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/iterative_pruner.py", line 89, in compress
    self.update_mask()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/dependency_aware_pruner.py", line 82, in update_mask
    self._dependency_update_mask()
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p37/lib/python3.7/site-packages/nni/algorithms/compression/pytorch/pruning/dependency_aware_pruner.py", line 149, in _dependency_update_mask
    _names = [x for x in self.channel_depen[wrapper.name]]
KeyError: 'bn1'

Config List used=[{'sparsity': 0.18710986222101633, 'op_types': ['BatchNorm2d']}]
Model: ResNet18
Is this a bug or am I doing something wrong? Let me know if you need more info. Thanks!
Environment:

NNI version: 2.3
Training service (local|remote|pai|aml|etc): local
Client OS: ubuntu
Server OS (for remote mode only):
Python version: 3.7
PyTorch/TensorFlow version: 1.8.1
Is conda/virtualenv/venv used?: conda
Is running in Docker?: no

How to reproduce it?:
Running SlimPruner on ResNet18 using PyTorch (with cuda).

The reason why this error raised is that SlimPruner only supports BatchNorm2d module while dependency_aware model only supports Conv2d module. You can consider that SlimPruner doesn't support dependency_aware mode right now. If you want to use dependency_aware mode on ResNet, I recommend you using TaylorFOWeightFilterPruner, ActivationAPoZRankFilterPruner or ActivationMeanRankFilterPruner.

The reason why this error raised is that SlimPruner only supports BatchNorm2d module while dependency_aware model only supports Conv2d module. You can consider that SlimPruner doesn't support dependency_aware mode right now. If you want to use dependency_aware mode on ResNet, I recommend you using TaylorFOWeightFilterPruner, ActivationAPoZRankFilterPruner or ActivationMeanRankFilterPruner.

Thanks @linbinskn for digging out the route cause, @crawlingcub hope the answer helps you.
@QuanluZhang /@linbinskn - possible to change the error message to more informational ones?

KeyError: 'bn1'


ImportError: cannot import name 'IPAttnProcessor2_0' from 'ip_adapter.attention_processor'

Hi,
Trying the demo, looks like a bug here?
ImportError                               Traceback (most recent call last)

[<ipython-input-12-eeef1f01030c>](https://localhost:8080/#) in <cell line: 11>()
      9
     10 from insightface.app import FaceAnalysis
---> 11 from pipeline_stable_diffusion_xl_instantid import StableDiffusionXLInstantIDPipeline, draw_kps

[/content/InstantID/pipeline_stable_diffusion_xl_instantid.py](https://localhost:8080/#) in <module>
     47         AttnProcessor2_0 as AttnProcessor,
     48     )
---> 49     from ip_adapter.attention_processor import (
     50         IPAttnProcessor2_0 as IPAttnProcessor,
     51     )

ImportError: cannot import name 'IPAttnProcessor2_0' from 'ip_adapter.attention_processor' (/content/InstantID/ip_adapter/attention_processor.py

Is everything ok? @whiterose199187

yes, this error is fixed

RuntimeError: Could not infer dtype of NoneType

Getting this error on line:  images = pipe(prompt, image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8).images

What is the input image? It seems that the face model cannot find a face from it, and the face_emb is None. Try another image.

---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

[<ipython-input-55-66e60fe9245f>](https://localhost:8080/#) in <cell line: 7>()
      5 print(len(face_info))
      6 face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
----> 7 face_emb = face_info['embedding']
      8 # face_emb = face_info.normed_embedding
      9 print(face_emb)

KeyError: 'embedding'

This is the error when running the below snippet:
# load an image
face_image = load_image("https://m.media-amazon.com/images/M/MV5BODg3MzYwMjE4N15BMl5BanBnXkFtZTcwMjU5NzAzNw@@._V1_FMjpg_UX1000_.jpg")
# prepare face emb
face_info = app.get(cv2.cvtColor(np.array(face_image), cv2.COLOR_RGB2BGR))
print(len(face_info))
face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
face_emb = face_info['embedding']
# face_emb = face_info.normed_embedding
print(face_emb)
face_kps = draw_kps(face_image, face_info['kps'])
# face_kps.save('kps.jpg')
pipe.set_ip_adapter_scale(0.8)

prompt = "RAW photo, A portrait photograph of a woman aged between 21 and 35 as a renowned mixologist, crafting cocktails in a trendy Manhattan bar, 16k uhd, dslr, photorealistic, natural lighting, high quality, film grain, Fujifilm XT3"
negative_prompt = "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured (lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch,deformed, mutated, cross-eyed, ugly, disfigured"

# generate image
images = pipe(prompt, image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8).images

---------------------------------------------------------------------------

KeyError                                  Traceback (most recent call last)

[<ipython-input-55-66e60fe9245f>](https://localhost:8080/#) in <cell line: 7>()
      5 print(len(face_info))
      6 face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
----> 7 face_emb = face_info['embedding']
      8 # face_emb = face_info.normed_embedding
      9 print(face_emb)

KeyError: 'embedding'

This is the error when running the below snippet:
# load an image
face_image = load_image("https://m.media-amazon.com/images/M/MV5BODg3MzYwMjE4N15BMl5BanBnXkFtZTcwMjU5NzAzNw@@._V1_FMjpg_UX1000_.jpg")
# prepare face emb
face_info = app.get(cv2.cvtColor(np.array(face_image), cv2.COLOR_RGB2BGR))
print(len(face_info))
face_info = sorted(face_info, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # only use the maximum face
face_emb = face_info['embedding']
# face_emb = face_info.normed_embedding
print(face_emb)
face_kps = draw_kps(face_image, face_info['kps'])
# face_kps.save('kps.jpg')
pipe.set_ip_adapter_scale(0.8)

prompt = "RAW photo, A portrait photograph of a woman aged between 21 and 35 as a renowned mixologist, crafting cocktails in a trendy Manhattan bar, 16k uhd, dslr, photorealistic, natural lighting, high quality, film grain, Fujifilm XT3"
negative_prompt = "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured (lowres, low quality, worst quality:1.2), (text:1.2), watermark, painting, drawing, illustration, glitch,deformed, mutated, cross-eyed, ugly, disfigured"

# generate image
images = pipe(prompt, image_embeds=face_emb, image=face_kps, controlnet_conditioning_scale=0.8).images

It may be an issue with your environment. I test it. It's normal


Refactored and simplified requests to webservice

I appreciate your pull request, but I've found these issues:

Tests are not passing with this refactoring.

Savon gem itself and its gem dependencies are throwing several security and deprecation warnings when tests or web server are running.

This gem (bcn_ni) intends to work with the less gem dependencies as possible to make it lighter. So it would be nice to make refactoring or bug fixes with the 3rd parties gems as less as possible.

When running bcn_ni methods, if a request is made and no exchange rates are given (exchange_rates null) an exception is raised in this line and code section: exchange_rates.any?

Once these observations and issues are resolved, feel free to resend your pull request.
Regards my friend.


Tile predict methods

A predict tile method should

[x] take in a raster object
[x] generate crops for prediction (in memory?)
[x] produce predictions for crops
[x] recombine predictions to the origin coordinate system
[x] perform some kind of NMS edge filter.

To do, only perform NMS on edge annotations.

merge boxes in overlapping area, not IoU threshold.

I've settled on more overlap per tile and lower IoU threshold. Can revisit, but considering closed. Trying to rewrite a center box method was expensive and not worth it.


Fix chunked upload of large files to blob

Bug description:
When uploading large files to blob store, the  _upload_chunk implementation actually overwrites the entire file instead of just a single chunk.
This PR fixes that by implementing the _initiate_upload and taking the final arg into account.

@cjalmeida -- Can you comment on whether #35 fixes this for you as well?

@AlbertDeFusco I'll check and report back.

@AlbertDeFusco It does not, the issue is in _upload_chunk calling create_blob_from_bytes. The expected behavior for _upload_chunk is to push a partial block into blob store, create_blob_from_bytes is designed to push an entire file.
An alternative would be to buffer the entire file in-memory for the flush calls except for the final one. But that's not the expected behavior from the user side when calling flush().

Your changes look compatible with mine. #35 does not touch the AzureBlobFile class. I believe these PRs solve different problems.

@hayesgb Sorry mentioned wrong @ but Albert helped here :)


[Bug]: Unable to check bank balance

Description
No response from Bank while fetching accounts
Steps to reproduce

Open the Stax app
On the home screen select the appropriate account (Bank)
Click on the check balance refresh icon
Select the sim to complete the transaction
Wait for the transaction to complete

Expected result
The bank balance is displayed on the balance screen
Actual result
Pending status stalls or a fetching account error (No response from Bank) is displayed.
Link to screenshots/screen recordings
https://drive.google.com/file/d/10yb2GYZl-l27NwPCi8inNRIVEMYqAP-D/view?usp=sharing
Device name
Samsung Galaxy A20s
Android OS
11
Is your phone dual SIM or single SIM?
Dual SIM
Which telco are you using for this test?
Safaricom
Stax device ID or email address
<EMAIL_ADDRESS>

Merging this with #708


Add router support for WRONGDIRECTION SPACING

Description
Add router support for the LEF construct:

Suggested Solution
Update gc and possibly dr.
Additional Context
No response

Rule supported by merging PR #4303


#1000 enabling integration tests for apt plugin

Proposed changes

integration test workflow now requires amd64 to be installed first

Types of changes

[x] Bugfix (non-breaking change which fixes an issue)

Paste Link to the issue

#1000

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have signed the CLA (in all commits with git commit -s)

Further comments

Hi @abelikt please test and let me know if this fixes the issue. I wasn't able to test this because of restrictions on the branch

Hi, we still need to install tedge on that VM. Code is located here : https://github.com/abelikt/thin-edge.io/tree/bugfix/1000/apt-integration-tests


Have haulers pick up dropped resources when they are near the mining container

This Pull Request has been merged by GitConsensus.
Vote Totals

Yes
No
Abstain
Voters

4
0
0
4

Vote Breakdown

User
Yes
No
Abstain

tedivm
✔

dragoonreas
✔

ViezeVingertjes
✔

PostCrafter
✔

Vote Results

Criteria
Result

Has Quorum
True

Has Votes
True


Task created from a method with an isolated parameter breaks actor isolation

Description
A Task created from a method that executes within the isolated context of a passed-in actor should inherit the execution context of the passed-in actor, but does not.
Steps to reproduce
The following code fails to compile:
func execute<ActorType: Actor>(
    on isolatedActor: isolated ActorType,
    task: @Sendable (isolated ActorType) -> Void)
{
    // Compiler correctly allows this task to execute synchronously.
    task(isolatedActor)
    // Start a task that inherits the current execution context (i.e. that of the isolatedActor)
    Task {
        // BUG: compiler insists that we must `await` the `task`, meaning that this Task is not executing in the isolatedActor's execution context
        task(isolatedActor) // Compiler error: Expression is 'async' but is not marked with 'await'
    }
}

Here's a screenshot of the same code with the inlined compiler error for folk who like Xcode's syntax hilighting:

Expected behavior
I would expect the above code to compile without modification.
Per the Task documentation, a not-detached Task "inherits the priority and actor context of the caller", meaning that a Task created in an isolated ActorType method should execute within the isolated context of the passed-in actor.
Environment

Swift compiler version info

I was able to reproduce this issue on the following swift compiler versions (I did not try other versions):
swift-driver version: 1.62.15 Apple Swift version 5.7.1 (swiftlang-<IP_ADDRESS>.3 clang-14<IP_ADDRESS>)
Target: arm64-apple-macosx13.0

swift-driver version: 1.62.15 Apple Swift version 5.7.2 (swiftlang-<IP_ADDRESS>.5 clang-14<IP_ADDRESS>)
Target: arm64-apple-macosx13.0

Xcode version info

I was able to reproduce this issue on the following Xcode versions (I did not try other versions):
Xcode 14.1
Build version 14B47b

Xcode 14.2
Build version 14C18

Deployment target:

.macOS(.v10_15),
.iOS(.v13),
.tvOS(.v13),
.watchOS(.v6),

Additional context
This bug report was first posted as a question in the Swift forums.
Unlike #62503 and #62505, I have not found a way to force the compiler to execute the Task above in the expected execution context.

Thanks for the report!
Linked to rdar://103337121


In route finder typeahead stops on a space

When entering stars for selecting routes, the system stops reading if you type a space.  For example, using the above dataset, try to enter "HD 156384".When you press space after HD, the selection box goes away and the combo box is blank. Obviously, this is a problem if you have lots of stars from the same catalog.

space is consumed deep within the combo box and so I can't add it as a filter for the name.
I did add digit and * so you could use these other values to search.

Sorry, I think I understand the limitations.  It sounds like a user info box may want to be nationally assigned to this (sounds like user entered *, rather than a typeahead),.
Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Androidhttps://aka.ms/ghei36

From: LJ Ramones<EMAIL_ADDRESS>Sent: Wednesday, February 24, 2021 4:40:02 PM
To: ljramones/trips<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: Re: [ljramones/trips] In route finder typeahead stops on a space (#210)
space is consumed deep within the combo box and so I can't add it as a filter for the name.
I did add digit and * so you could use these other values to search.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/ljramones/trips/issues/210#issuecomment-785402355, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQPXLNUTIGKT4OQV4PYEOR3TAVW3FANCNFSM4W5YJH2Q.

the type ahead is used for the on screen display of stars of at most you are looking at about 200. typing a few characters weans that down to a handful so not having a space is not detrimental.

however, I didn't put a typeahead for the entire 300k dataset. I would have to use a much more sophisticated technology for that.
The typeahead works on what's in memory. As such, you have real limitations on how many stars you can have in memory and keep the requirements for running TRIPS accessible to most people. More memory -> more powerful computer.

Got it.
Sent from my Verizon, Samsung Galaxy smartphone
Get Outlook for Androidhttps://aka.ms/ghei36

From: LJ Ramones<EMAIL_ADDRESS>Sent: Wednesday, February 24, 2021 4:57:59 PM
To: ljramones/trips<EMAIL_ADDRESS>Cc: cegannon1<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [ljramones/trips] In route finder typeahead stops on a space (#210)
however, I didn't put a typeahead for the entire 300k dataset. I would have to use a much more sophisticated technology for that.
The typeahead works on what's in memory. As such, you have real limitations on how many stars you can have in memory and keep the requirements for running TRIPS accessible to most people. More memory -> more powerful computer.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/ljramones/trips/issues/210#issuecomment-785411691, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQPXLNXEMNCFINZISJJYOY3TAVY6PANCNFSM4W5YJH2Q.


Implement constraint importer from gnark for usage within Groth16 prover

As raised in this comment, it would be great if we could validate the algorithms with intermediate results from other libraries, such as gnark.
The idea would be:
-> Implement a CSV importer of R1CS (for example, https://play.gnark.io/ outputs such a CSV representing the constraints, select cubic equation on the top right dropdown)
-> Convert said R1CS into a QAP representation, that we can then forward to the Groth16 prover
-> Write a test for said conversion, proving and verification

Solved


Prometheus in agent mode fails to send data to Thanos Receiver via remote_write way too often.

What did you do?
Running Prometheus in k8s with agent mode enabled and have remote_write endpoint configured to receive metrics from prometheus. Running prometheus v2.36.2, with below remote_write configuration
`prometheus.yml: |-
global:
external_labels:
monitor: prometheus
replica: '${HOSTNAME}'
pod: '${HOSTNAME}'
scrape_interval: 15s
remote_write:
- url: 'http://:19291/api/v1/receive'
queue_config:
capacity: 6000
      # Maximum number of shards, i.e. amount of concurrency.
      max_shards: 1500

      # Minimum number of shards, i.e. amount of concurrency.
      min_shards: 1

      # Maximum number of samples per send.
      max_samples_per_send: 2000

      # Maximum time a sample will wait in buffer.
      batch_send_deadline: 5s

      # Initial retry delay. Gets doubled for every retry.
      min_backoff: 30ms

      # Maximum retry delay.
      max_backoff: 5s
    `

What did you expect to see?
Prometheus to send data without any gap to remote_write endpoint.
What did you see instead? Under which circumstances?
Metrics are been dropped after certain interval of time, with no log messages in prometheus. But, it recovers after a short interval by itself.
System information
No response
Prometheus version
prometheus v2.36.2

Prometheus configuration file
`prometheus.yml: |-
    global:
      external_labels:
        monitor: prometheus
        replica: '${HOSTNAME}'
        pod: '${HOSTNAME}'
      scrape_interval: 15s
    remote_write:
      - url: 'http://<thanos-receive-endpoint>:19291/api/v1/receive'
        queue_config:
          capacity: 6000

          # Maximum number of shards, i.e. amount of concurrency.
          max_shards: 1500

          # Minimum number of shards, i.e. amount of concurrency.
          min_shards: 1

          # Maximum number of samples per send.
          max_samples_per_send: 2000

          # Maximum time a sample will wait in buffer.
          batch_send_deadline: 5s

          # Initial retry delay. Gets doubled for every retry.
          min_backoff: 30ms

          # Maximum retry delay.
          max_backoff: 5s
        `

Alertmanager version
No response
Alertmanager configuration file
No response
Logs
No response

🆙
Are there any updates about this issue ? I have the same problem !

Hello from the bug scrub.
Apologies for the delay in getting to this issue.
There are some troubleshooting guides that may be helpful:

https://prometheus.io/docs/practices/remote_write/
https://grafana.com/blog/2021/04/12/how-to-troubleshoot-remote-write-issues-in-prometheus/

If you find evidence that there is a bug in Prometheus then please follow up with this issue.


RuntimeError: The size of tensor a (4) must match the size of tensor b (69) at non-singleton dimension 2

When I ran this case, I encountered the following error. Can you help me? Thank you!
Stage III: Downstream Adaptation:
Case I: Multitask Learning: To train a multitask generalist policy on LIBERO-90:
python train_prise.py exp_name=${EXP_NAME} replay_buffer_num_workers=4 stage=3 exp_bc_name=${DOWNSTREAM_EXP_NAME} multitask=true downstream_task_suite=libero_90 num_train_steps=30010 eval=false save_snapshot=true vocab_size=${VOCAB_SIZE} &
Resuming Snapshopt
========= Tokenizing the downstream data... ==========
Error executing job with overrides: ['exp_name=./experiments/run_0', 'replay_buffer_num_workers=0', 'stage=3', 'multitask=true', 'downstream_task_suite=libero_90', 'num_train_steps=30010', 'eval=false', 'save_snapshot=true', 'vocab_size=200']
Traceback (most recent call last):
  File "train_prise.py", line 447, in main
    workspace.downstream_adapt()
  File "/home/a/Videos/bin/boot/PRISE/train_prise.py", line 329, in downstream_adapt
    u = self.agent.PRISE.module.action_encoder(z, action.float())
  File "/home/a/anaconda3/envs/LIBERO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/a/anaconda3/envs/LIBERO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/a/Videos/bin/boot/PRISE/prise_agent.py", line 71, in forward
    return self.sa_embedding(z.detach()+u)
RuntimeError: The size of tensor a (4) must match the size of tensor b (69) at non-singleton dimension 2

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Hi, thanks for your question. Indeed there was an error in our original code. In the original code, z = self.agent.compute_transformer_embedding(z) is missing. Thus the z there has not been passed through the transformer layer. This has been fixed now.
Feel free to ask if you have any other questions about the code, and we will be more than happy to address them. Thanks again!

Thanks for your help！And I find another issue and provide it for you . In following function,  return cls  may be missing. Thanks again!
def register_benchmark(cls):
    BENCHMARK_MAPPING[cls.__name__.lower()] = cls
    return cls


[WIP] [Review] Separate some features of "ExecuteOneAsync" into the "TransactionExcu…

Separate some features of "ExecuteOneAsync" into the "TransactionExcuteEntry" class and the "ProcessTransactionExecution" function

:x: Build AElf <IP_ADDRESS>16 failed (commit https://github.com/AElfProject/AElf/commit/996c6ca135 by @IamWenboZhang)

:x: Build AElf <IP_ADDRESS>45 failed (commit https://github.com/AElfProject/AElf/commit/1000fbbba5 by @EanCuznaivy)

Unit test failed.

:x: Build AElf <IP_ADDRESS>38 failed (commit https://github.com/AElfProject/AElf/commit/a99cfa9aee by @rosona)

:white_check_mark: Build AElf <IP_ADDRESS>63 completed (commit https://github.com/AElfProject/AElf/commit/40604d88c1 by @IamWenboZhang)


Make auto-quoting toggle rather than just applying

Issue Type: Feature Request
I highlight a word and hit some for of quotes, single, double, backtick, and it will wrap the word with the quote.  However I can't get rid of them. I would like the code to check to see if the item is currently wrapped with the quote you are applying and if it is currently quoted, it should unquote, thus becoming a toggle feature rather than just an apply feature. i.e.
original - myWord
after hitting ' - 'myWord'
after hitting ' - myWord
VS Code version: Code 1.32.3 (a3db5be9b5c6ba46bb7555ec5d60178ecc2eaae4, 2019-03-14T22:46:48.547Z)
OS version: Darwin x64 17.7.0

I don't believe this is a dupe of the items shown above.


UX Suggestions

(Copied per @lidel 's suggestion from: https://github.com/ipfs-shipyard/pm-ipfs-gui/issues/28#issuecomment-359202705)
(Apologies in advance if there's been a discussion about some or all of these points already/if this is the wrong place to post these thoughts. I've been loosely tracking things with IPFS, but this was me capturing my experience going in blind just testing stuff out)
Through my work I get to work with users across the technical spectrum (from those who get confused about the difference between a browser and chrome) to full on devs who are way more technical than I am. My concern is that for engineers (who understand IPFS) the current UI may be useful, but for non technical users theres a lot of jargon that would intimidate them from using IPFS.
Current UX notes:
Home page:

Swapping the "Home Page" with the "Files Page" in the WebUI to get users closer to their content
Details: When I open up the WebUI, I see my Node Info and my Node Addresses. I would assume for most users this is actually meaningless, or at the very least less likely the reason they opened the WebUI. I think for most users, the primary interaction they're expecting from the UI is to upload or download files. We could make it easier for users to interact with the page, if we make the Home page the files page, and rename what is currently the home page to "Information" or something.

Files page:

Loading or status bar between an uploading action and the completion view (minor polish)
Details: I had my Dad play around with IPFS, and immediately he tried uploading a file in between the initial upload the "upload successful message" there was a brief pause, where he was confused as to whether the file was actually uploaded.

Automatically triggering a refresh after an upload (so the file appears in the bottom portion of the screen)
Details: Right after the upload was successful, there was no clear indication where the file went. It was only on triggering a refresh that the file appeared.

Allowing for an optional naming of the hash during an upload so a user can identify which hash links to which content.
Details: Having tried uploading a few files sequentially, it became hard to track which hash was related to which file. I found the only way to identify which was which was to click on RAW to see what the hash was. Ideally if on upload you could name the hash too. Even if these are just for local reference, it would be insanely given the ordering of uploads and the ordering on the page are not consistent (its not clear how the hashes are sorted to me).

Clicking on "Remove" for content linked me back to the home page, without removing the item from my Files page.

Renaming "Action" terminology to something more accessible to users.
Details: Specifically, I don't think "RAW" or "DAG" are inherently meaningful for non-technical users. Internally I've found my self equating RAW to something like "Open" or "Preview", and I don't know if have a strong association for DAG in the context of this page.

Something that might also make this easier for users is adding a "Share" link, which autogenerates the structure of the ipfs/hash for them to dump into the WebUI (or browser once theres broader support)

DAG Page:

Given the primary purpose of this page is to explore content, maybe rename this to something more meaningful for non-technical users - something like "Search" or "Explore"?

This feels a little odd to have a search page, when the top bar is also a search. If the primary purpose is to have an entry point for exploring content, maybe add this to the files page as an exploration option? Given the search bar is persistent at the top, I don't know if a separate page is necessary.

 I'd also note that the download didn't trigger when I clicked on the second link.

A more macro question: is there limitations with support for different file types? I tried uploading an MP4 and when I went to click "RAW" or "Download" I got a weird hash file.

Config Page:

I probably am missing how to use this as an IPFS newbie, but this seems like a good home for some of the stuff that's currently on the Home page. Useful and easy to find for users who know they need that info, hidden away from users who have no idea what it is.

General notes:
Apologies if this is just me fundamentally misunderstanding stuff, but one thing I wasn't confident of was whether when I uploaded stuff it was actually being uploaded to IPFS or if it was just locally mounted (the equivalent of things existing on a local branch in git). In the WebUI it says I'm uploading files to a "Local Folder", but when the content is uploaded, anyone can access it right (ie anyone could see /ipfs/QmZbjw3n2WRhaE2Aueqo4kYh7SjNYWiEoXpZ6MZKfsaSUe ) if they wanted to? If there is a way of formally publishing from your local folder to the rest of IPFS I don't think the UI makes it inherently obvious that you've done so. If by default everything is being shared, I think the UI needs to make this a little more explicit (especially before someone does something silly and uploads something inherently personal)!

As I said on IRC:

whyrusleeping: the webui I want is one that is focused on adding content to your node, making it easy for people to have a server of some sort. Similar to the desktop orion client for ipfs. Rn the webui greets the user with setttings, and the upload interface isn't very informative, and doesn't display what's already there.

Hey @makeworld-the-better-one 😄 I think you'll find on the revamp branch a brand new file browser where you can add content very easily to your node!

@jnthnvctr We've just gone through a round of redesign work on the IPFS Web UI. It'd be great to get your feedback on it if you have time. From your original feedback. It's not deployed with go-ipfs yet, but you can see a preview at https://webui.ipfs.io

Swapping the "Home Page" with the "Files Page"

This is good idea. We don't have this yet. The home page is still the status page, but I'm going to create an issue to make this change. Any empty files page is an opportunity to add an introduction / walk through, and I imagine, as you say, the first thing a new user is going to want to do here is add / share / get files from the network, so we should avoid showing them a bunch of stats and graphs at that point.

Files page:

The files page has been completely rebuilt, with upload progress feedback and showing files you just uploaded. A share action has been added and in a future release will be accompanied with a web app that's focused just on that https://share.ipfs.io

DAG Page:

The DAG page has been rebuilt and renamed "explore", and starts with some example datasets to give a clue as to what can be done with it.

Apologies if this is just me fundamentally misunderstanding stuff, but one thing I wasn't confident of was whether when I uploaded stuff it was actually being uploaded to IPFS or if it was just locally mounted

This is something we have to do a much better job of explaining and demonstrating. "Uploading" is misleading here as when you "add to IPFS" you are adding it to your local IPFS repo. Doing so means it's possible that others may fetch those files and blocks from you at some point, but until then they only exist on your machine.  They are not automatically backed up to some other nodes. It does make it easy to share files with others and be sure that you got exactly the same file back if you ask them for it later. It opens up the possibilty of communally co-hosting files and websites that are important to you, but right now you have to understand enough about IPFS to encourage some friends to do the same. https://awesome.ipfs.io/categories/datasets/ has some examples of datasets that are now hosted by many IPFS users.
Also of note, added files are not private, as other nodes could request them, but they can only be retrieved if the other user knows the content address for them. They can only get that if you share it with them, or they already have the files, as the content address is uniquely generated from the raw data in the files. You can think of like "unlisted" on services like youtube, where only people you share the link with can get the data. The current recommendation for sensitive data is to encrypt it before adding it to IPFS.

Closing due to the fact that there have been some pretty substantial changes made in the interim, and this might be best approached with fresh eyes. Please feel free to re-open if needed. Thank you!


PS2 Optimisations

Some future ideas for more optimal circumstances on PS2:

The PS2 GPU memory is fast enough to flush and refill textures multiple times per frame, according to the developers of Metal Gear Solid 2 - who had up to 10MB of textures on-screen at times inside the 4MB VRAM (total, not including memory used for buffers). It may be possible to use similar techniques to achieve better results in this demo.
I feel better looking textures could be used with the above method and storing these in the main 32MB RAM and sending them to the GPU as needed - the PS2 has a 3.6GB/s memory bus so the potential is undoubtedly there.
Using recent gsKit tools available on github may also give the code some cleaner syntax and possibly expand the available libraries?
Different video modes; I would say that a 640x448i mode and a 640x480p mode would be optimal, however 640x540 at 1080i would also be very interesting - a few games used this for a 1080i mode, and this could look great on HDTVs possibly with antialiasing.

Keep this project up, though. It's nice to see demos like this arise on the PS2. I recommend you check out the gsKit "hi-res" sample code for an idea of what you might be able to do - this seems like a relatively simple scene, and one programmer was able to get a rotating 3D teapot running at native 1920x1080, a 1920x1080 background texture and all with antialiasing at 60FPS. Would be cool to see some improvements here from code like that.

Oh and a footnote; the emulator statistics are telling me that GPU is below 10% for the majority of the time, sometimes spiking to just under 15%, and the CPU load is in the 50% region for most of the time (~25% at lowest, ~60% at highest). The vector units are completely unused (perhaps you could use these for on-the-fly decompression or geometry?). Just thought I would mention the statistics in case it helps.

This was my first PS2 homebrew tentative, so I might have missed some best practices and more complex techniques.  I discovered the PS2 hardware during this project, so I didn't feel confident enough to immediately use the VUs and juggle with RAM/VRAM per-frame. I've stayed close to the basic examples available w/ gsKit and other resources on the web, there is clearly room for many improvements!
Thank you for all these details and suggestions, and for your pull requests!

No problem. I've been trying to get this demo running at a close to HD resolution (960x720 or 720x1080) single buffered with better, lower bit depth textures but I haven't had the time to finish it. I'll submit another pull once I've done so!

In the meantime, a note; by lowering the bit depth of images (textures etc.) and the framebuffer itself (ie. to  16-bit) you save a significant amount of memory. For example:
640x448x24 (24-bit): 6881280
640x448x16 (16-bit): 4587520
You can see you could probably get away with nicer textures and higher resolution by finding a sweet spot with colour/bit depth.

Thank you, that's great! I won't have the time to come back to this demo in the next few weeks, but I will try to see how I can better use the freed RAM/VRAM in order to improve the quality of the textures and the rendering resolution.

No problem. I haven't had time to compile this and I think some of the code expects certain bit depths for different textures, so I think it'll require a bit of trial and error with a working ps2sdk to get it working correctly.

Something I forgot to mention months ago and remembered when I saw this on my homepage; here's a fundamental thing about PS2 hardware design - the VRAM is not meant to be used for storing textures. The VRAM is purely for pixel data (eg. framebuffers). The 32MB of main RAM is main RAM, but the bus between it and the GPU is specifically designed to be able to zip textures from RAM to the GPU. So essentially, you could have many more textures at a higher resolution than what you could fit in 4MB.
Dumb of me to overlook such a thing, but I digress 😅

A VU1 sample would be awesome! We need more VU1 sample in the ps2 scene


Reopen bug #135721

Please reopen this bug, it was not resolved: https://github.com/microsoft/vscode/issues/135721

Maybe related to #139877

dupe


Export common type for BaseSchema and BaseSchemaAsync

As a developer creating integrations with valibot (not using valibot directly, but creating library solutions for other developers on top of valibot), I am missing an option to accept abstract valibot schema, either sync or async. Currently, I have to use v.BaseSchema<Input, Output> | v.BaseSchemaAsync<Input, Output> boilerplate.
Example (simplified) code:
import * as v from "valibot"

/**
 * Vue composable: validate input and submit if it's valid.
 * Simplified for brevity.
 */
export function useForm<Input, ValidInput = Input>(options: {
  fields: Input
  schema: v.BaseSchema<Input, ValidInput> | v.BaseSchemaAsync<Input, ValidInput>
  submit: (data: ValidInput) => void | PromiseLike<void>
}) {
  const { schema, fields } = options

  return async function submit() {
    const res = await v.safeParseAsync(schema, fields)
    if (res.success) {
      await options.submit(res.output)
    }
  }
}

I propose to export a new common type (such as BaseSchemaMaybeAsync), so that in the code above it'd possible to write:
schema: v.BaseSchemaMaybeAsync<Input, ValidInput>

Thanks for your feedback on this. I'm not sure yet if I want to add such a type to the library. I think the explicit assignment is better in most cases. As a workaround for now you can create this type yourself.
type BaseSchemaMaybeAsync<Input, ValidInput> =
  | v.BaseSchema<Input, ValidInput>
  | v.BaseSchemaAsync<Input, ValidInput>;

Thank you for the response. Of course I understand that I can create types and other extensions on my own, especially as trivial as this one.
If you don't think the use case is relevant (even though I provided an example) and/or that is an anti-pattern, I suppose this issue should be just closed as not planned. I'm inclined to publish some kind of valibotx anyway, which will re-export everything from valibot along with the unobtrusive extensions that you don't want (e.g. #171). :)

My spontaneous feedback is that I prefer an explicit assignment. This is also the approach I currently use in the Valibot source code. However, I understand your approach and depending on the context, it may be better.
I'm open to new ideas, and even if I don't directly agree with a change or extension, I leave it open to change my mind if I get feedback from more people. So I will leave the labels as they are for now and maybe change them in a few months.
Feel free to create a package like valibotx that extends the functionality of Valibot. Maybe in the long run I will take some ideas and implement them directly into Valibot.

At the moment I still prefer an explicit declaration. Therefore I close this issue. I am still open to discussion and may change my mind in the future.


Allow using IPC to handle navigations

I think before we start dealing with bi-direction communication API, there should be a dedicated IPC channel that could send from core crate to verso crate. Could you test if this could be done upon we start the IPC channel here?

I thought about it before, but now I think I would like to go with the event listener approach that we only send an IPC sender to the core crate when we need info from it, instead of always listen to messages from the core crate


Serious memory leak caused by websocket package

The example server uses ws to implement the websocket server. I wrote my own live server based on the example, and noticed that the server suffers from serious memory leak when network bandwidth is insufficient.
The issue is not difficult to reproduce. Run the server in live mode, and let multiple clients (i.e. streetscape.gl front end) connect to it. When the bandwidth required totally by all the clients exceeds the maximum supported by your device (the network interface, the router, etc), the memory occupied by the server will keep increasing, eventually crashing the machine.
At the beginning, the process occupied 5% of my total memory:

After openning enough clients to exhaust the network bandwidth, the memory footprint began to boost:

Even if I closed all the clients, the memory footprint never decreased:

After some googling, I found that the  ws package has a historical memory leak problem, which has not been fully resolved today. I know it's unbelievable: a widely-used package with 10000+ github stars has an easy-to-reproduce memory leak.
I switched to the uWS package and it worked like a charm. The memory footprint stays below 8% when the bandwidth is exhausted:

And drops back to 5% after I close all the clients:

uWS is only my personal choice, but I think ws should definitely be avoided due to the memory leak.
Here is my uWS server implementation in case it may help. The only thing worthy of note is the idleTimeout option. uWS requires that the server receive some msg from the client every idleTimeout seconds. If no msg arrives, the client is considered dead and closed by the server. Since streetscape.gl frontend does not send any message to the server, I set the number to an extremely large number to prevent the client from being closed.
const uWS = require('uWebSockets.js/uws.js');
const app = uWS.App()
  .ws('/*', {
    /* Options */
    compression: 0,
    maxPayloadLength: 16 * 1024 * 1024,
    idleTimeout: 100000000,
    /* Handlers */
    open: (ws, req) => {
      console.log('A WebSocket connected via URL: ' + req.getUrl());
      this.ws_clients.add(ws);
      this.send_data_one_client(ws, this.metadata_frame);
      console.log("sent metadata");
    },
    message: (ws, message, isBinary) => {
      console.log("server receives some msg");
    },
    drain: (ws) => {
      console.log('drain: WebSocket backpressure: ' + ws.getBufferedAmount());
    },
    close: (ws, code, message) => {
      console.log(`WebSocket closed: code ${code}, message: ${message}`);
      this.ws_clients.delete(ws);
    }
  }).any('/*', (res, req) => {
    res.end('Nothing to see here!');
  }).listen(port, (token) => {
    if (token) {
      console.log('Listening to port ' + port);
    } else {
      console.log('Failed to listen to port 10000' + port);
    }
  });
send_data_one_client(ws, data) {
  ws.send(data, true);
}

send_data_all_clients(data) {
  this.ws_clients.forEach(client => {
    this.send_data_one_client(client, data);
  });
}

Thanks for the details.  I was planning on allowing the server implemention to be swapped out (by providing an instance of another server.  Looks like I should accelerate this feature and look at other websocket server implementations.

If you set the idleTimeout to 0 it is disabled. However you really should use it - behavior can be changed to allow the server to send pings (not as distributed as the server need to to more work, but hey).
Other than that I agree, ws is lacking. Feel free to contact me for quotation / brainstorming.


add isGesture prop to onRegionChangeComplete callback on iOS MapKit p…

…rovider

Does any other open PR do the same thing?

no
What issue is this PR fixing?
#4112
How did you test this PR?

Test on 3 iOS devices and 1 simulator

You need to declare OnPanDrag in MapView props otherwise {"isGesture": true} do not trigger

Eagerly awaiting this functionality, so that we aren't forced to use google maps on iOS devices.


nannyml can confuse months with days on some rows of a dataset

Describe the bug
NannyML gets the month and the date values wrong and can make a date of 01-09-2018 (1st September 2018) be thought of 09-01-2018 (9th January 2018).
To Reproduce
Install NannyML and run the following code in a jupyter notebook:
import wget
from pathlib import Path
import pandas as pd
import nannyml as nml

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00618/Steel_industry_data.csv'
download_foler = Path.home().joinpath("Downloads")
filename = wget.download(url,out = str(download_foler))

data = pd.read_csv(filename, header=0)
display(data.head())

features_selected = list(data.columns)[2:]

data['partition'] = 'train'
ind_ = data.shape[0]//3
data.loc[ind_:2*ind_, 'partition'] = 'reference'
data.loc[2*ind_: , 'partition'] = 'analysis'

reference = data.loc[data.partition == 'reference', :].reset_index(drop=True)
analysis = data.loc[data.partition == 'analysis', :].reset_index(drop=True)

calc = nml.UnivariateStatisticalDriftCalculator(
    feature_column_names=[features_selected[0]],
    timestamp_column_name='date'
)
calc.fit(reference)
results = calc.calculate(analysis)

drift_fig = results.plot(kind='feature_drift', feature_column_name=features_selected[0], plot_reference=True)
drift_fig.show()

display(results.data)
display(analysis)

A partial screenshot of the results.data can be seen below:

A partial screensgot of the first and last line of the analysis dataframe can be seen below:

By comparing the two screenshots you can see that some dates have been altered.
Expected behavior
The dates would be recognized correctly. The misidentification of dates can be seen by comparing the start and end dates of the results data and the start and end dates of the analysis dataset.
Screenshots & scripts
The problem is also visible on the following univariate drift plot:

Apart from issues mentioned here keep in mind there might be issues coming from naive datetimes (without time zones). In extreme cases for high frequency data when someone wants to have hourly chanks this can generate problems when the time changes (we will have the same hour twice or one hour of data will be missing).

Hey,
it seems like the date format in the data was a bit too exotic to be parsed correctly.
If I manually parse the timestamps during preprocessing like data['date'] = pd.to_datetime(data['date'], format='%d/%m/%Y %H:%M'), I end up with the following plot:

The question is if we consider this to be a part of the data requirements, as currently documented or provide some options in the library to deal with this? I'd lean towards the first.

Hey,
it seems like the date format in the data was a bit too exotic to be parsed correctly.
If I manually parse the timestamps during preprocessing like data['date'] = pd.to_datetime(data['date'], format='%d/%m/%Y %H:%M'), I end up with the following plot:

The question is if we consider this to be a part of the data requirements, as currently documented or provide some options in the library to deal with this? I'd lean towards the first.

Issue still reproducible under the latest version
import wget
from pathlib import Path
import pandas as pd
import nannyml as nml

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00618/Steel_industry_data.csv'
download_foler = Path.home().joinpath("Downloads")
filename = wget.download(url,out = str(download_foler))

data = pd.read_csv(filename, header=0)
display(data.head())

features_selected = list(data.columns)[2:]

data['partition'] = 'train'
ind_ = data.shape[0]//3
data.loc[ind_:2*ind_, 'partition'] = 'reference'
data.loc[2*ind_: , 'partition'] = 'analysis'

reference = data.loc[data.partition == 'reference', :].reset_index(drop=True)
analysis = data.loc[data.partition == 'analysis', :].reset_index(drop=True)

calc = nml.UnivariateDriftCalculator(
    column_names=[features_selected[0]],
    timestamp_column_name='date',
    continuous_methods=['kolmogorov_smirnov',],
    categorical_methods=['chi2']
)
calc.fit(reference)
results = calc.calculate(analysis)

drift_fig = results.plot(kind='drift', column_name=features_selected[0], method='kolmogorov_smirnov', plot_reference=True)
drift_fig.show()

display(results.data)
display(analysis)

and date fix as suggested successfully resolves the issue
import wget
from pathlib import Path
import pandas as pd
import nannyml as nml

url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00618/Steel_industry_data.csv'
download_foler = Path.home().joinpath("Downloads")
filename = wget.download(url,out = str(download_foler))

data = pd.read_csv(filename, header=0)
display(data.head())

features_selected = list(data.columns)[2:]

data['partition'] = 'train'
ind_ = data.shape[0]//3
data.loc[ind_:2*ind_, 'partition'] = 'reference'
data.loc[2*ind_: , 'partition'] = 'analysis'

# date fix
data['date'] = pd.to_datetime(data['date'], format='%d/%m/%Y %H:%M')

reference = data.loc[data.partition == 'reference', :].reset_index(drop=True)
analysis = data.loc[data.partition == 'analysis', :].reset_index(drop=True)

calc = nml.UnivariateDriftCalculator(
    column_names=[features_selected[0]],
    timestamp_column_name='date',
    continuous_methods=['kolmogorov_smirnov',],
    categorical_methods=['chi2']
)
calc.fit(reference)
results = calc.calculate(analysis)

drift_fig = results.plot(kind='drift', column_name=features_selected[0], method='kolmogorov_smirnov', plot_reference=True)
drift_fig.show()

display(results.data)
display(analysis)


Assert Fail: TASK [control_plane_common : Assert management_net_dhcp_start_range]

Describe the bug
This check fails when using only 2 NICs for control plane. It should not fail.
TASK [control_plane_common : Assert management_net_dhcp_start_range] ***************************************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {
    "assertion": "dhcp_start_mgmnt == mngmnt_network_subnet",
    "changed": false,
    "evaluated_to": false,
    "msg": "Failed. Incorrect range assigned for dhcp for management network"
}

@j0hnL This is targeted at 1.2

Solved in devel branch in Issue https://github.com/dellhpc/omnia/pull/834


[Bug] JVMService-produce high cpu 100%

Search before asking

[X] I had searched in the issues and found no similar issues.

Apache SkyWalking Component
Java Agent (apache/skywalking-java)
What happened

After the service runs for a period of time, SkywalkingAgent-5-JVMService-produce-0 will occupy 100% CPU
"SkywalkingAgent-5-JVMService-produce-0" #9 daemon prio=5 os_prio=0 cpu=199370183.56ms elapsed=606196.06s tid=0x00007feb31359800 nid=0x7c5 runnable [0x00007feae9e99000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x0000000700399d28> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
What you expected to happen
nothing
How to reproduce
Not all services will happen, only one can be seen。
environment：
jdk: openjdk version "1.8.0_312"
runtime: k8s-container
Anything else
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

We can't do an analysis from a CPU usage, especially you just posts a stack showing queue blocked.