CircuitPython no such option: --no-compile

Thanks for working on this!
I'm on Fedora Silverblue 34, using Python version 3.9.6 and pip 21.0.1.
When I attempt to install CircuitPython libraries, I receive the following error message.
$ minipip install --port /dev/ttyACM1 --target /lib adafruit-circuitpython-framebuf
Processing 'adafruit-circuitpython-framebuf'
Querying package metadata from https://micropython.org/pi/adafruit-circuitpython-framebuf/json
Could not find 'adafruit-circuitpython-framebuf' from https://micropython.org/pi
Querying package metadata from https://pypi.org/pypi/adafruit-circuitpython-framebuf/json
Inspecting version 1.4.7
Installing with pip: ['adafruit-circuitpython-framebuf']

Usage:
  /usr/bin/python3 -m pip <command> [options]

no such option: --no-compile

Thank you for the report!
 Fixed now and uploaded as version 0.1b4. Please pip install minipip==0.1b4 and try again!

Well, that does get a bit farther but I'm still hitting an error.
$ minipip install --port /dev/ttyACM1 --target /lib adafruit-circuitpython-framebuf
Processing 'adafruit-circuitpython-framebuf'
Querying package metadata from https://micropython.org/pi/adafruit-circuitpython-framebuf/json
Could not find 'adafruit-circuitpython-framebuf' from https://micropython.org/pi
Querying package metadata from https://pypi.org/pypi/adafruit-circuitpython-framebuf/json
Inspecting version 1.4.7
Installing with pip: ['adafruit-circuitpython-framebuf']
Collecting adafruit-circuitpython-framebuf
  Using cached adafruit_circuitpython_framebuf-1.4.7-py3-none-any.whl
Collecting Adafruit-Blinka
  Using cached Adafruit_Blinka-6.10.3-py3-none-any.whl
Collecting pyftdi>=0.40.0
  Using cached pyftdi-0.53.2-py3-none-any.whl (141 kB)
Collecting Adafruit-PlatformDetect>=3.13.0
  Using cached Adafruit_PlatformDetect-3.14.2-py3-none-any.whl
Collecting Adafruit-PureIO>=1.1.7
  Using cached Adafruit_PureIO-1.1.9-py3-none-any.whl
Collecting pyusb<1.2.0,>=1.0.0
  Using cached pyusb-1.1.0-py3-none-any.whl
Collecting pyserial>=3.0
  Using cached pyserial-3.5-py2.py3-none-any.whl (90 kB)
Installing collected packages: pyusb, pyserial, pyftdi, Adafruit-PureIO, Adafruit-PlatformDetect, Adafruit-Blinka, adafruit-circuitpython-framebuf
Successfully installed Adafruit-Blinka-6.10.3 Adafruit-PlatformDetect-3.14.2 Adafruit-PureIO-1.1.9 adafruit-circuitpython-framebuf-1.4.7 pyftdi-0.53.2 pyserial-3.5 pyusb-1.1.0
Using buffer-size of 32
Connecting to /dev/ttyACM1 (buffer-size 32)...
Trying to connect to REPL . connected
Retrieving sysname ... b'\r\nAuto-reload is on. Simply save files over USB to run them or enter REPL to disable.\r\ncode.py output:\r\n'
Traceback (most recent call last):
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1383, in connect
    ip_address = socket.gethostbyname(port)
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/home/jordan/.local/bin/rshell", line 33, in <module>
    sys.exit(load_entry_point('rshell==0.0.30', 'console_scripts', 'rshell')())
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/command_line.py", line 4, in main
    rshell.main.main()
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 3084, in main
    real_main()
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 3046, in real_main
    connect(args.port, baud=args.baud, wait=args.wait, user=args.user, password=args.password)
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1389, in connect
    connect_serial(port, baud=baud, wait=wait)
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1413, in connect_serial
    dev = DeviceSerial(port, baud, wait)
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1744, in __init__
    Device.__init__(self, pyb)
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1497, in __init__
    self.sysname = self.remote_eval(sysname)
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1632, in remote_eval
    return eval(self.remote(func, *args, **kwargs))
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/main.py", line 1610, in remote
    self.pyb.enter_raw_repl()
  File "/var/home/jordan/.local/lib/python3.9/site-packages/rshell/pyboard.py", line 209, in enter_raw_repl
    raise PyboardError('could not enter raw repl')
rshell.pyboard.PyboardError: could not enter raw repl

It looks like rshell could connect to your device, but could not get the REPL. I don't understand, why it later wants to treat your port as host address, though.
Which device are you using? From Simply save files over USB to run them or enter REPL to disable I infer it's running CircuitPython. In this case it should mount its filesystem on your PC (Linux?) and you should be able to omit --port and give the path to the mounted lib directory as --target.
There's something more -- if you're installing Adafruit libraries from PyPI, they came with set of dependencies which are not relevant in CircuitPython (Blinka, platform-detect and maybe something more). I'll try to make minipip smarter so that it omits these when installing for CircuitPython device.

I'm using CircuitPython 7.0.0 alpha 4 on a Feather RP2040. It automatically mounts the drive's filesystem to /run/media/$USER/CIRCUITPY. I was just following the example provided in the README, and I wasn't sure why it needed the --port option. Thanks for the help, it makes sense not to include that. It would definitely help if it omits those host-specific libraries. Thanks!

Created separate issue for skipping Blinka: #10


Sortablejs Widget Question

When I visit the sortable website and view their examples, the list items all sort within the list.  But within the SurveyJS flow, you need to drag each item into a separate results box.
During user testing, our users consistently re-order within the lower section but never drag the items into the top results box.  So I'm wondering if there is a surveyJS flag or option that exists to not render a separate results box?
The current functionality allows you to select which items to include while ranking.  But in our case, the users need to rank all items.

You can use the CSS to hide source:
.sjs-sortablejs-source {
  display: none;
}

and the following code to fill results at the start:
survey.getAllQuestions().forEach(function(q) {
  if(q.getType() === "sortablelist") {
    q.value = q.choices.map(function(c) { return c.value; });
  }
});

as shown in this example - https://plnkr.co/edit/bhAMb2zx5QExhMgGmjqd?p=preview

Thank you for your quick help!  This is exactly what I was trying to accomplish.


Change default theme from obsidian to onyx

As of: https://github.com/HTPC-Manager/HTPC-Manager/issues/26

It also matches the default in the readme. There is only one colour issue when editing the menu bar. I'll fix that before merging into master.


[WIP] ApplicationRoute - mixin de-entanglement

This is a super-rough hack that I'll refine later on, but I wanted to leave it here for the purposes of discussion.
By refactoring two functions out of the mixin, I'm now able to do the ApplicationRouteMixin's job on a service or any other ember singleton.
TODO:

[ ] Figure out intuitive function names
[ ] Break these two top-level functions out into a single ./utils/___
[ ] Docs
[ ] Tests for non-mixin use

Hi @mike-north I'd like to help. I think I could look at unentangling AuthenticatedRoute mixin? I want to make sure you're not already working on it.

Hi @mike-north I'd like to help. I think I could look at unentangling AuthenticatedRoute mixin? I want to make sure you're not already working on it.

I'm not currently working on it. Go ahead 🚀


App info

Hello.
On android after selecting a previously selected icon, app does not start. App info launches

How can this be fixed?

upd


java-sdk has no method about policy

It is possible to add policies visually in casdoor, but it is inconvenient. I hope that the API about policies can be provided and can be called in Java-sdk.

Hello. Is there any update progress on nodejs sdk? It appears that there is no policy-related SDK part in the node js SDK.

@MoonsuCha create a github issue


XSS - javascript execution when pasting certain links

I am currently doing the PortSwigger XSS labs and taking notes in Obsidian. When I (carelessly) pasted my payloaded links into Obsidian I got an alert box. The alert did not trigger when I pasted the link with Shift+Ctrl+V, so it has to have been this Plugin.
This only happens when the lab is up and running, so you would have to start them to recreate it.
Lab: Reflected XSS into a JavaScript string with single quote and backslash escaped | Web Security Academy
I pasted the following link triggering an alert in Obsidian:
https://[LAB-ID].web-security-academy.net/?search=AAAA%00\';%0A%0Aalert(1);//
Lab: Reflected XSS into a JavaScript string with angle brackets and double quotes HTML-encoded and single quotes escaped | Web Security Academy
I pasted the following link triggering an alert in Obsidian:
https://[LAB-ID].web-security-academy.net/?search=\\\%27;alert();//

Also works for this one: Lab: Reflected XSS into a template literal with angle brackets, single, double quotes, backslash and backticks Unicode-escaped | Web Security Academy
with this link https://[LAB-ID].web-security-academy.net/?search=AAAAAAAA${alert()}
Seem a a lot of reflected XSS can trigger this? It's not all of them however; for Lab: Reflected DOM XSS | Web Security Academy the link https://[LAB-ID].web-security-academy.net/?search=%5c%22%7d%3balert(1)%3b%2f%2f works on the lab, but does not trigger an alert in Obsidian when fetching the title.

Also works for this one: Lab: Reflected XSS into a template literal with angle brackets, single, double quotes, backslash and backticks Unicode-escaped | Web Security Academy
with this link https://[LAB-ID].web-security-academy.net/?search=AAAAAAAA${alert()}
Seem a a lot of reflected XSS can trigger this? It's not all of them however; for Lab: Reflected DOM XSS | Web Security Academy the link https://[LAB-ID].web-security-academy.net/?search=%5c%22%7d%3balert(1)%3b%2f%2f works on the lab, but does not trigger an alert in Obsidian when fetching the title.
[closing issue was by accident]

Also works for this one: Lab: Reflected XSS into a template literal with angle brackets, single, double quotes, backslash and backticks Unicode-escaped | Web Security Academy
with this link https://[LAB-ID].web-security-academy.net/?search=AAAAAAAA${alert()}
Seem a a lot of reflected XSS can trigger this? It's not all of them however; for Lab: Reflected DOM XSS | Web Security Academy the link https://[LAB-ID].web-security-academy.net/?search=%5c%22%7d%3balert(1)%3b%2f%2f works on the lab, but does not trigger an alert in Obsidian when fetching the title.
[closing issue was by accident]

Also works for this one: Lab: Reflected XSS into a template literal with angle brackets, single, double quotes, backslash and backticks Unicode-escaped | Web Security Academy
with this link https://[LAB-ID].web-security-academy.net/?search=AAAAAAAA${alert()}
Seem a a lot of reflected XSS can trigger this? It's not all of them however; for Lab: Reflected DOM XSS | Web Security Academy the link https://[LAB-ID].web-security-academy.net/?search=%5c%22%7d%3balert(1)%3b%2f%2f works on the lab, but does not trigger an alert in Obsidian when fetching the title.
[closing issue was by accident]

This is a fantastic plugin and saves me a lot of time!
@kevNii It looks like the plugin conditionally loads links in electron windows, so JS will always execute. This would solve issues of dynamic page titles (please correct me if I'm wrong). You could replace the conditional tt the bottom of scraper.ts with just the nonElectronGetPageTitle function if this is a concern.
Although-- nodeIntegration is enabled and webSecurity is disabled in these windows, which seems it could be a security issue?

@blueputty01 That does seem to be the issue.
What kind of saves this from being a RCE vulnerability is the introduction of context isolation in electron, which is enabled by default. Without this any website you paste would be able to use the node API and have full RCE on your machine.
However this is still an issue, since executing any website's JS is not what I expect when I paste a link. It still enables tracking users, leaking IPs, opening new browser windows,...

As a workaround, I can suggest implementing a backend server that parses the title tag from the body and nothing more. This may cause some site titles to be broken, but introducing headless options to parse web page titles can solve this problem. See #77


Fetcher uses too much memory

pyspider version: latest commit on master (14th of June, 2018) 3fb9167983f0123b9cac6615e543e9fbaf68cc04
Operating system: debian stretch in a docker container

I am using pyspider deployed with a similar docker-compose file as shown in the doc
I have configured a lot of projects : a bit more than 200 and I recrawl most of them every 6hours to check for news pages.
It works pretty well.
But I noticed that the container for the fetcher service uses a lot memory,and does not releases it even when it has no task to fetch.
When every task is completed, the container for each services uses less than 200MiB of memory, except for the fetcher container which uses more than 2Gib !
It became an issue because I have other apps running on the same server also using a significant amount of memory and it led to the crash of the fetcher container (and therefore the entire process of pyspider).
I first tried to limit the memory usage of the fetcher container using docker, but when the fetcher reached that limit it stopped working (not processing any task)
I tried to configure 2 fetcher container, but it just doubled the memory usage.
I then suspected tornado to have memory leak issues. I managed to adapt the tornado_fetcher code to use tornado in it's version 5.1 and built my own docker image with tornado 5.1 installed. The crawl goes well but I couldn't find significant improvement in the memory usage. However the debug mode of the webui does not work due to tornado breaking changes and I couldn't fix it.
I guess that some objects created during the fetching of a web page aren't removed by the garbage collector once the fetching is completed.
The fetcher uses more memory when crawling new pages but not when recrawling, i suppose that those objects can be reused by the fetcher on a new crawl. But due to the large amounts of pages i want to crawl I would have prefer the fetcher to free memory when the crawl of a project is done (after on_finished)
Have you met the same issue ?
How can I fix this ?

I met the same issue too. May be it's a litle of memory leak .  you can use crontab to restart  the process.


Add some authors to CITATION.cff files

I would like to add the lists below to the CITATION.cff files in both esmvaltool and esmvalcore:
esmvaltool citation.cff:
    "affiliation": "Netherlands eScience Center"
    "family-names": "Kalverla"
    "given-names": "Peter"
    "orcid": "https://orcid.org/0000-0002-5025-7862"

    "affiliation": "Netherlands eScience Center"
    "family-names": "Alidoost"
    "given-names": "Sarah"
    "orcid": "https://orcid.org/0000-0001-8407-6472"

    "affiliation": "Netherlands eScience Center"
    "family-names": Verhoeven
    "given-names": Stefan
    "orcid": "https://orcid.org/0000-0002-5821-2060"

    "affiliation": "Netherlands eScience Center"
    "family-names": Vreede
    "given-names": Barbara
    "orcid": "https://orcid.org/0000-0002-5023-4601"

    "affiliation": "Netherlands eScience Center"
    "family-names": "Smeets"
    "given-names": "Stef"
    "orcid": "https://orcid.org/0000-0002-5413-9038"

esmvalcore citation.cff:
    "affiliation": "Netherlands eScience Center"
    "family-names": Vreede
    "given-names": Barbara
    "orcid": "https://orcid.org/0000-0002-5023-4601"

More information about List of authors.

Thanks @SarahAlidoost! Do you mind if I jump in and add myself for both? I forgot to do that earlier :)

Thanks @SarahAlidoost! Do you mind if I jump in and add myself for both? I forgot to do that earlier :)

Sure, please add commit to those PRs :)


Can not make the tag Unique

if i want to create a friend selector,
my data point is like:
[
{ id : 1, name : 'bob"  ,avatar: "1.jpg" } ,
{ id : 2, name : 'bob"  ,avatar: "2.jpg" } ,
{ id : 3, name : 'bob"  ,avatar: "3.jpg" } ,
]
as U see, 3 friends have the same name, but diffrent avatar, with custom "autocomplete.render" method I can distinguish them.
but in custom stringToItem , i don't know  how to distinguish them ~~

@filod This is more of application design issue rather than TextExt. Are there functionality issues that you are experiencing caused by displaying the same name?

TextExt is very good, i like it .
As i have said above,  tag ( & other plugins ) use [ stringToItem ] to set formvalue , when user press enter key，via user input I can‘t know  the [ id ]  of user( in a website, we allow same username, but different [ id ].
maybe my problem is， how can I get the element i chose in autocomplete dropdown ? so that i can use it in the [ stringToItem ] to get the right item.
i find a way to do this in  [ stringToItem ], it's like :
               stringToItem : function(str){
               		var item;
               		item =  $('#txtid').textext()[0].autocomplete().selectedSuggestionElement().data('text-suggestion');
					if(item){
						return item;
					}
                            // other filter to get item;
               		return item;
               }

the way i used will be allright in the future TextExt ?

There's currently no way to know in stringToItem() which particular HTML element was clicked... You would have to hack up autocomplete to pass that down the chain.


move mirage application serializer to addon to avoid conflicts

Exporting a serializer named application overwrites the application serializer in consuming apps causing much confusion.  Moving this to addon seems like the most prudent solution, at least for now until there is a better way to do this in mirage.

This doesn't actually work...


Fix binary nokogiri gems on musl based Linux

What problem is this PR intended to solve?
As described in #1983 there are two issues when using the nokogiri binary gem on musl based Linux.
 The the commit description for more details.
Have you included adequate test coverage?
Not yet. Tests on Alpine were run per docker run --rm -it ruby:alpine /bin/sh.
Does this change affect the C or the Java implementations?
The C version only.

The test failure is also present, when nokogiri is native compiled on Alpine Linux. It is not specific to the binary gems.

THIS looks awesome. I would love to have nokogiri binary gems on alpine.

Just a note - I want to add alpine to the CI pipeline (per note in #1845) before merging this; I'd like to go red (see the segfault reliably on alpine) first, then merge and go green.

I've added two new things to the CI pipelines (master and PRs):

run tests and valgrind on a musl system (alpine)

https://ci.nokogiri.org/teams/nokogiri-core/pipelines/nokogiri/jobs/ruby-musl-system
https://ci.nokogiri.org/teams/nokogiri-core/pipelines/nokogiri/jobs/ruby-musl-valgrind

build a native linux gem and run it against all supported rubies and a musl system (alpine)

https://ci.nokogiri.org/teams/nokogiri-core/pipelines/nokogiri/jobs/native-gem-test

These should go red, and I'll manually merge these two commits to make each red build go green.

Red #1: https://ci.nokogiri.org/teams/nokogiri-core/pipelines/nokogiri/jobs/ruby-musl-system/builds/1
  1) Error:
Nokogiri::HTML::TestDocumentEncoding#test_encoding_without_charset:
ArgumentError: invalid byte sequence in UTF-8
    /tmp/build/4d9a0f57/nokogiri/test/html/test_document_encoding.rb:26:in `test_encoding_without_charset'

First commit merged: ff99fa5

Second red build: https://ci.nokogiri.org/teams/nokogiri-core/pipelines/nokogiri/jobs/native-gem-test/builds/1

Second commit merged: a3d9438

OK - additional test coverage is all green, after manual merges. Closing.


Add blosc2 to CLI, libcramjam features and own CI

Adds blosc2 to CLI
Adds a libcramjam CI
Adds each codec in libcramjam as a feature flag (default w/ all enabled).

Why was Cargo.lock removed?

Sorry, it was not intentional. While developing against local copies of libs like blosc2-rs and isal-rs it becomes a bit of a hazard when those libs are moving quickly. I'll add it back on the next PR I get time for. Thanks for noticing. :)


Sample code for usage

Hi,
I am trying to use the packages and I am a little confused. My task is to find the similarity scores between sentences.
Can you provide a sample code how can we use this in python code to train and test?

Hi. context2vec is not suitable for computing sentence similarity (at least not in a straight forward manner). I suggest you look for other alternatives, e.g. https://radimrehurek.com/gensim/models/doc2vec.html


Min and Max date

I took a stab at implementing min and max dates but there must be more to it than what I have here; neither the getters nor the setters are working.
https://github.com/cookieguru/tabris-calendar/commit/a500b2b20287b0771c7b2f15fb21d6df74e5363d

@cookieguru Thanks for the contribution. :) You are right that there is something missing. You would need to make the new properties available in https://github.com/cookieguru/tabris-calendar/blob/master/www/calendar.js

@mpost Awesome, it works now, thanks!  Can you tell me a little bit about the type and cache properties?  If I'm understanding this correctly the type is set to any so that either a string, int, or even a float can be passed and java.lang.Long.longValue() will take care of parsing the value.

There is documentation on he Properties system here: https://tabrisjs.com/documentation/latest/custom-widgets#properties

Of course!  Thanks @mpost


Regenerate node_modules missing in v1.0.2

As mentioned in https://github.com/dlang-community/setup-dlang/pull/32,
this should have been done for v1.0.2

@mihails-strasuns : How did you regenerate them ? A naive npm install doesn't seem to be the way to go :)
CC @WebFreak001 (yes my JS game is weak).

@Geod24 if you need to use the ones currently specified in package-lock, the easiest is to just delete node_modules and run npm install - probably there is a better way but I am a bit ignorant here :) Or just npm update to get the latest version of everything.

Right, just forgot to use --production, hence the +800k LoC. Fixed now.

No more annotations, seems like this is working.

👍


Some environment has timestamp related error with add method

Only skeet-solana-mobile template web front end has this issue.
https://github.com/elsoul/skeet-solana-mobile-stack/issues/47
skeet functions + add method from @skeet-framework/firestore didn't work suddenly.
Before it worked well. (This repository didn't updated for this moment thought)
Also asking for Firebase support team.
>  Error: Error adding document: Error: Value for argument "data" is not a valid Firestore document. Couldn't serialize object of type "rme" (found in field "createdAt"). Firestore doesn't support JavaScript objects with custom prototypes (i.e. objects that were created via the "new" operator).
>      at Poa (/Users/ktm/dev/skeet-solana-mobile-stack/functions/skeet/dist/index.js:1324:711)
>      at /Users/ktm/dev/skeet-solana-mobile-stack/functions/skeet/dist/index.js:1353:1058
>      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
>      at async runFunction (/Users/ktm/dev/skeet-solana-mobile-stack/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:9)
>      at async runHTTPS (/Users/ktm/dev/skeet-solana-mobile-stack/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:531:5)
>      at async /Users/ktm/dev/skeet-solana-mobile-stack/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:694:21

For now, this is the way to use
https://github.com/firebase/firebase-functions/issues/1496


Add priority list command

Add priority list command

Add command to sort contact list by priority level of their tags.
Add unit test for this command.
Minor edits for testcases due to change in implementation for Tag class.

Codecov Report

Merging #64 (82b2343) into master (3ed70f1) will increase coverage by 0.08%.
The diff coverage is 96.87%.

@@             Coverage Diff              @@
##             master      #64      +/-   ##
============================================
+ Coverage     66.58%   66.66%   +0.08%
- Complexity      577      586       +9
============================================
  Files            99      101       +2
  Lines          1975     2007      +32
  Branches        217      224       +7
============================================
+ Hits           1315     1338      +23
- Misses          597      604       +7
- Partials         63       65       +2

Impacted Files
Coverage Δ

.../seedu/address/logic/parser/AddressBookParser.java
61.53% <0.00%> (-2.47%)
:arrow_down:

src/main/java/seedu/address/model/Model.java
100.00% <ø> (ø)

...du/address/logic/commands/PriorityListCommand.java
100.00% <100.00%> (ø)

src/main/java/seedu/address/model/AddressBook.java
92.30% <100.00%> (+0.64%)
:arrow_up:

...rc/main/java/seedu/address/model/ModelManager.java
100.00% <100.00%> (ø)

...a/seedu/address/model/person/UniquePersonList.java
93.02% <100.00%> (+0.34%)
:arrow_up:

...odel/person/comparators/TagPriorityComparator.java
100.00% <100.00%> (ø)

src/main/java/seedu/address/model/tag/Tag.java
83.33% <100.00%> (+1.51%)
:arrow_up:

...rson/predicates/TagsContainsKeywordsPredicate.java
62.50% <0.00%> (-25.00%)
:arrow_down:

.../java/seedu/address/model/util/SampleDataUtil.java
0.00% <0.00%> (-20.00%)
:arrow_down:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ed70f1...82b2343. Read the comment docs.

LGTM!


Sampling of clips for the testing phase

Hi!
I had a question concerning the sampling procedure of input clips for the lta testing phase. When running the evaluation code, I get the following Debug messages image. Having looked at the code of the ptv_dataset_helper, I am not sure I completely understand how clips for testing are sampled. From my understanding, the following happens:
In the ptv_dataset_helper, where input clips are sampled, a counter self._next_clip_start_time is kept, which keeps track of the time of the last clip sampled from the particular video. During training and validation, since sub-clips from annotated clips are subsampled at random, the parameter self._next_clip_start_time is not taken into account. During testing, however, this parameter is added to all the sub-clips which are sampled from the same video. But looking at the way the sequences of the input clips are sampled, I do not see why this counter is required. For example, given clips (i1, i2, i3, i4, i5) from a video v, if we sample 4 clips for the encoder, we get the following input tuples t1 = (i1, i2, i3, i4) and t2 = (i2, i3, i4, i5). If we process t1 before t2 (or t2 before t1 also) and the ending time of clip i4 is greater than the starting time of clip i2 (which is true most of the time), then input clips from t2 won't be loaded correctly. (I add a more concrete example in the photos)

So, I wonder if I misunderstand something about how clips are sampled or is there a way to ignore this counter?
Thank you for your answer!

Hello, I see this issue is now old, but I guess the author and other readers may still benefit from some clarifications. Maybe @Tushar-N can comment on this issue since it is related to LTA?

Hi, I am also interested in this. Is there any update? Thanks!

Sorry for the delay on this, and thanks for your detailed report on the issue.
As you pointed out, the issue stems from the fact that the same clip needs to be loaded for multiple instances, causing an inconsistency with self._next_clip_start_time. In short, the start time would go out of the clip bounds, and result in clips being sampled from arbitrary points in the video (but in a deterministic manner).
The simplest solution to this is always selecting the center clip during testing using a custom sampler like below. The old code was testing with a single view per clip anyway, so this is consistent with the original test loop.
from pytorchvideo.data.clip_sampling import ClipSampler, ClipInfo
from typing import Dict, Any

class CenterClipVideoSampler(ClipSampler):
    """
    Samples just a single clip from the center of the video (use for testing)
    """

    def __init__(
        self, clip_duration: float
    ) -> None:
        super().__init__(clip_duration)

    def __call__(
        self, last_clip_time: float, video_duration: float, annotation: Dict[str, Any]
    ) -> ClipInfo:

        clip_start_sec = video_duration / 2 - self._clip_duration / 2

        return ClipInfo(
            clip_start_sec,
            clip_start_sec + self._clip_duration,
            0,
            0,
            True,
        )

and then in the Ego4dLongTermAnticipation class:
if mode == 'test':
    clip_sampler = CenterClipVideoSampler(clip_duration)

@Tushar-N
Hello, I have replaced your new codes.
But it seems to have some errors for submitting the output.json.
Some keys are missing in the Stdout file on the EvalAI website.
{'85f21d37-73c8-42e8-bdda-1ff89dd6eb15_8', '7a6cdfe4-db07-4807-a3c7-6faca1ce9be5_30', '366b9e0e-a924-4098-8fd0-c480ab1d4972_29', '43effa66-0aea-46ca-be77-a3498395cbee_178', '515486af-f327-47ca-a1fc-28549f913e47_12', 'aac468fa-eb38-4d85-bf68-20eb452b2d27_21', 'a70e725a-3c7e-45b6-b602-85d48f6752e8_29', 'e2ddee72-8dd1-464e-bad0-ce55d51e75c3_23', '506792a1-a4fe-475e-a358-665150095b81_54', 'a70e725a-3c7e-45b6-b602-85d48f6752e8_97', '21280453-ee88-4271-aa97-a111983bfef9_11', '5eed19de-ab08-4015-980d-8e75fe411094_46', '43effa66-0aea-46ca-be77-a3498395cbee_104', '89422cf4-ffa4-4d3c-b469-6d899f8379da_63', '4a71ef05-d236-46ec-8a4c-8a384a2b1d23_41', 'bbc51aad-3e80-48e8-8338-ee1828ede7b9_41', '814baefc-d043-40f8-bb52-3573266f613f_25', 'b4c14f3c-c8d7-45ba-bad4-d69438eab173_10', 'ccd39504-2a28-4b1c-ad85-d75b3ca622ad_54', '82fe2903-49b0-42f2-b9f5-f8b383ba2a96_29', '675de519-10e4-41ef-965a-6397b4218cdc_46', 'dee0dd43-f7bf-4035-8c37-cf8ea49f5d1e_45', '02d5e4b5-71a8-4469-b5ed-ae22055443ed_25', 'cf7e712c-aaf6-404f-bea5-cea214548c92_37', 'a70e725a-3c7e-45b6-b602-85d48f6752e8_44', '6d56d91c-f1b1-46c6-9456-e07d1dbbeb55_35', 'fc4f23c7-ba2e-4d9f-afca-32f15e47ef92_19', 'bf74ed2d-889b-49df-a023-dc4ce6a346f3_16', '6d8c9de4-65e5-4b5a-a1ed-d63de485099b_33', '279bdfb0-9dce-457b-adf0-621c4da187ac_71', 'fb5606be-9917-424e-83c8-192dd05ce959_15', '7535f4ff-8469-41ad-8f93-9d59315db900_21', '13683720-d092-4f6f-a438-75701f4ddfb3_31', '1c29b7e7-f7d3-4b90-b6a1-94347b9ac91a_33', '9b212da8-6dba-4684-995e-16f7d4eaa95e_100', '43effa66-0aea-46ca-be77-a3498395cbee_116', '279bdfb0-9dce-457b-adf0-621c4da187ac_65', '8e68ee6f-f53e-4003-8a1d-e0eed5797a8f_10', '99861853-4c7d-4e48-b146-688c5d16b929_28', '1a2592aa-f26b-4159-9afb-e58040de5d0c_11', '413ea6df-0042-4502-8e5b-4c2de084bf50_19', '515486af-f327-47ca-a1fc-28549f913e47_15', '9b212da8-6dba-4684-995e-16f7d4eaa95e_43', 'a49f4840-a0a1-438d-b474-9a471f55753c_112', '5eed19de-ab08-4015-980d-8e75fe411094_106', 'f4862bca-5a5a-435c-a791-08a0c79b7b08_60', 'b497d0a0-0d14-4c67-8f01-e1aa91e5cbd0_15', '6590d652-42b7-4b10-a08b-37ac378facbf_18', '191729c0-bf2c-4daf-8cb0-f600eec7a17a_17', '339ee84b-6c34-4358-9bd7-b7020cb409a5_36', '1b7b9187-91ac-4ded-86ca-be93043ab5ab_21', '1e0f4f35-c83d-4287-96c6-6de2f5841099_45', '5777b466-b52f-499f-bc8a-2fd0ba33cad9_62', '6e8b9a4b-b7fb-4487-b686-94a325d330d2_56', 'bbc51aad-3e80-48e8-8338-ee1828ede7b9_14', 'b79c7f48-54b4-4e8d-bc75-7b94a8250d7d_25', '85f21d37-73c8-42e8-bdda-1ff89dd6eb15_10', 'a5a9f54a-6b94-4711-867a-76ea90cd3008_7', 'f5916f54-5690-48b8-891b-2a1ba7e73b1a_19', '6db5b987-3c95-4751-92a2-1437c3539b38_37', '1c29b7e7-f7d3-4b90-b6a1-94347b9ac91a_25', 'b497d0a0-0d14-4c67-8f01-e1aa91e5cbd0_16', 'cd0326e0-1025-4596-aec1-8b08b91e732e_13', '6db5b987-3c95-4751-92a2-1437c3539b38_39', '85f21d37-73c8-42e8-bdda-1ff89dd6eb15_21', '6e8b9a4b-b7fb-4487-b686-94a325d330d2_26', 'f9c2847e-2bfa-4f73-ac25-e4ef68c8fbda_52', '9b212da8-6dba-4684-995e-16f7d4eaa95e_175', 'ae54c452-e7b3-4c7f-a056-7e95f42cad05_41', '4214e748-52e3-423e-bdbf-7a9150641e59_76', 'e9200dc7-ef75-4f30-ae23-bae236853994_12', 'b039d40f-d281-455f-83af-b0b9f801e3ad_11', 'bf74ed2d-889b-49df-a023-dc4ce6a346f3_10', 'b79c7f48-54b4-4e8d-bc75-7b94a8250d7d_7', '0d543b21-24a7-4cea-bfa8-74462a968cb0_20', '37773c5d-3cb1-42ee-bc6e-5e61675eb2bb_37'}

@takfate The distributed sampler likes to throw away instances when it can't evenly split them among GPUs. Running inference with batch_size = 1 per GPU will avoid this. i.e., NUM_GPUS 8 TRAIN.BATCH_SIZE 8 TEST.BATCH_SIZE 8

@takfate The distributed sampler likes to throw away instances when it can't evenly split them among GPUs. Running inference with batch_size = 1 per GPU will avoid this. i.e., NUM_GPUS 8 TRAIN.BATCH_SIZE 8 TEST.BATCH_SIZE 8

Thank you.

@Tushar-N
hi,
I have one other problem.
There is an overlap between adjacent clips in the annotations.
It means that the end part of observable clips includes some information from the start part of the forecasting clips.
Does this also mean that when I predict 20 clips, I should be able to use all the information up to the end time of the previous clip?

Yes, that's how the benchmark is defined. The actions are sorted based on PNR timestamps and predictions happen over a much longer horizon (20 actions). If the input is clips[0:i-1], the predictions are for actions in clips[i:i+20]

@Tushar-N
So,
How should I forecast the first action clip(idx=0)?

@Tushar-N So, How should I forecast the first action clip(idx=0)?

Ok.
I find your code passes the len(clips)<20+2.
Thank you.


[TESTING] Clean conn address behaviour to make it clear

Thank you, great chage, these  (void *)((uintptr_t)conn | 1), & ~0xffl and etc  were disgusting...


Remove the unnecessary use of render props in the component composition example.

The use of render props is misguided in the Card example. Render props allow a parent component to provide data to a render without having to explicitly know what its children are. If a render prop takes no arguments it should just be swapped out for static JSX.

To be clear, I'm referring to the use of arrow functions to wrap the jsx. Not the passing of children into properties.

I agree that in this case arrow function is not really doing anything but the goal really was to show that you can pass function as props that will return what to render. Maybe instead of getting rid of the arrow function it's better to change the exercise slightly so that the caller is passing some data to the render props!

Yep, and that makes since. Although I would still suggest you should remove the functions in favor of a simpler, more direct approach. Most of your tutorial focuses on a thorough understanding of the fundamentals. Which is good, because that's actually where most people struggle. I just wouldn't want someone to think composition actually requires render props specifically. Composition through children passing is a strong enough tool to get a dedicated section.


[Question] About websockets

Previously i had a class for handling websocket messages.
ws.onEvent([](AsyncWebSocket * server, AsyncWebSocketClient * client, AwsEventType type, void * arg, uint8_t *data, size_t len){
    hshSocket.handleEvent(server, client, type, arg, data, len);
});

void HsHSocket::handleEvent(AsyncWebSocket *server, AsyncWebSocketClient *client, AwsEventType type, void *arg, uint8_t *data, size_t len) {
    if (type == WS_EVT_DATA) {
        AwsFrameInfo *info = (AwsFrameInfo *)arg;
        for(int i = 0; i < len; i++){
            buffer[socketBuffIndex++] = data[i];
        }
        if (socketBuffIndex >= info->len) {
            std::string message(buffer, info->len);
            messages.push_back( std::move(message) );
            socketBuffIndex = 0;
        }
    }else if (type == WS_EVT_CONNECT) {
        if( debug ){
            logger.write("[HsH_Socket] - (%d) Client connected.\n", client->id());
        }
    } else if (type == WS_EVT_DISCONNECT) {
        if( debug ){
            logger.write("[HsH_Socket] - (%d) Client disconnected.\n", client->id());
        }
    } else if (type == WS_EVT_ERROR) {
        if( debug ){
            logger.write("[HsH_Socket] - (%d) Error: %s\n", client->id(),(char *)data);
        }
    } else if (type == WS_EVT_PONG) {
        if( debug ){
            logger.write("[HsH_Socket] - (%d) Pong received.\n", client->id());
        }
    }
}

I had a global buffer which was allocated in the external RAM so i can accept any size of request at once. For this i had to buffer the data which was separated by AsyncWebsocket like this
if (type == WS_EVT_DATA) {
    AwsFrameInfo *info = (AwsFrameInfo *)arg;
    for(int i = 0; i < len; i++){
        buffer[socketBuffIndex++] = data[i];
    }
    if (socketBuffIndex >= info->len) {
        std::string message(buffer, info->len);
        messages.push_back( std::move(message) );
        socketBuffIndex = 0;
    }
}

I want to keep this class with PsychicHttp.
Does this lib passes the data chunked or it buffers the entire data and give it at once?

If it is almost the same, in theory i could do something like this
void HsHSocket::handleEvent(PsychicWebSocketRequest *request, httpd_ws_frame *frame) {
    for(int i = 0; i < frame->len; i++){
        buffer[socketBuffIndex++] = frame->payload[i];
    }
    if (socketBuffIndex >= frame->len) {
        std::string message(buffer, frame->len);
        messages.push_back( std::move(message) );
        socketBuffIndex = 0;
    }
}

@zekageri if you are dealing with websocket requests outside of the onFrame callback, you need to store the client->socket() and use that to get your websocketclient pointer.  Otherwise you might have issues if a client disconnects between sending you a message and your response.
Docs are up here: https://github.com/hoeken/PsychicHttp#websockets
To check the socket:
//make sure our client is still connected.
PsychicWebSocketClient *client = websocketHandler.getClient(socket);
if (client != NULL)
  client->send("Your Message")

Same applies to eventsource clients.  Always store that socket # and not the client pointer

Thank you very much. I will try your suggestions. Right now the websocket client can not connect for some reason. I investigate it first. And i need to adjust my session logic too. I want to serve requests only to the logged in users but i need some exception path like login and auth.

i don't want to deal with socket clients. I just receive a json message and put it into a lifo std::deque buffer.
I process the messages one by one in the main loop.
void HsHSocket::loop(){
    if( messages.empty() ){return;}
    std::string message = messages.front();
    emit(message.c_str());
    messages.pop_front();
}

void HsHSocket::handleEvent(PsychicWebSocketRequest *request, httpd_ws_frame *frame) {
    if (frame->len <= sizeof(buffer)){
        memcpy(buffer, frame->payload, frame->len);
        std::string message(buffer, frame->len);
        messages.push_back( std::move(message) );
    }
}

At my server init method i do this
hshSocket.init();
server.on(WS_SOURCE, &ws);

#define WS_SOURCE "/hshSocket"

void HsHSocket::init(){
    if (!hshSystem.alloc(buffer, MAX_FRAME_SIZE) && debug) {
        logger.write("[HsH_Socket] - Failed to allocate memory!\n");
    }
    ws.onOpen([](PsychicWebSocketClient *client) {
        Serial.printf("[socket] connection #%u connected from %s\n", client->socket(), client->localIP().toString());
    });
    ws.onClose([](PsychicWebSocketClient *client) {
        Serial.printf("[socket] connection #%u closed from %s\n", client->socket(), client->localIP().toString());
    });
    ws.onFrame([this](PsychicWebSocketRequest *request, httpd_ws_frame *frame) {
        handleEvent(request, frame);
        return true;
    });
}

Client can not connect for some reason.
Events also can not connect.
Strange...

Okay the problem was that i added the handlers before server.listen and not after. Now my events and ws can connect.


Improve error reporting for malformed hashtags and attributes

Based on #269, originally reported by @mcarans
If more than 75% of the hashtag specs in a row are malformed, libhxl will simply assume that it's not meant to be the hashtag row, but that error isn't helpful if the hashtag specs were accidentally malformed, like in this example:
#affected+idps+ind+year+2013+2014
Instead, see if there's a row that might be meant as the hashtag row, and if so, provide more-meaningful messages about syntax errors.

I'm not sure if I should put this here or in a new ticket: perhaps in addition to for each row calculating a percentage until a fuzzy threshold is exceeded, also store those percentages and if you hit the max number of rows (25 I think it is) without the threshold being reached, use the row with the highest percentage of the 25 provided it exceeds a lower threshold. Just an idea to handle malformed hashtags (but the error reporting is still needed).

I'm not sure if I should put this here or in a new ticket: perhaps in addition to for each row calculating a percentage until a fuzzy threshold is exceeded, also store those percentages and if you hit the max number of rows (25 I think it is) without the threshold being reached, use the row with the highest percentage of the 25 provided it exceeds a lower threshold. Just an idea to handle malformed hashtags (but the error reporting is still needed).


Added browser support.

Ability to zip and unzip with JSZip library (https://github.com/Stuk/jszip).

Hi there @fefc,
Sorry for the delay and thanks for the contribution. I try to test this but didn't figure out how to. Any suggestions (I'm using browser platform and try with both, input file and absolute paths)

I think this implementation is based on chrome's only api https://developer.mozilla.org/en-US/docs/Web/API/Window/requestFileSystem

Hi @jjdltc
,
I used the code this way:
JJzip.zip(this.file.dataDirectory + 'toZipDirectory', {target: this.file.cacheDirectory, name: 'zipName'}, (data) => {
 if (data.success) {

} else {

}

}, (error) => {

});

Based on the File Plugin (https://cordova.apache.org/docs/en/latest/reference/cordova-plugin-file/)
this.file.dataDirectory => localhost/persistant
this.data.cacheDirectory => localhost/temporary
So the content of 'toZipDirectory' will be at localhost/temporary/zipName.zip. I doubt you can access the zip trough the url because of security implementations on web browsers.
However you could use the fileOpener Plugin (https://github.com/pwlin/cordova-plugin-file-opener2) where I implemented the Browser version with following code:
        this.fileOpener.open(this.file.cacheDirectory + 'zipName' + '.zip', 'application/zip').then(() => {
        }).catch((error) => {
          alert("Something went wrong while exporting: " + error);
        });

Which will open a download dialog where you can get the zip.
At time of coding zip/unzip where working on Firefox and Safarai. Chrome has permission issues so it doesn't work properly.
Does that helps in any point?


[Bug]: Annotation file names can collide

Joulescope serial number

UI version
0.9.11
What OS are you seeing the problem on?
Windows 11
What happened?
"{filename}.jls" files match with separate "{filename}.anno.jls" files containing annotations.  The UI allows for multiple annotations per base JLS file, so really "{filename}*.anno.jls" will match.  However, this can cause an issue with some filenames.  For example, if you have capture0.jls and capture0_0.jls, the annotations capture0.anno.jls and capture0_0.anno.jls with both match capture0.jls.
What was expected?
Annotations should only match with their respective base file.
How to reproduce?

Make a capture.  Add annotations.  Save as capture0.jls
Make a capture.  Add annotations.  Save as capture0_0.jls in the same directory.
Load capture0.jls and see both sets of annotations.

Extra information
I can see the potential value in having multiple annotation files.  You could perform some automatic detection and then manually add other annotations to a separate file.  Merging annotation files might also be useful.  However, if we keep the same scheme, we could match on "{filename}.anno*.jls" rather than "{filename}*.anno.jls" to minimize the likelihood of this conflict.

The code is in joulescope_ui.widgets.waveform.annotations.annotation_filename.

Fixed in 1.0.15.


Loggregator dropping logs

We run acceptance tests in our CI for any change to our Cloudfoundry setup. We are currently using CF v253 which includes loggregator v79. This runs in AWS eu-west-1.
Recently they have started failing randomly and this has seriously decreased our productivity since almost none of our changes were going through the pipeline.
The failing tests are always related to logs. Many tests connect via cf logs and expect a specific message. And this message never arrives.
We have run the loggregator smoke tests to isolate the problem and we indeed loose a significant number of messages.
We send 10000 messages with a delay between 5us and 1ms. We sometimes get the 10000 messages back, but often we receive much less (down to 3000).
It is not obvious that it is related to the load as it is very irregular: sometimes it fails for 1ms and succeeds with 5us.
We have obeserved the metron agent on the cells drops UDP packets most when messages are lost.
We can force the error by:

Stressing the CPU on the cells up to 0% idle
Saturating the network between the cells and dopplers

However, this creates an unrealistic load, much higher than the acceptance tests create.
What we have tried so far:

Increase the number of cells from 2 to 6 (Recommended by the loggregator team)
Increase the number of dopplers from 2 to 6
Increase the doppler drain buffer size to 100000
Recompiled metron to stop recycling connections (looked better for some time but maybe redherring)
Reduced the acceptance test nodes from 10 to 5. The tests take longer but this is the only thing working right now.
Revert CF to v250. The behaviour looks slightly different. We still loose messages, but in a more consistent way. For example for 50-100us delay we always loose about 1000 messages. For 5 us, we always loose about 3000.

@saliceti thanks for this issue. It's a helpful piece of feedback for us. A few thoughts.

The predictability seems logical given the changes introduced in Loggregator 72. Essentially all loss is moving to UDP which is consistent with your results.
The controlled experiments results are worth focusing on further. A few more things to consider.

Do you have firehose nozzles? Are they appropriately scaled?
Are you using syslog drains? How many are they fast enough to keep up?

We recommend running the loggregator smoke tests regularly to determine ongoing message reliability and look for any potential patterns to message reliability fluctuations.
There are other reasons we see cf logs fail at times for tests. The websocket connection can be slow to connect and timeout. it is worth ensuring you have appropriate timeouts on tests that rely on cf logs.

Also, are you using any tools for monitoring metrics? There are several metrics to look at while these tests run as well that can help us further isolate loss and help with scaling.

Hi @ahevenor, thanks for your help.

We use graphite nozzle and datadog nozzle. I've stopped both and logs are still dropped
We don't use any syslog drains, except the ones the acceptance tests may configure. We have syslog configured on the VMs to ship logs to logsearch but I believe this is different.
I have added a job to run the smoke tests continuously. Let's see if we can find a pattern.
Regarding the cf logs timeouts, we only use the acceptance tests defaults.
We use datadog for monitoring. Which metrics would you recommend we should monitor?

@saliceti here some charts I setup in datadog for monitoring - and they do show increased loss starting when we deployed CF 253. The increase in doppler.loggregator.dropped is not so surprising but the doppler.shedEnvelopes increase is unexpected.  I'd be curious to know if you are seeing the same.
{
	"viz": "timeseries",
	"requests": [{
		"q": "ewma_20(per_second(avg:datadog.nozzle.doppler.loggregator.dropped{*}))",
		"aggregator": "avg",
		"style": {
			"palette": "cool"
		},
		"type": "line",
		"conditional_formats": []
	}, {
		"q": "ewma_20(per_second(avg:datadog.nozzle.DopplerServer.doppler.shedEnvelopes{*}))",
		"style": {
			"palette": "cool"
		},
		"type": "line"
	}],
	"autoscale": true
}

We also post our smoke tests results into datadog. They show more volatility but the ramp is not as as obvious as it is with the above chart. We happen to have a separate bug already created on this here which is currently at the top of our backlog. I'll keep you posted on what we have found out.

I can't see the metrics *.doppler.loggregator.dropped and *.DopplerServer.doppler.shedEnvelopes in our datadog.
They don't even appear in cf nozzle -no-filter either.
Any idea?

@saliceti that implies you are not dropping anything out of these particular buffers (it requires a bit of effort to force). To confirm that you are configured correctly can you also look at doppler.loggregator.ingress and ensure it is coming through the firehose.

Is it that one?
origin:"doppler" eventType:CounterEvent timestamp:1491431932910955545 deployment:"" job:"" index:"" ip:"" tags:<key:"metric_version" value:"2.0" > tags:<key:"prefix" value:"loggregator" > counterEvent:<name:"loggregator.ingress" delta:0 total:894620 >
origin:"metron" eventType:CounterEvent timestamp:1491431933978336986 deployment:"colin2017" job:"uaa" index:"80b2258e-af01-4dfe-a032-8b51c1831579" ip:"" tags:<key:"metric_version" value:"2.0" > tags:<key:"prefix" value:"loggregator" > counterEvent:<name:"loggregator.ingress" delta:0 total:238324 >

Those are correct.
Are you able to monitor network io on dopplers? One thing we are investigating is an increase in network io there.
Also, how have the smoke tests looked the last ~24 hours?

I'm seeing significant network usage on our 2 doppler VMs when the smoke test runs, up to 8-9 Mb/s. But the doppler VMs are m3.medium and should handle that easily?
The smoke tests are failing at random and I couldn't find any pattern. However, I tried a few things:

Stopped the graphite nozzle
Stopped a test app with 180 instances
Stopped our CF smoke tests that run continuously
Stopped the datadog nozzle (2 instances)

Finally there was no message dropped.
I started 1 instance of the datadog nozzle and I started to see dropped messages, like 2000-4000. So it definitely looks suspicious.
Following the scaling nozzle doc, I added the other instance. It looked a bit better, I could see more tests passing, and when it was failing there was about 2500 dropped messages.
I then added 2 more nozzles but the performance degraded and tests didn't pass.
I'm going to update the datadog nozzle now as we're not using the latest version.

The latest datadog nozzle hasn't improved things
This is how the smoke tests looked today:

The drain (via POST) is really bad but I'd rather focus on fixing the cf logs first.

We are also seeing corresponding network usage increase.  It seems within healthy bounds but we noticed at the upgrade path point from CF252 -> CF253. It's not conclusively linked to anything but is something we are noting with our test environments.

Agreed. Let's focus on cf logs. There are some known deficiencies with drain we are working on a new architecture for and it should be completed soon.
What is the above delay set in the smoke test for the charts above?

The delay is set to 5us. But I was seeing dropped messages for 100us as well.

Thanks for sharing your results.
We have been running smokes against several versions of loggregator over the weekend with a consistent delay (we're running 2us but the specific delay is less important than the historical comparison).
Our results are showing that there is a 4-5% degradation in message reliability between Loggregator 76 vs later versions. We are testing starting some additional tests now to test some theories on root cause.

@saliceti we have identified a fix for this and brought our smokes back to equivalent performance of Loggregator 76. This is currently on our develop branch and we'll be cutting a new release soon.

The fix for this now available in Loggregator 84. I'll plan to close this issue when this is packaged in cf-release.
cc @dsabeti

Thanks @ahevenor . We'll test with the new cf release when it's included.

@saliceti this is now included in CF-257

@ahevenor Thanks. Would you be able to link us to the commits that fixed the problem so that we can understand it better?

@dcarley there is a list available in this story

Excellent, thank you.

@saliceti I am going to close this based on our results of running CF-257 vs CF-253. Please let us know if you don't see an improvement in message reliability in your deploy.

Thanks, I'll test with CF 257 soonish.


bug(connector): error message was not propagated to frontend

Describe the bug
Take this as an example:

The actual error is
Caused by: java.sql.SQLSyntaxErrorException: Table 'test.count_by_value' doesn't exist

But it can only be found in connector-node.log.
To Reproduce
No response
Expected behavior
No response
Additional context
No response

@tabVersion Please take a look and feel free to assign to others


Update stop-error-or-blue-screen-error-troubleshooting.md

Updates to improve article clarity and accuracy

@jasonepperly Thanks for your contribution, we'll proceed to handle the changes.

@jasonepperly The changes are live at https://learn.microsoft.com/en-us/troubleshoot/windows-client/performance/stop-error-or-blue-screen-error-troubleshooting. This PR will be closed.


dpc implementation

In the implementation of dpc algorithm, for code of this line,
as we know the size of  dist_matrix is B N N, because we want to get the max distance of each token, but if we flat the dist_matrix, we will only get the max distance of each batch.
dist_max = dist_matrix.flatten(1).max(dim=-1)[0][:, None, None]
we can change the code to
dist_max = dist_matrix.max(dim=-1)[0][:, :, None]

Thanks for your code. I'll go over my previous code tomorrow.


BouncingScrollPhysics breaks further scrolling after content was scrolled off viewport

Is there an existing issue for this?

[X] I have searched the existing issues
[x] I have read the guide to filing a bug

Steps to reproduce

Run code sample
Scroll content off the screen in any direction as in video
Release fingers of screen

Expected results
After releasing fingers ListView's content should return back to top of the screen and scrolling should keep applying physics.
Actual results
After releasing fingers ListView's content stays out of the screen in it's position and not jumped back to top of the screen. Also further scrolling stopped applying physics.
Code sample
Code sample
import 'package:flutter/material.dart';

class HomeScreen extends StatelessWidget {
  const HomeScreen({
    super.key,
  });

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('AppBar'),
      ),
      body: SafeArea(
        child: ListView.builder(
          physics: const BouncingScrollPhysics(),
          itemCount: 100,
          itemBuilder: (_, index) {
            return ListTile(
              title: Text('Element $index'),
            );
          },
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
https://github.com/flutter/flutter/assets/50376680/e68936ba-0f54-44dd-b2a6-504fa6946e87

Logs
No response
Flutter Doctor output
Doctor output
[!] Flutter (Channel unknown, 3.10.5, on Microsoft Windows [Version 10.0.19045.3208], locale ru-RU)
    ! Flutter version 3.10.5 on channel unknown at Y:\SDKs\flutter
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at
      https://flutter.dev/docs/get-started/install.
    ! Unknown upstream repository.
      Reinstall Flutter by following instructions at https://flutter.dev/docs/get-started/install.
    • Framework revision 796c8ef792 (6 weeks ago), 2023-06-13 15:51:02 -0700
    • Engine revision 45f6e00911
    • Dart version 3.0.5
    • DevTools version 2.23.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly
      to perform update checks and upgrades.

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    • Android SDK at Y:\SDKs\Android
    • Platform android-33-ext5, build-tools 33.0.2
    • ANDROID_HOME = Y:\SDKs\Android
    • Java binary at: Y:\Programs\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694)
    • All Android licenses accepted.

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[√] Visual Studio - develop for Windows (Visual Studio Community 2022 17.2.6)
    • Visual Studio at O:\Programs\Microsoft Visual Studio\2022\Community
    • Visual Studio Community 2022 version 17.2.32630.192
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2022.2)
    • Android Studio at Y:\Programs\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694)

[√] IntelliJ IDEA Community Edition (version 2023.1)
    • IntelliJ at Y:\Programs\IntelliJ IDEA
    • Flutter plugin version 74.0.4
    • Dart plugin version 231.9392.3

[√] Connected device (2 available)
    • Windows (desktop) • windows • windows-x64    • Microsoft Windows [Version 10.0.19045.3208]
    • Edge (web)        • edge    • web-javascript • Microsoft Edge 115.0.1901.183
    ! Device <IP_ADDRESS>:5555 is offline.

[√] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.

You need to wrap the ListView.builder with a SingleChildScrollView to make the list scroll back to the top. The SingleChildScrollView will provide the bounce-back effect when you scroll the list.

Wrap the ListView.builder() with a SingleChildScrollView() to make the list scroll back to the top. The SingleChildScrollView() will provide the bounce-back effect when you scroll the list.

No, I do not need to do this and this does not solve the problem.

Hi @mozochek. I'm unable to take all the content of the list off the screen on my physical iOS device but with only one element on the screen, the list still bounces back on release, can you provide more information on this issue?
Most specifically:

Does this reproduce only on Android? Also, can you provide more information on the device you're reproducing this on.
Can you switch to the latest master channel to confirm this issue still persists?

Thank you!

@dam-ease
I have reproduced this issue on master channel.
I don't have iOS device so I am unable to reproduce this issue on iOS.
My device is Honor 50 with Magic UI 6.0(Android 12).

Screencast

debug
release

flutter doctor -v
[√] Flutter (Channel master, 3.13.0-11.0.pre.27, on Microsoft Windows [Version 10.0.19045.3208], locale ru-RU)
    • Flutter version 3.13.0-11.0.pre.27 on channel master at Y:\SDKs\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision e81907e07a (35 minutes ago), 2023-07-28 08:04:19 -0700
    • Engine revision e97014c710
    • Dart version 3.2.0 (build 3.2.0-11.0.dev)
    • DevTools version 2.25.0

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    • Android SDK at Y:\SDKs\Android
    • Platform android-33-ext5, build-tools 33.0.2
    • ANDROID_HOME = Y:\SDKs\Android
    • Java binary at: Y:\Programs\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694)
    • All Android licenses accepted.

[X] Chrome - develop for the web (Cannot find Chrome executable at .\Google\Chrome\Application\chrome.exe)
    ! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.2.6)
    • Visual Studio at O:\Programs\Microsoft Visual Studio\2022\Community
    • Visual Studio Community 2022 version 17.2.32630.192
    • Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2022.2)
    • Android Studio at Y:\Programs\Android Studio
    • Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-9586694)

[√] IntelliJ IDEA Community Edition (version 2023.2)
    • IntelliJ at Y:\Programs\IntelliJ IDEA
    • Flutter plugin version 74.0.5
    • Dart plugin version 232.8660.129

[√] Connected device (3 available)
    • NTH NX9 (mobile)  • <IP_ADDRESS>:5555 • android-arm64  • Android 12 (API 31)
    • Windows (desktop) • windows            • windows-x64    • Microsoft Windows [Version 10.0.19045.3208]
    • Edge (web)        • edge               • web-javascript • Microsoft Edge 115.0.1901.183

[√] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const App());
}

class App extends StatelessWidget {
  const App({
    super.key,
  });

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const HomeScreen(),
    );
  }
}

class HomeScreen extends StatelessWidget {
  const HomeScreen({
    super.key,
  });

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('AppBar'),
      ),
      body: SafeArea(
        child: ListView.builder(
          physics: const BouncingScrollPhysics(),
          itemCount: 100,
          itemBuilder: (_, index) {
            return ListTile(
              title: Text('Element $index'),
            );
          },
        ),
      ),
    );
  }
}

Hi @mozochek. Thanks for these. I tried reproducing this on an Android device (Android 12) and still can't reproduce it (video below). On iOS, I am unable to scroll off the viewport. Are there any steps in reproducing this that I might have missed?
https://github.com/flutter/flutter/assets/53122008/fee1d92d-9ea0-4cd1-bdf7-b0f75266b7ab

@dam-ease I think you need to push it little bit deeper, so it should be out of cache extent
https://github.com/flutter/flutter/assets/50376680/5c5c167d-7c66-41ce-81ba-5103d65fbbe6

Hi @mozochek. Thanks for your patience but I still can't reproduce it on my devices, not sure if this might be device-specific. Also, can you please provide the code sample used here https://github.com/flutter/flutter/issues/131479#issuecomment-1658119409 maybe it could be easily reproduced with that.

Hi @mozochek. Thanks for your patience but I still can't reproduce it on my devices, not sure if this might be device-specific. Also, can you please provide the code sample used here #131479 (comment) maybe it could be easily reproduced with that.

Since this video was captured on iOS, BouncingScrollPhysics was applied by default.
class HomeScreen extends StatelessWidget {
  const HomeScreen({
    super.key,
  });

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('AppBar'),
      ),
      body: SafeArea(
        child: SizedBox(
          height: 300,
          child: ListView.builder(
            itemCount: 100,
            cacheExtent: 0,
            itemBuilder: (_, index) {
              return ListTile(
                tileColor: Colors.blue.withOpacity(0.3),
                title: Text('Element $index'),
              );
            },
          ),
        ),
      ),
    );
  }
}

Hi @mozochek. Thanks for your response. I tried reproducing this on my iOS device (iPhone X) as well. Tried pushing completely out of cache on Android as well but the list bounces back as it should. Are there any additional reproduction steps to take the blue container out of screen?

Android
iOS

Code Sample Used

import 'package:flutter/material.dart';
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:google_maps_flutter/google_maps_flutter.dart';

void main() {
  runApp(const App());
}

class App extends StatelessWidget {
  const App({super.key});

  @override
  Widget build(BuildContext context) {
    return const MaterialApp(
      home: Scaffold(
        body: HomeScreen(),
      ),
    );
  }
}

class HomeScreen extends StatelessWidget {
  const HomeScreen({
    super.key,
  });

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('AppBar'),
      ),
      body: SafeArea(
        child: SizedBox(
          height: 300,
          child: ListView.builder(
            itemCount: 100,
            cacheExtent: 0,
            physics: BouncingScrollPhysics(),
            itemBuilder: (_, index) {
              return ListTile(
                tileColor: Colors.blue.withOpacity(0.3),
                title: Text('Element $index'),
              );
            },
          ),
        ),
      ),
    );
  }
}

stable, master flutter doctor -v

[✓] Flutter (Channel stable, 3.10.6, on macOS 13.0 22A380 darwin-arm64, locale en-NG)
    • Flutter version 3.10.6 on channel stable at /Users/damilolaalimi/sdks/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f468f3366c (3 weeks ago), 2023-07-12 15:19:05 -0700
    • Engine revision cdbeda788a
    • Dart version 3.0.6
    • DevTools version 2.23.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/damilolaalimi/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.12.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.80.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (4 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554                        • android-arm64  • Android 14
      (API 34) (emulator)
    • iPhone 14 (mobile)          • 1A122DE2-0CAB-4C3E-A395-691BF27D626F • ios            •
      com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)             • macos                                • darwin-arm64   • macOS 13.0
      22A380 darwin-arm64
    • Chrome (web)                • chrome                               • web-javascript • Google
      Chrome 115.0.5790.114

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[✓] Flutter (Channel master, 3.13.0-16.0.pre.21, on macOS 13.0 22A380 darwin-arm64, locale en-NG)
    • Flutter version 3.13.0-16.0.pre.21 on channel master at /Users/damilolaalimi/fvm/versions/master
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision e8ebbfa2a8 (33 minutes ago), 2023-08-02 07:29:08 +0000
    • Engine revision 9dae7b708b
    • Dart version 3.2.0 (build 3.2.0-22.0.dev)
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/damilolaalimi/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E300c
    • CocoaPods version 1.12.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.80.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.50.0

[✓] Connected device (4 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554                        • android-arm64  • Android 14 (API 34) (emulator)
    • iPhone 14 (mobile)          • 1A122DE2-0CAB-4C3E-A395-691BF27D626F • ios            • com.apple.CoreSimulator.SimRuntime.iOS-16-4 (simulator)
    • macOS (desktop)             • macos                                • darwin-arm64   • macOS 13.0 22A380 darwin-arm64
    • Chrome (web)                • chrome                               • web-javascript • Google Chrome 115.0.5790.114

[✓] Network resources
    • All expected network resources are available.

• No issues found!

I dont know what else to say. I can't understand how you still unable to repeat it. You don't need any additional info, all info I already gave. You just need to try it more then 1 time.
https://github.com/flutter/flutter/assets/50376680/dedfe539-bbae-43dc-a6ca-28e8ef6dc71f

Thanks for your patience @mozochek. Unfortunately, I still can't reproduce scrolling with 3-4 fingers. I'd label for further insights. Thank you!
Video

https://github.com/flutter/flutter/assets/53122008/a365139b-edd8-443c-b8fa-e067e1858325


[ENH] Define "modality" and clarify "data type"

Closes #584.
Closes #586.
Reworded slightly from initial suggestion.

WDYT of also slightly updating the definition of "data types" to clarify that they're reflect in folder names?

That one was actually a bit harder to word, IMO. Let me know what you think.

Unless anyone raises any concerns about it, this PR should be good to merge tomorrow (5 days after second approval).

Unless anyone raises any concerns about it, this PR should be good to merge tomorrow (5 days after second approval).

I agree that it's good to be merged, it's a nice clarification.


Add header with empty nav

Add header component

Hey @vikas924 , your project is completed. You can merge it


yoffset is not respected in dynamicprompt function

If i change yoffset to some positive number large enough to move image few rows below info text, prompt is drawn over image.
I think that dynamicprompt should use yoffset to move prompt further down if needed:
(image height + yoffset) / font height + 4

Nice find!

Fixed.


WIP: CLDC-731 bypass tasklist from check answers

Save and go to next incomplete section when the case log still has subsections to complete

Save and go to submit when the case log only has the declaration left to complete

@paulrobertlloyd Could you have a play around with this and see if the following scenarios behave like you'd expect
Scenario 1: About this log completed, go to check answers about this log, "Save and go to next incomplete section" is on page and redirects to the first question in household characteristics (even if that first question has been completed)
Scenario 2: About this log completed, household characteristics completed, go to check answers for about this log, "Save and go to next incomplete section" is on page and redirects to the first question in household situation (even if that first question has been completed).
Scenario 3: About this log completed, household characteristics not completed, household situation completed, go to check answers for household situation, "Save and go to next incomplete section" is on page and redirects to the first question in household needs (even if that first question has been completed) and not back to household characteristics.
Scenario 4: All the sections completed (except the declaration), go to check answers for any section, "Save and go to submit" is on page and redirects to the first question in the declaration subsection.

@baarkerlounger Will write a test for that behaviour and then make sure it's implemented. Will ping you for a quick re-review when that's done 🙂


coinbase wallet sdk includes solana/web3.js in my non solana app

Describe the bug
solana included
Steps
npm ls

<EMAIL_ADDRESS>/Users/timdaub/Projects/attestate/kiwistand/src/web
└─┬<EMAIL_ADDRESS>└─┬<EMAIL_ADDRESS>└─┬<EMAIL_ADDRESS>└─┬<EMAIL_ADDRESS>└─┬<EMAIL_ADDRESS>└─┬<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>
Expected behavior
it must not include it in my bundle
Version
see above
Additional info
No response
Desktop
No response
Smartphone
No response

So you are the first one right

Ok yes
Sent from Yahoo Mail for iPhone
On Thursday, July 27, 2023, 4:47 PM, LuisLopez44431 @.***> wrote:
So you are the first one right
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>

What to help what are you saying honey

Ok is tha end buddy can explain

So now what are you guys think tha is going to happened I don't have or involve in all this so god is perfectly perfect.

hi. sorry for the late response. we are working on this and it will be removed soon.

Cuestion is that you are that having access to this accounts end way?


Expose Map currentKey  as public property.

Please provide a reason for your request

I wanted to create a merge functionality and in my case i needed the current key so I could test against it.

Oh ok. Feel free to create a pull request with your functionality. You can change the visibility of the currentKey variable within your Pull Request

I still trying to figuring out how to do this.
The way I'm doing it right now (without actually relaying on ObjectMapper existing functionality),
is by having a Dictionary extension:
extension Dictionary  {

    public func merge(dictionary: Dictionary) -> Dictionary {

        var result = Dictionary()

        self.forEach {(key,value) -> Void in
            _ = result.updateValue(value, forKey: key)
        }

        dictionary.forEach { (key, value) -> Void in
            if value is Dictionary, let v1 = result[key] where v1 is Dictionary {
                let dicValue = (v1 as! Dictionary).merge(value as! Dictionary) as! Value
                _ = result.updateValue(dicValue, forKey: key)
            }else{
                _ = result.updateValue(value, forKey: key)
            }
        }

        return result
    }
}

and then creating an extension for Mappable:
extension Mappable {
    mutating func merge<T:Mappable>(object:T) {
        let json = self.toJSON().merge(object.toJSON())
        mapping(Map(mappingType:.FromJSON, JSONDictionary: json))
    }
}

There's must be a way to create the merge functionality without going back and forth,  Mappable object -> JSON and merged JSON to Map object..
Your thoughts?


Add a license to the repository

I recently came across your project shor_qiskit and I noticed that the repository github.com/kazawai/shor_qiskit currently does not have a license specified.
Why Adding a License is Important?
As GitHub documents

Public repositories on GitHub are often used to share open source software. For your repository to truly be open source, you'll need to license it so that others are free to use, change, and distribute the software.

Would be possible to add a valid license to this repository? Probably something really OpenSource such as the Apache 2.0 license would fit really well.
Thanks!

Done ! Thank you for your advice !

Thanks!


Link account with Github

Would be nice to have option to link laddr account with Github account, and also login via Github

The development branch has initial support for linking an existing account to a GitHub account and storing an access token with user+repo scope


[Bug] The provider is unable to obtain custom URL parameters

Pre-check

[X] I am sure that all the content I provide is in English.

Search before asking

[X] I had searched in the issues and found no similar issues.

Apache Dubbo Component
Java SDK (apache/dubbo)
Dubbo Version
Dubbo java 2.7.22
Steps to reproduce this issue
@DubboReference(version = "1.0.0",
            retries = 0,
            timeout=5000,
            parameters = {"responseFilter=true","securityFilter=false"},
            loadbalance = "roundrobin",
            check = false,
            cluster="failsafe",
            protocol = "dubbo",
            mock = "com.tiny.notify.api.mock.NotifyServiceApiMock")

I want to pass some parameters from the parameter to the provider, so that the consumer can obtain custom parameter keys and value values, but the provider cannot obtain these parameters from the URL
What you expected to happen
Did the built-in interceptor filter out my custom URL parameters？
Looking forward to your reply. Thank you very much
Anything else
No response
Are you willing to submit a pull request to fix on your own?

[X] Yes I am willing to submit a pull request on my own!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Have you tried with parameters = {"responseFilter", "true", "securityFilter", "false"}?
Also, can't you get the value by invoker.getUrl().getParameter("responseFilter")?
BTW, configuring xxxFilter with true or false in parameters cannot enable or disable a certain Filter.

I only want to obtain the parameters of the URL in the provider's interception. Currently, I am unable to obtain custom parameters (responsiveFilter) in the provider's interception, but I can obtain them in the consumer's interception. Is there a problem? I am looking forward to your reply very much
@Activate(group = {CommonConstants.PROVIDER})
public class SecurityFilter extends ListenableFilter {

  @Override
  public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
    String responseFilter = url.getParameter("responseFilter");
    return invoker.invoke(invocation);
  }
}

I only want to obtain the parameters of the URL in the provider's interception. Currently, I am unable to obtain custom parameters (responsiveFilter) in the provider's interception, but I can obtain them in the consumer's interception. Is there a problem? I am looking forward to your reply very much
CONSUMER
@DubboReference(version = "1.0.0",
            retries = 0,
            timeout=5000,
            parameters = {"responseFilter", "true", "securityFilter", "false"},
            loadbalance = "roundrobin",
            check = false,
            cluster="failsafe",
            protocol = "dubbo",
            mock = "com.device.notify.api.mock.NotifyServiceApiMock")
    NotifyServiceApi notifyServiceApi;

PROVIDER
@Activate(group = {CommonConstants.PROVIDER})
public class SecurityFilter extends ListenableFilter {

  @Override
  public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
    String responseFilter = url.getParameter("responseFilter");
    return invoker.invoke(invocation);
  }
}

I only want to obtain the parameters of the URL in the provider's interception. Currently, I am unable to obtain custom parameters (securityFilter) in the provider's interception, but I can obtain them in the consumer's interception. Is there a problem? I am looking forward to your reply very much
CONSUMER
@DubboReference(version = "1.0.0",
            retries = 0,
            timeout=5000,
            parameters = {"securityFilter", "false","responseFilter", "true"},
            loadbalance = "roundrobin",
            check = false,
            cluster="failsafe",
            protocol = "dubbo",
            mock = "com.device.notify.api.mock.NotifyServiceApiMock")
    NotifyServiceApi notifyServiceApi;

PROVIDER
@Activate(group = {CommonConstants.PROVIDER})
public class SecurityFilter extends ListenableFilter {

  @Override
  public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
    String securityFilter= invoker.getUrl().getParameter("securityFilter");//value is NULL
    return invoker.invoke(invocation);
  }
}

I hope this helps you  https://dubbo.apache.org/zh-cn/overview/tasks/develop/context


[midi] can't reach end of track

[midi] object skips the empty space at the end of the midi file, not respecting the length of the clip.
Here, I highlighted the space that is being skipped at the end.

And, here's a screen capture of the looped audio vs the looped midi.
I'm using the midi-seq.mid example from the library.

Any advice?

same happens for cyclone/seq, right?

i just tested on latest else-midi, cyclone-seq and the end of clip is equivalent with the end of the last note. i tried to artificially update end of track in midi file to correspond with end of 4 bars (above example plays twice) hence clear shift error. immediate start of second track repetition is directly attached to the end of the last node from the previous one.
@porres what would be your suggestion to tell else-midi to loop to a track end? if it cannot be done using midi message maybe directly in pd?
I also found this in midi specification:

I bet that the late Marin Peach's [midifile] object deals nicely with this as it is much more sophisticated. I see the "End of track" case being treated in the line 681 of its source code downloaded from deken (btw, 2F = 47).

Thanks for the tip Alex.
RIP Mr Peach

Is else/midi considered supporting MIDI 1 types or only MIDI 0?

I dunno, I'm basically just stealing this from cyclone/seq and I don't how that deals with this.

Could anyone confirm end-of-track can be forced to a position that is not the end of last note in a way that it would be respected by a current else/midi or even cyclone/seq implementations?

how can one do that? I don't have a MIDI editor that shows all these raw data.  I once used a MIDI editor that would show me tempo in BPM, bar signs, tracks, lots of stuff, and also the raw data. Can’t remember what it was though…
can you guys give me a simple MIDI file with a couple of monophonic notes into a single track, that is with minimal information, and then a pause withthe "end of track" specification? Just so I can test it...

mid file is zipped as github did not like .mid extension
one-bar-4x-8th-notes.mid.zip

how do you assemble midi files manually?
here's what  I get when opening your file with sublime text editor
4d54 6864 0000 0006 0000 0001 0060 4d54
726b 0000 0039 00ff 0301 0000 ff58 0404
0224 0800 ff58 0404 0224 0800 903c 6430
803c 4030 9043 6430 8043 4030 9040 6430
8040 4030 903e 6430 803e 4000 ff2f 00

Can’t remember what it was though…

It was something like this https://jeffbourdier.github.io/midiopsy/ but for macOS
this one seems to do the job, but not free/open source https://www.canato.se/my_home/ and I couldn't even open the trial version

I updated endOfTrack message and recreated the midi file. new file zipped is here.
one-bar-4x-8th-notes.384.mid.zip
py-midicsv output
0, 0, Header, 0, 1, 96
1, 0, Start_track
1, 0, Title_t, "\000"
1, 0, Time_signature, 4, 2, 36, 8
1, 0, Time_signature, 4, 2, 36, 8
1, 0, Note_on_c, 0, 60, 100
1, 48, Note_off_c, 0, 60, 64
1, 96, Note_on_c, 0, 67, 100
1, 144, Note_off_c, 0, 67, 64
1, 192, Note_on_c, 0, 64, 100
1, 240, Note_off_c, 0, 64, 64
1, 288, Note_on_c, 0, 62, 100
1, 336, Note_off_c, 0, 62, 64
1, 384, End_track
0, 0, End_of_file

And with ToneJs/Midi
{
  "header": {
    "keySignatures": [],
    "meta": [],
    "name": "\u0000",
    "ppq": 96,
    "tempos": [],
    "timeSignatures": [
      {
        "ticks": 0,
        "timeSignature": [
          4,
          4
        ],
        "measures": 0
      },
      {
        "ticks": 0,
        "timeSignature": [
          4,
          4
        ],
        "measures": 0
      }
    ]
  },
  "tracks": [
    {
      "channel": 0,
      "controlChanges": {},
      "pitchBends": [],
      "instrument": {
        "family": "piano",
        "name": "acoustic grand piano",
        "number": 0
      },
      "name": "\u0000",
      "notes": [
        {
          "duration": 0.25,
          "durationTicks": 48,
          "midi": 60,
          "name": "C4",
          "ticks": 0,
          "time": 0,
          "velocity": 0.7874015748031497
        },
        {
          "duration": 0.25,
          "durationTicks": 48,
          "midi": 67,
          "name": "G4",
          "ticks": 96,
          "time": 0.5,
          "velocity": 0.7874015748031497
        },
        {
          "duration": 0.25,
          "durationTicks": 48,
          "midi": 64,
          "name": "E4",
          "ticks": 192,
          "time": 1,
          "velocity": 0.7874015748031497
        },
        {
          "duration": 0.25,
          "durationTicks": 48,
          "midi": 62,
          "name": "D4",
          "ticks": 288,
          "time": 1.5,
          "velocity": 0.7874015748031497
        }
      ],
      "endOfTrackTicks": 384
    }
  ]
}

You can see now correct value of (track=1 bar=4xPPQ=4x96=384) as on the screenshot. However, it is not respected in else/midi  I have not tested with [midifile] yet. Its past midnight in London.

here's your new file opened in the sublime text editor
4d54 6864 0000 0006 0000 0001 0060 4d54
726b 0000 0039 00ff 0301 0000 ff58 0404
0224 0800 ff58 0404 0224 0800 903c 6430
803c 4030 9043 6430 8043 4030 9040 6430
8040 4030 903e 6430 803e 4030 ff2f 00

4d54 6864 0000 0006 0000 0001 0060 4d54
726b 0000 0039 00ff 0301 0000 ff58 0404
0224 0800 ff58 0404 0224 0800 903c 6430
803c 4030 9043 6430 8043 4030 9040 6430
8040 4030 903e 6430 803e 4000 ff2f 00

this was the old one

I used these:
ToneJs/Midi: https://tonejs.github.io/Midi/
PyMidiCsv: https://pypi.org/project/py-midicsv/
Ableton11: https://www.ableton.com/en/trial/

I don't think I can deal with ToneJs/Midi & PyMidiCsv. I'm still looking around for that MIDI Editor that showed metadata, events and the whole MIDI stream of a file as raw MIDI and what it meant... you could then edit and play, it was so cool...

Yes i would prefer that but I don't know any solid ones.

midifile gives us 384 as the last tick on your file as I did bet so

I'm moving the discussion to https://github.com/porres/pd-cyclone/issues/557 please join us

I have a fix for this in [cyclone/seq], see https://github.com/porres/pd-cyclone/issues/557#issuecomment-917020285
please test it @MikeMorenoDSP and @styk-tv

I'm pretty confident I've fixed this i cyclone, plus I made other improvements, the same are being made to else/midi and ELSE, which will probably be updated sooner. I also added a built in loop function in else/midi, which makes things much more convenient

This is already part of the new release!


Logitech G203 LIGHTSYNC support

Is your feature request related to a problem? Please describe.
I've bought a new Logitech G203 mouse which has a different USB device ids than the G103-203 profile that is in the sources. My model shows as
 $ lsusb
 [...]
 Bus 003 Device 011: ID 046d:c09d Logitech, Inc. G203 LIGHTSYNC Gaming Mouse
 [...]

Button-wise the mouse looks like the previous G203 so it should als work with the existing G103-G203 profile of libratbag.
Describe the solution you'd like
If possible allow this USB id too.

Duplicate: #1406 (fixed by #1389, but not in a release yet).
See Additional context at the bottom of #1406 to see how to apply the change until new libratbag version is released.

@staticssleever668 Thank you very much!


R.equals takes a long time for same objects

assume:
const a = {}
const eq = (a, b) => a === b ? R.equals(a, b)

I ran eq(a, a) and R.equals(a, a) 100 million each and eq took about 17 ms but the latter took around 7800 ms which is 400 times slower. I wonder why?!
It's VERY important because R.equals is used in some other functions such as uniq which is in the order of n^2.

Hmm.
Usually when I see questions like this, they're from people who don't quite grasp why value identity is so much more important to FP than is reference equality.
But this is more interesting.
It would be a change in behavior:
eq(-0, 0) //=> true
R.equals(-0, 0) //=> false

But by the order of operations, it wouldn't affect the edge case that concerns me more:
eq(NaN, NaN) //=> true
R.equals(NaN, NaN) //=> true

And I don't think that's necessarily a change for the worst.  -0 is always an odd issue.
What do people think?  @ramda/core?

S.equals (-0) (0) has evaluated to true since support for Ord was added in sanctuary-js/sanctuary-type-classes#46. To be lawful, -0 needs to be less than (strange!) or equal to 0.

Does NaN simply not compare?  That is, in native, NaN < NaN //=> false, NaN === NaN //=> false, and NaN > NaN //=> false.  Or do you exclude NaN from your Number type?

S.equals (NaN) (NaN) evaluates to true.

Sorry, that was the wrong question.  How about comparing 42 and NaN?

The behaviour is currently incorrect (as I discovered a few days ago while working on sanctuary-js/sanctuary-type-classes#53). I decided, arbitrarily, that NaN will be less than any other Number value, but I have not yet had a chance to open a pull request for this change.

@davidchambers Why does -0 have to be equal to 0? Couldn't -0 be strictly less than 0?
Having R.equals(-0, 0) be true would simplify implementation quite a bit which is nice IMO. But one downside is that it would violate the principle.
if R.equals(x, y), then R.equals(f(x), f(y)) for any f.

Because R.equals(-0, 0) === true but R.equals(R.divide(1, -0), R.divide(1, 0)) === false.

Why does -0 have to be equal to 0? Couldn't -0 be strictly less than 0?

That's right. As I mentioned, there are two possibilities:

To be lawful, -0 needs to be less than (strange!) or equal to 0.

Ah, yes. I see :smile:

@CrossEye

Usually when I see questions like this, they're from people who don't quite grasp why value identity is so much more important to FP than is reference equality.

Could you point me into the path of understanding the full meaning?
Thank you.

We seem to have veered from the main question here.  @ramda/core: are we interested in updating R.equals by adding a short-circuiting reference equality check?  In many places it could certainly improve performance.

@CrossEye Are you ok with it breaking the
if R.equals(x, y), then R.equals(f(x), f(y)) for any f.

property?
If we change this we should also change the internal Set implementation which would be simplified quite a bit since native Sets has the same equality behavior as ===.

@paldepind: I'd much rather not, even though we haven't gotten it working everywhere because of key-ordering issues.
How would this proposal change that?  (The only suggestion was to make the first check in equals be a reference check.  If the two arguments are === then they should be equals according to our current code, right?
(Or is there some trickiness we can do with getters to update a secondary property so that each access of identical references actually returns different object properties?  Pehaps we could create something like this, but I don't know that we'd care.)

@CrossEye

How would this proposal change that?

Did you see the example I posted above? If we do === in R.equals then R.equals(0, -0) is true but R.equals(R.divide(1, -0), R.divide(1, 0)) is not true. That is because dividing by 0 gives positive infinity and dividing by -0 gives negative infinity. It's certainly an edge case but it does violate the principle.
On the other hand, the fact that -0 is not equal to 0 seems to violate basic knowledge from math. It's really just implementation artifacts from the floating point numbers leaking out.
I think doing === in R.equals would be nice. It will bring us closer to JS behavior, simplify the implementation and improve the performance of R.equals and _Set. Improving the performance of the later would in turn improve the performance of uniq and uniqWith.
If this is something that we want I'd be happy to do a PR.

Did you see the example I posted above?

Sorry, just being dense.  I've discussed the same example before, but just wasn't thinking of it here, even though I'm the one who brought up -0.  D'oh!

On the other hand, the fact that -0 is not equal to 0 seems to violate basic knowledge from math. It's really just implementation artifacts from the floating point numbers leaking out.

I've always had a hard time with this.
Perhaps we need a slightly more sophisticated version that checks for a -0 before checking primitive equality.  That wouldn't be hard to write, but is somehow less convincing.


TSPLIB parser

Environment

Qiskit Optimization version: 0.4.0
Python version: 3.9
Operating system: Mac OS

What is happening?
The TSPLIB parser in TSP application do not parse TSPLIB files from http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/index.html.
How can we reproduce the issue?
Download a tsp instance st70 and extract st70.tsp
from qiskit_optimization.applications.tsp import Tsp

read_path = "st70.tsp"
tsp = Tsp.parse_tsplib_format(read_path)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "[...]/py39/lib/python3.9/site-packages/qiskit_optimization/applications/tsp.py", line 165, in parse_tsplib_format
    raise QiskitOptimizationError(
qiskit_optimization.exceptions.QiskitOptimizationError: 'This supports only "TSP" type. Actual:  TSP\n'

What should happen?
Should parse the file and create the TSP object.
Any suggestions?
strip() is not an inplace method line 163 thus the bug line 165. Simple fixes like this would suffice
name = name.strip()
typ = typ.strip()

Then TSPLIB may have End Of File entry like in st70.tsp, see TSPLIB documentation. Adding an extra condition is sufficient.
elif line.startswith("EOF"):
    continue

Happy to be assigned on this issue.

Thank you for pointing out this issue. If you are happy to contribute to qiskit optimization, feel free to work on it.


Fix the content of resource-types.xml.

Fix the content of resource-types.xml.
It should have "yarn.resource-types" instead of "yarn.nodemanager.resource-plugins" inside.

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).
📝 Please visit https://cla.developers.google.com/ to sign.
Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.
What to do if you already signed the CLA
Individual signers
* It's possible we don't have your GitHub username or you're using a different email address on your commit. Check [your existing CLA data](https://cla.developers.google.com/clas) and verify that your [email is set on your git commits](https://help.github.com/articles/setting-your-email-in-git/).

Corporate signers
* Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to [go/cla#troubleshoot](http://go/cla#troubleshoot) ([Public version](https://opensource.google/docs/cla/#troubleshoot)).

* The email used to register you as an authorized contributor must be the email used for the Git commit. Check [your existing CLA data](https://cla.developers.google.com/clas) and verify that your [email is set on your git commits](https://help.github.com/articles/setting-your-email-in-git/).

* The email used to register you as an authorized contributor must also be [attached to your GitHub account](https://github.com/settings/emails).

ℹ Googlers: Go here for more info.

@googlebot I signed it!

/gcbrun

Thank you for the fix!