[FEATURE] [💎 $400 bounty] Integrate SearchApi as a WebSearchEngine and as a Tool

Feature Overview
Integrate SearchApi as a WebSearchEngine and as a Tool for function calling.
Requirements

Support Google Search via SearchApi
Add SearchApi tool documentation
Write at least 1 example of using SearchApi as a tool

Adhere to langchain4j contribution guidelines
Related work

https://github.com/langchain4j/langchain4j/pull/642 - ground work for adding search tools
https://github.com/langchain4j/langchain4j/pull/641
https://github.com/langchain4j/langchain4j/pull/676

Related issues:

https://github.com/langchain4j/langchain4j/issues/146

Existing SearchApi integrations

SearchApi LangChain Python
SearchApi LangChain JS
SearchApi Summarization example

Design considerations
SearchApi support not only Google Search, but 30+ other APIs such as Youtube Search, Transcripts, Bing Search (similar JSON response keys). In a statically typed language, it is probably better to have separate tools for separate APIs, but implementation could be flexible enough to extend SearchApi and easily adapt to other engines. All engines use the same HTTP GET request. The only difference is the parameters they accept. And how you want the response to be parsed.
Other notes

Ping us if you need more search credits when implementing.
Someone from the SearchApi Team will help review the functionality.

Bounty
There is a $400 bounty on this awarded by SearchApi to the community.
In the event of multiple attempts, the one that is going to be merged by the maintainers of the repository will pick the bounty.

💎 SearchApi is offering a $400 bounty for this issue
👉 Got a pull request resolving this? Claim the bounty by adding @algora-pbc /claim #1132 in the PR body and joining algora.io

@mark-searchapi  can i get this assigned?

Hello @mark-searchapi, I just started work on the SearchApi integration as a search engine and tool.
If anyone else ( and @abhishek818 ) has already started work on this please let me know as soon as you can.

Please adhere to these two implementations:
https://github.com/langchain4j/langchain4j/tree/main/web-search-engines

Yes of course.

If anyone else ( and ...) has already started work on this please let me know as soon as you can.

@jemiluv8
I've but I've not publicly added an "attempt", yet. Since you've also started with the issue, what difference does it make if you know or not?

@ayewo. Just wanted to know so I may halt my attempt since I just started.
I'm aborting my attempt now so as to not duplicate your efforts.
Happy hunting ...

Hi, was setting this up. Just confused as to how I am supposed to set up env like OPENAI_API_KEY.

Hi, was setting this up. Just confused as to how I am supposed to set up env like OPENAI_API_KEY.

I have raised  a draft PR with my current work. Need help with this to continue. Also pls assign this to me if satisfactory

@Ahmadkhan02
The comment by @mark-searchapi above clearly says:

Given the speed at which we're moving, we don't assign issues or "give" issues to anyone.

@abhishek818 why do you need open ai key? In any way, you can register with openai and get it there: https://github.com/langchain4j/langchain4j?tab=readme-ov-file#how-to-get-an-api-key

Hi @mark-searchapi
Looking at the docs for the Google Search API, it seems the API will return JSON to match the type of search query. In other words, the endpoint returns differently shaped JSON depending on whether the search is for a well-known fact, a local business, a product search etc.
The docs itself lists about 34 API examples which I pulled out into the list of JSON elements below:

"knowledge_graph"#"knowledge_graph_type": "Kp3 verticals"
"knowledge_graph"#"knowledge_graph_type": "Local nav"
"knowledge_graph"#"knowledge_graph_type": "Shopping entity"
"knowledge_graph"#"knowledge_graph_type": "Company"
"answer_box"#"type": "organic_result"
"sports_results"#"type": "team_sport"
"answer_box"#"type": "open_hours"
"answer_box"#"type": "population_graph"
"answer_box"#"type": "finance_results"
"answer_box"#"type": "currency_converter"
"answer_box"#"type": "local_time"
"weather_result"
"local_map"
"job_results"
"salary_estimates"
"shopping_ads"
"ads"
"local_ads"
"inline_shopping"
"from_sources_across_the_web"
"discussions_and_forums"
"related_questions"
"questions_and_answers"
"explore_brands"
"about_this_result"
"things_to_do"
"events"
"scholarly_articles"
"inline_videos"
"inline_tweets"
"inline_images"
"inline_recipes"
"top_stories"
"courses"

Since the task here is to "Integrate SearchApi as a WebSearchEngine and as a Tool for function calling", the common format between the two classes is the WebSearchResults object.
The WebSearchResults object in turn is hydrated from 3 things from a search:

"list of organic search results, information about the search, and pagination information"

Can you clarify which of the JSON elements are to scope and which ones are to be skipped for this issue?

Hey @mark-searchapi, looks like the num parameter for pagination is not working properly, the endpoint always returns 4 results if the parameter is present, could you take a look?

how many tests should one write for this?
and also is validation of parameters based on selected 'engine' required?

@mark-searchapi I opened a PR with the idea of adding new search api engines in the future using an interface that handles the requests and responses

Hey @mark-searchapi, looks like the num parameter for pagination is not working properly, the endpoint always returns 4 results if the parameter 5 is present, could you take a look? This happens when using the query "LangChain4j", which is being tested in WebSearchEngineIT

@zambrinf, you can try using the default num parameter and verify that the total amount of organic results is greater than 0. This issue happens with the "LangChain4j" query because Google counts different elements toward the num parameter. WebSearchEngine results only consider organic results, so elements like inline videos are not included as organic results.

@mark-searchapi I’ve already implemented it as you described in my PR. Please have a look at #1215.

Hi @mark-searchapi  and everybody,
It looks as a interesting challenge and I think that SearchAPI would be helpful for a lot of users.
I was who define the v1 of WebSearchEgine, I would like to clarify some premises to take into account in your PRs and provide additional info to what is already explained here
WebSearchEngine v1 was thought to retrieve organic web content where we can include (searchAPI and imagesAPI) directly from the web search engine (google, bing, yahoo) or using a wrapper solution that searches for the same thing. It does not include news, shopping, videos, maps, books, etc.) since they are another type of object. However it could be relevant to some users and could be included in the v2  that dynamically includes an object structure of other content categories.
Speaking only of the organic contents. If the integration supports URL scrapping and is capable of retrieving the complete content of the website in text, it should populate it in 'WebSearchOrganicResult.content'. If integration can only extract one snippet into its corresponding WebSearchOrganicResult.snippet field.
For now, the Tool and the WebSearchContentRetriever do not include the metadata field of either the WebSearchContentRetriever.metadata or the WebSearchOrganicResult.metadata (we did not see it necessary until now). However, it is planned in v2 to include it and each implementation can set which metadata is fetched.
You can check in this post the flow included in the v1 https://x.com/c_zela/status/1785522559791808650
I hope it gave a little more context and I welcome your ideas to include in v2
I'll put some comments on your PRs :)
Thank you!

/attempt #1132

/claim #1316

/claim #1132

@czelabueno I will then assign both PRs to you? 🙏

Hello gentlemen, #1216 has been reviewed and receives a green light from my side, #1215 still requires changes. Decision about the winner, who "takes it all" 😄 is yours 😄

@Heezer thanks a lot for the reviews! 🙏

Hey everyone,
We (SearchApi) don't know Java well, but we will review PRs from a logical standpoint this week. 😊

@SebastjanPrachovskij aha, finally 🙂. Looking forward to your feedback.
I believe I've addressed all of the issues—most of which were trivial code changes—that @Heezer pointed out during review in #1215.
Thanks for your review.

Hello everybody!
I'll have more time this week, so can I add a second review if I'm still on time @langchain4j ?
I understand that there 2 PRs open so far #1215 and #1216 right?
in advance, thank you for the great job you have done in your PRs @ayewo @zambrinf 🙌🏼

@czelabueno

I understand that there 2 PRs open so far https://github.com/langchain4j/langchain4j/pull/1215 and https://github.com/langchain4j/langchain4j/pull/1216 right?

Correct.
Nice that you are also offering to review the PRs, but no pressure if you can't make it.
Thanks!

@czelabueno please do if you'll have time 🙏

Hello @langchain4j,
TL;DR
We’ve gotten to a point where you need to take a final decision between #1215 and #1216 as repo owner.
Both PRs have been open for 6+ weeks now. Deciding now would relieve stress for both PR authors and more importantly: cut the down the time needed for subsequent reviews in half i.e. @czelabueno and anyone else would only need to make time to review 1 PR instead of 2.
 You don't have to leave individual comments.
When done, simply close the PR that will not be merged with the comment:
Closing #this-PR in favor of #that-PR due to <reasons>

Pull Request Assessment
The sponsor explicitly stated that “In the event of multiple attempts, the one that is going to be merged by the maintainers of the repository will pick the bounty”, but 6+ weeks after the first PR was opened there are still 2 competing PRs because the decision on which of the two PRs should be accepted has been deferred for too long. (I totally understand that this is due to the difficulty of getting a maintainer to make such an assessment.)
After looking at the current code in #1216, I have a few concerns about how the assessment itself will be made, as I will explain.
Development Feedback
@Heezer was the first to provide lots of feedback on both PRs (thanks once again for your time!) He dwelt primarily on adherence to coding conventions, java docs, test failures, linting etc.
In other words, his feedback mostly raised development concerns, it didn’t raise any concerns with respect to design i.e. things like how the library will be used by other developers and future maintainability.
Design Feedback
The PR feedback by @mark-searchapi on both PRs focused mainly on design: tight coupling, how the code will be used in practice etc.
The extensive feedback he left in #1216 had the side-effect of completely reshaping the design of that PR. Essentially what I’m saying is that the code at the start of the PR is markedly different from the code there now:

PR
Initial PR
Current PR (after Design Feedback)

#1215
491a655
latest

#1216
1080630
latest

If you look at commit 1080630 above, you'll see that a core part of the design uses the Factory pattern, with each search engine requiring its own enum and 3 supporting classes. This made the library very tightly coupled to the search engines it supports.
In this design, the addition of a youtube, bing, baidu search engine would require adding enums to the appropriate class, updating the Factory and of course adding 3 supporting classes for each search engine. In other words, a total of 9 new classes will need to be added to support 3 search engines.
In spite of the absence of Java expertise[^1][^2] among folks from the SearchApi team, @mark-searchapi was still able to point out the original design as being too restrictive to support multiple search engines. It also meant they would have to update the Java library each time there is an upcoming search engine.
More Similar than Different
With the expert guidance of @mark-searchapi, his code has eventually converged on the same design I started with when I opened my PR.
Both PRs are now closer in design than when they were first opened. It’s now a really close call. It can go either way.
As things stand, the decision on which PR gets merged into the langchain4j repo has changed from technical merit (objective) to feel (subjective).
By feel I mean it will now come down to which coding style is liked best rather than the original differentiator of technical merit—which PR has the least coupling that can support multiple search engines and yet to be announced search engines.
Of course I would be stating the obvious if I said I’m now at a disadvantage here: I figured out the design for minimal coupling on my own but the other PR author didn’t have to do as much work: his current design was handed down to him during design review.
Please Pick a Winner
Now that it is a close call, there’s no point in kicking the can down the road any further. The maintainers need to pick a winner.
Rather use the latest commit, I think the only equitable thing at this point is to use a point-in-time commit (prior to design review) to assess both PRs.
I think this will save everyone involved any further anguish or sense of false hope …
The sponsors can then further shape the code of the accepted PR prior to it being merged.
(If you got to the end, thanks for attending my TED talk :))
[^1]: We (SearchApi) don't know Java well, but we will review PRs from a logical standpoint this week.
[^2]: The example above is crafted with the help of ChatGPT as I don't know enough Java myself.

Hi @ayewo, since I have limited time and other priorities, I will be able to provide more information only next week. Thank you for your understanding.

Hi @ayewo and @zambrinf ! I've added some reviews to https://github.com/langchain4j/langchain4j/pull/1216 and https://github.com/langchain4j/langchain4j/pull/1215 and they still require changes.
You've done a good job so far. We're pretty close 👍

Hey @czelabueno,
Thanks for you review🙏!
@langchain4j wont be able to decide on which PR he will be accepting until next week, so I'll wait till then before working on the changes you requested.

Hi @mark-searchapi and @SebastjanPrachovskij, please let us know if you have any preference in terms of completeness/supported functionality

So, after reviewing both PRs, it is indeed a very close call.
In my view, both authors did a good job and deserve the bounty.
So, I would either split the bounty 50/50 or make a choice using some online randomizer.
What do you think?

@langchain4j
In the rules defined by the sponsors, they made it clear that you will be the one to decide the winner if there are multiple PRs:

In the event of multiple attempts, the one that is going to be merged by the maintainers of the repository will pick the bounty.

But you are now deferring to the sponsors 🤷♂?
Anyway, suggesting a 50/50 split wouldn't be fair to me AT ALL because the 2 PRs were not even close when they were opened in June. That they are close now is a much recent event.

@langchain4j I'm okay to split if my PR is chosen

🎉🎈 @zambrinf has been awarded $300! 🎈🎊

Share on socials
Give feedback

🎉🎈 @ayewo has been awarded $200! 🎈🎊

Share on socials
Give feedback

@mark-searchapi could you please provide some free credits for us so we can continue testing the integration on our CI?
Right now I am getting this error:
java.lang.RuntimeException: status code: 429; body: {
  "error": "You have used all of the searches for the month. Please upgrade your plan on SearchApi.io."
}

Thanks!

@mark-searchapi thanks a lot!


can't click next page in REViewer2

Hi,
Really glad that REViewer2 exists and eager to use it to go through the svg files I created with Reviewer!
While in the folder with only svg files and running REViewer like so: python3 -m reviewer2 -t REViewerPlots_EHv402_150bp_notes.xls, I can open and judge the first svg file in that folder.
However, I can't click on the 'next' file and also when I go back to the table in the browser (firefox, chrome, safari, doesn't matter which one) and click on the next file it opens the first file.
Any idea why this is? I suspect I need to install some more software, because I also had to install pip (I always work with conda), xlwt and xlrd to install REViewer and get it to work up til this point.
My colleague does get it to work fine on her macbook, so it is definitely something on my end that is missing/not working.
Any hints are very welcome!
Kind regards,
Joke van Vugt

Hi Joke, thanks for trying out reviewer2.
I'm having trouble understanding the cause of this issue. In the first screenshot, where it shows

the "page #1 of 1" text tells me only 1 image was found by that reviewer2 server. However, the "last=407" in the url tells me the server did at one point see 407 images.
Then in the 2nd screenshot

tells me this server most recently got requests for page #2, which shouldn't exist.

To rule out issues due to an older version of reviewer2, could you please try uninstalling reviewer2 and then installing the latest version with
python3 -m pip install --upgrade reviewer2

Hi,
Thanks for your help on this.
I uninstalled REViewer2, then updated conda (conda update conda), then installed flask and configargparse packages with conda (conda install -c anaconda flask, conda install -c anaconda configargparse), then installed REViewer2:
$ python3 -m pip install --upgrade reviewer2
Collecting reviewer2
Using cached reviewer2-0.9.9-py3-none-any.whl (23 kB)
Requirement already satisfied, skipping upgrade: xlwt in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (1.2.0)
Requirement already satisfied, skipping upgrade: flask in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (1.1.2)
Requirement already satisfied, skipping upgrade: configargparse in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (1.4)
Requirement already satisfied, skipping upgrade: xlrd in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (1.2.0)
Requirement already satisfied, skipping upgrade: pandas in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (1.1.5)
Requirement already satisfied, skipping upgrade: gunicorn in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (20.1.0)
Requirement already satisfied, skipping upgrade: flask-cors in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (3.0.10)
Requirement already satisfied, skipping upgrade: jinja2 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (2.11.3)
Requirement already satisfied, skipping upgrade: openpyxl in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (3.0.7)
Requirement already satisfied, skipping upgrade: wcmatch in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from reviewer2) (8.1.2)
Requirement already satisfied, skipping upgrade: click>=5.1 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from flask->reviewer2) (7.1.2)
Requirement already satisfied, skipping upgrade: Werkzeug>=0.15 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from flask->reviewer2) (1.0.1)
Requirement already satisfied, skipping upgrade: itsdangerous>=0.24 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from flask->reviewer2) (1.1.0)
Requirement already satisfied, skipping upgrade: numpy>=1.15.4 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from pandas->reviewer2) (1.19.2)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from pandas->reviewer2) (2.8.1)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from pandas->reviewer2) (2021.1)
Requirement already satisfied, skipping upgrade: setuptools>=3.0 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from gunicorn->reviewer2) (52.0.0.post20210125)
Requirement already satisfied, skipping upgrade: Six in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from flask-cors->reviewer2) (1.15.0)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from jinja2->reviewer2) (1.1.1)
Requirement already satisfied, skipping upgrade: et-xmlfile in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from openpyxl->reviewer2) (1.1.0)
Requirement already satisfied, skipping upgrade: bracex>=2.1.1 in /Users/jvugt2/Documents/ALS_Data/Software_and_Databases/miniconda3/lib/python3.6/site-packages (from wcmatch->reviewer2) (2.1.1)
Installing collected packages: reviewer2
Successfully installed reviewer2-0.9.9
When opening REViewer2 I see this:
$ python3 -m reviewer2 -t /Users/jvugt2/Documents/ALS_Data/Articles/PM_disease_STRs/Data/REViewerPlots_EHv402_April2021_only_LessConsistentThanNone_150bp_notes.xls
Looking for svg, png, jpeg, jpg, gif, webp, reviewer2_content.html, and reviewer2_metadata.json files in /Users/jvugt2/Documents/ALS_Data/Articles/PM_disease_STRs/Data/REViewerPlots_EHv402_April2021_only_LessConsistentThanNone_150bp
Found 845 svg files in 845 sub-directories
Will save form responses to /Users/jvugt2/Documents/ALS_Data/Articles/PM_disease_STRs/Data/REViewerPlots_EHv402_April2021_only_LessConsistentThanNone_150bp_notes.xls  (columns: Verdict, Confidence, Notes)

Running on http://<IP_ADDRESS>:8080/ (Press CTRL+C to quit)
<IP_ADDRESS> - - [07/May/2021 09:38:51] "GET / HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:39:01] "GET /page?last=845&i=1 HTTP/1.1" 200 -
Sending /Users/jvugt2/Documents/ALS_Data/Articles/PM_disease_STRs/Data/REViewerPlots_EHv402_April2021_only_LessConsistentThanNone_150bp LP6008093-DNA_A05_C11orf80m1.svg
<IP_ADDRESS> - - [07/May/2021 09:39:01] "GET /LP6008093-DNA_A05_C11orf80m1.svg HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:39:16] "GET / HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:39:34] "GET /page?last=845&i=1 HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:39:37] "POST /save HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:40:23] "POST /save HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:40:26] "POST /save HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:40:32] "GET / HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:40:42] "GET /page?last=845&i=2 HTTP/1.1" 200 -
<IP_ADDRESS> - - [07/May/2021 09:40:52] "GET / HTTP/1.1" 200 -

In Chrome and/or Firefox I opened the http link and I see the same as before:
First the table appears with all svg files in there, see screenshot_1.
When I click on the first file it opens and I can fill in the verdict and notes, see screenshot_2.
But I cannot click on ‘prev’ or ‘next’, I can go back to the table when clicking on ‘home’ and I see the verdict in the table.
When I click on the next file in the table (or any other file in the table) it opens the first file (again).
Any thoughts on how to tackle this?
Attached you’ll also find a txt file with the conda packages that are installed in case it helps in any way?
Thanks!
Joke
From: bw2 @.>
Reply-To: broadinstitute/reviewer2 @.>
Date: Friday, 7 May 2021 at 05:51
To: broadinstitute/reviewer2 @.>
Cc: "Vugt-2, J.F.A. van (Joke)" @.>, Author @.***>
Subject: Re: [broadinstitute/reviewer2] can't click next page in REViewer2 (#1)
Hi Joke, thanks for trying out reviewer2.
I'm having trouble understanding the cause of this issue. In the first screenshot, where it shows
[Image removed by sender. image]https://user-images.githubusercontent.com/6240170/117394524-50f52c00-aec4-11eb-87e2-8cb816647184.png

the "page #1https://github.com/broadinstitute/reviewer2/issues/1 of 1" text tells me only 1 image was found by that reviewer2 server. However, the "last=407" in the url tells me the server did at one point see 407 images.
Then in the 2nd screenshot
[Image removed by sender. image]https://user-images.githubusercontent.com/6240170/117394933-150e9680-aec5-11eb-9f4b-067fdfc5c687.png
tells me this server most recently got requests for page #2, which shouldn't exist.

To rule out issues due to an older version of reviewer2, could you please try uninstalling reviewer2 and then installing the latest version with
python3 -m pip install --upgrade reviewer2
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/broadinstitute/reviewer2/issues/1#issuecomment-834043569, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AII336V27STKUAPENJT5NYTTMNPUNANCNFSM44HA3N4A.

De informatie opgenomen in dit bericht kan vertrouwelijk zijn en is
uitsluitend bestemd voor de geadresseerde. Indien u dit bericht onterecht
ontvangt, wordt u verzocht de inhoud niet te gebruiken en de afzender direct
te informeren door het bericht te retourneren. Het Universitair Medisch
Centrum Utrecht is een publiekrechtelijke rechtspersoon in de zin van de W.H.W.
(Wet Hoger Onderwijs en Wetenschappelijk Onderzoek) en staat geregistreerd bij
de Kamer van Koophandel voor Midden-Nederland onder nr. 30244197.
Denk s.v.p aan het milieu voor u deze e-mail afdrukt.

This message may contain confidential information and is intended exclusively
for the addressee. If you receive this message unintentionally, please do not
use the contents but notify the sender immediately by return e-mail. University
Medical Center Utrecht is a legal person by public law and is registered at
the Chamber of Commerce for Midden-Nederland under no. 30244197.
Please consider the environment before printing this e-mail.
This file may be used to create an environment using:
$ conda create --name  --file
platform: osx-64
@EXPLICIT
https://conda.anaconda.org/conda-forge/osx-64/coreutils-8.31-h0b31af3_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/blas-1.0-mkl.conda
https://repo.anaconda.com/pkgs/main/osx-64/bzip2-1.0.8-h1de35cc_0.conda
https://conda.anaconda.org/anaconda/osx-64/ca-certificates-2020.10.14-0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/intel-openmp-2021.2.0-hecd8cb5_564.conda
https://repo.continuum.io/pkgs/free/osx-64/krb5-1.13.2-0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/libcxx-10.0.0-1.conda
https://repo.anaconda.com/pkgs/main/osx-64/libsodium-1.0.18-h1de35cc_0.conda
https://repo.continuum.io/pkgs/free/osx-64/ncurses-5.9-10.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/pandoc-2.12-hecd8cb5_0.conda
https://repo.continuum.io/pkgs/free/osx-64/readline-6.2-2.tar.bz2
https://repo.continuum.io/pkgs/free/osx-64/sqlite-3.13.0-0.tar.bz2
https://repo.continuum.io/pkgs/free/osx-64/tk-8.5.18-0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/xz-5.2.5-h1de35cc_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/yaml-0.2.5-haf1e3a3_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/zlib-1.2.11-h1de35cc_3.conda
https://repo.anaconda.com/pkgs/main/osx-64/libffi-3.3-hb1e8313_2.conda
https://repo.anaconda.com/pkgs/main/osx-64/libpng-1.6.37-ha441bb4_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/mkl-2019.4-233.conda
https://conda.anaconda.org/anaconda/osx-64/openssl-1.0.2u-h1de35cc_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/zeromq-4.3.4-h23ab428_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/freetype-2.10.4-ha233b18_0.conda
https://repo.continuum.io/pkgs/free/osx-64/libssh2-1.8.0-0.tar.bz2
https://repo.continuum.io/pkgs/free/osx-64/python-3.6.1-2.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/appnope-0.1.2-py36hecd8cb5_1001.conda
https://repo.anaconda.com/pkgs/main/noarch/asn1crypto-1.4.0-py_0.conda
https://repo.anaconda.com/pkgs/main/noarch/attrs-20.3.0-pyhd3eb1b0_0.conda
https://conda.anaconda.org/anaconda/osx-64/certifi-2020.6.20-py36_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/chardet-4.0.0-py36hecd8cb5_1003.conda
https://conda.anaconda.org/anaconda/noarch/click-7.1.2-py_0.tar.bz2
https://conda.anaconda.org/anaconda/noarch/configargparse-1.2.3-py_0.tar.bz2
https://repo.continuum.io/pkgs/free/osx-64/curl-7.54.1-0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/decorator-5.0.6-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/noarch/defusedxml-0.7.1-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/entrypoints-0.3-py36_0.conda
https://repo.anaconda.com/pkgs/main/noarch/idna-2.10-pyhd3eb1b0_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/ipython_genutils-0.2.0-pyhd3eb1b0_1.conda
https://conda.anaconda.org/anaconda/osx-64/itsdangerous-1.1.0-py36_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/kiwisolver-1.3.1-py36h23ab428_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/markupsafe-1.1.1-py36h1de35cc_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/mistune-0.8.4-py36h1de35cc_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/pandocfilters-1.4.3-py36hecd8cb5_1.conda
https://repo.anaconda.com/pkgs/main/noarch/pickleshare-0.7.5-pyhd3eb1b0_1003.conda
https://repo.anaconda.com/pkgs/main/noarch/prometheus_client-0.10.1-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/noarch/ptyprocess-0.7.0-pyhd3eb1b0_2.conda
https://repo.anaconda.com/pkgs/main/osx-64/pycosat-0.6.3-py36h9ed2024_0.conda
https://repo.anaconda.com/pkgs/main/noarch/pycparser-2.20-py_2.conda
https://repo.anaconda.com/pkgs/main/noarch/pyparsing-2.4.7-pyhd3eb1b0_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/pyrsistent-0.17.3-py36haf1e3a3_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/pysocks-1.7.1-py36hecd8cb5_0.conda
https://conda.anaconda.org/conda-forge/osx-64/python_abi-3.6-1_cp36m.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/pyzmq-20.0.0-py36h23ab428_1.conda
https://repo.anaconda.com/pkgs/main/osx-64/ruamel_yaml-0.15.100-py36h9ed2024_0.conda
https://repo.anaconda.com/pkgs/main/noarch/send2trash-1.5.0-pyhd3eb1b0_1.conda
https://repo.anaconda.com/pkgs/main/osx-64/simplegeneric-0.8.1-py36_2.conda
https://repo.anaconda.com/pkgs/main/osx-64/six-1.15.0-py36hecd8cb5_0.conda
https://repo.anaconda.com/pkgs/main/noarch/testpath-0.4.4-pyhd3eb1b0_0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/tornado-6.1-py36h9ed2024_0.conda
https://repo.anaconda.com/pkgs/main/noarch/tqdm-4.59.0-pyhd3eb1b0_1.conda
https://repo.anaconda.com/pkgs/main/noarch/typing_extensions-<IP_ADDRESS>-pyha847dfd_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/wcwidth-0.2.5-py_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/webencodings-0.5.1-py36_1.conda
https://conda.anaconda.org/anaconda/noarch/werkzeug-1.0.1-py_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/wheel-0.36.2-pyhd3eb1b0_0.conda
https://conda.anaconda.org/anaconda/osx-64/xlrd-1.2.0-py36_0.tar.bz2
https://conda.anaconda.org/anaconda/osx-64/xlwt-1.2.0-py36h5ad1178_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/zipp-3.4.1-pyhd3eb1b0_0.conda
https://conda.anaconda.org/bioconda/osx-64/bcftools-1.7-0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/cffi-1.14.5-py36h2125817_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/conda-package-handling-1.7.3-py36h9ed2024_1.conda
https://repo.anaconda.com/pkgs/main/osx-64/cycler-0.10.0-py36hecd8cb5_0.conda
https://conda.anaconda.org/bioconda/osx-64/htslib-1.7-0.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/importlib-metadata-3.10.0-py36hecd8cb5_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/mkl-service-2.3.0-py36h9ed2024_0.conda
https://repo.anaconda.com/pkgs/main/noarch/packaging-20.9-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/noarch/pexpect-4.8.0-pyhd3eb1b0_3.conda
https://repo.anaconda.com/pkgs/main/noarch/python-dateutil-2.8.1-pyhd3eb1b0_0.tar.bz2
https://conda.anaconda.org/bioconda/osx-64/samtools-1.7-2.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/setuptools-52.0.0-py36hecd8cb5_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/terminado-0.9.4-py36hecd8cb5_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/traitlets-4.3.3-py36_0.conda
https://repo.anaconda.com/pkgs/main/noarch/bleach-3.3.0-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/brotlipy-0.7.0-py36h9ed2024_1003.conda
https://repo.continuum.io/pkgs/free/osx-64/cryptography-1.8.1-py36_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/importlib_metadata-3.10.0-hd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/noarch/jinja2-2.11.3-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/jupyter_core-4.7.1-py36hecd8cb5_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/numpy-base-1.19.2-py36hcfb5961_0.conda
https://conda.anaconda.org/anaconda/osx-64/pip-20.2.4-py36_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/pygments-2.8.1-pyhd3eb1b0_0.conda
https://conda.anaconda.org/bioconda/osx-64/pysam-0.14.1-py36_htslib1.7_0.tar.bz2
https://conda.anaconda.org/anaconda/noarch/flask-1.1.2-py_0.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/jsonschema-3.2.0-py_2.conda
https://repo.anaconda.com/pkgs/main/noarch/jupyter_client-6.1.12-pyhd3eb1b0_0.conda
https://conda.anaconda.org/conda-forge/noarch/prompt_toolkit-1.0.15-py_1.tar.bz2
https://repo.continuum.io/pkgs/free/osx-64/pyopenssl-17.0.0-py36_0.tar.bz2
https://conda.anaconda.org/conda-forge/osx-64/ipython-5.8.0-py36_1.tar.bz2
https://repo.anaconda.com/pkgs/main/noarch/nbformat-5.1.3-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/noarch/urllib3-1.26.4-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/ipykernel-5.3.4-py36h5ca1d4c_0.conda
https://repo.anaconda.com/pkgs/main/noarch/nbconvert-5.5.0-py_0.conda
https://repo.anaconda.com/pkgs/main/noarch/requests-2.25.1-pyhd3eb1b0_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/conda-4.10.1-py36hecd8cb5_1.conda
https://conda.anaconda.org/conda-forge/osx-64/notebook-5.7.10-py36h9f0ad1d_0.tar.bz2
https://conda.anaconda.org/conda-forge/osx-64/matplotlib-3.1.1-py36_1.tar.bz2
https://conda.anaconda.org/conda-forge/osx-64/matplotlib-base-3.1.1-py36h3a684a6_1.tar.bz2
https://repo.anaconda.com/pkgs/main/osx-64/mkl_fft-1.3.0-py36ha059aab_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/mkl_random-1.1.1-py36h959d312_0.conda
https://repo.anaconda.com/pkgs/main/osx-64/numpy-1.19.2-py36h456fd55_0.conda


[New OXS backend] Window maximized on double click

When I double click anywhere in the window, it is maximized (or restored)
<PackageReference Include="Avalonia" Version="0.6.2-build6362-beta" />
<PackageReference Include="Avalonia.Desktop" Version="0.6.2-build6362-beta" />
<PackageReference Include="Avalonia.AvaloniaEdit" Version="0.2.0-build204-alpha" />
<PackageReference Include="Avalonia.Native" Version="0.7.0-build4201810097" />

Update to latest nuget (the wont be ordered correctly due to screw up with build numbers) see one dated 12 october

This is fixed

Where can I find the list of available versions ?

Yep thx


NEXT-8065 - Change tmp filename of sitemap

1. Why is this change necessary?
Currently when the sitemap is generated, it writes a temporary file with the name sitemap-product-1.xml.gz to the temp location that is configured within php ( sys_get_temp_dir()). However in case of a shared server with multiple server users and Shopware installations the temporary file be owned by just one server user. Because the temporary filename is always the same, the other installations cannot generate the sitemap because they get a permission denied error when trying to write to the temporary file.
2. What does this change do, exactly?
The change adds the saleschannelid to the temporary filename and it is unlikely that there are installations with the same saleschannelid.
3. Describe each step to reproduce the issue or behaviour.
Install multiple shopware installations under different server users on the same server. Then try to generate the sitemap on each installation.
4. Please link to the relevant issues (if any).
https://issues.shopware.com/issues/NEXT-8065
5. Checklist

[x] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[x] I have written or adjusted the documentation according to my changes
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-9963
Please use this issue to track the state of your pull request.

Thanks for your contribution 💙 @alexbaat

Hi @bneumann97 , I see the issue is closed with version set to <IP_ADDRESS> but I can't find it the <IP_ADDRESS> release. Is this postponed?

@alexbaat the ticket was assigned wrongly. It will be part of the release after the <IP_ADDRESS>


'Circular structure in "getInitialProps" result of page' error after npm install

After a recent npm i in the repo, when I ran npm run dev it always gave me the error:
Error: Circular structure in "getInitialProps" result of page "/x". https://err.sh/vercel/next.js/circular-structure
    at Function.getInlineScriptSource (...\deno_website2\.next\server\pages\_document.js:717:15)
    at NextScript.render (...\deno_website2\.next\server\pages\_document.js:774:28)
    ...

I looked around, and found these two issues:
https://github.com/vercel/next.js/issues/17854
https://github.com/developit/nextjs-preact-demo/issues/25
Changing next version to 9.5.3 as mentioned in a comment, seems to resolve this problem for me.
Dont know, if thats the solution, to be taken.

That sounds good to me. I think this issue will be addressed in a few weeks.


Retry Failed Test with Protractor Flake

Hi @jan-molak, Greetings!
Its being a long since i have started working with Serenity JS, which really helped us achieve the best results into test automation. Thanks again.
Now, i am trying my hands with Protractor Flake with serenity framework. I saw many posts.. even here with the configurations to retest the failed scenarios.
When i run the test with protractor flake  it do gives me the failed scenarios. Now if a feature which has two scenarios then also it runs the whole feature instead of the failed test scenario. Any suggestions please.
Thanks,
Abhineet Sharma

Hi @AbhineetSharmax, welcome back!
Have you looked at the new @serenity-js/mocha module? It integrates with Mocha 8 which supports re-running failed scenarios.
Let me know if that solves your issue!
Thanks,
Jan

@jan-molak , Thank you for your help. We somehow managed to run flakes and its running the failed scenarios in a sequence.


Bakery-specific datasets

One question around the interaction between datasets and bakeries: will a bakery "owner" be able to opt in / out of specific datasets?
If AI for Earth wanted to run a bakery, would we be expected to process and host every dataset that comes along? I ask because a.) we have a focus on sustainability, and b.) we need some way to forecast budget.
I suppose the hypothetical NCAR bakery makes it clear that certain recipes would require certain bakeries.

Good question.
I imagine using prefect "labels" for this: https://docs.prefect.io/orchestration/agents/overview.html#labels
The recipes will get labeled (e.g. bakery:ldeo-gcp-us-central1 or something like that). This will route recipes to bakeries. When we review and add new recipes, we can make sure that the bakery owner is okay with supporting that particular recipe.
However, this does require the bakery owner (e.g. AI for Earth) to trust the pangeo-forge github org and recipe review process sufficiently.


Improve is_valid_day_of_month function

use moment lib to test validity

@gsvr please review

Tests?

@gsvr second pass review please

@gsvr second pass review please.  For now, I didn't do much with the function's tests apart from changing test descriptions and adding simple comments... should I open a separate issue to improve our testing of this function?  Then when we get to it, we can discuss how to effectively approach it...

@gsvr thorough testing! 👍


⚠ Bofrost.de has degraded performance

In 068ccad, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1364 ms

Resolved: Bofrost.de performance has improved in c91e8f9 after 25 minutes.


[Gardening] transferable_throws_oom_test crashing after upgrade from Ubuntu 16 -> 18 (Bionic)

From Dart infra chat: @wheese: "their attempt to transfer a 1 TB object is causing an OOM crash in the test process, instead of being caught by the Dart VM before it causes a crash, and throwing a Dart exception instead".
I've approved the failure while we wait for a fix.

cc @aam


feat: add support for tuple type

add support for tuple type.
var tuple: [string, number]

Could you please add a generic type and an array to the tuple ?

@vicb I added test cases for that. is it ok?

Merged. Thanks for the PR.


ri URL has changed

Sentry Issue: COURTLISTENER-7EK
HTTPError: 404 Client Error: Not Found for url: https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOrders/Forms/20232024.aspx
(2 additional frame(s) were not displayed)
...
  File "cl/scrapers/management/commands/cl_scrape_opinions.py", line 387, in handle
    self.parse_and_scrape_site(mod, options)
  File "cl/scrapers/management/commands/cl_scrape_opinions.py", line 350, in parse_and_scrape_site
    site = mod.Site().parse()

We need to update the scraper to the new endpoint
https://www.courts.ri.gov/Pages/ood.aspx?k=(RIJCourt:'Supreme') AND (ContentType:'RIJOpinion')
Explaning the changes
We have 2 ri scrapers, ri_u.py and ri_p.py. They get these URLs, where the years used to change depending on the court terms. From a comment on the script: "This court hears things from mid-September to end of June. This defines the "term" for that year, which triggers the website updates.", The term ends on :term_end = datetime(this_year, 9, 15)
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOpinions/Forms/20232024.aspx
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOrders/Forms/20232024.aspx
So, we may think that the URLs were updated before the term ended, but they do not exist neither for future nor past terms
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOpinions/Forms/20242025.aspx
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOrders/Forms/20242025.aspx
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOpinions/Forms/20222023.aspx
https://www.courts.ri.gov/Courts/SupremeCourt/SupremeOrders/Forms/20222023.aspx

isnt that wonderful

Ugh - I took a look at this and have a few thoughts.

this should be an easy rewrite - they still expose a JSON endpoint
the U/P distinction should be merged into one scraper.  the current website provides published opinions and published orders that I think should be collected
The miscellaneous court orders etc can be ignored imho.
The endpoint is pliable to allow for scraping 100s of opinions at a time and is quite responsive.

the only weirdness is the schemas.microsoft.com/sharepoint xml that is required for the parameters.

partially fixed.  but appears to stop early.

Closing this because the scraper is back up


Add fast_beam_search_with_nbest_rescoring_LG

hello, Did fast_beam_search_with_nbest_rescoring_LG do some tests? How about WER?

Currently for phone based training and decoding.
| decoding method | dev & test | checkpoint  |
  |-- | -- | --  |
  | fast_beam_search_with_LG | 13.15 & 13.46 | epoch 30 avg 7  |
  | fast_beam_search_with_nbest_rescoring_LG | 12.90 & 13.19 | epoch 30 avg 7  |


terms for causes of premature death

Starting to work on osteosarcoma papers for KidsFirst project and I have a couple of questions:

Should we create premature death child terms for specific causes of premature death? There are many annotations for 'tumor-free survival' time. We could create a new term for premature death due to tumor burden.
Should we create a term for abnormal tumor-free survival time? In J:326989 the addition of another mutation to an osteosarcoma model increases tumor-free survival time. Currently I've annotated this to abnormal survival but I think it would be helpful to have an increased tumor-free survival time (and decreased) to capture this.

suggest we add terms for
abnormal tumor-free survival time
def: any anomaly in the span of life of an organism or population of organisms spends free from the signs, symptoms, or occurrence of neoplasms
ref: https://www.cancer.gov/publications/dictionaries/cancer-terms/def/disease-free-survival
child of abnormal survival
increased tumor-free survival time
def: increase in the length of time an organism or population of organisms spends free from the signs, symptoms, or occurrence of neoplasms
child of abnormal tumor-free survival time
decreased tumor-free survival time
def: decrease in the length of time an organism or population of organisms spends free from the signs, symptoms, or occurrence of neoplasms
child of abnormal tumor-free survival time and premature death

MP:0014235
MP:0014236
MP:0014237


⚠ Processors has degraded performance

In 5117d22, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 85 ms

Resolved: Processors performance has improved in 86d238e.


[1.21] Issue Title

DO NOT REPORT AN ISSUE IF YOU HAVEN'T TESTED THE ISSUE WITH THE LATEST VERSION IN CURSEFORGE
**Industrial Foregoing Version:1.21-3.6.15
**Titanium Version:4.0.21
Crashlog If Applicable (Upload to pastebin/gist):
seeder does not plant(only plant berry,but potatoes and carrots and wheat not plant)

this does not plant,only plant berry,fix it quickly!!!

It works for me, using dirt and Hydroponic beds


Latex doesn't render in header

Steps to reproduce the problem

Go to "https://learn.qiskit.org/course/ch-labs/lab-3-accuracy-of-quantum-phase-estimation"
Scroll down to section 1
See error in the header

What is the current behavior?
the text contains "$2^{-2}$" which should be rendered as a math equation instead of standard text.
What is the expected behavior?
Render the text as latex, which is correctly done in the original textbook version of this page.
Screenshots (optional but very helpful)
current behavior:

If applicable, please add screenshots or video captures to help explain the problem.
Environment (optional but very helpful)

Device: desktop and mobile
OS: macos and android
Browser firefox and brave

this bug is probably on all devices, OSs and browsers
Additional context (optional)
Add any other context that could help debug the problem.

Possibly related to https://github.com/Qiskit/platypus/issues/980


Updating some old code to new ProcedureKit

This is more of a how-to question than an issue.
I've been working on updating an older codebase to Swift 3, and concurrently from a pretty old version of Operations to beta 5 of ProcedureKit. For the most part, this hasn't been too hard, but I admit that I'm a bit stumped when it comes to non-trivial Conditions, which work very differently now. In particular, I have one that looks essentially like this:
struct IsSyncedCondition: OperationCondition {
  let name = "IsSynced"
  let isMutuallyExclusive = false

  func dependencyForOperation(_ operation: Operation) -> NSOperation? {
    var dependency: NSOperation?
    if /* … the operation's state isn't synced up … */ {
      dependency = GetSyncedOperation(/* … */)
    }
    return dependency
  }

  func evaluateForOperation(operation: Operation, completion: OperationConditionResult -> Void) {
    if /* … the operation's state isn't synced up … */ {
      completion(.Satisfied)
    }
    else {
      completion(.Failed)
    }
  }
}

How does the framework currently handle what dependencyForOperation(_:) used to do?

Hi @sixten - this is a really good question.
Essentially, conditions can still have dependencies, because they are just Procedure subclasses. It's just that you would add your dependency to the condition like any normal operation. You could for example do this as part of its initialiser? For example:
class IsSyncedCondition: Condition {

    override init() {
        super.init()
        if /* … the operation's state isn't synced up … */ {
            add(dependency: GetSyncedOperation(/* … */))
        }
    }

    func evaluate(procedure: Procedure, completion: @escaping (ConditionResult) -> Void) {
        // etc - as before
    }
}

Also, so that you're aware - we are thinking about further changes to Condition to support a richer programmer model for conditional procedures, see the discussions in #584

Yeah, for some reason, the notion of having the condition configure its dependencies in its initializer hadn't really occurred to me. I'd been sort of stuck on the fact that evaluate(_:_) was too late, because the condition was running by then…
There is one important difference here from before: dependencyForOperation(_:) was given the target operation—the one to which the condition would be attached—as a parameter. What our existing code does is to examine that operation to determine whether things are synced up before it runs. I suppose I could define an initializer for my condition that took the target operation as a parameter, but that seems kind of smelly. On the other hand, I should also be able to pull out the state that the target operation has that the condition needs to know, and pass that to the initializer. That's also somewhat cleaner than the old approach, since the dependency only flows in one direction.

@sixten - yes, exactly. I think ultimately, the goal should be to reduce the key elements of the business logic into the smallest possible unit/type - which you then inject into procedures/conditions as needed. I think the old/original API made this less obvious.

Yeah, the new approach makes sense. It's just a little disorienting jumping straight from the old idiom to the new. (And at the same time as fixing hundreds of other migration issues…)
Thanks!


Update UI and add view feature

What's change:

Implement new command: view
Improve UI/UX of the SOCket

Functional
Logic.java, LogicManager.java

Add getViewedPerson(): To get a person that user wants to view their detail information
Add setViewedPerson(): Set the person that the user wants to view

ViewCommand.java, ViewCommandParser.java

Similar to delete; parse and identify the selected index and update the viewPerson attribute in model class

AddressBookParser.java

Add case for view command

Model.java, ModalManager.java

Add getViewedPerson(): To get a person that user wants to view their detail information
Add updateViewedPerson(): update the viewPerson attribute such as only this particular person is in the list

Add test for ViewCommand and ViewCommandParser
UI/UX
Update MainWindow:

Shift the command input and result display to the bottom of the page
Split the initial contact table into split table:

Show brief information of the contact such as index, name, github username(display "Not available" when its empty) and tags
Show all the information about the person
Add personDetailCard, personDetailPane: to display the detailed information of the person that is selected

Minor change on the CSS

The codecov failes is because most of the misses came from GUI and it is very hard to test.


[PE-D][Tester E] 2 different people are recognized as the same person when adding a new job

Even though the recipient and the sender are clearly different from the addressbook, since they share the same id they are treated as the same person.
It might not be a good idea to use substrings of input values as uniquely identifying features of model objects. Perhaps hashing the data would make such collisions rarer/ not easy to replicate.

Labels: type.FunctionalityBug severity.High
original: vaidyanaath/ped#5

fix by #250


import mammoth error

Describe the bug
import mammoth from "mammoth" 报错
Reproduction
https://github.com/dazuo-art/viteAndMammoth/tree/master
System Info
window10,npm@8.1.2,vite@2.6.14(2.9.12),chrome

Used Package Manager
npm
Logs
sax.js:221 Uncaught TypeError: Cannot read properties of undefined (reading 'prototype')
    at sax.js:221:46
    at node_modules/sax/lib/sax.js (sax.js:1574:1)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at node_modules/mammoth/lib/xml/reader.js (reader.js:2:11)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at node_modules/mammoth/lib/xml/index.js (index.js:6:22)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at node_modules/mammoth/lib/docx/office-xml-reader.js (office-xml-reader.js:4:11)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at node_modules/mammoth/lib/docx/docx-reader.js (docx-reader.js:11:26)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at node_modules/mammoth/lib/index.js (index.js:3:18)
    at __require2 (chunk-OROXOI2D.js?v=fa264171:16:50)
    at dep:mammoth:1:16

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Closing as it worked after upgrading Vite to 3.0.0-alpha.10. It was fixed by #8338.
related: #7555


.table-responsive when Flexbox is enabled

Is it possible to have a table responsive in flex mode ?
  The issue is definitely related to flexbox.  I was able to solve it in a really hacky way by basically disabling flex box for smaller screens.  In this case, I wanted a single column layout for smaller screens anyway, so it worked out.
HTML:

Sass:
#mainContent {
  .row {
    display: block;

    @include breakpoint($md) {
      display: flex;
    }
  }

  .col-xs-12 {
    width: 100%;

    @include breakpoint($md) {
      width: auto;
    }
  }
}

Like I said... hacky.  Certainly not a permanent solution, but it works in a pinch.  Hoping for a more permanent solution in the released version of Bootstrap 4.

Hi,
Thank you for your answer
I read an article (+ demo http://dfmcphee.com/2016/03/flex-items-and-min-width-0/) two days ago about that. Maybe this could help ?

By adding min-width: 0 to the item, it will resize at the correct ratio and still apply the overflow as desired

Yes - that totally helped.  Far less hacky.  Thank you!
Updated Sass:
#mainContent {
  .col-xs-12 {
    min-width: 0;
  }
}

This appears to be functional to me: https://jsbin.com/nilura/edit?html,output. Tested with the latest code from v4-dev which has flexbox enabled by default.

not working when the wrapper is a flex display, .table-responsive don't work at all :/

Anyone has a solution?

I came across this a while ago.  It works with Bootstrap 3.  Maybe someone (with more time than me right now :) ) can fork this and make it work with Bootstrap 4?  http://gergeo.se/RWD-Table-Patterns/

Use min-width: 0; on table wrapper with flex. Worked for me.


Please help with AOT

Hey guys! Thank you @qdouble fir this great repo!
One question though - http://stackoverflow.com/questions/40450604/aot-function-calls-are-not-supported.
Could anyone help please? I don't get it what it wants from me...

Think it would be better for you to do a search of this issue on the Angular repo as this error is just a general AOT issue and not specific to this repo.


Support Dark Keyboard on iOS 13 when Dark Mode is enabled

Keyboard has white background on iOS 13 when in dark mode.
For reference, if this helps speed up implementation:
        if #available(iOS 13, *) {
            if (vc.traitCollection.userInterfaceStyle == .dark) {
                // Make keyboard background dark.
            }
        }

Looks like this is on line 217 of EmojiView.swift
I think something like this will do it:
private func setupView() {
    if #available(iOS 13, *) {
        backgroundColor = UIColor.secondarySystemBackground
    }
    else {
        backgroundColor = UIColor(red: 249/255.0, green: 249/255.0, blue: 249/255.0, alpha: 1)
    }
}

Will also have to check the other view classes for equivalent stuffs.

Thanks for the ideas @thejeff77 , I pushed some code to the master branch and updated the example, it looks good, please try it.

Wow cool screenshots. Thanks for doing this! You're like 12 hours ahead of me, was gonna do this today. I'll try it out right now. Thanks :)
Tried to add Swift Package Manager to this project but ran into issues. Any help with that would be pretty great.
I'll report back in a few.

Works like a charm. Thanks 🥇
Will close this out after 0.2.6 tag if that works.

Released v0.2.6

You're the best. Thanks.

The Pod is not updated yet. Please help.
-> Installing ISEmojiView (0.2.6)
 > Git download
 > Git download
     $ /usr/bin/git clone https://github.com/isaced/ISEmojiView.git
     /var/folders/l6/8nk8j_9j02dggl3_0hl29x_40000gn/T/d20200330-14308-g32s22 --template= --single-branch --depth 1 --branch
     0.2.5
     Cloning into '/var/folders/l6/8nk8j_9j02dggl3_0hl29x_40000gn/T/d20200330-14308-g32s22'...
     Note: switching to 'cd7fdd9be435b9545ca0d9a08c5e02dbcb8cc603'.

@isaced

hey guys this is not working on iOS14 / latest pod version.


Add new api to open all devices in cluster

Need a new api to open all devices in the cluster with a single call.
for mmio_device in mmio devices:
  open mmio_device
  for tunnel_count in mmio_device:
    for remote_chips in mmio_device.tunnel_number
      open remote device

I think this is resolved


WIP: Add edu::StoreProgram()

Fixes #133

Codecov Report
Attention: Patch coverage is 0% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 87.15%. Comparing base (8f73af6) to head (5f898e9).
Report is 8 commits behind head on master.

Files
Patch %
Lines

Sts1CobcSw/Edu/EduMock.cpp
0.00%
2 Missing :warning:

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #271   +/-   ##
=======================================
  Coverage   87.15%   87.15%
=======================================
  Files          18       18
  Lines         514      514
=======================================
  Hits          448      448
  Misses         66       66

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


THREE.Shape() & <extrudeGeometry args={[shape, options]} />

First and foremost, what an incredible feat you're achieving with this repo, it's the perfect solution for using R3F in a maps environment!
I've been playing with the library for a week or so, everythings been absolutely delightful being able to add R3F components as an overlay and the ability to extend THREE graph with drei!
one limitation that I've stumbled on is some sort of matrix mismatch when trying to use a THREE.Shape() as a basis for <extrudeGeometry/>, it looks like maybe one of two (or both) things are happening, a matrix transformation is being applied on the shape before it's passed as a prop to the extrusion ( afaik, THREE's native shapes are drawn on an x,y plane to be extruded on the z plane, it seems like deriving a vector2 from MapboxGl.MercatorCoordinate.fromLngLat([lng, lat], 0) is maybe missing a relative matrix transform resulting in an incorrect evaluation of the final geometry ) and / or a matrix transform is missing from the eventual ExtrudeGeometry which is currently being extruded on the z axis ( as it would be in a typical THREE environment ) but in this maps environment ( I'd expect ) would rather be extuded on the Y ( or up ) axis away from the  maps "floor" with the same behaviour as map boxes native 2.5D layer's "fill-extrusion-height"
would you know or atleast have any idea why this may be happening? Is it left down to user land to implement the correct transforms for this? if so would you know how I'd apply the correct matrix transforms and and which stage? If it's not intentional and not intentionally user land issues is this something valuable worth pointing out for a potential update to support <extrudeGeometry/> with shapes?
Thanks for your effort on creating this Library, it's a real powerhouse in itself!

As an update, the <Coordinates> component works perfectly for individual meshes, I'm wondering if there's a way to simply expose the underlying function behond the <Coordinates> component so an array of vectors (3 or 2) could be set from the same function and passed to Shape and or BufferGeometry respectively, this way geometries can be absolutely positioned on the map from their vertex positions...

Hi, @forerunrun, thanks! Do you have maybe a sandbox to replicate the issue? R3M (react-three-map) doesn't alter geometries or the scene object matrices, only the camera matrices.

@forerunrun the issue is that you are using coordinate points to define a ThreeJS mesh, but as soon as you are in ThreeJS, evertyhing is in meters units. So you need to convert your coordinates to meters. I wrote an example of how to do that here.

I wrote an example of how to do that here.

oh @RodrigoHamuy that's amazing, i completely overlooked that! one thing is that there seems to be an odd rotation to the shape, the back of the shape, based on the coordinates, should be parralel to the blue line in the following image...

would you have an idea why that rotation is off? I'm going to have a proper look into the coordsTovector3 method you've written to try to look into it now...

@forerunrun maybe this version  helps?

@RodrigoHamuy that's great thank you so much for looking into this, yes helps, i still can't tell why the extrude geometry is off, it should be right, if for instance on the extrude geometry mesh we use <mesh rotation={[90 * MathUtils.DEG2RAD, 0, 0]}> rather than -90 the orientation looks correct but the mesh is effectively upside down, both lines and extrusions are really useful in different cases so i'll try getting to the bottom of this! thanks again for your help!

@forerunrun, in coordsTovector3 change from - angle to + angle to flip the interpretation. Then you can have your mesh facing Z-up  (sandbox) ✌

in coordsTovector3 change from - angle to + angle to flip the interpretation.

Hey @RodrigoHamuy the problem with that is it then changes the orientation of the  in the same way... i ended up just changing the scale of the up axis to -1 for the extrude geometry mesh which seems to work...
one other thing i've noticed is the raycasting accuracy is strangely off when the map is zoomed out and an object appears small... for instance in an official example here...

in the above image the mouse is currently over the lowest left cube (although you can't see it, it is ) but the onPointerOver event is not being detected for some reason, i'm thinking it's possibly a raycaster misprojection maybe?


Add myself as a contributor, the right way this time

:tada: This PR is included in version 4.1.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[buildbot] Use python3 and more detailed logs for GithubStatusPush

Useed python3 and more detailed logs for GithubStatusPush

@azhogov What do you mean?
If you mean to use Past Simple there are some recommendations about good commit messages like:
https://github.com/erlang/otp/wiki/writing-good-commit-messages#do
It is better to write in imperative style.


Typo "Git Hub"→"GitHub"

https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/tutorial-gitops-flux2-ci-cd
https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/azure-arc/kubernetes/tutorial-gitops-flux2-ci-cd.md
#PingMSFTDocs

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Error when running TorchRuntimeClient

Describe the bug
(Initially submitted to the qiskit-machine-learning GitHub. It was suggested that it should be posted here)
When using TorchRuntimeClient, it fails with the following error (taken from the log of the job):
2022-11-16T07:37:37.332393231Z --- Logging error ---
2022-11-16T07:37:37.335383631Z Traceback (most recent call last):
2022-11-16T07:37:37.335438588Z File "/provider/programruntime/program_starter_wrapper.py", line 88, in execute
2022-11-16T07:37:37.335466993Z final_result = self.main(backend, self.messenger, **self.user_params)
2022-11-16T07:37:37.335499686Z File "/code/./program.py", line 502, in main
2022-11-16T07:37:37.335520910Z ret = trainer.train()
2022-11-16T07:37:37.335545397Z File "/code/./program.py", line 221, in train
2022-11-16T07:37:37.335625843Z loss, t_fwd, t_bckwd = self.train_step(train_batch)
2022-11-16T07:37:37.335644144Z File "/code/./program.py", line 293, in train_step
2022-11-16T07:37:37.335662319Z self.optimizer.step()
2022-11-16T07:37:37.335679267Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/optimizer.py", line 140, in wrapper
2022-11-16T07:37:37.335744542Z out = func(*args, **kwargs)
2022-11-16T07:37:37.335790758Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/optimizer.py", line 23, in _use_grad
2022-11-16T07:37:37.335813894Z ret = func(self, *args, **kwargs)
2022-11-16T07:37:37.335837831Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/adam.py", line 214, in step
2022-11-16T07:37:37.335863581Z if self.defaults['capturable'] or self.defaults['fused']
2022-11-16T07:37:37.335885565Z KeyError: 'fused'
2022-11-16T07:37:37.335929568Z
2022-11-16T07:37:37.335946623Z During handling of the above exception, another exception occurred:
2022-11-16T07:37:37.335963172Z
2022-11-16T07:37:37.335991995Z Traceback (most recent call last):
2022-11-16T07:37:37.336010325Z File "/usr/lib64/python3.8/logging/init.py", line 1085, in emit
2022-11-16T07:37:37.336049729Z msg = self.format(record)
2022-11-16T07:37:37.336065098Z File "/usr/lib64/python3.8/logging/init.py", line 929, in format
2022-11-16T07:37:37.336082152Z return fmt.format(record)
2022-11-16T07:37:37.336198160Z File "/usr/lib64/python3.8/logging/init.py", line 668, in format
2022-11-16T07:37:37.336366939Z record.message = record.getMessage()
2022-11-16T07:37:37.336391059Z File "/usr/lib64/python3.8/logging/init.py", line 373, in getMessage
2022-11-16T07:37:37.336410227Z msg = msg % self.args
2022-11-16T07:37:37.336459186Z TypeError: not all arguments converted during string formatting
2022-11-16T07:37:37.336477951Z Call stack:
2022-11-16T07:37:37.336596920Z File "/usr/lib64/python3.8/threading.py", line 890, in _bootstrap
2022-11-16T07:37:37.336679861Z self._bootstrap_inner()
2022-11-16T07:37:37.336700948Z File "/usr/lib64/python3.8/threading.py", line 932, in _bootstrap_inner
2022-11-16T07:37:37.336725819Z self.run()
2022-11-16T07:37:37.336744522Z File "/opt/app-root/lib64/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
2022-11-16T07:37:37.336762596Z result = context.run(func, *args)
2022-11-16T07:37:37.336780238Z File "/provider/server/main.py", line 75, in execute_program
2022-11-16T07:37:37.336797858Z starter.execute()
2022-11-16T07:37:37.336816243Z File "/provider/programruntime/program_starter_wrapper.py", line 92, in execute
2022-11-16T07:37:37.336833844Z logger.error(
2022-11-16T07:37:37.336851598Z Message: 'Error executing program job cdq96djcgdedhj0apec0'
2022-11-16T07:37:37.336873987Z Arguments: (KeyError('fused'),)
2022-11-16T07:37:37.336940229Z webserver-starter - ERROR Failed to execute program: 'fused'
2022-11-16T07:37:37.336959480Z Traceback (most recent call last):
2022-11-16T07:37:37.336977111Z File "/provider/server/main.py", line 75, in execute_program
2022-11-16T07:37:37.336998211Z starter.execute()
2022-11-16T07:37:37.337016918Z File "/provider/programruntime/program_starter_wrapper.py", line 96, in execute
2022-11-16T07:37:37.337035462Z raise ex
2022-11-16T07:37:37.337057385Z File "/provider/programruntime/program_starter_wrapper.py", line 88, in execute
2022-11-16T07:37:37.337075587Z final_result = self.main(backend, self.messenger, **self.user_params)
2022-11-16T07:37:37.337093184Z File "/code/./program.py", line 502, in main
2022-11-16T07:37:37.337108520Z ret = trainer.train()
2022-11-16T07:37:37.337127089Z File "/code/./program.py", line 221, in train
2022-11-16T07:37:37.337143146Z loss, t_fwd, t_bckwd = self.train_step(train_batch)
2022-11-16T07:37:37.337160520Z File "/code/./program.py", line 293, in train_step
2022-11-16T07:37:37.337176835Z self.optimizer.step()
2022-11-16T07:37:37.337197548Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/optimizer.py", line 140, in wrapper
2022-11-16T07:37:37.337214349Z out = func(*args, **kwargs)
2022-11-16T07:37:37.337232286Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/optimizer.py", line 23, in _use_grad
2022-11-16T07:37:37.337544979Z ret = func(self, *args, **kwargs)
2022-11-16T07:37:37.337579685Z File "/opt/app-root/lib64/python3.8/site-packages/torch/optim/adam.py", line 214, in step
2022-11-16T07:37:37.337602418Z if self.defaults['capturable'] or self.defaults['fused']
2022-11-16T07:37:37.337627948Z KeyError: 'fused'
2022-11-16T07:37:37.352319665Z jaeger_tracing - INFO Span publisher exited
2022-11-16T07:37:37.352881613Z webserver-starter - DEBUG Writing job status to termination marker file: '{"status": "Failed"}'
Steps to reproduce
I have used the code of the TorchRuntimeClient that used to be on Qiskit's website (it fails both with ibmq_qasm_simulator and all real devices I have tried):
import numpy as np
import matplotlib.pyplot as plt
from torch import Tensor, manual_seed, is_tensor
from torch.nn import Linear, CrossEntropyLoss, MSELoss
from torch.optim import LBFGS, Adam
from torch.utils.data import Dataset, DataLoader
from qiskit import Aer, QuantumCircuit
from qiskit.utils import QuantumInstance, algorithm_globals
from qiskit.opflow import AerPauliExpectation
from qiskit.circuit import Parameter
from qiskit.circuit.library import RealAmplitudes, ZZFeatureMap
from qiskit_machine_learning.neural_networks import CircuitQNN, TwoLayerQNN
from qiskit_machine_learning.connectors import TorchConnector
from qiskit_machine_learning.runtime import TorchRuntimeClient, TorchRuntimeResult
seed = 42
manual_seed(seed)
algorithm_globals.random_seed = seed
Expected behavior
It should run without errors
Suggested solutions
Additional Information

qiskit-ibm-runtime version: 0.8.0
Python version: 3.7.15
Operating system: Google Colab

Hi, any chance of finding a solution to this issue? Thanks so much in advance!!!

Are you still running into the same error? I was able to run a torch-train job based on the code in 527. This snippet doesn't actually use qiskit-ibm-runtime so if it's not an issue in qiskit-machine-learning it's a server side issue in which case I can create an issue there.

Hi, thanks for taking a look at it. I have just run the code in https://github.com/Qiskit/qiskit-machine-learning/issues/527 and it still gives an error. However, the error now seems to be different. Namely, I obtain this:
2022-11-28T18:10:03.188378499Z XXX lineno: 94, opcode: 120
2022-11-28T18:10:03.188613639Z --- Logging error ---
2022-11-28T18:10:03.190209521Z Traceback (most recent call last):
2022-11-28T18:10:03.190247776Z   File "/provider/programruntime/program_starter_wrapper.py", line 88, in execute
2022-11-28T18:10:03.190264607Z     final_result = self.main(backend, self.messenger, **self.user_params)
2022-11-28T18:10:03.190279652Z   File "/code/./program.py", line 434, in main
2022-11-28T18:10:03.190295873Z     optimizer = str_to_obj(kwargs.get("optimizer", None))
2022-11-28T18:10:03.190312774Z   File "/code/./program.py", line 533, in str_to_obj
2022-11-28T18:10:03.190327234Z     obj = dill.loads(base64.b64decode(string.encode()))
2022-11-28T18:10:03.190342775Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 387, in loads
2022-11-28T18:10:03.190368802Z     return load(file, ignore, **kwds)
2022-11-28T18:10:03.190384573Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 373, in load
2022-11-28T18:10:03.190399451Z     return Unpickler(file, ignore=ignore, **kwds).load()
2022-11-28T18:10:03.190414230Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 646, in load
2022-11-28T18:10:03.190443554Z     obj = StockUnpickler.load(self)
2022-11-28T18:10:03.190499499Z   File "/usr/local/lib/python3.7/dist-packages/torch/optim/adam.py", line 94, in setstate
2022-11-28T18:10:03.190516344Z SystemError: unknown opcode
2022-11-28T18:10:03.190623151Z
2022-11-28T18:10:03.190642471Z During handling of the above exception, another exception occurred:
2022-11-28T18:10:03.190658358Z
2022-11-28T18:10:03.190683951Z Traceback (most recent call last):
2022-11-28T18:10:03.190698909Z   File "/usr/lib64/python3.9/logging/init.py", line 1083, in emit
2022-11-28T18:10:03.190714201Z     msg = self.format(record)
2022-11-28T18:10:03.190729079Z   File "/usr/lib64/python3.9/logging/init.py", line 927, in format
2022-11-28T18:10:03.190744080Z     return fmt.format(record)
2022-11-28T18:10:03.190760782Z   File "/usr/lib64/python3.9/logging/init.py", line 663, in format
2022-11-28T18:10:03.190774361Z     record.message = record.getMessage()
2022-11-28T18:10:03.190793344Z   File "/usr/lib64/python3.9/logging/init.py", line 367, in getMessage
2022-11-28T18:10:03.190810777Z     msg = msg % self.args
2022-11-28T18:10:03.190825816Z TypeError: not all arguments converted during string formatting
2022-11-28T18:10:03.190842505Z Call stack:
2022-11-28T18:10:03.190918966Z   File "/usr/lib64/python3.9/threading.py", line 937, in _bootstrap
2022-11-28T18:10:03.190939580Z     self._bootstrap_inner()
2022-11-28T18:10:03.190952620Z   File "/usr/lib64/python3.9/threading.py", line 980, in _bootstrap_inner
2022-11-28T18:10:03.190967463Z     self.run()
2022-11-28T18:10:03.190983386Z   File "/opt/app-root/lib64/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
2022-11-28T18:10:03.190998735Z     result = context.run(func, *args)
2022-11-28T18:10:03.191013239Z   File "/provider/server/main.py", line 75, in execute_program
2022-11-28T18:10:03.191028601Z     starter.execute()
2022-11-28T18:10:03.191043892Z   File "/provider/programruntime/program_starter_wrapper.py", line 92, in execute
2022-11-28T18:10:03.191059667Z     logger.error(
2022-11-28T18:10:03.191073501Z Message: 'Error executing program job ce2fiu4gc7i7i090mu1g'
2022-11-28T18:10:03.191088677Z Arguments: (SystemError('unknown opcode'),)
2022-11-28T18:10:03.191116789Z webserver-starter - ERROR Failed to execute program: unknown opcode
2022-11-28T18:10:03.191133460Z Traceback (most recent call last):
2022-11-28T18:10:03.191147343Z   File "/provider/server/main.py", line 75, in execute_program
2022-11-28T18:10:03.191160682Z     starter.execute()
2022-11-28T18:10:03.191177891Z   File "/provider/programruntime/program_starter_wrapper.py", line 96, in execute
2022-11-28T18:10:03.191191998Z     raise ex
2022-11-28T18:10:03.191206735Z   File "/provider/programruntime/program_starter_wrapper.py", line 88, in execute
2022-11-28T18:10:03.191221387Z     final_result = self.main(backend, self.messenger, **self.user_params)
2022-11-28T18:10:03.191235824Z   File "/code/./program.py", line 434, in main
2022-11-28T18:10:03.191252561Z     optimizer = str_to_obj(kwargs.get("optimizer", None))
2022-11-28T18:10:03.191265914Z   File "/code/./program.py", line 533, in str_to_obj
2022-11-28T18:10:03.191301022Z     obj = dill.loads(base64.b64decode(string.encode()))
2022-11-28T18:10:03.191315569Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 387, in loads
2022-11-28T18:10:03.191328690Z     return load(file, ignore, **kwds)
2022-11-28T18:10:03.191342775Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 373, in load
2022-11-28T18:10:03.191359011Z     return Unpickler(file, ignore=ignore, **kwds).load()
2022-11-28T18:10:03.191374378Z   File "/opt/app-root/lib64/python3.9/site-packages/dill/_dill.py", line 646, in load
2022-11-28T18:10:03.191389895Z     obj = StockUnpickler.load(self)
2022-11-28T18:10:03.191402921Z   File "/usr/local/lib/python3.7/dist-packages/torch/optim/adam.py", line 94, in setstate
2022-11-28T18:10:03.191416821Z SystemError: unknown opcode
2022-11-28T18:10:03.207245704Z jaeger_tracing - INFO Span publisher exited
2022-11-28T18:10:03.207516047Z webserver-starter - DEBUG Writing job status to termination marker file: '{"status": "Failed"}'
2022-11-28T18:10:03.207889348Z /pod-data/ CLOSE_WRITE,CLOSE terminated
2022-11-28T18:10:03.209925665Z Termination marker file found. Kill process (7).
2022-11-28T18:10:03.257637109Z /bin/bash: line 3:     7 Killed                  python -m uvicorn server.main:app --port 8081
2022-11-28T18:10:03.257918971Z Termination signal received, exited.

So it fails with both Qiskit Machine Learning 0.4.0 and Qiskit Machine Learning 0.5.0, but the error message is different depending on the version.
The one I copied in the OP is with 0.5.0. The one I copied today (two messages above this) is with 0.4.0.

Hmm weird, I get the same error with 0.4.0 but it runs fine with 0.5.0 - have you tried updating all your dependencies?

I'm using a clean Google Colab session in which I just install Qiskit 0.39.2 and Qiskit Machine Learning 0.5.0.

Hi! It seems that the problem was that Google Colab uses PyTorch 1.12.1 and version 1.13 is needed. After updating, it runs without error!!!
Thanks for your help!

@EliasCombarro Just to give you a heads up, the torch Qiskit Runtime programs (used by TorchRuntimeClients) were deprecated more than a month ago and will be removed in the near future.

Thanks so much for the info. But, if I do not update my Qiskit version I will still be able to use it TorchRuntimeClient, right?

@EliasCombarro Unfortunately no since the programs will be removed from the server. However, qiskit-machine-learning has been updated to use Qiskit Runtime primitives, so I suggest you check out the tutorials there.

Oh, that's too bad, really :( We are finishing writing a book on QML that uses Qiskit and we use TorchRuntimeClient for an example of how to train QNNs on actual quantum hardware.
Thanks anyway

Hi, again. We've been trying to train a QNN with the new Runtime services but it seems to send a bunch of independent circuit jobs instead of a Runtime program. I'd really appreciate it if you could point me to a specific tutorial in which it is explained how to train a  QNN with Runtime (we were not able to find any among the Qiskit Machine Learning tutorials). Thanks in advance!

@ElePT do you know if there are tutorials on training a QNN using runtime?
@EliasCombarro , Elena also just gave a demo on that yesterday during Qiskit Demo day. The recording should be posted shortly if you're interested.

Hi again!
This is the code that we have tried in order to train a VQC with Runtime:
from qiskit_machine_learning.algorithms.classifiers import VQC

# channel = "ibmq_quantum" gives us access to IBM's quantum computers.
service = QiskitRuntimeService(channel = "ibm_quantum")

with Session(service = service, backend = "ibmq_lima"):
    sampler = Sampler()
    vqc = VQC(sampler = sampler, num_qubits = 2)
    vqc.fit(x_tr, y_tr)´´´

However, it spawns hundreds (literally, I counted more than 400) of Runtime Sampler jobs. We did not expect that. We expect the program to be submitted in just one go and to queue just once.

Any hint and what we could be doing wrong will be really appreciated. Thanks in advance.

Hi @EliasCombarro, it looks like you were expecting the primitives to behave like an "upload yourself" runtime program, but this is not the case.
Primitive jobs (in this case, sampler jobs) are submitted every time you do .run(), the idea behind the sessions is that they provide a sort of "dynamical reservation" where your jobs will not be interrupted by other jobs in the queue, but there are still "individual jobs" so to speak. So you are not doing anything wrong, this is the expected behavior.

OK, I see. Thanks so much for the clarification!


Compensation concepts

Currently, the master thesis concepts are implemented as follows:
Retry Task
A retry task has a map of properties isRetryTask, retryCooldown and retryCount embedded in its extensionElements. If an error occurs, the task checks if it's a retryTask and, if true, ignores the error and restarts the execution with the same instance parameters. This is done by simply resetting the timestamp and as currently only user tasks are tackled, the user task can be attempted again. The failedAttempts are tracked and if they exceed the retryCount, the error is propagated and no longer ignored. The cooldown is always applied before the error is propagated, or the task create time reset. See TaskEntity.java .
Non-vital Task
The Non-vital tasks has an extensionElement property isVital. If a task is vital, a BPMNError occurs, the taskEntity will check, if it is a vitalTask. If it is not the case, the error is ignored. This means the task will be completed and by immediately returning, no error handling is undertaken. See TaskEntity.java .
Savepoints
For savepoints, an extensionElement isSavepoint is used. This property is propagated from the savepoint task to the compensation catch event subscription. If then a compensation event later is thrown, all time-based ordered event subscriptions are signalled. The difference here is now, that all task before the savepoint are no longer signalled to trigger the compensation behaviour. This allows for partial compensation.
To continue the execution of the process instance, the compensation handler of the savepoint task has its compensation reaction surpressed. Then after a successfull compensation, if there was a savepoint present, a processInstanceModification is undertaken to start another execution after the savepoint task. Therefore, the execution of the compensation throw event is successfully completed, whilst also the new execution for the re-execution after the compensation is enabled.
For the changes, see:

CompensationUtil → ordering of the compensation event subscriptions and triggering of post savepoint compensation executions.
AbstractBpmnActivityBehaviour → Propagation of isSavepoint property to attached compensation catch event (isSavepointcompanion) and setting of start position of the re-execution
BpmnParse → Setting of task properties as task elements were not always available.

Alternative Paths
The alternative paths gateway is an adaption of the exclusive gateway. It could be made standalone by duplicating every exclusive gateway file and moving the changed lines of code as well as adding a new xml tag. This would require alone for the parsing around 10 functions to be written, hence I omitted this step.
Because gateways cant handle extensionElements, the alternative path gateway is detected, when the conditions on the outgoing flow contain the term APGateway. From this, the condition is matched against a Regex, to retrieve the index of the alternative. The exclusive gateway received an instance variable, an instanceExecutionCount, that tracks how often this gateway instance has been executed. Based on this, the fitting alternative is selected.
The savepoint behaviour is added explicitly using a dummy savepoint task, marked with the extensionElement isAPSavepoint = true. From there on, using multiple flags in the CompensationUtil, APSavepoints can be revisited multiple times whilst normal savepoints are disabled after one usage. Furthermore, the APs savepoint is disabled, when the gateway has successfully joined the incoming flow.
For the relevant changes, see:

Condition.java to retrieve the condition text on which the regex to detect APGateways is applied to
ExclusiveGatewayActivityBehavior.java to choose the alternativePathIndex based on the gateways instanceExecutionCount.
Calls to setFlagSavepointIrrelevant(true), which marks that the AP Gateway has successfully joined and the savepoint is disregarded.

Discussion
Advantages of the implementation is that everything could also be included in the camunda modeller, as either extensionElements or hard coded strings in the conditions have been used.
The limitations of the implementation currently are the following:

I have only tested the implementation using User Tasks. Other task types have not been tested. I think accidentally, User tasks were a lucky choice, as Camunda sets transaction boundaries whenever a user task occurs, reducing problems when manipulating the process instance state.
As I am using flags to show if a savepoint in the process instance has been triggered, only one savepoint/AP construct per process is currently supported. Mixing of the concepts also won't work. If one wanted to add support for this, the flags would have to be replaced with ActivityId based maps, that track if a savepoint is still regarded or if it has been disabled. But as then multiple savepoints exists, the question arises, which savepoint should be taken as base for re-execution. This requires a complex graph algorithm that tracks back the sequence flow from the compensation throw event. This would have exceeded the scope of this implementation.

Testing
I've created an extensive test suite in CompensationConceptsTests.java. I am not able to recreate the CI of Camunda, as errors occur during the package command of maven.
On my local machine, executing the follwoing commands, yields the follwoing results (on commit 78fa84901be170abce9ab4a6a0ee2353df54ac67)
/compensation-conecept-extension-of-camunda-bpm-platform/engine$ mvn test -Dtest=CompensationConceptsTests -DforkCount=0 -f pom.xml

...
[INFO] Results:
[INFO]
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  15.230 s
[INFO] Finished at: 2024-08-23T16:31:03+02:00
[INFO] ------------------------------------------------------------------------

The following command fails due to errors in unrelated test suites.
/compensation-conecept-extension-of-camunda-bpm-platform$ mvn install

[INFO] Running org.camunda.bpm.engine.test.bpmn.event.compensate.helper.CompensationConceptsTests
[INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.222 s - in org.camunda.bpm.engine.test.bpmn.event.compensate.helper.CompensationConceptsTests

(The error thrown, referring to the process model for testing not being found)
[ERROR] Errors:
[ERROR] org.camunda.bpm.engine.test.api.form.FormDataTest.testMissingFormVariables(org.camunda.bpm.engine.test.api.form.FormDataTest)
[ERROR]   Run 1: FormDataTest.testMissingFormVariables » ProcessEngine ENGINE-01009 Error while...
[ERROR]   Run 2: FormDataTest.testMissingFormVariables:258 » NullValue no processes deployed wi...

The CI fails because of the dependency graph build (idk), but the build works. In the logs, I found following line:
2024-08-23T15:31:12.2526187Z [INFO] Running org.camunda.bpm.engine.test.bpmn.event.compensate.helper.CompensationConceptsTests
2024-08-23T15:31:16.4358547Z [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.182 s - in org.camunda.bpm.engine.test.bpmn.event.compensate.helper.CompensationConceptsTests
``


Patch to fix code warnings

Attached is code-warnings.patch from Debian package <IP_ADDRESS>-10.  It fixes a code warning in compress42.c.
KEN
# Description: Fix compiler warnings in the code.
# Author: Kenneth J. Pronovici<EMAIL_ADDRESS>Index: ncompress-<IP_ADDRESS>/compress42.c
===================================================================
--- ncompress-<IP_ADDRESS>.orig/compress42.c
+++ ncompress-<IP_ADDRESS>/compress42.c
@@ -1007,8 +1007,12 @@ comprexx(fileptr)

               if (infstat.st_nlink > 1 && (!force))
               {
-                 fprintf(stderr, "%s has %d other links: unchanged\n",
-                             tempname, infstat.st_nlink - 1);
+                 /* There's no portable way to print an nlink_t, because it
+                     changes size by platform.  So, I just cast to unsigned
+                     long int and hope for the best.  This is basically the
+                     StackOverflow recommendation: http://stackoverflow.com/questions/1401526 */
+                 fprintf(stderr, "%s has %ld other links: unchanged\n",
+                             tempname, (unsigned long int) infstat.st_nlink - 1);
                  exit_code = 1;
                  return;
               }

This patch is included in the pull request referenced above.

as mentioned in the pull request, using long for nlink_t won't work as some platforms have 8 byte nlink_t and 4 byte long (like the x32 ABI)
fixed w/intmax_t in commit add92bc8f2061700872f770ef99c7b9c8beb028d

Ok, make sense.  I had tested i386 and amd64.  I'll adjust the Debian patch to match your implementation.


ResponseTemplating: now format='epoch' is read as zero when passed as input to math

The problem
"body" : "[{{#each (range 0 20) as |index|}} [ {{math (now format='epoch') '+' (math index '*' 86400000)}}, {{randomDecimal lower=190.0 upper=220.0}} ] {{#if @last}}{{else}},{{/if}}{{/each}}]",

This will generate a body that looks like
[ [ 0,<PHONE_NUMBER>809619 ] , [ 86400000,<PHONE_NUMBER>7016043 ] , [ 172800000,<PHONE_NUMBER>4155523 ] , [ 259200000,<PHONE_NUMBER>4502032 ] , [ 345600000,<PHONE_NUMBER>9839516 ] ..... ]

As you can see it seems like math treats the output from (now format='epoch') as zero.  If I use now format='epoch' outside the context of math, it works like a charm.
Might it be that now format='epoch' returns a string or something else that makes math lay down and die?
Details

Version: 2.33.2
Starting the server docker run -d --name wiremock-timeseries --network host -v $PWD/timeseries-wiremock:/home/wiremock -e uid=$(id -u) wiremock/wiremock --port="8083" --local-response-templating --verbose
The full stub (I removed a bunch of stuff I do not want to leave the company)

{
  "id" : "04056434-098e-3775-a070-1ca35778aa25",
  "request" : {
    "urlPattern": "/.*",
    "method" : "POST"
  },
  "response" : {
    "transformers": ["response-template"],
    "status" : 200,
    "body" : "[{{#each (range 0 20) as |index|}} [ {{math (now format='epoch') '+' (math index '*' 86400000)}}, {{randomDecimal lower=190.0 upper=220.0}} ] {{#if @last}}{{else}},{{/if}}{{/each}}]",
    "headers" : {
      "Date" : "Mon, 05 Sep 2022 14:17:55 GMT",
      "Content-Type" : "application/json",
      "X-Content-Type-Options" : "nosniff",
      "X-XSS-Protection" : "1; mode=block",
      "Cache-Control" : "no-cache, no-store, max-age=0, must-revalidate",
      "Pragma" : "no-cache",
      "Expires" : "0"
    }
  },
  "uuid" : "04056434-098e-3775-a070-1ca35778aa25"
}

Confirmed this is an issue. The maths helper can't cope with a date object as an input.
You can work around this by assigning it to a variable, which will cast it to a String e.g.
{{#assign 'currentEpoch'}}{{now format='epoch'}}{{/assign}}
{{math currentEpoch '+' (math index '*' 86400000)}}


[xDS] require EDS resource name in CDS resources with xdstp names

Implements the change described in https://github.com/grpc/proposal/pull/377.

/easycla


Should we add docker-compose to the installation?

I'm not sure what's the best way to do this on Unraid, since docker-compose it's just a single binary, shouldn't we just install as part of the plugin installation?
It should be as simple as:
curl -L https://github.com/docker/compose/releases/download/1.29.2/docker-compose-Linux-x86_64 --output /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

Hmmm, I'm really not a friend running compose on Unraid because, well Unraid is a little different and it has it's own app plugin (CommunityApplication).
Wouldn't it be better to reach out to Squid on the forums or even here on GitHub and create corresponding templates and point to the issue tracker from GoW?
I think not that tech savvy users will really appreciate that move.

Discussion moved to: https://github.com/games-on-whales/unraid-plugin/issues/17


[red-knot] Add the special assignability logic for int/float/complex in parameter annotations

Given this Python function:
def f(x: float = 42): ...

red-knot currently issues this complaint:
error[lint:invalid-parameter-default] /Users/alexw/dev/experiment/foo.py:1:7 Default value of type `Literal[42]` is not assignable to annotated parameter type `float`

This is incorrect. Although Literal[42] is not a subtype of float, the typing spec carves out a special case for numeric types when used specifically in function parameter annotations:

Python’s numeric types complex, float and int are not subtypes of each other, but to support common use cases, the type system contains a straightforward shortcut: when an argument is annotated as having type float, an argument of type int is acceptable; similar, for an argument annotated as having type complex, arguments of type float or int are acceptable.

We need to implement this special case to avoid false-positive errors like the one above. Note that the special case only applies in function parameter annotations, not in any other context. Note also that all subtypes of int should also be considered assignable to float (and, transitively, complex) in this context: Literal[42], bool and Literal[True] are also therefore assignable to float and complex in the context of parameter annotations.

Wow interesting. Should we add an optional rule that warns about such "incompatible" default values? I don't think I'd want that behavior 😆

Should we add an optional rule that warns about such "incompatible" default values? I don't think I'd want that behavior 😆

It would conflict with https://docs.astral.sh/ruff/rules/redundant-numeric-union/ ;)

I suppose we could make that rule configurable so that you can "reverse" the behaviour and enforce the opposite...

You're correct that mypy and pyright do sometimes extend this behaviour to other contexts. It can be pretty inconsistent and surprising when they do, however! For example:

The odd behavior you point out here is inherent to the special case, and the way mypy implements it; it's not related to applying the special case to variable annotations. Exactly the same behavior appears with a function parameter annotation, too: https://mypy-play.net/?mypy=latest&python=3.12&gist=d1d1c0ed731ca0afe425f427fbb7e302
I think the only reason the spec implies this is only for parameter annotations is because the text predates variable annotations; I don't believe there is any good reason to limit it to parameter annotations, nor does any existing type checker do that. So I don't believe we should do so either.

I believe mypy when mypy sees float as an annotation it actually constructs an int | float pseudo-union behind the scenes.

Yes, I think this is right; and pyright does this too. I think this is the best way to implement this special case, as compared to the alternative of actually treating int as a subtype of float, which is problematic because int doesn't preserve Liskov relative to float. Applying it as a special case in the interpretation of the annotation float limits the scope of the special case, and means you can still infer a type of "float but not int", which would not be possible if int were treated generally as a subtype of float. Based on previous discussions, I think if/when we do clarify the spec, we will specify this "treat float annotation as int | float" behavior.
The strangeness you observe in the above example really comes entirely from the fact that mypy tries to "hide" this implicit int | float union type, and display it as float. This means that a display type of float might be "actually just float" or might be "implicit int | float" -- the type display doesn't allow you to tell which it is. And in your example, it's the implicit union in the first reveal_type, and actually-just-float in the third.
My feeling is that we should treat a float annotation as int | float (in any annotation, not particular to function parameter annotations), and that unlike mypy, we should not try to hide this from the user. Yes, it might be confusing for the user to annotate something as float and later see it revealed as int | float, but it's more confusing to have two different types both reveal as float. The special case exists, we can't get rid of it, let's embrace it and not try to hide it.
It's possible this will get us in trouble with the conformance suite, but I'll be happy to advocate for changing the conformance suite so it doesn't require hiding this int | float union.
(Also of course we must treat an annotation of complex as int | float | complex.)

And from the perspective of users, I would encourage them to only make use of the special case in parameter annotations. The intuition that led to this special case was that "nearly all functions that accept floats will also work fine with ints in practice" -- for all the flaws of the special case (and there are many! if only Python had a better runtime numeric tower so we didn't have to hack around it in the type system) it does make parameter annotations a lot less fiddly in many situations. But I don't think there's nearly the same benefits for users outside of the context of parameter annotations.
This isn't an argument against what you're saying -- I agree consistent behaviour is probably more important here.

it does mean there's no way to express in a stub file, for example, that an instance attribute really will be exactly a float

Yeah, this is the main downside of the special case. If we had intersections you could say float & ~int