[SPARK-27699][SQL] Partially push down disjunctive predicated in Parquet/ORC

What changes were proposed in this pull request?
Currently, in ParquetFilters and OrcFilters, if the child predicate of Or operator can't be entirely pushed down, the predicates will be thrown away.
In fact, the conjunctive predicates under Or operators can be partially pushed down.
For example, says a and b are convertible, while c can't be pushed down, the predicate
a or (b and c)
can be converted as
(a or b) and (a or c)
We can still push down (a or b).
We can't push down disjunctive predicates only when one of its children is not partially convertible.
How was this patch tested?
Unit test

Test build #105380 has started for PR 24598 at commit 490e00c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10643/
Test PASSed.

Test build #105380 has finished for PR 24598 at commit 490e00c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105380/
Test PASSed.

@dbtsai @cloud-fan @dongjoon-hyun

looks reasonable to me.

Thank you for pinging me, @gengliangwang . I'll take a look today~

Test build #105397 has started for PR 24598 at commit 38519d3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10660/
Test PASSed.

Test build #105397 has finished for PR 24598 at commit 38519d3.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105397/
Test FAILed.

retest this please.

Test build #105406 has started for PR 24598 at commit 38519d3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10671/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10672/
Test PASSed.

Test build #105409 has started for PR 24598 at commit 3b88687.

Test build #105406 has finished for PR 24598 at commit 38519d3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105406/
Test PASSed.

Test build #105409 has finished for PR 24598 at commit 3b88687.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105409/
Test PASSed.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10678/
Test PASSed.

Test build #105415 has started for PR 24598 at commit c65e026.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10680/
Test PASSed.

Test build #105417 has started for PR 24598 at commit caeb64d.

Test build #105415 has finished for PR 24598 at commit c65e026.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105415/
Test PASSed.

Test build #105417 has finished for PR 24598 at commit caeb64d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105417/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10720/
Test PASSed.

Test build #105458 has started for PR 24598 at commit 5f412bd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10725/
Test PASSed.

Test build #105463 has started for PR 24598 at commit b8cb843.

Test build #105458 has finished for PR 24598 at commit 5f412bd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105458/
Test PASSed.

Test build #105463 has finished for PR 24598 at commit b8cb843.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105463/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10737/
Test PASSed.

Test build #105476 has started for PR 24598 at commit 4d84060.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/10738/
Test PASSed.

Test build #105477 has started for PR 24598 at commit 90b0b69.

Test build #105476 has finished for PR 24598 at commit 4d84060.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105476/
Test PASSed.

Test build #105477 has finished for PR 24598 at commit 90b0b69.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/105477/
Test PASSed.

thanks, merging to master!


[SPARK-22341][yarn] Impersonate correct user when preparing resources.

The bug was introduced in SPARK-22290, which changed how the app's user
is impersonated in the AM. The changed missed an initialization function
that needs to be run as the app owner (who has the right credentials to
read from HDFS).

Test build #83019 has started for PR 19566 at commit d65e8f2.

Test build #83019 has finished for PR 19566 at commit d65e8f2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83019/
Test PASSed.

@vanzin
I tested your patch. It worked.

Test build #83054 has started for PR 19566 at commit de3755a.

Test build #83054 has finished for PR 19566 at commit de3755a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83054/
Test PASSed.

Merged build finished. Test PASSed.

Merging to master (since YARN is pretty broken without this).


PackageLoading: correct relative path construction

We would construct a RelativePath without rooting it, which would then
be normalized and thus cause problems.  Alter the path to validate the
path is relative and then construct the AbsolutePath by rooting the
relative path.

@compnerd can this be closed given https://github.com/apple/swift-package-manager/pull/5910?

Yeah, I think that we can close this PR, I checked the current PackageLoading test failures - they seem to be related to path normalisation rather than relative path construction.  Thanks for the reminder!


Navigate between components with vue-router

Describe the issue / bug.
I'm trying to navigate between components by using the electron menu. To do so, I create events and send them via something like mainWindow.webContents.send('navigate', 'about'). I currently receive those events on my landing page (main component) and the call this.$router.push(page).
All this works fine, if I always return back to the landing page before trying to navigate to one of the other pages.
When I try to go from non-landing page A to B via the menu, nothing happens, because the event listeners are on the non-landing page.
How can I manage page navigation in a centralized way? Can I manipulate the router from the electron menu directly? I'm trying to achieve this without every component having the same event listeners that handle navigation
How can I reproduce this problem?
Add A.vue and B.vue to the router and a menu to electron. From the menu call mainWindow.webContents.send('navigate', 'a') and in the landing page ipc.on('navigate', this.navigate, (event, page) => { this.$router.push(page) }).
While on A call mainWindow.webContents.send('navigate', 'b') from the menu. It won't redirect, as A doesn't have an event listener for page navigation.
Tell me about your development environment.

Node version: 10.15.1
NPM version: 6.5.0
vue-cli version: 3.3.0
Operating System: Windows 10

Why not have the code handling the routing in the same file as your <router-view></router-view> element?
Use that as your main layout, and then change out the contents in the middle using the router view.
I hope this made sense, and if not, feel free to ask questions. :)

Thanks, that indeed works just fine. Should have thought of that 🤦♂


refactor TriggerManager by smart Intellij

refactor TriggerManager by smart Intellij

Nice. Thanks for making it a separate pull request.


Update copyright years in LICENSE

Went through the license file in order to fix #577 but not sure why Github isn't recognizing it? Can it be because of the LICENSES folder?

LGTM


[CONSOLE] GDA Pool table address filter does not accept letters

When users would try to look up a pool by their address they would not be able to input the address, as the field only accepts numbers and e

Fixed in #210


Bug fix 3.7/make aql fast again 12

Scope & Purpose
Try creating specialized key bounds for full collection index scanning.
This allows removing a special case and a branch for scanning all values in an index (though the branch will still be present in the VPackComparator, but the code there already existed).
Suggested by @graetzer via the comments in https://github.com/arangodb/arangodb/pull/12266

[x] Bug-Fix for devel-branch (i.e. no need for backports?)
[x] Strictly new functionality (i.e. a new feature / new option, no need for porting)
[x] The behavior in this PR can be (and was) manually tested (support / qa / customers can test it)
[x] The behavior change can be verified via automatic tests

Testing & Verification
This change is already covered by existing tests, such as shell_server_aql.
http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/11215/

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/11222/

Tests blue.

Obsoleted by https://github.com/arangodb/arangodb/pull/12330


Applying a v1.List fails with "resourceVersion should not be set on objects to be created"

[2019-02-14T21:59:39.711] [INFO] kubernetes-bootstrap - v1.Secret kube-lego-account: smart failed: resourceVersion should not be set on objects to be created
(node:7817) UnhandledPromiseRejectionWarning: Error: Cannot apply resource v1.Secret kube-lego-account through 'smart'
    at applyResource (/home/travis/build/Collaborne/collaborne/node_modules/kubernetes-bootstrap/src/bootstrap.js:441:9)
    at processTicksAndRejections (internal/process/next_tick.js:81:5)
(node:7817) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:7817) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

(repeating for each element of the list)

Work-around: Convert the v1.List into a multi-document.

Actually that isn't the problem, but rather that the v1.List was made directly from kubectl get -o yaml -- which includes a bunch of things that we need to drop.


docs: ambiguous link to settings from rule N802

The current version of the documentation for rule invalid-function-name (N802) has a link to option pep8-naming.ignore-names, but it leads to rule flake8-self.ignore-names, not pep8-naming.ignore-names

Good catch!
Would you like to contribute to fix this?

If I'm not mistaken, according to the code - here need not just add a link in multiple the comments, but change something in the autogenerated documentation.
I'm afraid that I'm not able to figure it out now :(

You are right, the auto-generated-documentation needs to be updated.

@JonathanPlasse I have a PR incoming to fix this!


Create github actions for maven central publish

+1 to @aastha25, @rzhang10 could you update the PR description to include the test done?

@rzhang10 could you please add the test done? It would be great if this PR could be merged before #86

Yes, I plan to use a empty commit to trigger build with this github action and see if it publish successfully.

I think there's no good way to test "github actions" before they are merged.. so i think we can check in now and see if it can successfully publish. whether it succeed or not itself serves as a test.

https://github.com/linkedin/linkedin-calcite/commit/2500b128456bd67179e3642fe32d0d24407d365d this is the final rebased commit.


Create a heirarchical deterministic multisig addresses

This gives a convenience class BTC::KeychainGroup that allows the easy generation of multisig addresses based on an index and required number of keys.
Addresses https://github.com/oleganza/btcruby/issues/27

@oleganza not sure what kind of API you want this class to expose, but what's here is the minimum I was thinking of in issue #27.


removing limits in axios for large requests

https://stackoverflow.com/questions/56868023/error-request-body-larger-than-maxbodylength-limit-when-sending-base64-post-req

@theoephraim testei, tá funcionando, manda bala no merge!

Thanks for contributing. I was in the middle of a big typescript rewrite that took longer than expected...
Sorry I did not merge  the PR, it was easier to just copy the changes - it is now fixed!


help : (

Enter the maximum size of the model (by default, enter nothing and we will use 3GB as the limit). You can specify a unit (e.g. 3GB, 300Mb). If no unit is given, we assume the size is given in bytes.
Maximum model size set to<PHONE_NUMBER>.0 bytes.
Retrieving model...
 In this task, the input is a string that consists of both a question and a context passage. The context is a descriptive passage related to the question and contains the answer. And the question can range from Math, Cultural, Social, Geometry, Biology, History, Sports, Technology, Science, and so on.
Here are examples with input questions and context passages, along with their expected outputs:
input="Question: What city did Super Bowl 50 take place in? Context: Super Bowl 50 was an American football game to determine the champion of the National Football League (NFL) for the 2015 season. The American Football Conference (AFC) champion Denver Broncos defeated the National Football Conference (NFC) champion Carolina Panthers 24–10 to earn their third Super Bowl title. The game was played on February 7, 2016, at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California. As this was the 50th Super Bowl, the league emphasized the "golden anniversary" with various gold-themed initiatives, as well as temporarily suspending the tradition of naming each Super Bowl game with Roman numerals (under which the game would have been known as "Super Bowl L"), so that the logo could prominently feature the Arabic numerals 50."
output="Santa Clara"
input="Question: What river runs through Warsaw? Context: Warsaw (Polish: Warszawa [varˈʂava] ( listen); see also other names) is the capital and largest city of Poland. It stands on the Vistula River in east-central Poland, roughly 260 kilometres (160 mi) from the Baltic Sea and 300 kilometres (190 mi) from the Carpathian Mountains. Its population is estimated at 1.740 million residents within a greater metropolitan area of 2.666 million residents, which makes Warsaw the 9th most-populous capital city in the European Union. The city limits cover 516.9 square kilometres (199.6 sq mi), while the metropolitan area covers 6,100.43 square kilometres (2,355.39 sq mi)."
output="Vistula River"
input="Question: The Ottoman empire controlled territory on three continents, Africa, Asia and which other? Context: The Ottoman Empire was an imperial state that lasted from 1299 to 1923. During the 16th and 17th centuries, in particular at the height of its power under the reign of Suleiman the Magnificent, the Ottoman Empire was a powerful multinational, multilingual empire controlling much of Southeast Europe, Western Asia, the Caucasus, North Africa, and the Horn of Africa. At the beginning of the 17th century the empire contained 32 provinces and numerous vassal states. Some of these were later absorbed into the empire, while others were granted various types of autonomy during the course of centuries."
output="Europe"
"""done
done
Parsing prompt...
Prompt parsed.
Retrieving dataset...
config.json: 100%|████████████████████████████████████████████████████████████████████████████| 781/781 [00:00<?, ?B/s]
C:\Python310\lib\site-packages\huggingface_hub\file_download.py:147: UserWarning: huggingface_hub cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\antdx.cache\huggingface\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the HF_HUB_DISABLE_SYMLINKS_WARNING environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
warnings.warn(message)
tokenizer_config.json: 100%|██████████████████████████████████████████████████████████████████| 420/420 [00:00<?, ?B/s]
vocab.txt: 100%|████████████████████████████████████████████████████████████████████| 228k/228k [00:00<00:00, 13.0MB/s]
special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████| 125/125 [00:00<?, ?B/s]
pytorch_model.bin: 100%|████████████████████████████████████████████████████████████| 440M/440M [00:08<00:00, 49.2MB/s]

Here are the datasets I've retrieved for you:
Name    Size[MB]        Description
1):     yulongmannlp/dev_para   76.72   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
2):     yulongmannlp/dev_orig   76.71   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
3):     yulongmannlp/adv_para   75.72   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
4):     yulongmannlp/adv_ori    75.72   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
5):     squad   85.66   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
6):     lhoestq/squad   85.66   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
2023-12-30 05:04:19,962 - dataset_utils - ERROR - Error occurred in fetching size: 500
7):     lhoestq/custom_squad    NA      Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
8):     hapandya/sqnnr  0.00    Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
9):     Yulong-W/squadpararobustness    82.14   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
10):    Yulong-W/squadpara      75.82   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
11):    Yulong-W/squadorirobustness     82.14   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
12):    Yulong-W/squadori       75.82   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
13):    MajdTannous/Test3       0.03    Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
14):    MajdTannous/Test2       122.45  Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
2023-12-30 05:04:20,715 - dataset_utils - ERROR - Error occurred in fetching size: 500
15):    MajdTannous/Dataset1    NA      Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
16):    BerMaker/test   85.66   Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
17):    SajjadAyoubi/persian_qa 18.98   \\\\Persian Question Answering (PersianQA) Dataset is a reading comprehension dataset on Persian Wikipedia.  The crowd-sourced dataset consists of more than 9,000 entries. Each entry can be either an impossible to answer or a question with one or more answers spanning in the passage (the context) from which the questioner proposed the question. Much like the SQuAD2.0 dataset, the impossible or unanswerable questions can be utilized to create a system which "knows that it doesn't know the answer".
18):    web_questions   0.78    This dataset consists of 6,642 question/answer pairs. The questions are supposed to be answerable by Freebase, a large knowledge graph. The questions are mostly centered around a single named entity. The questions are popular ones asked on the web (at least in 2013).
19):    race    333.27  Race is a large-scale reading comprehension dataset with more than 28,000 passages and nearly 100,000 questions. The  dataset is collected from English examinations in China, which are designed for middle school and high school students. The dataset can be served as the training and test sets for machine comprehension.
20):    EleutherAI/race 2.61    Race is a large-scale reading comprehension dataset with more than 28,000 passages and nearly 100,000 questions. The  dataset is collected from English examinations in China, which are designed for middle school and high school students. The dataset can be served as the training and test sets for machine comprehension.
21):    TenzinGayche/Demo-datasets      1.34    Stanford Question Answering Dataset (DemoDatasets) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.
22):    the-coorporation/the_squad_qg   42.47   A preprocessed version of the Standford Question Answering Dataset (SQuAD) version 2.0 consisting of contexts and questions only.  Duplicate contexts have been removed and corresponding questions have been merged into an array per context.  Stanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.  SQuAD 2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD 2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering.
2023-12-30 05:04:21,508 - dataset_utils - ERROR - Error occurred in fetching size: 401
23):    imagenet-1k     NA      ILSVRC 2012, commonly known as 'ImageNet' is an image dataset organized according to the WordNet hierarchy. Each meaningful concept in WordNet, possibly described by multiple words or word phrases, is called a "synonym set" or "synset". There are more than 100,000 synsets in WordNet, majority of them are nouns (80,000+). ImageNet aims to provide on average 1000 images to illustrate each synset. Images of each concept are quality-controlled and human-annotated. In its completion, ImageNet hopes to offer tens of millions of cleanly sorted images for most of the concepts in the WordNet hierarchy. ImageNet 2012 is the most commonly used subset of ImageNet. This dataset spans 1000 object classes and contains 1,281,167 training images, 50,000 validation images and 100,000 test images
24):    AlexFierro9/imagenet-1k_test    4774.63 ILSVRC 2012, commonly known as 'ImageNet' is an image dataset organized according to the WordNet hierarchy. Each meaningful concept in WordNet, possibly described by multiple words or word phrases, is called a "synonym set" or "synset". There are more than 100,000 synsets in WordNet, majority of them are nouns (80,000+). ImageNet aims to provide on average 1000 images to illustrate each synset. Images of each concept are quality-controlled and human-annotated. In its completion, ImageNet hopes to offer tens of millions of cleanly sorted images for most of the concepts in the WordNet hierarchy. ImageNet 2012 is the most commonly used subset of ImageNet. This dataset spans 1000 object classes and contains 1,281,167 training images, 50,000 validation images and 100,000 test images
25):    dbpedia_14      191.44  The DBpedia ontology classification dataset is constructed by picking 14 non-overlapping classes from DBpedia 2014. They are listed in classes.txt. From each of thse 14 ontology classes, we randomly choose 40,000 training samples and 5,000 testing samples. Therefore, the total size of the training dataset is 560,000 and testing dataset 70,000. There are 3 columns in the dataset (same for train and test splits), corresponding to class index (1 to 14), title and content. The title and content are escaped using double quotes ("), and any internal double quote is escaped by 2 double quotes (""). There are no new lines in title or content.

If none of these are relevant to your prompt, we'll only use generated data. Are any of these datasets relevant? (y/N)
y
Which dataset would you like to use? Give the number between 1 and 25.
6

Loaded dataset. Example row:
{
"id": "5733be284776f41900661182",
"title": "University_of_Notre_Dame",
"context": "Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend "Venite Ad Me Omnes". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.",
"question": "To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?",
"answers_text": [
"Saint Bernadette Soubirous"
],
"answers_answer_start": [
515
]
}
Will use the columns ["context", "question"] as input.
Will use the column "answers_text" as our target.

Enter the maximum size of the model (by default, enter nothing and we will use 3GB as the limit). You can specify a unit (e.g. 3GB, 300Mb). If no unit is given, we assume the size is given in bytes.
Maximum model size set to<PHONE_NUMBER>.0 bytes.
Retrieving model...
Traceback (most recent call last):
File "C:\Users\antdx\prompt2model\prompt2model_demo.py", line 535, in
main()
File "C:\Users\antdx\prompt2model\prompt2model_demo.py", line 214, in main
retriever = DescriptionModelRetriever(
File "C:\Users\antdx\prompt2model\prompt2model\model_retriever\description_based_retriever.py", line 101, in init
self.load_model_info()
File "C:\Users\antdx\prompt2model\prompt2model\model_retriever\description_based_retriever.py", line 134, in load_model_info
urllib.request.urlretrieve(
File "C:\Python310\lib\urllib\request.py", line 251, in urlretrieve
tfp = open(filename, 'wb')
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/model_info.tgz'
C:\Users\antdx\prompt2model>

Hi @AntDX316, thank you for the issue. Unfortunately, our code was developed for use on Linux, and therefore we assumed that all users would have a directory called /tmp/ available.
As a temporary fix, you can change lines 136 and 138 in C:\Users\antdx\prompt2model\prompt2model\model_retriever\description_based_retriever.py to replace "/tmp/model_info.tgz" to another path that works on Windows (e.g. C:\Users\antdx\prompt2model\model_info.tgz).
Note that I don't use Windows for development so I might have formatted my suggested path for you incorrectly - take care to make sure that the path is formatted correctly.
After making this change locally, you should reinstall prompt2model from source, as shown here: https://github.com/neulab/prompt2model/blob/main/CONTRIBUTING.md

ty for the replies everyone but I need to install Linux first so I will see those later.

Hi @AntDX316, thank you for the issue. Unfortunately, our code was developed for use on Linux, and therefore we assumed that all users would have a directory called /tmp/ available.
As a temporary fix, you can change lines 136 and 138 in C:\Users\antdx\prompt2model\prompt2model\model_retriever\description_based_retriever.py to replace "/tmp/model_info.tgz" to another path that works on Windows (e.g. C:\Users\antdx\prompt2model\model_info.tgz).
Note that I don't use Windows for development so I might have formatted my suggested path for you incorrectly - take care to make sure that the path is formatted correctly.
After making this change locally, you should reinstall prompt2model from source, as shown here: https://github.com/neulab/prompt2model/blob/main/CONTRIBUTING.md

What Linux software do you recommend I use for these AI developments?
What Linux are you using? I want to install that on my 96GB PC that cannot install Windows 11 and Windows 10 has lost future Microsoft support.

@viswavi : maybe we could replace /tmp/ with a temporary directory created with the tempfile package?

@neubig I put a fix here - https://github.com/neulab/prompt2model/pull/388. It's actually simpler than I thought.

@AntDX316 Windows should actually be able to work (after #388 is merged). You may need to make a few small tweaks to get it to work on Windows - other users have reported success in training models with Prompt2Model on Windows.
We tested our code on Ubuntu. For training models with Prompt2Model, you will likely need access to an Nvidia GPU to train in a reasonable time, but otherwise, this should work on any machine

Closing because #388 is now merged


No Tabs Appearing

Hello, as per the Tabalonia example I have tried to integrate it into my application. Problem is that no tabs appear.
My repository: https://github.com/ScreenCrowStudios/CrowEngine
Files that follow the example: https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineUI/ViewModels/TabItemViewModel.cs
https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineUI/ViewModels/MainViewModel.cs
https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineUI/Views/BottomTabs.axaml
Thanks in advance!

Have you added the theme on App.xaml?

This is a bug) I did not test the lack of tabs, since by default the window closes after closing the last tab.
I'll try to fix it soon

i see what you are referring to but for me there were no tabs to begin with.

This is a bug) I did not test the lack of tabs, since by default the window closes after closing the last tab.

@egorozh  Can you make the closing of the window optional ? Like @HeadMonitor  I have no tabs half of the time (both at startup and during runtime)

This is a bug) I did not test the lack of tabs, since by default the window closes after closing the last tab.

@egorozh Can you make the closing of the window optional ? Like @HeadMonitor I have no tabs half of the time (both at startup and during runtime)

It's working now. You can set closed action like:

Thank you for your reply. It seems that was the issue! Apologies for the confusion. One more thing I am not sure how to fix is this:
With the same code no tabs appear. Only the plus sign. Any idea why that might be happening?

Fixed in 0.9.12

https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineProjectManager/ViewModels/MainWindowTabsViewModel.cs https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineProjectManager/ViewModels/TabItemViewModel.cs https://github.com/ScreenCrowStudios/CrowEngine/blob/master/CrowEngineProjectManager/Views/MainWindow.axaml
Hello, thank you for your replies. I have followed the demo but I cannot figure out how to add tabs. No tabs appear at all. Any idea why this is happening?

ViewModel not setted to DataContext. Try like this:

Thanks a lot, that worked!


Runtime error

Getting this issue

Hi @aryanmanik013 can you provide us a minimal reproducible example via a JSFiddle or something similar? I'm afraid a screenshot of the error isn't going to be enough information here to help

Closing this thread as it is outdated. Please re-open if it is still relevant. Thank you for your contribution!


[error]An object at the specified path C:\Users\OD42B~1.BOR does not exist

Type: Bug
Enter Task Name: PublishSymbols@2
Environment

Server - Azure Pipelines

account name, team project name, build definition name/build number:
quipu-cwnet
packages
impersonationRemoting/#20190604.6

Agent - Private:

WindowsServer 2012 R2 6.3 build 9600
agent version 2.150.3
Issue Description
PublishSymbols task fails to execute with error

[error]An object at the specified path C:\Users\OD42B~1.BOR does not exist

when private agent is running as windows service using o.borodai domain user. I've tried to understand how OD42B~1.BOR is related to the o.borodai but failed.
Task works fine if network service account is used to run Azure pipelines agent service
Task logs
logs_1406.zip
Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Error logs
##[debug]Asserting leaf path exists: 'E:\buildagent\network\_work\_tasks\PublishSymbols_0675668a-7bba-4ccb-901d-5ad6554ca653\2.0.14\pdbstr.exe'
##[debug]Leaving Invoke-IndexSources.
##[debug]Leaving E:\buildagent\network\_work\_tasks\PublishSymbols_0675668a-7bba-4ccb-901d-5ad6554ca653\2.0.14\PublishSymbols.ps1.
##[debug]Caught exception from task script.
##[debug]Error record:
##[debug]Push-Location : An object at the specified path C:\Users\OD42B~1.BOR does not exist.
##[debug]At E:\buildagent\network\_work\_tasks\PublishSymbols_0675668a-7bba-4ccb-901d-5ad6554ca653\2.0.14\IndexHelpers\IndexFunctions.ps1:26 char:9
##[debug]+         Push-Location $env:TEMP
##[debug]+         ~~~~~~~~~~~~~~~~~~~~~~~
##[debug]    + CategoryInfo          : InvalidArgument: (:) [Push-Location], PSArgument    Exception
##[debug]    + FullyQualifiedErrorId : Argument,Microsoft.PowerShell.Commands.PushLocat    ionCommand
##[debug]
##[debug]Script stack trace:
##[debug]at Invoke-IndexSources, E:\buildagent\network\_work\_tasks\PublishSymbols_0675668a-7bba-4ccb-901d-5ad6554ca653\2.0.14\IndexHelpers\IndexFunctions.ps1: line 26
##[debug]at <ScriptBlock>, E:\buildagent\network\_work\_tasks\PublishSymbols_0675668a-7bba-4ccb-901d-5ad6554ca653\2.0.14\PublishSymbols.ps1: line 138
##[debug]at <ScriptBlock>, <No file>: line 1
##[debug]at <ScriptBlock>, <No file>: line 22
##[debug]at <ScriptBlock>, <No file>: line 18
##[debug]at <ScriptBlock>, <No file>: line 1
##[debug]Exception:
##[debug]System.Management.Automation.PSArgumentException: An object at the specified path C:\Users\OD42B~1.BOR does not exist.
##[debug]   at Microsoft.PowerShell.Commands.FileSystemProvider.NormalizeThePath(String basepath, Stack`1 tokenizedPathStack)
##[debug]   at Microsoft.PowerShell.Commands.FileSystemProvider.NormalizeRelativePathHelper(String path, String basePath)
##[error]An object at the specified path C:\Users\OD42B~1.BOR does not exist.

I have the exact same problem

We have the same issue here and some colleagues now wasted some hours as the other guy. So I analyzed it a bit and it was not very hard to find this tricky thing. So please MS fix it!
The good thing is, that it is easy to fix. But first some words about the issue:
The issue in Windows
It seems there is some "hack" in Windows, which fills the TEMP and TMP variable in some cases with the short name representation of the real path. So if you set TEMP to "%USERPROFILE%\AppData\Local\Temp" and USERPROFILE does contains some special characters, you get the shortened "8.3" path. Just check this out: Why does the system convert TEMP to a short file name?
Sample:
TEMP="c:\users\john" -> no issue
TEMP="c:\users\johnma.yer" -> no issue
TEMP="c:\users\johnm.ayer" -> "C:\Users\JOHNM~1.AYE" -> boom
The issue in PowerShell
As with most errors, there are several problems that make them appear:
PowerShell seems to unable to switch into a directory if the name is the shortened one.
If you try to Set-Location on such a shortened path, you get this object does not exist error, which is not the same as "path does not exist":

Here is the unfixed and also unanswered PowerShell issues:

https://github.com/PowerShell/PowerShell/issues/7760
https://github.com/PowerShell/PowerShell/issues/2369

The issue in PublishSymbols
So as TEMP and TMP are no safe variables in PowerShell, they shouldn't be used in any PowerShell Code. So this is the top layer issue here, even if the two other issue layers are also all in Microsoft components.
So this and all usages of TEMP and TMP has to changed:
https://github.com/microsoft/azure-pipelines-tasks/blob/e2387b5ec14e14fcfa487800420193e0b20b3f17/Tasks/PublishSymbolsV2/IndexHelpers/IndexFunctions.ps1#L26.
To one of the known working variants:

Set-Location ([System.IO.FileInfo]$env:TEMP).FullName
or much better: Set-Location ([System.IO.Path]::GetTempPath())

Can you recheck this issue please @bryanmacfarlane @aasim @damccorm ?

We have the same issue here and some colleagues now wasted some hours as the other guy. So I analyzed it a bit and it was not very hard to find this tricky thing. So please MS fix it!
The good thing is, that it is easy to fix. But first some words about the issue:
The issue in Windows
It seems there is some "hack" in Windows, which fills the TEMP and TMP variable in some cases with the short name representation of the real path. So if you set TEMP to "%USERPROFILE%\AppData\Local\Temp" and USERPROFILE does contains some special characters, you get the shortened "8.3" path. Just check this out: Why does the system convert TEMP to a short file name?
Sample:
TEMP="c:\users\john" -> no issue
TEMP="c:\users\johnma.yer" -> no issue
TEMP="c:\users\johnm.ayer" -> "C:\Users\JOHNM~1.AYE" -> boom
The issue in PowerShell
As with most errors, there are several problems that make them appear:
PowerShell seems to unable to switch into a directory if the name is the shortened one.
If you try to Set-Location on such a shortened path, you get this object does not exist error, which is not the same as "path does not exist":

Here is the unfixed and also unanswered PowerShell issues:

https://github.com/PowerShell/PowerShell/issues/7760
https://github.com/PowerShell/PowerShell/issues/2369

The issue in PublishSymbols
So as TEMP and TMP are no safe variables in PowerShell, they shouldn't be used in any PowerShell Code. So this is the top layer issue here, even if the two other issue layers are also all in Microsoft components.
So this and all usages of TEMP and TMP has to changed:
https://github.com/microsoft/azure-pipelines-tasks/blob/e2387b5ec14e14fcfa487800420193e0b20b3f17/Tasks/PublishSymbolsV2/IndexHelpers/IndexFunctions.ps1#L26.
To one of the known working variants:

Set-Location ([System.IO.FileInfo]$env:TEMP).FullName
or much better: Set-Location ([System.IO.Path]::GetTempPath())

Can you recheck this issue please @bryanmacfarlane @aasim @damccorm ?

I found a workaround, set indexSources to false.

I found a workaround, set indexSources to false.

I found a workaround, set indexSources to false.
Thanks for this hint!
It is of course also possible to set TEMP for the user running the agent to a value that is not affected by this issue, like "c:\user-temp" or something.

I found a workaround, set indexSources to false.
Thanks for this hint!
It is of course also possible to set TEMP for the user running the agent to a value that is not affected by this issue, like "c:\user-temp" or something.

I found a workaround, set indexSources to false.

Thanks for this hint!
It is of course also possible to set TEMP for the user running the agent to a value that is not affected by this issue, like "c:\user-temp" or something.

Thanks, this might be easier then my existing workaround 😀

I found a workaround, set indexSources to false.

Thanks for this hint!
It is of course also possible to set TEMP for the user running the agent to a value that is not affected by this issue, like "c:\user-temp" or something.

Thanks, this might be easier then my existing workaround 😀


Roadmap for allowing browser-embedded nodes to connect to Substrate networks

[ ] Make it possible for node operators to attach a TLS certificate to their libp2p WebSocket server. I believe this can be done by adding a CLI option that passes a certificate.

[ ] Add a WebRTC "client" in nodes.

[ ] Implement some mechanism for nodes to relay WebRTC SDPs, in other words to act as signalling servers. This point is quite vague and needs more thoughts.

cc https://github.com/libp2p/rust-libp2p/issues/1066

Make it possible for node operators to attach a TLS certificate to their libp2p WebSocket server. I believe this can be done by adding a CLI option that passes a certificate.

I put that item here because it believe that it isn't super hard to implement.
However, it might be easier to just ask people to put a reverse proxy with a certificate in front of the WebSocket endpoint.

Make it possible for node operators to attach a TLS certificate to their libp2p WebSocket server. I believe this can be done by adding a CLI option that passes a certificate.

I put that item here because it believe that it isn't super hard to implement.
However, it might be easier to just ask people to put a reverse proxy with a certificate in front of the WebSocket endpoint.

For the first bullet point, a "quick fix" could be to use a reverse proxy (like nginx or something) that handles the SSL cert. Would definitely be nice to support within the libp2p library, though.

Make it possible for node operators to attach a TLS certificate to their libp2p WebSocket server. I believe this can be done by adding a CLI option that passes a certificate.

I put that item here because it believe that it isn't super hard to implement.
However, it might be easier to just ask people to put a reverse proxy with a certificate in front of the WebSocket endpoint.

You could also do what js-libp2p does in their docs for the webrtc-star server, and provide a docker-compose file to automatically setup the reverse proxy. The nice thing is this way letsencrypt can be automated with this as well.

However, it might be easier to just ask people to put a reverse proxy with a certificate in front of the WebSocket endpoint.

We're already doing this for parity nodes, so this is probably the way to go instead of modifying the client.

Since this issue has been opened, we realized that in the context of a browser extension, non-secure WebSockets worked.
Therefore, another thing that can be considered is that nodes that nodes all listen through the WebSocket protocol by default, unless for example they were started using --validator.

Issue still relevant and important.

cc https://github.com/libp2p/specs/issues/220

https://github.com/ipfs/js-ipfs/issues/611 seems like a big problem.

Progress is done here:
https://github.com/libp2p/rust-libp2p/pull/2622
https://github.com/libp2p/specs/pull/412

Would be closed by https://github.com/paritytech/substrate/pull/12529


Breaking API change: Update validation report information to be in report key

Breaking API change: Update validation report information to be in report key of the response object. This object identifies the subject node of validation, whether it is valid, error counts, messages, and if an assertion where recipient_profile is used, the confirmed identity of the recipient.
"report": {
        "validationSubject": "https://api.badgr.io/public/assertions/1c6de3ea-b91e-4ce8-b735-ca9504faafe5",
        "valid": true,
        "messages": [],
        "errorCount": 0,
        "warningCount": 0
    }

With recipient_profile:
"report": {
        "validationSubject": "https://api.badgr.io/share/badge/e5f0c5f1-efb2-4ecb-90d4-cf7fc8131e5e?v=2_0",
        "valid": true,
        "messages": [],
        "errorCount": 0,
        "warningCount": 0,
        "recipientProfile": {"email"<EMAIL_ADDRESS>    }

Previously, validationSubject and recipientProfile did not exist, and valid, messages, errorCount and warningCount were top-level properties of the API response.

LGTM


fs: littlefs: Use flash map macro in automount

To identify the flash partition id, the macro DT_FIXED_PARTITION_ID()
was used previously.
Now the macro FLASH_AREA_ID() is used. This also supports the usecase
when a different flash map implementation is used that redefines the
macros in include/storage/flash_map.h, e.g. the nordic partition
manager.
Signed-off-by: Christian Taedcke<EMAIL_ADDRESS>

@nvlsianpu ping


NIFI-11319 Fixing TestStatelessBootstrap::testClassNotAllowed

The test was broken. My understaning is that the test supposed to ensure that classes from outside the allowed NAR directory should not be loaded expect if they are part of a spefic set of classes (java or jdk "modules"). The original configuration of the test used the "target" folder as NAR folder, which contained every product of the build, including the class "org.apache.nifi.stateless.bootstrap.RunStatelessFlow" used as test subject, thus the class was loaded and the test broke. I narrowed down the "nar folder" in order to not have the test subject included, this I would believe solves the issue.
NIFI-11319
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[x] Apache NiFi Jira issue created

Pull Request Tracking

[x] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[x] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[x] Pull Request based on current revision of the main branch
[x] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[x] Build completed using mvn clean install -P contrib-check

[x] JDK 11
[ ] JDK 17

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

Thanks @simonbence that does appear to address the issue. Interestingly, I didn't encounter it when running mvn clean install because all of the directories under target hadn't yet been completed when that runs. But if you run in the IDE after building or run via mvn test after the build has completed, it failed. +1 merged to main and support branches


Founders tile animations

from Brad:
Founders tile animation - love the slight scale as hover state on the tiles. 2 small things. Is it possible to separate the logo from the image so JUST the image scales but the logo stays in place?
Second request... we have a nice ease into the scale on hover. However we have a hard snap back to neutral when the user scrolls away. Is it possible to animate the scale back to neutral the same way it animates in?
Just noticed, you have it perfect on the buttons. Is it possible to duplicate what you did there?

@bradneal77 I got just the background image to scale, but I'm having trouble figuring out why the ease transition isn't working. I'll keep at this one and keep the issue open.

I like the second option with the bleed thru of the home page hero image. Thanks, Brad!


Do not parse empty strings for inputs of type number

This is migrating a fix done by @jayphelps for v6 to v5.

Hi, @erikras,
Could you take a look at this one when you get a chance? It's a fix that's already in v6, just need it in v5. Could I also trouble you to publish a new version of v5 when you get a chance?
Cheers, Jeremy.

Published in v5.3.4.


Memory leak when using EF + Oracle: OracleParameter(s) are not disposed

Greetings.
Faced with memory leak when using EF + Oracle.
My Windows Service application consumes about 7 GB memory after 1 day of uptime. There's no unusual activity - service just polls database for some data.
Memory dump from production machine looks like this:

Analysis shows, that every OracleConnection and OracleCommand are disposed, but OracleParameters are not. Yes, they (Oracle) had implemented IDisposable in OracleParameter.
After some googling and EF source code research, think I found the reason:

EF creates DbConnection and DbCommand (Oracle ones in my case) using factory approach.
EF executes command and calls Dispose for command, but not for command parameters, because it doesn't know that particular DbParameter implementation is IDisposable.
Disposing of context disposes connection.
Since OracleParameter.Dispose had not been called, parameters hold their resources. Looks like they hold references to command and connection.

This confirms my conclusion:
https://blogs.msdn.microsoft.com/asiatech/2013/01/14/case-study-net-memory-leakage-caused-by-un-released-oracle-objects/
The same service works fine with SQL Server and PostgreSQL.
Is it possible to add this workaround before disposing command:
            foreach (var parameter in command.Parameters)
            {
                (parameter as IDisposable)?.Dispose();
            }

            command.Parameters.Clear();
            command.Dispose();

?
EF version: 6.2.0
Database Provider: Oracle.ManagedDataAccess.Client/Oracle.ManagedDataAccess.EntityFramework, 12.2.1100
Operating system: Windows 10
IDE: Visual Studio 2017 15.7.5

Ooops. OracleCommand are not disposed too. But DbCommand implements IDisposable, at it must be called. What's happening? Does EF ever call DbCommand.Dispose?

See #639


Very slow qt widgets in Spyder ipython console.

Problem Description
Today, after a conda update (and then a full Anaconda reinstall), my QtWidgets have become incredibly slow when launching from the spyder ipython console, taking a second or so to update when I click around. This never was the case before. By contrast, executing in a system terminal with the default python (or ipython), it is all the usual level of fast and responsive.
What steps reproduce the problem?
Here is a test script (sorry, it relies on pyqtgraph to get at the Qt library):
# Try to run the qt magic to take it out of inline mode.
# This is required to avoid a freeze.
try:    get_ipython().run_line_magic('matplotlib', 'qt')
except: pass # Non-ipython terminal.

# Get access to Qt4 API (this is still a Qt5 environment, however)
import pyqtgraph
QtGui  = pyqtgraph.QtGui
QtCore = pyqtgraph.QtCore

# Make the app.
_qtapp = QtCore.QCoreApplication.instance()
if not _qtapp: _qtapp = QtGui.QApplication([])

# Create a button and show it.
b = QtGui.QPushButton('Pants')
b.setGeometry(0,0,200,200)
b.show()

What is the expected output? What do you see instead?
This pops up a button as usual, but when I click and hold, it takes a second or so to switch to the "pushed" state, and then another second to "unpush" after I've released my mouse button. Again, in other terminals outside of spyder (and before today), the response is immediate.
Paste Traceback/Error Below (if applicable)
No errors in traceback.
 Please update to ipykernel 5.3.4, that should fix your error.
If that version is not yet available in Anaconda, please downgrade to 5.3.2.

Perfect, thank you! I hope someone is leaning on Conda about this...


Command "go mod vendor" does not include files tagged as "ignored" on 1.13

I'd like to bring to attention that beginning with Go 1.13, the command go mod vendor will not include files tagged with // +build ignore so Prometheus CI script will fail with vendor dirs created with this version.
More info on https://github.com/golang/go/issues/33353 and https://github.com/golang/go/issues/31088.

Thanks for the heads-up! I'm keeping the issue to remember that CONTRIBUTING.md would need an update when switching to Go 1.13.

Yes, I've been fighting this over last days (as you've been following too) where my Go 1.13 (the version that works on Risc-V) generates the vendor dir with some files that are ignored and was failing the CI.

@simonpasquier Is this still relevant?


Link Members Connected Studies Badges to Projects

It would be really useful if the badges on the Open Humans Profile Page shoudl link to the projects:
https://www.openhumans.org/members/
https://www.openhumans.org/member/me/
https://www.openhumans.org/member/mpball/
A lot of people will not no what the moves, fitbit, runkeeper icons, mean, and those are the obvious ones.
It would be wonderful if it linked the badges to the study info pages. It might create a few more ms of  overhead on the server, but the site is wicked quick!
Change:

To:

This doesn't quite work because of the floating bar at the top, so with a px offset it should be fine.

ah, i like this idea!

I like this too -- I think there's a bigger/better solution needed though.
Regarding popping up under the navbar -- I believe the .anchor-tag class on a div corrects this problem.
But I think an underlying problem may we haven't created a good view for "details about a project", period. Just the activities list on the home page.
Option 1: We could add a URL that shows an information page for each project -- easy solution, easy to link? But clunky user experience for exploring project info?
Option 2: Anchor link shennanigans that jump the user to the home page, jumps to the project, and opens the project details as a pop-up. (Looks cool? Needs JS magic. Maybe annoying to not have a simple link.) c.f. https://github.com/PersonalGenomesOrg/open-humans/blob/master/static/js/lib/nav-pills.js


Command to select an executable to profile?

Is there a way to select an executable file to profile? I only see the list of already running apps in DrDotnet to choose from.

Hi @vborovikov ,
No this is not possible currently but it would be a good addition for the DrDotnet desktop version 🙂


bg_BG  plugins

bg_BG  plugins

Hi, you don't need to translate the version, author, just name and description.

Sorry , I will fix this immediately,

-------- Оригинално писмо --------
От: Diego Najar<EMAIL_ADDRESS>Относно: Re: [bludit] bg_BG plugins (#128)
До: dignajar/bludit<EMAIL_ADDRESS>Изпратено на: 24.11.2015 01:01

Hi, you don't need to translate the version, author, just name and description.
—Reply to this email directly or view it on GitHub.


Update tapir-cats, tapir-json-circe to 0.18.0-M17

Updates

com.softwaremill.sttp.tapir:tapir-cats
com.softwaremill.sttp.tapir:tapir-json-circe

from 0.18.0-M15 to 0.18.0-M17.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.softwaremill.sttp.tapir" } ]

labels: library-update, semver-pre-release

Superseded by #8.


Possibility to have Global Teams/Groups

Gitea version (or commit ref): 1a83581
Git version:
Operating system: Linux
Database (use [x]):

[ ] PostgreSQL
[x] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[x] Not relevant

Log gist:

Description
Right now is not possible to create global group/teams that can be assigned to Organisation as collaborator.
Use case: We are a small  team (< 15 dev) and we do many project with many company. What we want is to have a way of creating a dev team (all of us) and have all the permission over all organisation/Repository.  After that we want to have other people (like our customer) having access to only their Organisation
Currently, we need to assign all the dev to all the Organisation manually. Since we want to use LDAP for authentification, it will be good to have also the possibility to reuse specific groups from LDAP. Imagine we have a LDAP group GIT_DEV that can be use in the collaborator of an organisation.
Screenshots

I'd also like to be able to flick a switch and have a team be able to access all repos of an organisation.

Any news on this?

anyone have a workaround? Or looked into auto adding new users to groups via some hook?

anyone have a workaround? Or looked into auto adding new users to groups via some hook?

Yes, this is currently discussed at #1395 -> until Gitea data model changes (no plans yet?), one needs to use a tool like gitea-group-sync which automates mapping LDAP / AD groups to Gitea's existing teams within organizations.

[ ] In admin panel, there is a group management UI to add/remove users to the group. There is an internal group which is owners. All the members of this group have the permissions to manage the Gitea instance.
[ ] Group could be mapping with a LDAP/AD group, so that the members will be sync in background
[ ] A group could be added to some organization as a special team. you cannot change the team's members because it will be synced with some group. But you can change the permissions of the team like a normal team.
[ ] There is a config item that mapping all organization's owner team as global group

This feature would also be super important to me - organizations are currently the only way to "group repositories" and separate teams have to be created for each of these organizations (and managed/updated).

I would also love to have this feature available in Gitea!
Is there a way to upvote this FR?

I would also love to have this feature available in Gitea! Is there a way to upvote this FR?

Add reactions. :) Answer your questions in 23262, I think yes. I would like this could be implemented. Maybe we need a proposal about how to implement it since we already have organization teams.

Any actions / new about this?
I would be glad to have a way to administrate owners of organizations by a global team management. Especially when new people join the team or someone leaves the team, a central / global team would be very helpful.

A workaround that I implemented on my instance is to sync the teams using a script:
#!/usr/bin/python3

import gitea

class GiTeamSync:
    def __init__(self, url: str, token: str):
        self.g = gitea.Gitea(url, token)
        print("Gitea Version: " + self.g.get_version())
        print("API-Token belongs to user: " + self.g.get_user().username)

    def get_team(self, org: str, team: str):
        return gitea.Organization.request(self.g, org).get_team(team)

    def sync_members(self, source_team: gitea.Team, target_team: gitea.Team):
        source_members = set(source_team.get_members())
        target_members = set(target_team.get_members())

        inter = source_members & target_members
        to_be_removed = target_members - inter
        to_be_added = source_members - inter

        print(f"Syncing members from {source_team.organization.name}/{source_team.name} to "
              f"{target_team.organization.name}/{target_team.name}")

        print("Users removed:")
        for u in to_be_removed:
            print(f"- {u.login}")
            target_team.remove_team_member(u.username)

        print("Users to added:")
        for u in to_be_added:
            print(f"- {u.login}")
            target_team.add_user(u)

gs = GiTeamSync("https://my.url", "TOKEN123")
gs.sync_members(gs.get_team("org_a", "Owners"), gs.get_team("org_b", "team_123"))

@denyskon as mentioned in #27194
See also: #4822 #23262

I think actually the only thing needed here would be allowing to access org teams from outside the organization, which might actually be not that hard to implement....


Slow when creating font for the first time

I was looking for the reason why creating a pdf is so slow and I was able to find out that creating a font,
(e.g. PDFont font = PDType1Font.HELVETICA) is very slow. In my case it takes about 10s. I might pull request sometime in the future for the fix of this one (if it is possible and if  I can do it), but right now I'm very busy and can't help. Just letting you know.
Btw, after creating the first pdf, the following pdf creations are fast.

This is because the first time a font is created, a lot of IO is done to initialize some of the font classes. If you haven't already, try adding PDFBoxResourceLoader.init(getApplicationContext()); to your application somewhere before you start using PDFBox classes. This allows the library to use Android style asset loading instead of Java resource loading, which is much slower.


NODEJS_14_X is deprecated and causes an error on cdk deploy

Describe the bug
NODEJS_14_X is deprecated and causes an error on cdk deploy. This version is no longer supported by AWS, therefore the version should be upgraded to at least NODEJS_16.
Expected Behavior
Successful cdk deploy
Current Behavior
"The runtime parameter of nodejs14.x is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs20.x) while creating or updating functions.
Reproduction Steps
line 24 of hitcounter.ts runtime: lambda.Runtime.NODEJS_14_X,
When running cdk deploy the error for unsupported runtime in AWS Lambda functions is received.
Possible Solution
Replace:
runtime: lambda.Runtime.NODEJS_14_X,
with:
runtime: lambda.Runtime.NODEJS_16_X,
Additional Information/Context
No response
CDK CLI Version
No response
Section
No response
Browser
No response
Language
No response

@GryphonInfoSec Thanks for reporting the issue. I do recognize that Node 14.x is deprecated and we should fix the code example to change it to Node 16.x (at least). Thanks for the PR. Could you update your you PR to make similar change for Japanese version here.
Thanks,
Ashish

The NODEJS_16_X is also deprecated as of Jun 12, 2024 and function create will be blocked on Feb 28, 2025.
Lambda Supported Runtimes https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
It should be updated to NODEJS_20_X


Another exception for disclaimer

they all seemed to download fine except for Civilization V, which just made an empty directory.

Endless Space behaves like this too. The download it trigger is empty (0 bytes/0 bytes downloaded) and the empty game directory. Let me know if I can help test/debug anything.

Another one: You Need A  Budget 4. Is there any workaround to this at all?

Civ5's appmanifest file is huge, I don't know what fields are required (Them folks at valve aren't exactly open about their software or api's); I would suggest taking the time to install the game manually and close steam right after, send us the manifest file it made before it finished downloading the game; Or if that doesn't expose exactly what steam needs (It probably won't) then I'd try, copying its manifest file (appmanifest_8930.acf), delete (or move) the game, make the appmanifest file with the tool, and then add properties from your real manifest file one at a time till you find exactly what Civ5 (Or whatever game) needs, and get back to us with that; then we can add exceptions so the tool can do it automatically.

Here's a chuck of my real manifest file, I've censored some fields as I don't know if these values are sensitive or not.
"AppState"
{
	"appID"		"8930"
	"Universe"		"1"
	"name"		"Sid Meier's Civilization V"
	"StateFlags"		"4"
	"installdir"		"Sid Meier's Civilization V"
	"LastUpdated"		"1450075528"
	"UpdateResult"		"0"
	"SizeOnDisk"		"6495012312"
	"buildid"		"656471"
	"LastOwner"		"******"
	"BytesToDownload"		"4549276368"
	"BytesDownloaded"		"4549276368"
	"AutoUpdateBehavior"		"0"
	"AllowOtherDownloadsWhileRunning"		"0"
	"UserConfig"
	{
		"Language"		"english"
	}
	"MountedDepots"
	{
		****
	}
	"DlcDownloads"
	{
		****
	}
}

Steam online operations uses your Steam ID 64 among others which, while unique to you, is very easy to find and not supposed to be private anyway. If you don't have a custom profile URL, it even shows up as a default URL for your profile. LastOwner is that Steam ID.
MountedDepots is probably why the games like Civ V and EndlessSpace creates empty folders, because that's usually what Steam uses to know which language and game files to download.
You can see which depot a game has using SteamDB ( https://steamdb.info/app/8930/depots/ ) and figure out most of the needed depots from there.

Where DO i paste it

Sorry for taking so long responding, I didn't look at Github in a while. Once you figured out the depots you need from the SteamDB page, you should past their IDs under "MountedDepots".
It has a format of "Depot ID" "Manifest ID".
For example, XCOM 2's first DLC (Anarchy's Children) goes like this:
"433080" "7321043562789976597"
Which is valid compared to this page (at date of writing this): https://steamdb.info/depot/433080/


error: failed to run custom build command for tikv-jemalloc-sys v0.4.3+5.2.1-patched.2

Is there an existing issue?

[X] I have searched the existing issues

Experiencing problems? Have you tried our Stack Exchange first?

[X] This is not a support question.

Bug report for compiling, code snippets, templates, etc.
Hi. need help. I am on :
Intel core i5 CPU M430 2.27GHz x4
Memory: 3.6 GiB
graphic card NVA5 intel HD Graphics (ILK)
Ubutu 20.04.4 LTS --- 64 bit --- gnome 3.36.8
Tutoriel
Prepare a Substrate node using the node template
cargo build --release
I have the following error:
Compiling rand_distr v0.4.3
error: failed to run custom build command for tikv-jemalloc-sys v0.4.3+5.2.1-patched.2
Caused by:
process didn't exit successfully: /home/nonqme/substrate-node-template/target/release/build/tikv-jemalloc-sys-b789a6952291f23f/build-script-build (exit status: 101)
--- stdout
TARGET=x86_64-unknown-linux-gnu
HOST=x86_64-unknown-linux-gnu
NUM_JOBS=4
OUT_DIR="/home/nonqme/substrate-node-template/target/release/build/tikv-jemalloc-sys-68994192a6deaa0f/out"
BUILD_DIR="/home/nonqme/substrate-node-template/target/release/build/tikv-jemalloc-sys-68994192a6deaa0f/out/build"
SRC_DIR="/home/nonqme/.cargo/registry/src/github.com-1ecc6299db9ec823/tikv-jemalloc-sys-0.4.3+5.2.1-patched.2"
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
CFLAGS_x86_64-unknown-linux-gnu=None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
CC="cc"
CFLAGS="-O3 -ffunction-sections -fdata-sections -fPIC -m64 -Wall"
JEMALLOC_REPO_DIR="jemalloc"
running: "sh" "/home/nonqme/substrate-node-template/target/release/build/tikv-jemalloc-sys-68994192a6deaa0f/out/build/configure" "--disable-cxx" "--with-private -namespace=rjem" "--host=x86_64-unknown-linux-gnu" "--build=x86_64-unknown-linux-gnu" "--prefix=/home/nonqme/substrate-node-template/target/release /build/tikv-jemalloc-sys-68994192a6deaa0f/out"
checking for xsltproc... false
Steps to reproduce the problem
Tutoriel
Prepare a Substrate node using the node template
cargo build --release

I think #1081 and #1026 should solve this

Hi all I have followed the instructions within the tutorial and ensured that make is installed; however I continue receiving the error as above:
error: failed to run custom build command for tikv-jemalloc-sys v0.4.3+5.2.1-patched.2
Windows 11 running WSL:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
--- stdout
TARGET=x86_64-unknown-linux-gnu
HOST=x86_64-unknown-linux-gnu
NUM_JOBS=16
OUT_DIR="/mnt/c/Users/anric/substrate-node-template/target/release/build/tikv-jemalloc-sys-589f2417f775c6fa/out"
BUILD_DIR="/mnt/c/Users/anric/substrate-node-template/target/release/build/tikv-jemalloc-sys-589f2417f775c6fa/out/build"
SRC_DIR="/home/anrichp/.cargo/registry/src/github.com-1ecc6299db9ec823/tikv-jemalloc-sys-0.4.3+5.2.1-patched.2"
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
CC="cc"
CFLAGS="-O3 -ffunction-sections -fdata-sections -fPIC -m64 -Wall"
JEMALLOC_REPO_DIR="jemalloc"

i have the exact same problem, tried to fix it with those propositions but it's not working


small visual bug

in chapter 3/3.mdx. There is a misgenerated HTML code block:

This may be resolved by turning this code block
```
trainer.train()
```

into this:
```py
trainer.train()
```

But I don't know how to test this on my machine.

Hello @osbm, this issue has been fixed by https://github.com/huggingface/course/pull/370


Remove testbed port check for binding file

This code is a Contribution to the OpenConfig Feature Profiles project ("Work") made under the Google Software Grant and Corporate Contributor License Agreement ("CLA") and governed by the Apache License 2.0. No other rights or licenses in or to any of Nokia’s intellectual property are granted for any other purpose. This code is provided on an "as is" basis without any warranties of any kind.

Pull Request Functional Test Report for #2015 / f070bbdce3c600547a660d400ec1064eb00f8dc9
No tests identified for validation.
Help

Can you remove the one line in the unit test that checks for this error being removed?  The test was looking for the error message.
https://github.com/openconfig/featureprofiles/blob/d697583000f6714bd91447556e050f3637599655/topologies/binding/binding_test.go#L211
Otherwise this SGTM.

Thanks @bstoll - Just did that, once all the checks are OK I'll merge main into my branch and do another push.


Build page component (About)

This Issue relies on the completion of #2, #3, #4, #6 and #9
Build a replicate of the About section in a component as seen in the website's design prototype

Build and style the section
Fetch media asset from cloudinary

Can you assign this to me? @BolajiAyodeji

Can you assign this to me? @BolajiAyodeji

Yes, please, I just did.
How long will this take you?

24hrs after the completion of #2, #3, #4, #6 and #9

I'm I building a replica of what is above? coz the about section on the prototype isn't displaying

I'm I building a replica of what is above? coz the about section on the prototype isn't displaying

Hello @Olawoore, This issue has been fixed in #56.
My apologies, please check for other issues you'll like to work on.


Issue with icon color

Hi,
I'm having a little issue with the icon's color.
Before a tab bar is selected, its icon's color is wrong.
For example: my TextColor is dark grey, and my TextSelectedColor and IconSelectedColor are white.
Before I select a tab bar it appears with dark grey text (which is correct) and white icon (which is wrong).

Have I configured something wrong? Or is it a bug?
Thanks!

I'm having the same issue 😑

I'm having a similar issue... All by icons are blue and text are black.  when I first run the app the selected icon is black instead of blue.  When I go to another tab then it fixes itself.

I fixed this in my fork and sent a pull request. For the meantime, here it is: https://github.com/aalzanki/animated-tab-bar

There are a lot of changes in your fork that are not needed for this fix.
Here is the fix. Go into RAMAnimatedTabBarController.swift.

Set your deselected color @IBInspectable var textColor = myDeselectedColorWhichIsLikelyAShadeOfGray

Replace the following code:
         let icon = UIImageView(image: item.image)
         icon.translatesAutoresizingMaskIntoConstraints = false
         icon.tintColor = UIColor.clearColor()

With this:
            let renderImage = item.image?.imageWithRenderingMode(.AlwaysTemplate)
            let icon = UIImageView(image: renderImage)
            icon.translatesAutoresizingMaskIntoConstraints = false
            icon.image = renderImage
            icon.tintColor = item.textColor

Actually those change only fixes "rotation item" color. With other type of animations. Its doesn't work :(

This problem is still here

Fixed

i want to change selected image of tab bar item. In fact i had applied tabbar.selected Image Property

I am not getting what people are saying .I have two assets not selected and selected icons for Tab Bar Item Images .I am adding selected image and unselected image to the Tab bar item but after running I am getting selected image colour changing.i checked my assets I don't have that king of blue colour selected asset mine is pink colour

can you help me is anything wrong


Update Faker version

Hi! I see the Faker version is v5 whilst v8 is out already. Are there plans to update the faker version?

Agree, please update


Updated packages

The dependencies are up-to-date (Minor and patch versions).
I've added "types": [] in the tsconfig.json to let it work, or else tsc tried to compile all the types in node_modules/@types.
Let me know what you think
closes #5

@Stukongeluk I didn’t know about types: [], thanks for fixing it!


[PAPRIKA] Host-Guest Free Energy Gradients

Description
This PR adds support for calculating the free energy gradients for host-guest systems with the paprika protocol.
Status

[ ] Ready to go

Question: I noticed that the code in _evaluate_energies converts the potential energy to pint units but keeps it in kilojoule/mol. So the gradient I'm getting for energy parameters like LJ_epsilon becomes kilojoule/kilocalorie. What is the reason for keeping the potential energy in kilojoule/mol?
https://github.com/openforcefield/openff-evaluator/blob/ef37ac8231e07f24dc97a3fba9a4ef03ed8651b8/openff/evaluator/protocols/openmm.py#L117-L122

Question: I noticed that the code in _evaluate_energies converts the potential energy to pint units but keeps it in kilojoule/mol. So the gradient I'm getting for energy parameters like LJ_epsilon becomes kilojoule/kilocalorie. What is the reason for keeping the potential energy in kilojoule/mol?
https://github.com/openforcefield/openff-evaluator/blob/ef37ac8231e07f24dc97a3fba9a4ef03ed8651b8/openff/evaluator/protocols/openmm.py#L117-L122

Thanks @jeff231li! I've added this to my queue to review within the next week or so. Please ping if you require it more urgently!

@SimonBoothroyd I'm reviewing this PR this week; are there any areas of concern preventing merge? I'll prioritize getting the CI to pass first.

@SimonBoothroyd I'm reviewing this PR this week; are there any areas of concern preventing merge? I'll prioritize getting the CI to pass first.

I think the implementation looks like a great first pass, but currently contains large areas where code either duplicates already present functionalities (ComputePotentialEnergyGradient) or which can be significantly reduced (the schema definitions) which will take some careful thinking through.
Unfortunately I don't have the time at the moment to look over this, and suggest the changes which will be required to be merge, so it may be some time before this will be able to be merged - my sincerest apologies for this delay!


Release 0.10?

We're currently using Mongobee 0.9. That version still eats most of the exceptions, making debugging of problems on remote systems hard for us. We would like to move to a version that passes the cause of the exceptions, like the current Mongobee git master does. When do you plan on releasing the next version of Mongobee, including the exception changes?

Hi, Actually this version is ready to release so I will do it today. Thanks for the reminder :)

That's great news, thanks!

released


Loop Video in toro v3

How can I make the video play again once it is finished? Is available some function like
@Override
public Target getNextTarget () {
     Return Target.THIS_PLAYER;
   }
  Available in version 2.3.0? I'm using SimpleExoPlayerViewHolder.java

@davisoqt v3 will closely tight to ExoPlayer 2, you can use its official LoopingMediaSource for loop playback.


Workflow prototype

Add prototype workflows.
The main dc6 argo workflow runs on cleaned input from a single GCM, SSP, for "tasmax". It currently fails on the final downscale step.
Cleaned data are required input to the main dc6 workflow and so additional workflows clean CMIP6 and ERA-5:

The cleaning CMIP6 workflow is in dev. It gets cleaned output for "tasmin", "tasmax", "pr", and a rough "dtr". This requires raw CMIP6 data downloaded to co-located storage via workflows/downloadraw-workflow.yaml.
The ERA-5 workflow is a mess of old, minimum changes required to get old, preliminary reanalysis data clean enough to input to the main dc6 workflow.

The dodola:dev image in the workflow refers to the development image between v0.2.0 and v0.3.0. Other images used should be tagged with the appropriate version.
Close #91.

@dgergel You wanted the workflow-prototype branch merged into master?

@brews ready to merge! Changes needed are all documented in issues


Fix create-element-to-jsx failing when props argument is missing

React.createElement(Component), without the second props argument,
was failing with TypeError: Cannot read property 'type' of undefined
error. As React.createElement() allows emitting the props argument
altogether as far as I have understood, the codemod shouldn't fail when
props argument is indeed missing.

Thank you! :)


doc: improve instanceRole, podRole and role definitions

Closes: #4241

/ok-to-merge documentation


[SPARK-30225][core] Correct read() behavior past EOF in NioBufferedFileInputStream

This bug manifested itself when another stream would potentially make a call
to NioBufferedFileInputStream.read() after it had reached EOF in the wrapped
stream. In that case, the refill() code would clear the output buffer the
first time EOF was found, leaving it in a readable state for subsequent
read() calls. If any of those calls were made, bad data would be returned.
By flipping the buffer before returning, even in the EOF case, you get the
correct behavior in subsequent calls. I picked that approach to avoid keeping
more state in this class, although it means calling the underlying stream
even after EOF (which is fine, but perhaps a little more expensive).
This showed up (at least) when using encryption, because the commons-crypto
StreamInput class does not track EOF internally, leaving it for the wrapped
stream to behave correctly.
Tested with added unit test + slightly modified test case attached to SPARK-18105.

Test build #116056 has started for PR 27084 at commit 0489fca.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20848/
Test PASSed.

Test build #116056 has finished for PR 27084 at commit 0489fca.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116056/
Test FAILed.

retest this please

Test build #116061 has started for PR 27084 at commit 0489fca.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20853/
Test PASSed.

Test build #116061 has finished for PR 27084 at commit 0489fca.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116061/
Test PASSed.

Merged to master and branch-2.4.