Improve spec/validation of participants.tsv

TLDR
In the participants.tsv file, the age and sex columns are sometimes not well defined, and this leads to (unnecessary) issues on the side of tool developers (and thus eventually the users). We should improve either the spec or the validator, or both.
cc @jasmainak @agramfort @adam2392 @hoechenberger
came up in: https://github.com/mne-tools/mne-bids/issues/396
Intro
The specification says the following about the participants.tsv file:

In case of single session studies this file has one compulsory column participant_id that consists of sub-, followed by a list of optional columns describing participants.

so strictly speaking, all columns that are not participant_id are OPTIONAL, and thus SHOULD be described in an accompanying participants.json.
For optional columns that are not described, the validator currently emits a warning such as this:
1: [WARN] Tabular file contains custom columns not described in a data dictionary (code: 82 - CUSTOM_COLUMN_WITHOUT_DESCRIPTION)
  ./participants.tsv
    Evidence: Columns: group not defined, please define in: /participants.json

Yet, the validator treats some "optional" columns differently, i.e., these columns are accepted WITHOUT warning. Examples of these are:

age
sex

However, the specification does not cover that these two variables are "expected optional columns". The expected behavior would be to raise a warning also for age and sex.
I could not pin down the exact part of the validator that is responsible for this behavior, but it may be this line:
https://github.com/bids-standard/bids-validator/blob/dfabbfb058daca406ed1d0897c3a25be059a5ad6/bids-validator/utils/summary/collectSubjectMetadata.js#L31
perhaps @nellh or @rwblair can help
The problem
The issue that arises from this (apart from inconsistency) is that users define their own levels for the sex column, and are NOT reminded by the validator to please define their levels further in a participant.json.
As a result, these values are hard (or impossible) to parse by software.
E.g., we may have the following participants.json:
participant_id	age	sex
sub-05	25	fem
sub-06	30	ma
sub-07	26	ma

what's fem? what's ma?
How to fix?
I think we should do one of the following:

fix the validator so that it emits a warning if age and sex are columns in participants.tsv but have no description in an accompanying participants.json

OR

Amend the participants.tsv part of specification and explicitly say that age and sex are "to-be-expected" columns ... and then also define the expected inputs:

age MUST be a float (years since birth)

if a user wants to specify age differently, they must make their own custom column, e.g. age_in_months

sex MUST be a string (here we need to discuss, which strings we accept. Most straight forward would perhaps be "male", "female", ~"undefined"~, "other", but I would like somebody with a bit more experience in inclusive language to make a suggestion here.

again: if a user wants to do their own sex column they can make their own custom column with a wide range of acceptable factor levels

sex MUST be a string (here we need to discuss, which strings we accept. Most straight forward would perhaps be "male, female, undefined", but I would like somebody with a bit more experience in inclusive language to make a suggestion here.

Typically the third option is other or diverse – undefined could probably be perceived as disrespectful by some. diverse could potentially lead to confusion with gender. So I guess, at this time, I'd be in favor of other. The latter would also capture inter, in case we decide not to add that option too.

age MUST be an integer (years since birth)

I don’t understand the prohibition against decimal ages.  It is not uncommon to have exact birthdate and scan date (before anonymisation) with which to compute an age with greater than integer precision.

Thanks @hoechenberger and @nicholst I updated the post with your sensible suggestions

@sappelhoff and @hoechenberger - i would suggest making a clearer difference between sex and gender. in bids, sex was adopted in the early days to be chromosomal, while datasets could easily add columns on gender to include and potentially differentiate preference. the language can definitely be improved to both clarify and provide support for including gender in addition to sex, when such information is collected in a study.
more broadly, #423 has some general conventions about augmenting BIDS keys. my vote would be in preference of the validator suggesting that age/sex have to have corresponding entries in participants.json. to map on to known units and levels (as is done in other parts of bids). this allows greater flexibility downstream, especially if BIDS is used for datasets containing in utero participants or non-human species where other age units are more appropriate.

Hello @satra, thanks for your comment! I fully agree that having a separate gender key could be beneficial in certain situations.
I could imagine sth like:

sex: female, male, other (as suggested above)
gender: female, male, diverse (as a catch-all), or
gender: female, male, trans-female, trans-male, inter, other (or diverse)

I fully agree that having a separate gender key could be beneficial in certain situations.

Currently we are limiting this discussion to sex and age, which have an undocumented "special role" in the validator (see my OP). I'd rather not extend this discussion to gender. If users want to specify gender, they can already do so by adding a column to TSV and describing it in JSON.
Note: for gender, the validator WILL WARN if no description is in JSON ... while for age and sex, it will not.

my vote would be in preference of the validator suggesting that age/sex have to have corresponding entries in participants.json. to map on to known units and levels (as is done in other parts of bids). this allows greater flexibility downstream

If I understand correctly, this preference maps to my suggestion 1 in the "how to fix" part of the original post.
I think that allowing downstream flexibility is a good point.
Another point is, that there are many datasets already out there ... and if we clarify now that we want age to be a float and sex to be one of (female, male, other [or whatever we would decide]) that would break several datasets.
Therefore I think it'd be best to:

Clarify the specification what we mean by sex and age as well as RECOMMEND (i.e., not REQUIRE) factor levels for sex
Amend the bids-validator to explicitly WARN if either age or sex are NOT described in an accompanying JSON file

--> This will not break existing datasets
--> This will hopefully result in more "sane" datasets in the future
HOWEVER
--> existing datasets with age and sex not described in JSON are still "broken" (i.e., cannot be parsed automatically in most cases)
--> users who create new datasets may simply ignore a bids-validator warning ... and hence produce new datasets where age or sex cannot be parsed automatically
I think that's a fair tradeoff and a pragmatic solution.

@sappelhoff - yes i was voting for 1. and my reference to gender was simply that the bids key sex should not be confused with gender.
regarding validator and specification interaction, each dataset has a specification version, so a validator should always be checking against the spec rules of that version.  this means that for datasets released prior to any change we make to the spec, it should simply accept it as valid (may not be useful, but that's not the validator's responsibility). once the age/sex descriptions are upgraded, current validator can conform to it.
the validator should always be able to make good recommendations at any time. and a new validator could always recommend how to upgrade the dataset from version X to version Y of the spec.

@sappelhoff These suggestions sound very sane to me.

hi guys, I would actually suggest going for error. It may sound a bit radical but I think there are parts of the BIDS specification that are too under-specified and flexible which makes it a nightmare to write software for it.
At least regarding the sex/gender issue, I think there is no good reason to leave it as optional. IMO it should be a compulsory field and if the researcher feels really compelled to leave it out, then they should fill it up as 'n/a'. I would also not introduce a new mechanism for unknown/others. We should try to be consistent across the specification.

In that sense, this should be considered a bug-fix ... the datasets that break should be retroactively fixed, otherwise we are stuck with a specification that is too flexible.

I would also not introduce a new mechanism for unknown/others.

But there are cases where the biological does not clearly fall into the male/female dichotomy, and these are different from unknown (which would be n/a to me). Designing a sex key limited to only two genders would be a little like having a handedness key without the option for left-handed participants…

yes sure! Sorry I read too quickly, it's fair to have options for sex/genders other than male and female of course.

For some context presently in bids-examples the following show up in the sex field:

D
f
F
Female
m
M
Male
n/a

ds102 being the sole dataset using D.
Ages are mostly numeric. The outliers being ieeg_epilepsy and ieeg_epilepsy_ecog which have XX as the age for their sole subject. genetics_ukbb uses 89+ for 4 subjects.

Also 6 datasets don't have an age column. 8 don't have a sex column, 2 of which use gender instead. Both the datasets using gender have participants.json defining the column. Posting one liners I used in case I made a mistake in them:
~/projects/bids-examples $ find . -name participants.tsv -exec grep -lL age {} \; | cut -d '/' -f 2
ieeg_motorMiller2007
ds009
eeg_rest_fmri
ieeg_visual
ds114
eeg_face13

~/projects/bids-examples $ for file in $(find . -name participants.tsv -exec grep -lL sex {} \;); do grep gender -lL $file; done | cut -d '/' -f 2
ieeg_motorMiller2007
ds009
eeg_rest_fmri
ieeg_visual
ds114
eeg_face13

~/projects/bids-examples $ for file in $(find . -name participants.tsv -exec grep -lL sex {} \;); do grep gender -l $file; done | cut -d '/' -f 2
ieeg_filtered_speech
eeg_rishikesh

~/projects/bids-examples $ find . -name participants.tsv -exec grep -lL sex {} \; | cut -d '/' -f 2
ieeg_motorMiller2007
ds009
eeg_rest_fmri
ieeg_filtered_speech
eeg_rishikesh
ieeg_visual
ds114
eeg_face13

Thanks @rwblair that's really helpful. Can you get the same info also for those datasets on OpenNeuro? And would it be possible to share the script you may have used for extraction for us to try as well?
re: @jasmainak's proposal to go one step further and make bids-validator return an error instead of a warning:
If we were to REQUIRE a set of inputs for sex and age, then we would probably be breaking only very few existing datasets if we allowed:

sex [chromosomal, nothing to do with gender]:

for female: female, f, F, FEMALE, Female
for male: male, m, M, MALE, Male
for other: other, o, O, OTHER, Other
n/a (which is always allowed)

age [always in years]:

int
float
n/a (as usual)
we could think about allowing syntax like int-int, e.g., 80-90, reflecting age between 80 and 90 and int+ reflecting "older than" --> but I am not sure whether this is a nice feature or an unnecessary burden

regarding "downstream flexibility" --> if in the future we see more and more need for additional "levels", we could add them. Else, there is once more the option for a custom column.

thanks a lot @rwblair for jumping in. This is great and @sappelhoff's proposal is very concrete and simple. I'm not sure about int+ either but agree with the rest of it.

@sappelhoff

sex [chromosomal, nothing to do with gender]:

Wonder if it shouldn't be "phenotypical" instead of "chromosomal", as there's people with XY chromosomes with a female phenotype, because their androgen receptors are non-functional :-) just my 2 ct

Published OpenNeuro participants.tsv files are a little bit noisier but make a case for a stronger validator warning/error
Outliers in age:
ds002345 has a list of ages for each participant in the age column, like 28,28,28,31,31. No participants.json.
ds002000 has NaN for all its ages.
ds001365 has positive and negative floating point numbers like -0.994. No participants.json, but the next column is mean reaction time, so maybe they forgot to add ages in?
Here are the unique values for sex for every public dataset:
https://gist.github.com/rwblair/b00cf41d6693b65d67e3c8c1fe73608c
ds001510, ds002655, ds000121, ds000119 Seem to have their age and sex columns swapped.
The "" result from ds002221 comes from the file not being properly terminated.
How I copied participants files:
~/datasets/participants $ aws s3 cp s3://openneuro.org . --recursive --exclude "*" --include "*participants.tsv" --no-sign-request

Age values by OpenNeuro dataset:
https://gist.github.com/rwblair/deabd683a4229277dd69aa54f39adb1c

It looks like age and sex were added as non-custom fields in https://github.com/bids-standard/bids-validator/pull/905. I don't see any justification for special status, but I would assume it's because they appear in the example, but as noted above there's no text describing their meaning or restricting their values.
So I agree with @sappelhoff and @satra that we should revert the validator to warning when they do not have metadata in participants.json.

In the long term, I think adding BIDS terms specifying an official default interpretation (#423) is a good idea, and that these terms should be linked to from the spec itself.

I'll go ahead and make a PR to revert those changes.
I was curious to see how many datasets would start issuing warnings, out of 194 datasets that have age in their participants.tsv, age only appears in 48 of their participants.json. For sex the numbers used in participants.tsv are 175 in  and 48 in participants.json. Over all I'm seeing participants.tsv in 244 datasets and participants.json in 66.


Page navigation

What 🕵 🔍
This adds page navigation, a bolt font and moves the folder structure up one level.

How to test 🥼 🔬
Run npm run dev and see if any errors pop up

References 📝 🔗
How to add fonts

looks solid nice font


feat(upload folder): init web and node upload folder with manifest implementations PE-4643

This PR adds a basic uploadFolder method with both Node and Web implementations. For now it is just a wrapper around the uploadFile in a sense. On nodeJS it will send each file found within the folder path provided. On Web we will accept an array of File type provided by the browser. After uploading we optionally create a manifest of the upload.
There are more features we could build onto here in the future including BDI support, multipart-uploads, and ephemeral signing on web to prevent signature authorization of each item.

:tada: This PR is included in version 1.10.0-alpha.1 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.10.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


serde_derive_cfg doesn't seem to work

It seems it's present in rust-protobuf/protobuf-codegen/src/serde.rs in master, but not in version 2.16. It's possible to set it in Customize struct, but it's not used anywhere.

This option was implemented for protobuf itself.
Anyway, it is too late to add this feature to rust-protobuf=2, and rust-protobuf=3 remove serde support, and relies on callbacks provided by users, and users can apply any annotations to generated code.
Also, rust-protobuf 3 has native json support.


Adding CouchDB-Scala to "Projects using http4s"

Hi All,
Would it be possible to add my project (https://github.com/beloglazov/couchdb-scala) to the list of projects using http4s on your web site? :) Thanks!
Cheers,
Anton

Gladly!  Looks like a nice project.

Thanks Ross!


Replaced some raw pointers with smart pointers

I'm not sure about this.
I think that if it's guaranteed that some object won't outlive something else, we don't need std::unique_ptr nor std::shared_ptr. I mean there is nothing in C++ that would be reference counted and non-atomic, so the question is, do we really want to use atomically incremented shared pointers in this project, even when the API around it is not thread safe?
What I mean is that both unique_ptr and shared_ptr are owning. So for example:
minio::creds::StaticProvider provider;
minio::s3::Client client(base_url, &provider);

can be changed to:
minio::creds::StaticProvider provider;
minio::s3::Client client(base_url, provider); // reference instead of pointer.

It would be the same and you won't need a dynamic memory allocation for the provider.
So my question is, do we mind more allocations just to remove * from some functions / members or is it okay to just use a reference?
Because for example in the provider example - do we want to share providers across clients? If so, then unique_ptr is not the right pointer and shared_ptr would be better I think.

Eradicating pointers to class Client and StaticProvider is a good thing, because these classes is passed around in function calls, the StaticProvider is currently not copyable AFAICT. Currently, the solution is to use raw pointer; I don't think it currently triggers bugs, but it is (in the current form) hard to make a class with a client, and have the constructor initialize this client, simply because StaticProvider has no default constructor (or is not copyable, etc). Some simple managed pointers fixes this. @piotr-topnotch what would be the least intrusive update to make Client 'usable'?

As I am not able to understand what bugs we are trying to address here. I am sharing my thoughts WRT current code.
In higher level, Client and BaseClient are not thread-safe, particularly region_map_ and Fetch() method of Provider. WRT StaticProvider is kind of thread-safe i.e. it returns copy of Credentials at the time of Fetch(). Note that null  credential provider is a valid/supported value for anonymous access.
Lifetime of provider has to be maintained along with Client or BaseClient. It is user's responsibility. I do not recommend sharing of Provider with different Client or BaseClient. Each client needs to be created with individual credential providers. Otherwise we need to make Provider thread-safe.


Mldb 1500 order by aggregators

Allow aggregator functions in order by expressions

Very good cleanup in sql_expression.  I feel however that the test coverage is a little thin.  Can you add tests that demonstrate that the ordering is as expected and possible tests that demonstrate that the validation of the order by clause in presence of aggregators is correct.
+1 once that's done


Peers not recognized as a section

v3.5.1
In the config you can define named peers and then have stick tables use the peers list.
If you edit the config then it will add the 'peers' into the previous section as oppose to having it has a separate expandable section:
Peers details:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3.5

Thank you, I will fix it too
Отправлено из Mail.ru для Android понедельник, 07 октября 2019г., 11:37 +03:00 от rabievdm<EMAIL_ADDRESS>:

v3.5.1
In the config you can define named peers and then have stick tables use the peers list.
If you edit the config then it will add the 'peers' into the previous section as oppose to having it has a separate expandable section:
Peers details:
https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#3.5
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly,  view it on GitHub , or  mute the thread .

I fixed it, please update and try

Working


0.8.3

test...

This pull request has not been acted on in over 100 days, and I am not sure what its purpose is.
@kksri19, is this still needed, or can you close this request?

Closed due to inactivity/no JIRA


The front-end controller is not updated to the new version 4 syntax

Route::get('/maintenance-mode', [MaintenanceModeController::class, 'index'])->name('plugrbase.maintenance.index');

Just opened a PR for this https://github.com/plugrbase/statamic-maintenance-mode/pull/3


Encrypted accounts still show a barcode when prompted.

The secret is blank, so there is no "leakage" of information, but it shouldn't be an option of the secret is still encrypted.

thanks, this bug has been fixed in develop branch, and the new version will be available soon.


Expense transactions not featuring original invoice

Describe the bug
When browsing, DEBIT transactions of a Collective, when clicking on Download (pdf), I get an invoice automatically generated by Open Collective, not the original one. Is that expected.
To Reproduce

Go to https://opencollective.com/engineering/transactions
Click on 'debits'
Click on 'view details'
Click on 'Download (pdf)'

Current behavior
Downloading an Open Collective generated invoice.
Expected behavior
Not sure. The original invoice?
Screenshots

This behaviour is indeed confusing and I don't see the need to have generated invoices here, we should definitively surface the invoice submitted by the person who submitted the expense if there's one.


Feature request: Graphviz (DOT) beautifier

Could you please provide a link to the project?

Link to Graphviz project: http://graphviz.org/


Change BanStick to be built automatically instead of manually

This is an example of how the rest of the plugins may be
automatically built from source. This approach means plugins do not
need to be manually compiled, and the plugin jars do not need to be
checked in to version control.
Updating plugins can also be automated by bumping the version and
SHA256 hash to the newer version. It is important to check the
SHA256 hash in order to verify the integrity of the source code
matches the tag we are trying to build.
Unforutunately many of the plugins don't seem to provide binaries,
so we must build the plugin jar ourselves. This requires Maven
packages from the GitHub Package Repository which requires a valid
Personal Access Token with read:packages.
It would be even nicer if the upstream plugins could provide their
own compiled jars, and all CivDocker had to do was download the jar
from the plugin instead of downloading the plugin's source code and
building. This would also avoid having to provide a GitHub Personal
Access Token.
It would also be nice to make sure these builds are reproducible,
but that is something to think about later.
See https://reproducible-builds.org/ and
https://reproducible-builds.org/docs/jvm/

This starts to address https://github.com/CivMC/CivDocker/issues/3

All the download+check+etc. logic could be in a shell script COPYed at the beginning, and then called with one line for each jar. This also helps keeping it DRY without splitting downloading from copying to the final image. I don't think we need to cache the jars so we can just download them all during every build.

Some code could be moved to a shell script, but it is important to make sure we take advantage of Docker's caching mechanism. Updating one plugin should not result in the rest of the plugins being updated. Updating each plugin currently involves downloading and building from the source code. This takes a long time and requires some build dependencies like Gradle. It is important to make sure this long process can be cached. Also, build dependencies should be kept out of the final stage so the image size remains low.
This approach seems complex and would involve a fair bit of code. It would be much more simple if the upstream plugins could provide the compiled binaries, and all we had to do here was download them.
@AngrySoundTech It seems like you maintain a lot of these upstream plugins. What do you think of providing the compiled jar with each release you publish? For example, if BanStick's v2.0.0-SNAPSHOT-2 release also provided a compiled jar as an asset, then all we would have to do here is download it.

We're in agreement. The shell script would just bundle the 4-5 lines of download+check so it's not repeated for every plugin in the Dockerfile. Building each plugin in their own CI and then just downloading them all during this Docker build process is the way to go. I don't know enough about Github CI capabilities to tell whether we can just download+commit the jars into this repo when they're updated in the other repos' releases, to make better use of Docker caching.

To publish each plugin, it seems we can just update this https://github.com/CivMC/.github/blob/master/.github/workflows/gradle_publish.yml#L29
to something like this: https://github.com/Gjum/SnitchMod/blob/master/.github/workflows/build.yml#L23

The changes to you are proposing to https://github.com/Gjum/SnitchMod/blob/master/.github/workflows/build.yml#L23 is exactly what would be helpful to CivDocker. I believe it would require a maintainer to make these changes.
Once this is done, my PR can be updated to download these compiled jars from each plugin in Docker.

We don't want to build plugins from source in Docker, here, we will however be providing the built jars on the plugin releases after we have a non-snapshot version


Compositing frames with different level of details

For now, all frames have the same level of details. I guess that’s wrong. We should have the choice to set arbitrary dimensions.

In order to make that change happen, we need to ask ourselves how we should set the dimensions. We could use the same idea we use for shadows: we use a configuration object that describes the dimensionality of the future frames.
Another way of doing that would be to automatically generate the frames without having to care that much about them. Or, we could simply create them by hand, like with genFrame. Or… we could also use fractions. Like, halves and quarters. That would enable us to make downsampled renders without caring too much.
We could set three levels of

Should be okay now.


[BUG] Create refine-app crashing with SAOError

Describe the bug
I am trying to create a refine project with the following command:
npm create refine-app@latest client
However, it systematically crashes at the moment of Installing packages with npm with this error:
superplate-cli has encountered an error.

If you think this is caused by a bug. Please check out:
https://github.com/pankod/superplate/issues

ERROR SAOError: Failed to install packages in /home/kevin/Projects/refine_dashboard/client
    at ChildProcess.<anonymous> (/home/kevin/.npm/_npx/98c66fac912b1cba/node_modules/sao/dist/chunk.T-GsVNZ1.js:6885:16)
    at ChildProcess.emit (node:events:526:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5) {
  sao: true
}

Steps To Reproduce

Open terminal
Run create refine-app with following command:
npm create refine-app@latest client
Select following options:

✔ Choose a project template · refine-react
✔ What would you like to name your project?: · client
✔ Choose your backend service to connect: · data-provider-custom-json-rest
✔ Do you want to use a UI Framework?: · mui
✔ Do you want to add example pages?: · inferencer
✔ Do you need any Authentication logic?: · auth-provider-google
✔ Do you need i18n (Internationalization) support?: · no
✔ Choose a package manager: · npm
✔ Would you mind sending us your choices so that we can improve superplate? · yes

Validate choices to proceed with installation

Expected behavior
It should properly create the project without errors
Screenshot
✔ Downloaded remote source successfully.
✔ Choose a project template · refine-react
✔ What would you like to name your project?: · client
✔ Choose your backend service to connect: · data-provider-custom-json-rest
✔ Do you want to use a UI Framework?: · mui
✔ Do you want to add example pages?: · inferencer
✔ Do you need any Authentication logic?: · auth-provider-google
✔ Do you need i18n (Internationalization) support?: · no
✔ Choose a package manager: · npm
✔ Would you mind sending us your choices so that we can improve superplate? · yes
superplate-cli has encountered an error.

If you think this is caused by a bug. Please check out:
https://github.com/pankod/superplate/issues

ERROR SAOError: Failed to install packages in /home/kevin/Projects/refine_dashboard/client
    at ChildProcess.<anonymous> (/home/kevin/.npm/_npx/98c66fac912b1cba/node_modules/sao/dist/chunk.T-GsVNZ1.js:6885:16)
    at ChildProcess.emit (node:events:526:28)
    at maybeClose (node:internal/child_process:1092:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5) {
  sao: true
}

Desktop

OS: Ubuntu 22.04
Browser: Google Chrome

Mobile
No response
Additional Context
No response

Hello @kevnoutsawo we'll check the issue and get back to you.

Hello again, @kevnoutsawo, unfortunately, couldn't reproduce the issue. It could be a connectivity problem, could you try it again or retry with pnpm? Alternatively, you can also try DEBUG=true npm create refine-app@latest client, it should log some errors about babel, ( starts with Error: Could not find source file.. you can ignore them, but if you still have error, you can send us latest error log, so we can help.

Hi @BatuhanW ,
I installed pnpm and ran the installation with it and it works. I am currently using a mobile connexion and it probably is part of the issue. pnpm seems more resilient than npm in this regard. I am marking this as solved.
Thanks a lot.


Revert should remove new properties

Create copy of original state only once
Iterate on own properties of current state instead of original
Allows new properties to be 'unset' by setting them to undefined

@warrenseymour Ping?

This has been in production for various clients and is battle tested. Merging.


Cost function is different to what appeared in the paper. Confidence value missed?

Hi,
Thanks for the implementation. I am trying to use the RegionProposalCriterion class in my framework.
In RegionProposalCriterion.lua line 57, the target description is: (x, y, \sqrt w, \sqrt h, class1, class2), but in the paper, it is (x, y, \sqrt w,\sqrt h, confidence value, class1, class2) (In Figure 2 description, the prediections are encoded as an S * S * (B * 5 + C)).
Or maybe it defined at somewhere else?
Thanks.

After taking a closer look, each grid output might be (x, y, \sqrt w, \sqrt h, confidence, class1). Based on line 127.


removes e2e tests

Several of these tests broke at some point in the past and we have not been able to fix them.
Additionally, once #928 is deployed (which is our new UI), they will be further broken.
I'm certainly open to re-adding end to end tests like these at some point in the future, but this will unblock deploying the new UI.
Signed-off-by: Nell Shamrell<EMAIL_ADDRESS>

I have disabled the e2e pipeline in Azure DevOps as well.

@disulliv good point, opening an issue now.


Undefined references in libdlib.a

I am working on Ubuntu 16.04 x64. Recently I have updated my system to CUDA 9.1 version and cudnn 7.1 version along with the new NVidia GPU driver 390.48. I should note that after installation all CUDA samples build and run properly. And I have successfully rebuild dlib library with CUDA support enabled. But... all my apps linked with the libdlib stop to compile. Compiler just says:
g++ -Wl,-O1 -o ../../bin/Trainer main.o   -L/usr/local/cuda-9.1/lib64 -L/usr/lib/x86_64-linux-gnu -lcudnn -lpthread -ldlib -L/usr/local/lib -lopencv_core -lopencv_objdetect -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lopencv_imgcodecs -lopencv_dnn
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_descriptor::set_size(int, int, int, int)':
cudnn_dlibapi.cpp:(.text+0x4b): undefined reference to `cudnnDestroyTensorDescriptor'
cudnn_dlibapi.cpp:(.text+0x6f): undefined reference to `cudnnCreateTensorDescriptor'
cudnn_dlibapi.cpp:(.text+0x99): undefined reference to `cudnnSetTensor4dDescriptor'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_descriptor::get_size(int&, int&, int&, int&) const':
cudnn_dlibapi.cpp:(.text+0x3de): undefined reference to `cudnnGetTensor4dDescriptor'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_conv::clear()':
cudnn_dlibapi.cpp:(.text+0x5c3): undefined reference to `cudnnDestroyFilterDescriptor'
cudnn_dlibapi.cpp:(.text+0x5d1): undefined reference to `cudnnDestroyConvolutionDescriptor'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_conv::~tensor_conv()':
cudnn_dlibapi.cpp:(.text+0x913): undefined reference to `cudnnDestroyFilterDescriptor'
...(over 1000 strings with the same looking errors)

Please, give me advice, what should I do to resolve this issue?

Version: master at 27.04.2018
Where did you get dlib: here, on github
Platform: Ubuntu 16.04.4 LTS x64
Compiler: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9)

Put -ldlib before -lcudnn

Still get same error:
g++ -Wl,-O1 -o ../../bin/Trainer main.o   -L/usr/local/cuda-9.1/lib64 -L/usr/lib/x86_64-linux-gnu -ldlib -lcudnn -lpthread -L/usr/local/lib -lopencv_core -lopencv_objdetect -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lopencv_imgcodecs -lopencv_dnn
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_conv::get_gradient_for_data(bool, dlib::tensor const&, dlib::tensor const&, dlib::tensor&)':
cudnn_dlibapi.cpp:(.text+0xaba7): undefined reference to `cudaGetDevice'
cudnn_dlibapi.cpp:(.text+0xb170): undefined reference to `cudaGetErrorString'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::tensor_conv::get_gradient_for_filters(bool, dlib::tensor const&, dlib::tensor const&, dlib::tensor&)':
cudnn_dlibapi.cpp:(.text+0xb5cf): undefined reference to `cudaGetDevice'
cudnn_dlibapi.cpp:(.text+0xbb98): undefined reference to `cudaGetErrorString'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::assign_conv_bias_gradient(dlib::tensor&, dlib::tensor const&)':
cudnn_dlibapi.cpp:(.text+0xc002): undefined reference to `cudaGetDevice'
cudnn_dlibapi.cpp:(.text+0xc37d): undefined reference to `cudaGetErrorString'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::pooling::get_gradient(dlib::tensor const&, dlib::tensor const&, dlib::tensor const&, dlib::tensor&)':
cudnn_dlibapi.cpp:(.text+0xc9ff): undefined reference to `cudaGetDevice'
cudnn_dlibapi.cpp:(.text+0xcd97): undefined reference to `cudaGetErrorString'
/usr/local/lib/libdlib.a(cudnn_dlibapi.cpp.o): In function `dlib::cuda::batch_normalize_conv_inference(double, dlib::resizable_tensor&, dlib::tensor const&, dlib::tensor const&, dlib::tensor const&, dlib::tensor const&, dlib::tensor const&)':
cudnn_dlibapi.cpp:(.text+0xd72f): undefined reference to `cudaGetDevice'
cudnn_dlibapi.cpp:(.text+0xda0f): undefined reference to `cudaGetErrorString'
...

Oh, you are linking to the static library.  You also need to link to the cuda runtime.  Those are cuda symbols, so you need to link to the cuda libraries that contain them.
If you installed dlib as a shared library you could just do -ldlib and it would pull in the right libraries, but since you are trying to link to a static library you aren't getting those automatically and have to link them yourself.  But really, you should either use cmake or install dlib as a shared library if you insist on not using cmake.

Thank you @davisking. My problem has been solved by the manual addition of cuda libraries in linking section of project:
g++ -c -pipe -O2 -std=gnu++11 -Wall -W -fPIC -DPATH_TO_DLIB_RES=\"\" -DOPENCV_DIR=\"\" -I../../Learner -I. -I/home/alex/Programming/Kaggle/Shared/dlibimgaugment -I/home/alex/Programming/Kaggle/Shared/opencvimgaugment -I/home/alex/Programming/Kaggle/Shared/dlibopencvconverter -I/home/alex/Qt/5.10.0/gcc_64/mkspecs/linux-g++ -o main.o ../../Learner/main.cpp
g++ -Wl,-O1 -o ../../bin/Trainer main.o   -ldlib -L/usr/local/cuda-9.1/lib64 -lcudnn -lpthread -lcuda -lcudart -lcublas -lcurand -lcusolver -lopencv_core -lopencv_objdetect -lopencv_highgui -lopencv_imgproc -lopencv_videoio -lopencv_imgcodecs -lopencv_dnn


Print page default banner rendering error

The links aren't displayed/interpreted as links.
Screenshot:

I'm using mkdocs material.

Yeah that's no good. Fixed and releasing a new versio now.
Thanks for reporting!

Released: https://github.com/timvink/mkdocs-print-site-plugin/releases/tag/v2.2.5


Cannot install signoz with Flux helm-controller

Flux is reporting error with helm chart. Looks like the template is malformed.
Helm install failed: error while running post render on files: map[string]interface {}(nil): yaml: unmarshal errors: line 23: mapping key "nodePort" already defined at line 20

corresponding helm release:
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: signoz
  namespace: signoz
spec:
  releaseName: signoz
  chart:
    spec:
      chart: signoz
      version: "0.4.2"
      sourceRef:
        kind: HelmRepository
        name: signoz
        namespace: flux-system
  interval: 5m
  install:
    remediation:
      retries: 30

Does not help. When running helm template test . in the cloned repo, it shows a bunch nodePort values nodePort: and more of  nodePort: null this is probably not correct. Not sure which is right yet.

Since these with empty value are the last ones, they are probably not correct. Will try earlier version of the chart.

Hmm, this looks weird:
# Source: signoz/templates/otel-collector-metrics/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: test-signoz-otel-collector-metrics
  labels:
    helm.sh/chart: signoz-0.4.3
    app.kubernetes.io/name: signoz
    app.kubernetes.io/instance: test
    app.kubernetes.io/component: otel-collector-metrics
    app.kubernetes.io/version: "0.11.2"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - name: health-check
      port: 13133
      nodePort: null
      targetPort: health-check
      protocol: TCP
      nodePort:
  selector:
    app.kubernetes.io/name: signoz
    app.kubernetes.io/instance: test
    app.kubernetes.io/component: otel-collector-metrics

or this:
# Source: signoz/templates/otel-collector/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: test-signoz-otel-collector
  labels:
    helm.sh/chart: signoz-0.4.3
    app.kubernetes.io/name: signoz
    app.kubernetes.io/instance: test
    app.kubernetes.io/component: otel-collector
    app.kubernetes.io/version: "0.11.2"
    app.kubernetes.io/managed-by: Helm
spec:
  type: ClusterIP
  ports:
    - name: jaeger-grpc
      port: 14250
      nodePort: null
      targetPort: jaeger-grpc
      protocol: TCP
      nodePort:
    - name: jaeger-thrift
      port: 14268
      nodePort: null
      targetPort: jaeger-thrift
      protocol: TCP
      nodePort:
    - name: metrics
      port: 8888
      nodePort: null
      targetPort: metrics
      protocol: TCP
      nodePort:
    - name: otlp
      port: 4317
      nodePort: null
      targetPort: otlp
      protocol: TCP
      nodePort:
    - name: otlp-http
      port: 4318
      nodePort: null
      targetPort: otlp-http
      protocol: TCP
      nodePort:
  selector:
    app.kubernetes.io/name: signoz
    app.kubernetes.io/instance: test
    app.kubernetes.io/component: otel-collector

node port defined multiple times in the map.

I just installed SigNoz in EKS using the latest helm. It worked fine

@petrmac Thanks for raising this issue.
In the scenario of redundant configuration, Helm itself seems to handle it.
However, Flux was able to detect it.
PR with the fix for the issue is out now.


getP() and translate() : add optional scope parameter

from #28
add polyglotScope notion.

waiting for the new implementation ( the curried way ).

@guillaumearm I don't really like the way getP can take the scope directly as a string ... I think it should receive a 'propsy' object as we discussed.
Only the translate enhancer should take either a string or an object IHMO.
This PR is already merged, so I will think about it and PR if I can propose a better implementation.


Refactoring select queries.

Added "only" method to select queries to load only a subset of the
document's fields.

LGTM


"Go to Definition" does not seem to work when using package folders

I like to use package folders to keep my code clean.
When I go to code which calls a function from a package and I try to "Go to Definition" in vscode, the tool tells me it cannot find the function.
To recreate the bug, I made the following simple workspace:

For all the functions in het script, the "Go to Definition" works, except for the package.function_in_package() function.
When I do the same thing from matlab, it works and it can open the right function

I'm able to peek and go to definition inside my custom packages:

Although my package itself is saved to my pathdef file so on any MATLAB startup it is automatically on the path
If i make a new package in a folder not on my default path and edit in in VSCode I get the same result as you did, unable to find definition

So it seems to me that this is an issue of the headless MATLAB instance not having the opened folder on the path.

And actually, if i move that second function into another folder, close VSCode, and open it again, it has not added my new folder nonPack to the path, so it does not see customFunc

However if i just expand the nonPack folder in my explorer, it is able to find it again as the folder is now added to the path

Expanding my testPack package still doesn't get it on the path though:

Thank you for identifying this issue! We will look into this and try to reproduce on our end.

This should be resolved in v1.1.1.


org: Update visual display on User and Group cards

Hey, I just made a Pull Request!

Closes #4211.

Updates the contents of the about cards in both User and Group pages using a <List> to fix some minor visual alignment
Grid/box fixes to improve alignment/reduce complexity. (avatar icons weren't truly 3 spaced and aligned with other content, I've tested varying sizes (sm/md/lg), resizing etc.)
Makes the user's e-mail clickable (to mailto:, firing up your local e-mail client).
Replaces the title of the Group about card with the group's description, if it's available. This better lines up with how the user card shows, with the user name (like adamharvey) or the group name (like backend-team) at the top of the page and the friendly version in the card (e.g., "Adam Harvey" or "Members of the Backend Team").

User card

Parent team with a child

Child team with a parent

Group with no description; name repeated in about card

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[x] Screenshots attached (for UI changes)

Think this one is ready for maintainers review. 🚀 Appreciate the feedback @Fox32. Thx!


Adding an event i am already attending results in server error

Clicking attend event on an event i am already attending results in an sever error message instead of an error like "You are already attending this event!"

Will be fixed in next release!


Installation should check if the user is part of the docker group

Problem
The current installation requires that the user who is running it is part of the docker group. In the installation manual, we have given instruction on how to add to the group and specifically that the user has to restart the shell before proceeding for the new addition to take effect.
However many users do not read the installation manual and try to install without the restart or without adding the user to group.
Requirement
We should check at the beginning of the script if the user is part of the docker group. If not, then we should ask him to add himself to the group, restart the shell and try again. We should not go ahead with the installation in this case.

adding a disclaimer:
to check about the pre-requisite at the beginning of the installation itself.

Duplicate of issue #208


Mui datagrid poc

Fixes #ISSUE_NUMBER
Fixes #156
Describe your changes
implemented datagrid for poll search page
added replaced privateId params to polIId in results page
link copied message is now shown in snack bar
NOTE - will once the PR is reviewed will incorporate same changes to user listing page
Checklist before requesting a review

[x] I have performed a self-review of my code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have added thorough tests

Codecov Report

Merging #167 (74c880f) into dev (63392bb) will decrease coverage by 0.39%.
The diff coverage is 2.43%.

@@            Coverage Diff             @@
##              dev     #167      +/-   ##
==========================================
- Coverage   46.54%   46.15%   -0.40%
==========================================
  Files          74       75       +1
  Lines        3246     3274      +28
  Branches      446      447       +1
==========================================
  Hits         1511     1511
- Misses       1701     1729      +28
  Partials       34       34

Flag
Coverage Δ

backend
59.66% <ø> (ø)

dev
46.15% <2.43%> (?)

frontend
10.45% <2.43%> (-0.34%)
:arrow_down:

production
?

staging
?

unit
46.15% <2.43%> (-0.40%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

frontend/src/components/AdminView.tsx
1.20% <0.00%> (+0.01%)
:arrow_up:

frontend/src/components/AdminViewPoll.tsx
0.00% <0.00%> (-3.58%)
:arrow_down:

frontend/src/components/PollEditView.tsx
2.08% <0.00%> (-0.05%)
:arrow_down:

frontend/src/components/PollResult.tsx
3.44% <0.00%> (+0.50%)
:arrow_up:

frontend/src/components/AdminViewPollDataGrid.tsx
3.03% <3.03%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


chore(4.5): release 1.8.32

:robot: I have created a release beep boop
1.8.32 (2024-03-20)
Features

add java memory arguments in helpers file (#628) (31de98f)

Miscellaneous Chores

prepare for 1.8.32 release (53546a0)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/GluuFederation/cloud-native-edition/releases/tag/v1.8.32 :sunflower:


Replace swiftcopt with feature

The wrapped swiftcopts aren't meant for external consumption, they should always be implied by public facing features where the strings are more stable.

@jerrymarino


Soft spans

This PR implements soft spans (an idea of @raphlinus). Soft spans are maybe best shown with an example of an actual use case:

In this example I am not using arrow keys to move the cursor over the closing parenthesis, I am inserting an actual closing parenthesis instead
Here are some characteristics I've implemented:

if a character is inserted before a soft span and the character matches with the first character of the soft span, the cursor is shifted over this character (implementation-wise the existing value of the soft span is overridden)
if more than one character is inserted (e.g. using paste), it works according the point above, but shifts the cursor for as long as the input string matches the soft span string
if the cursor is moved (arrow keys, mouse click, ...), all soft spans before the cursor are removed
if text is deleted, soft spans are shifted accordingly

The implementation of soft spans is similar to style spans, with the difference that the editor keeps the soft spans in place (instead of relying on the plugin to handle the lifetime of the soft span).
However, merging asynchronous inserts that come in late, is not implemented yet, e.g.:

While adding soft spans at the correct position is easy (since it is already implemented for style spans), as a user I would expect the additional plugin-inserted closing parenthesis to not show up at all.
This brings me to the main open question of this PR: How and where to implement the merging of soft spans? I kind of have the feeling that something like this should actual go into engine.rs?
A quick'n'dirty branch of xi-mac for testing purposes can be found here (I have added an entry into the Debug menu).
(commits will be squashed once the PR is ready to go)

This is neat. I imagine this is supposed to work like the equivalent feature in XCode, so I tested XCode's behaviour and found that any movement off of the line the soft close paren is on, the close paren becomes real (and XCode has a cool flash animation that calls attention to this).
I think this behaviour makes more sense than only removing soft spans before the cursor. Even if you move up a line, I think the spans on the line you are at should become real, at least for paren matching.
Another interesting thing I noticed is that XCode only creates the soft close paren after you type a character following the opening paren. I'm not sure the reason behind this, but one thing that often annoys me about bracket matching in other text editors is when I do some editing operation that messes up paren nesting, so I go to insert a missing open paren, but then the editor adds a close paren for it! The XCode behaviour avoids doing this, which is cool.
I also tested Sublime's behaviour and it seems that typing a ) with your cursor before any ) always just moves past it. Typing an opening paren in the middle of other characters doesn't insert a close paren, but typing one before whitespace in the middle of a line does.

My feeling on this is that we're not quite yet at a state where this would be ready to merge. That said, this is great as a sample implementation, and there's good discussion in the comments. Do people agree with closing this particular PR and opening an issue for "soft spans" which points to this?

I'd like to update this PR to work in async scenarios, thus, moving the logic from the editor into the engine. However, before investing to much time into the engine, I'd like to defer it for some weeks an await the changes to the engien/OT that might come.
That being said, I am fine with closing it and moving the discussion into an issue.


Details About ERF Visualization

@MzeroMiko Thank you for your outstanding work and generous open-source contributions! I have a question regarding the use of RandomSampler when drawing ERFs for each feature extraction backbone:
https://github.com/MzeroMiko/VMamba/blob/c07d2ea1322061cd52a505fdae735c91346a5d65/analyze/utils.py#L317
Doesn't this mean that each heatmap represents the average result of several different image batches? Or is this not a significant concern because each batch contains 50 images, which is sufficient to mitigate any bias? Thank you!

Yes  It is the average ERF for 50 images.
We adopted this code from RepLKNet. You may refer to that for more information.

Thank you very much!


Manually fail the test to see yamato behavior on origin code

This is a test PR to see if yamato behaves properly. One system.runtime test is set to be fail. Yamato test is expected to fail.

The yamato checks still pass with one test manually set to fail.

I think that we can close this PR.


[Icon Request]:  snippet about vscode-extension development.

Icon Type

[X] Folder
[ ] File

Folder names

snippet
snippets

Graphic ideas

or

Additional context
In the development environment for VSCode extensions related to programming languages or transpilers, the folder for defining snippets is conventionally named 'snippets'.
I believe having a dedicated folder icon for this would greatly aid in identification.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Merged in #2510 :)


🛠 Tooling: Create a common component for code tags

Bug Report Checklist

[X] I have tried restarting my IDE and the issue persists.
[X] I have pulled the latest main branch of the repository.
[X] I have searched for related issues and found none that matched my issue.

Overview
This comment exists in doc-content.tsx:

// TODO: create a component for code tag so that the styles are automatically applied

It's in reference to there being a lot of <code css={usageStyles.code}> tags in the page. Agreed, this would be a nice opportunity to refactor a bit to extract out a shared component for code.
Additional Info
No response

@JoshuaKGoldberg has done the required changes and ran the site locally, everything worked fine.
Let me know if anything needs to be changed here.


Remove use of python 2 compatibiliy APIs

Django 3.0 removes django.utils.encoding.python_2_unicode_compatible
(https://docs.djangoproject.com/en/dev/releases/3.0/#removed-private-python-2-compatibility-apis)
Since this package is python 3 only, there shouldn't be any issues removing these decorators.

Hi @jeremy-engel - sorry for missing this PR. I have addressed it in #24, which also drops support for Django111.


Validate CARDANO_NETWORK env matches Cardano.networkName otherwise throw

In the NetworkInfo feature a mismatch here is an undesirable state. Retaining the configured setting allows for application bundling optimisations.

{
  cardano {
    networkName
  }
}


feat(Business Glossary) Profile page for business glossary

Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[x] Links to related issues (if applicable)
[x] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable)

Important point to note -
Same set of UI code was being used in UserProfile page and GlossaryTermProfile page. So we  moved our code of UserProfile component  to a reusable component i.e EntitySearchResult component inside shared folder and used this component for GlossaryTerm profile and user profile
Uploaded a sample screenshot based on dummy data

The language of the ownership section seems a little confusing when applied to business glossary terms. A dataset isn't owned by a term, it just has that term applied to it. Can we communicate that more clearly in the glossary term's ownership section?

I Will work on the above feedbacks and update the PR.


Remove code offsets from Function

A future cleanup could remove the begin_* and end_codegen functions from CodeSink I think.

CI failures look like some warnings-as-errors for newly unused imports in various places -- hopefully an easy fix.

Fixed


Remove some stuff from mirrorButton function

Removed the duetNote array and copiedNote variable from it since they are never used

fixed some merge conflict


refactor OkWrap to not call .into_py(py)

Motivated by https://github.com/PyO3/pyo3/pull/3588/files#r1403933922
The goal here is to keep the OkWrap trait as simple as it can possibly be.
@wyfo what do you think? It conflicts a little bit with #3588, so that'd need rebasing if this merged, but I like how this keeps OkWrap simple.

I didn't dare to do this refactor as part of the async work, but I fully agree with this simplification

:+1: for me

Thanks again for review, as this touches more code which other contributors initially helped design I'll wait to merge a little bit in case of other opinions. But not too long, as you'll need to rebase off this 🙂


Errors with flow experimental.const_params=true

Is this a bug report?
Yes
Have you read the Contributing Guidelines?
Yes
Environment
Environment:
OS:  macOS High Sierra 10.13
Node:  6.11.2
Yarn:  1.1.0
npm:  3.10.10
Watchman:  4.7.0
Xcode:  Xcode 9.0 Build version 9A235
Android Studio:  Not Found
Packages: (wanted => installed)
react: 16.0.0-alpha.12 => 16.0.0-alpha.12
react-native: ^0.48.4 => 0.48.4
Steps to Reproduce

(Write your steps here:)

Create a new react-native app with create-react-native-app
yarn add --dev<EMAIL_ADDRESS>(same version specified in .flowconfig)
Add experimental.const_params=true under [options] in .flowconfig
./node_modules/.bin/flow

Expected Behavior
No errors.
(Write what you thought would happen.)
Actual Behavior
$ flow
node_modules/fbjs/lib/UnicodeBidi.js.flow:90
 90:   fallback = fallback || UnicodeBidiDirection.NEUTRAL;
       ^^^^^^^^ fallback. const param cannot be reassigned (see experimental.const_params=true in .flowconfig)
 89: function resolveBlockDir(str: string, fallback: ?BidiDirection): BidiDirection {
                                           ^^^^^^^^^^^^^^^^^^^^^^^^ const param fallback

node_modules/fbjs/lib/UnicodeBidi.js.flow:92
 92:     return fallback;
                ^^^^^^^^ null. This type is incompatible with the expected return type of
 89: function resolveBlockDir(str: string, fallback: ?BidiDirection): BidiDirection {
                                                                      ^^^^^^^^^^^^^ string enum

…[removed for brevity]…

... 88 more errors (only 50 out of 138 errors displayed)
To see all errors, re-run Flow with --show-all-errors

Reproducible Demo
https://github.com/ckknight/ReactNative-const_params-repro

This also affects<EMAIL_ADDRESS>

Some of there errors seem to come from RN dependencies (fbjs from the sample you provided). I think it would be nice to fix but will require changing stuff across repos. If someone want to send a PR to fix some errors that are in this repo it would be nice.

Hi @janicduplessis: I am a beginner in opensource contribution and like to work on this.
Could you please guide me to fix this issue?

@janicduplessis would like to work on this since it seems like an easy one for first timer. just to clarify, this would be to add experimental.const_params=true to react-native's .flowconfig and find/fix all the ones it outputs.

I have a PR inbound for this. Should be submited today or tomorrow.

@TakaGoto I'll tag you in my PR in case you'd like to submit any changes as well. You need to clone this repo, enable experimental.const_params and run flow. Most of the errors are fixed with a minor refactor.

I really really want to do this. Unfortunately, we have a monorepo at facebook and everything is under a single flow config. If we merge this, it wouldn't change how flow treats these files internally and we'd be likely to break flow for everyone in open source.
Fortunately, I actually wrote a pretty intensive codemod to do it at Facebook so we could convert our entire codebase and turn this flow config on. Unfortunately, there is an issue with the codemod that has kept me from converting our entire codebase. The codemod erroneously converts:
function foo(myVar) {
var myVar = 2; // valid shadowing of the function argument
}
into
function foo(myVar) {
let localMyVar = myVar;
var localMyVar = 2; // Uncaught SyntaxError: Identifier 'localMyVar' has already been declared
}
If you would be interested in detecting this scope issue and getting the codemod to handle it, we'd be able (and happy) to make this change to react native. The codemod would of course also be open source then. :)

Very interesting. I'll take a look, thanks!

Silly bot

👋 there - this issue is still technically "not fixed", for the reason Eli explained above.
Basically every instance similar to this in the main codebase
function foo(arg) {
  arg += 2;
  console.log(arg);
}

would need to be changed to
function foo(arg) {
  let localArg = arg;
  localArg += 2;
  console.log(localArg);
}

There is a codemod Eli worked on -> https://github.com/cpojer/js-codemod/pull/85 which would help a lot in automating this process, but there is a 1% cases in which it would conflict.
So, all in all, there is a lot of work related to "code hygiene" - we'd love to see some PRs to help with this effort but it's really low prior right now. So I'm going to close it as there are more important tasks to focus on.


Align the overview message for what aspire is with the website and new messaging

Align the overview with the website.
We want to get rid of the word orchestration and replace it with inner loop (and remove it from the App Host ToC item as well).
@maddymontaquila this is yours 😄

The official marketing website says "Dev-time orchestration":

Also, there's no mention of inner-loop in the dot.net site either?

https://github.com/dotnet/docs-aspire/blob/main/docs/get-started/aspire-overview.md @alistairmatthews I think using a mix of "dev-time orchestration" and "inner loop" is good, just general "orchestration" is where we start to get into k8s land

I think about this new slide:

We should be using this exact same language in the docs


updating user profile info

willl want handle user info update

Can I close this issue? I saw your PR but I'm not sure if its exactly related to this issue.

Am yet to finish some small touches so you can close

this issue is done so we can close


Checks for string equality to fix nltk.util.escape_xml doctest

Fixing #1577 by checking using string literals to check for equality.

Thanks @alvations.
What do you think about also demonstrating the use of xml_escape itself, in the docstring for xml_escape?
I think the current documentation is there to justify the need for xml_escape (which could go into a Python comment). Users expect to see usage examples in docstrings.

@stevenbird sorry I was a little busy for the past few days. I missed out the doctest for xml_escape. Sorry about it. Added in the PR #1592 and also corrected one of the escape symbol.


Playwright fixes - Work in progress

Motivation/Description of the PR

Fixing issues with playwright
Fixed waitForValue so far

Applicable helpers:

[ ] WebDriver
[ ] Puppeteer
[ ] Nightmare
[ ] REST
[ ] FileHelper
[ ] Appium
[ ] Protractor
[ ] TestCafe
[/ ] Playwright

Applicable plugins:

[ ] allure
[ ] autoDelay
[ ] autoLogin
[ ] customLocator
[ ] pauseOnFail
[ ] puppeteerCoverage
[ ] retryFailedStep
[ ] screenshotOnFail
[ ] selenoid
[ ] stepByStepReport
[ ] wdio

Type of change

[ ] :fire: Breaking changes
[ ] :rocket: New functionality
[ ] :bug: Bug fix
[ ] :clipboard: Documentation changes/updates
[ ] :hotsprings: Hot fix
[ ] :hammer: Markdown files fix - not related to source code
[ ] :nail_care: Polish code

Checklist:

[ ] Tests have been added
[ ] Documentation has been added (Run npm run docs)
[ ] Lint checking (Run npm run lint)
[ ] Local tests are passed (Run npm test)

@DavertMik hopefully this fixes some of the playwright tests

Any other failures i suspect are things i missed that were broken in in the playwright 0.12.1 API changes

Need some help with execureAsync waitForClickableand scrollTo think these might have been broken in 0.12 and I missed them

Thank you!


Add arm64 macOS wheel?

Hi!
Would it be possible to build and publish arm64 wheels to PyPI? Right now, on Apple Silicon, pip installing qdldl (or OSQP) triggers a build from source, which fails if the user doesn't have cmake installed.
cibuildhwheel supports building arm64 wheels: https://cibuildwheel.readthedocs.io/en/stable/faq/#apple-silicon

macOS arm64 wheels are now available through PyPi.


Make a Java 8 build

Make a Java8 build of the repository as well (it will be used in some Java8-projects).

No. Won't fix this. Too much work.


Automated cherry pick of #2495: Fix panic in stats.calculateRuleDiff

Cherry pick of #2495 on release-1.1.
#2495: Fix panic in stats.calculateRuleDiff
For details on the cherry pick process, see the cherry pick requests page.

Codecov Report

Merging #2505 (f60b882) into release-1.1 (dafc994) will decrease coverage by 6.73%.
The diff coverage is 100.00%.

@@               Coverage Diff               @@
##           release-1.1    #2505      +/-   ##
===============================================
- Coverage        62.01%   55.27%   -6.74%
===============================================
  Files              278      278
  Lines            21161    21162       +1
===============================================
- Hits             13122    11698    -1424
- Misses            6682     8221    +1539
+ Partials          1357     1243     -114

Flag
Coverage Δ

kind-e2e-tests
40.19% <0.00%> (-13.02%)
:arrow_down:

unit-tests
41.35% <100.00%> (+<0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/agent/stats/collector.go
98.47% <100.00%> (+0.01%)
:arrow_up:

pkg/apis/controlplane/helper.go
25.00% <0.00%> (-75.00%)
:arrow_down:

pkg/apis/controlplane/v1beta2/helper.go
25.00% <0.00%> (-75.00%)
:arrow_down:

pkg/controller/networkpolicy/mutate.go
0.00% <0.00%> (-71.77%)
:arrow_down:

pkg/controller/networkpolicy/store/group.go
5.00% <0.00%> (-70.00%)
:arrow_down:

pkg/controller/networkpolicy/validate.go
0.00% <0.00%> (-69.64%)
:arrow_down:

pkg/agent/controller/networkpolicy/packetin.go
4.36% <0.00%> (-68.45%)
:arrow_down:

pkg/util/k8s/name.go
33.33% <0.00%> (-66.67%)
:arrow_down:

pkg/controller/networkpolicy/convert.go
0.00% <0.00%> (-64.52%)
:arrow_down:

...formers/externalversions/security/v1alpha1/tier.go
0.00% <0.00%> (-64.29%)
:arrow_down:

... and 73 more

/test-all


use app-model for view menu

Description
Partially to keep moving forward, but also to keep testing the app-model pattern, I'm playing with some branches that use #4784 to start removing old code.  This refactors the menu bar view menu with the new pattern.
@Czaki, particularly interested to get your thoughts on this.  While I'm mostly very happy with the pattern of declaring commands that require dependencies to be injected (rather than always using methods bound to specific instances), I do recognize that some of it feels a little different as I go.  Your Qt expertise and critical eye is very much appreciated here.
diff relative to #4784: https://github.com/tlambert03/napari/compare/use-app-model...tlambert03:napari:appmodel-view-menu

ok, I think this is ready to go.  Since i think most of the other menu/app-model PRs are waiting on this pattern, we should try to get this reviewed and merged relatively soon.  (but if there's anything anyone wants changed, don't hesitate to point it out)

Feel free to take a look at it.  But I see that as likely permanent.

@napari/core-devs this PR should be ready to go. It would unblock migration of menus for other PRs as well, may I have a round of sanity check to get this in since the approval is a while back? thanks!

This may be unimportant but I noticed that the migrated menu item does not have an underscore under the first letter (view menu in screenshot below). Is this a keyboard shortcut thing that was not added?

Those come from adding an ampersand before the letter.  It's a (somewhat poorly documented) feature in Qt.
If you want to retain that behavior, you'll need to add the ampersand back in the string that declares the menu name.  Since it's Qt specific, I didn't "automatically" add the ampersand to menu names in app-model.
It's a little problematic since you kinda need to maintain two strings now: one "real" title of the menu ("View") and another one meant for the Qmenu ("&View").
Anyway, that's why they're different

@DragaDoncila I think this is ready for re-review. Have tested the menu items and seem to work (though I'm not sure what some of them are meant to do :sweat_smile: )
Just few remaining questions:
https://github.com/napari/napari/pull/4826#discussion_r987447469
https://github.com/napari/napari/pull/4826#discussion_r987448016

Thanks, everyone! I hope we can merge this before the end of the week!  Also checked opening a viewer and the injection seems to work fine for _get_current_activity_dock.
Re-running CI to see if that helps :woman_shrugging:

Tests (test_qapp_model_menus.py) still failing (with some CIs getting a 'Fatal Python error: Segmentation fault' part way through running the test) but definitely working locally for me. Would be nice if someone else could try it locally to check if it's just me.

@lucyleeow I checked out this PR on my mac (macOS 12.6, arm64) and every element of the menu works properly in the GUI. Running pytest:
╰─ pytest -vv napari/_qt/_qapp_model/_tests/                     (apple-TF29) ─╯
============================= test session starts ==============================
platform darwin -- Python 3.9.13, pytest-7.1.2, pluggy-1.0.0 -- /Users/piotrsobolewski/Dev/miniforge3/envs/apple-TF29/bin/python3.9
cachedir: .pytest_cache
PyQt5 5.15.4 -- Qt runtime 5.15.4 -- Qt compiled 5.15.3
rootdir: /Users/piotrsobolewski/Dev/napari, configfile: pyproject.toml
plugins: qt-4.1.0, napari-plugin-engine-0.2.0, npe2-0.6.1, napari-0.4.17rc4.dev131+g02ab6a55
collected 6 items

napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view] PASSED [ 16%]
napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/axes] PASSED [ 33%]
napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/scalebar] PASSED [ 50%]
napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/context] PASSED [ 66%]
napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/convert_dtype] PASSED [ 83%]
napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/project] PASSED [100%]

=============================== warnings summary ===============================
../miniforge3/envs/apple-TF29/lib/python3.9/site-packages/_pytest/config/__init__.py:1129
  /Users/piotrsobolewski/Dev/miniforge3/envs/apple-TF29/lib/python3.9/site-packages/_pytest/config/__init__.py:1129: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: __editable___napari_0_4_17rc4_dev131_g02ab6a55_finder
    self._mark_plugins_for_rewrite(hook)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================= slowest 10 durations =============================
0.10s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view]
0.07s call     napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view]
0.00s call     napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/axes]
0.00s call     napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/context]
0.00s call     napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/scalebar]
0.00s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/axes]
0.00s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/view/scalebar]
0.00s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/convert_dtype]
0.00s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/context]
0.00s setup    napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py::test_build_qmodel_menu[napari/layers/project]
========================= 6 passed, 1 warning in 0.24s

@psobolewskiPhD thank you :pray: !!
I this CI failure is outside my abilities :sweat_smile:

(I have encountered in the past that running a single test file passes but running all tests gets a segfault on the otherwise passing tests)
I think we shouldn't get into this seg fault rabbit hole for now. I suggest we pull the toggle commit out into another PR and investigate later so we don't block the merging of this PR. Thoughts? (unless everyone is ok with disabling this seg fault test temporarily?)

I believe in-n-out inspects type signatures of functions to know what to inject.  In this case it looks like something with _get_current_activity_dock(). It might be worth trying to reproduce in a clean environment.
I'm not seeing a segfault.

I believe in-n-out inspects type signatures of functions to know what to inject. In this case it looks like something with _get_current_activity_dock().

Agree, that is what the failure suggests to me too but just cannot reproduce. I did a clean reinstall of env recently (maybe this week) but can try again next week.

I'm not seeing a segfault.

Segfault doesn't happen in test_qapp_model_menus.py, it occurs later:
  napari/_qt/_qapp_model/_tests/test_qapp_model_menus.py F.....            [ 79%]
  napari/_qt/_tests/test_app.py s..                                        [ 79%]
  napari/_qt/_tests/test_plugin_widgets.py ...........                     [ 79%]
  napari/_qt/_tests/test_prune_qt_connections.py .                         [ 79%]
  napari/_qt/_tests/test_qt_notifications.py ...........                   [ 79%]
  napari/_qt/_tests/test_qt_provide_theme.py .                             [ 79%]
  napari/_qt/_tests/test_qt_utils.py ........                              [ 80%]
  napari/_qt/_tests/test_qt_viewer.py ..Fatal Python error: Segmentation fault

this was from https://github.com/napari/napari/actions/runs/3202726217/jobs/5232006304 (ubuntu-latest 3.8 pyqt5)
(Also note I am running ubuntu 20.04.5 LTS)

It might be worth trying to reproduce in a clean environment.

Tried again with clean environment (installed via pip install -e .[dev] using main) and could not reproduce the error.

For future reference:
The cause of all the problems seems to be because napari._app_model.get_app() always returns the same NapariApplication (checked with id()). This means that if a previous test registered any commands, providers etc they will remain for subsequent tests. This would explain:

segfault at varying points when running full suite of tests: NapariApplication may keep a reference to viewer or window that has since been cleaned up at the end of a previous test.
Running just test_qapp_model_menus.py passes (both locally and in CI) but running with full suite fails (for ease of testing we found running test_vispy_multiscale.py followed by test_qapp_model_menus.py causes the same failure): if a previous test builds napari, all commands get registered with NapariApplication. When build_qmodel_menu is run, _MenuMixin.__init__(self, menu_id, app) runs rebuild which rebuilds menu for all actions matching the MenuID given to build_qmodel_menu. This results in the following error for MENUBAR_VIEW:

TypeError: After injecting dependencies for NO arguments, _get_current_activity_dock() missing 1 required positional argument: 'window

This is because, although a window provider has been registered, there is no current _QtMainWindow.
When running just test_qapp_model_menus.py, CommandId.TOGGLE_ACTIVITY_DOCK is never registered because q actions are registered via init_qactions which is run on __init__ of _QtMainWindow. This is never run in build_qmodel_menu. Thus the test passes. Note that app actions are registered as this occurs in __init__ of NapariApplication.
If a MenuID only contains q actions (and test_qapp_model_menus.py is run in isolation), the test also fails with KeyError at get_menu() in:
 assert len(menu.actions()) >= len(app.menus.get_menu(menu_id))

as there are 'no' actions for that MenuID.
There are 2 problems to solve:

For tests a clean NapariApplication should be used for each test. This work has been started by Talley with the app fixture in napari/conftest.py. In this fixture we also need to mock.patch the return value for napari._app_model.get_app() (as Talley suggested) because napari._app_model.get_app() is called in many places. E.g., for test_build_qmodel_menu, even if we use the app fixture, build_qmodel_menu calls get_app():

    return QModelMenu(
        menu_id=menu_id, app=get_app(), title=title, parent=parent
    )

test_build_qmodel_menu only works for app actions, and not Q app actions. We could amend the test such that both actions are tested, or add another test that checks q actions. Not sure of best approach for this.

@DragaDoncila please correct any inaccuracies above!

@lucyleeow @DragaDoncila Is this ready to merge? Do we need to get another review from someone?

@nclack would be great if we could get someone's eyes (yours maybe... 👀) on how we ended up setting up the app fixture in conftest.py. Mainly:

we've made it autoused, so that every test gets an instance
we've made it a NapariApplication not a plain app-model Application. This is because we felt like for the majority of tests, we'd want the providers, accessors and actions registered in the NapariApplication.__init__. If someone wanted a completely clean unadulterated app in their test, they could always create one, or clear the existing one.
we didn't init_qactions in the fixture, as we didn't feel it would be best practice to have qt associated bits and bobs registered for every test. These do get registered whenever make_napari_viewer is used, and init_qactions can be called explicitly (as we've done in the build_qmodel_menu test) when you want all the bells and whistles, as long as the appropriate providers are mocked.

If all of this sounds sane and reasonable to you, then I think this is ready for merge? @lucyleeow can confirm as she's been doing all the heavy lifting here.

@nclack would be great if we could get someone's eyes (yours maybe... 👀) on how we ended up setting up the app fixture in conftest.py. Mainly:

we've made it autoused, so that every test gets an instance
we've made it a NapariApplication not a plain app-model Application. This is because we felt like for the majority of tests, we'd want the providers, accessors and actions registered in the NapariApplication.__init__. If someone wanted a completely clean unadulterated app in their test, they could always create one, or clear the existing one.
we didn't init_qactions in the fixture, as we didn't feel it would be best practice to have qt associated bits and bobs registered for every test. These do get registered whenever make_napari_viewer is used, and init_qactions can be called explicitly (as we've done in the build_qmodel_menu test) when you want all the bells and whistles, as long as the appropriate providers are mocked.

If all of this sounds sane and reasonable to you, then I think this is ready for merge? @lucyleeow can confirm as she's been doing all the heavy lifting here.

This sounds in line with what the group discussed before, I think we should start the 24 hour clock for merging

Thank you for the notes @DragaDoncila ! I was half way through typing some but didn't finish. Have added minor notes to the docstring of mock app fixture to add one of the things you mentioned. I agree that this deserves another pair of eyes due to the changes made.
The only thing I would note, for our future reference, is that we mock patched the NapariApplication get_app method and not the function napari._app_model.get_app() because of funny import behaviour observed when mocking napari._app_model.get_app() - we found you had to import napari._app_model.get_app() within the test function for it to work when mocking the function.

One of the CI's is still failing: https://github.com/napari/napari/actions/runs/3254935776/jobs/5343726097
It seems maybe unrelated?

E   ValueError: <enum 'NotificationSeverity'> may only be called with a str or an instance of <enum 'NotificationSeverity'>. Got <class 'shibokensupport.enum_310.auto'>

Okay seems unrelated and can probably be ignored - see https://github.com/napari/napari/issues/5217

Both failures here are unrelated, and I'm happy with the state of the PR atm. I've pulled it down and checked the menu behaves as expected, including the toggle functionality, so I'm going to merge this now. @lucyleeow @potating-potato I think next focus should be on #4977 and #4922 as they don't include any dynamic menu generation.

Thank you @DragaDoncila !!
I will update https://github.com/napari/napari/pull/4977 which should fix the current CI failures!


Add foldable panes into main webpages

This adapts and extends the Boostrap accordion example to have foldable sections in the main webpages of the study groups. This could be used to structure and organize content into subpages. The main files and structure:
_data/accordion.yml:
Each mapping (e.g., subspace) represents a main page (e.g., the main page for the multivariate subspace group). For each mapping, one can add one block per subpage (e.g., the 'resources' page for the subspace group). There are three fields on each block: 'title', 'url', and 'id'. These fields are to be used in the accordion template to generate the foldable sections.
_includes/accordion.html:
This contains the Boostrap code and Liquid/Jekyll code to build the foldable panes using the data from accordion.yml and attributes from the variables page and site.
In the page where the template accordion.html is used, one must define a shortid variable in the front matter indicating which mapping in accordion.yml will be used. The subspages has to be named as <id>.md, where <id> is the corresponding id field in the accordion.yml file. Also, this subpages have to be stored in a subfolder called <shortid>, inside the _study-groups folder.
In the foldable panes, a preview or excerpt of each subpage will be presented. By default, the first paragraph is displayed, but one can control how much to show by adding <!--more--> at the point in the subpage where one wants to stop the preview.

This is looking great, @chmendoza !
Thank you for doing the heavy lifting.
Noticed the html for the subspace page ended up included in you commit.
Could you make sure to remove it?
Here are instructions for that.

Adding this just for the record.
Technical description: here

Redoing this again. I had to do a hard reset and incorporate the changes again.


[BUG] Error: DOM element provided is null or undefined

With patches in the grid and patch embeddings in the panel, clearing the view bar results in the following error:
import fiftyone as fo
import fiftyone.brain as fob
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset("quickstart")
model = foz.load_zoo_model("clip-vit-base32-torch")

embeddings = dataset.compute_patch_embeddings(model, "ground_truth")
fob.compute_visualization(
    dataset,
    embeddings=embeddings,
    patches_field="ground_truth",
    brain_key="gt_viz",
)

session = fo.launch_app(dataset)

Error: DOM element provided is null or undefined
at Object.r [as getGraphDiv] (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.a5fcf1b2.js:5737:1224)
at Object.Et [as purge] (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.a5fcf1b2.js:5737:115837)
at C.value (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.a5fcf1b2.js:4523:5565)
at QAe (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:30550)
at Wu (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:27781)
at QAe (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:29241)
at Wu (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:27781)
at QAe (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:28093)
at Wu (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:27781)
at QAe (http://localhost:5151/datasets/quickstart-demo-0.19/assets/index.d880d008.js:63:27896)

Thanks for taking a look, @imanjra!
There's yet another avenue to this error that @brimoor showed me. Load the embeddings first, and the patches view, and then clear the view bar - or I might be getting the order wrong. I did try various permutations of it and couldn't reproduce it on my end, but let's confirm with @brimoor before we shut the door on this ticket.

@imanjra @sashankaryal I can still reproduce this. From the state pictured above, when I clear the view bar I get this:
Error: DOM element provided is null or undefined
    at Object.r [as getGraphDiv] (http://localhost:5151/datasets/zzz-dom-element/assets/index.66b826f7.js:5816:1224)
    at Object.Et [as purge] (http://localhost:5151/datasets/zzz-dom-element/assets/index.66b826f7.js:5816:115837)
    at C.value (http://localhost:5151/datasets/zzz-dom-element/assets/index.66b826f7.js:4602:5565)
    at o1e (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:30550)
    at tu (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:27781)
    at o1e (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:29241)
    at tu (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:27781)
    at o1e (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:28093)
    at tu (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:27781)
    at o1e (http://localhost:5151/datasets/zzz-dom-element/assets/index.9e69629f.js:63:27896)

Should be fixed by https://github.com/voxel51/fiftyone/pull/2783. Please feel free to re-open it if you see it again.


Difficulty with problem having multiple inputs and outputs

I am trying to load a model that has multiple inputs and outputs. I followed you example but I keep getting segmentation faults errors.
The model is created using the following python code:
def build_model(X_train, d , n ):

    n_inputs = X_train.shape[1]
    n_outputs = Y_train.shape[1]
    model = Sequential()
    model.add(Dense(128, input_dim=n_inputs, activation='elu', kernel_regularizer=regularizers.l2(l2=n)))
    model.add(Dropout(d))
    # model.add(Dense(32, activation='elu', kernel_regularizer=regularizers.l2(l2=n)))
    # model.add(Dropout(0))
    model.add(Dense(n_outputs))
    optimizer  = tf.keras.optimizers.Adam( learning_rate=0.001  , beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False, name='Adam') # These are the default values
    # Smaller learning rate will cause better accuracy, but with higher computaional power (larger training time)

    model.compile(loss ='mse',
                optimizer=optimizer,
                metrics=['mae', 'mse'])

    return model

and I saved the model using the code:
# Save Best Model
BestModel.save(f"{Model_Dir}/Model", save_format='tf')

It should be straight forward using the example that you provided but I keep getting errors.
Perhaps I should treat this as a single input and output but the data type would be a tuple or list?
Am I missing something?
Thanks

Hi @mohamedmkamra
How are then you running the model with cppflow? What error you getting?
As far as I can see is a single-input, single-output model, so I don't think it's a multi-input problem.

Actually the number of inputs is technically not 1.
X_train is a numpy array with number of rows =  ntrainData  and number of columns =  n_inputs
Y_train is a numpy array with number of rows =  ntrainData  and number of columns =  n_outputs
This is why I am not sure about this.

No, ntrainData is the size of training data

PyTorch models are prepared to take inputs of shape NxD (as yours). With N being number of samples, batch dimension, and D being the dimension of each data point.
Without seeing what are you trying is hard to make a guess.

I am not using PyTorch, I am using tensor flow.
I have used tensorflow::keras to train a neural network based on the model above.
def build_model(X_train, d , n ):

    n_inputs = X_train.shape[1]
    n_outputs = Y_train.shape[1]
    model = Sequential()
    model.add(Dense(128, input_dim=n_inputs, activation='elu', kernel_regularizer=regularizers.l2(l2=n)))
    model.add(Dropout(d))
    # model.add(Dense(32, activation='elu', kernel_regularizer=regularizers.l2(l2=n)))
    # model.add(Dropout(0))
    model.add(Dense(n_outputs))
    optimizer  = tf.keras.optimizers.Adam( learning_rate=0.001  , beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False, name='Adam') # These are the default values
    # Smaller learning rate will cause better accuracy, but with higher computational power (larger training time)

    model.compile(loss ='mse',
                optimizer=optimizer,
                metrics=['mae', 'mse'])

    return model

The NN model is simple: it has 5 inputs, and the size of training data is roughly 100K samples with every sample consisting of 5 values as inputs and 2 values as outputs.

Yeah, I meant TensorFlow sorry.
But I don't see how you trying to use cppflow, you only posted Python code. If it's a TensorFlow related issue, it shouldn't be posted here.

The tf model itself is good. It works fine for evaluation when the model files are read in Python.
But when I try to read the model files using cppflow, I get segmentation fault errors:
Basically, I tried using the load_model example
#include
#include "cppflow/ops.h"
#include "cppflow/model.h"
int main() {
auto input = cppflow::fill({10, 5}, 1.0f);
cppflow::model model(std::string(MODEL_PATH));
auto output = model(input);

std::cout << output << std::endl;

auto values = output.get_data<float>();

for (auto v : values) {
    std::cout << v << std::endl;
}
return 0;

}

Try following the answer on StackOverflow https://stackoverflow.com/a/71998020/3957794


[iOS] Sticky carousel view on swipe

I have a ListView with a image header and ImageCell on my CarouselViewControl for each page (max of 8 pages all the same layout of ListView). On iOS when swiping left or right the view "sticks" for a few seconds/milliseconds causing what seems like "lag". The scrolling then jumps when its "unstuck".
I cannot find a way to make this smooth, I'm guessing its how the carousel view locks on to a page and then releases the lock on the page to allow scrolling. Fixing this issue would give us the amazing smooth scroll we need.
If you moving the carousel view after the page is locked (scroll a little) and let it still sit on the same page, when scrolling from then its smooth again (no jumping affect as you already have "unlocked" the view).
Same app running on android and I do not get this sticking problem.
Any ideas for this?

How are you configuring the carousel source? That lag may be related to each view rendering but why there's a lag in iOS? No idea. I will have to see some code.

@alexrainman Hey, sorry for not showing any code. Its a project I am unable to share but I will try to get a example project so we can do some testing on this.

Just having a little look (will still try and post some code soon) but it seems to be where I have the listview itemsource binding. So instead I am now using OnBindingContextChanged and setting the itemsource that was but having the same slow, lagging experience.
Just as a test I have put a 5 second delay inside OnBindingContextChanged before setting the itemsource and the carousel view runs smooth until the listview is populated.
This could just be bad optimisation on the Xamarin side for ListView but Im not sure yet. Just a quick update before I forget.

Can i close this?

@alexrainman Sorry, I have been a bit busy. Will get back to this when I have an example to show.


Add option to select firewalld zone

Fix #6
This add the option to set the firewalld zone in which the ssh service should be mananged.
This option is only used if this role is configured to manage the firewall.
The default behavior doesn't change. Without any modifications this role configures the ssh
service in the public (default) zone. This is current behavior.
This is a new feature without any breaking changes.

No problem. Does this once again mean that somebody else has to do the review now?

I think you should be able to review again, since the PR is mine and the last commit was also made by me. But feel free to suggest another reviewer!


WIP: Please ignore, testing travis jobs

You can also enable Travis against your fork, so you don't have to create a PR, if that's what you want. :wink:

That's a good point, thanks :)


Added Matt's proof for ceilLog2's CLZ implementation

Features:

Included a new directory, labs/SAW/ceilLog2
Added Matt's source code and proof for the CLZ implementation of ceilLog2
Added a Makefile in the ceilLog2 directory to run the example
Updated SAW.md to mention the CLZ verification as a bonus exercise

Looking great! Would you be willing to make artifacts in the CI -- .github/workflows/python-saw.yml ?

Sure thing! Based on the previously included steps in python-saw.yml, I wrote out the explicit steps rather than just calling make after navigating via cd. I take it that it is better to be explicit in the steps that way someone looking at just python-saw.yml can understand exactly what is being checked, right?

Including this for future reference.
If you are pushing workflow changes in Gitpod and a "workflow scope" error appears like this:
gitpod /workspace/cryptol-course/labs/SAW (clz-proof) $ git push origin clz-proof
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 16 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 475 bytes | 475.00 KiB/s, done.
Total 5 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/weaversa/cryptol-course.git
 ! [remote rejected] clz-proof -> clz-proof (refusing to allow an OAuth App to create or update workflow `.github/workflows/python-saw.yml` without `workflow` scope)
error: failed to push some refs to 'https://github.com/weaversa/cryptol-course.git'

Update your Gitpod Integration permissions for GitHub. Click on the 3 dots for the GitHub permissions > select Edit Permissions > check the workflow permission > select Update Permissions. After doing so, GitHub will prompt you to confirm the permission changes and, after confirming them, you'll be able to push your commit.

Looking great! Would you be willing to make artifacts in the CI -- .github/workflows/python-saw.yml ?

Sure thing! Based on the previously included steps in python-saw.yml, I wrote out the explicit steps rather than just calling make after navigating via cd. I take it that it is better to be explicit in the steps that way someone looking at just python-saw.yml can understand exactly what is being checked, right?

I'd rather have the CI run a make prove command (see here for an example, though it doesn't use the python interface). I think this is better because if we decide to build differently (add some header file, change optimization flags, etc.) the proof will be linked to that build process. If you can come up with a good demonstration here, I'd like to change over all of the builds to work this way - we could still have two ci stages (if this is helpful, I'm not sure) - a make proofartifacts and a make prove or something...

Looking great! Would you be willing to make artifacts in the CI -- .github/workflows/python-saw.yml ?

Sure thing! Based on the previously included steps in python-saw.yml, I wrote out the explicit steps rather than just calling make after navigating via cd. I take it that it is better to be explicit in the steps that way someone looking at just python-saw.yml can understand exactly what is being checked, right?

I'd rather have the CI run a make prove command (see here for an example, though it doesn't use the python interface). I think this is better because if we decide to build differently (add some header file, change optimization flags, etc.) the proof will be linked to that build process. If you can come up with a good demonstration here, I'd like to change over all of the builds to work this way - we could still have two ci stages (if this is helpful, I'm not sure) - a make proofartifacts and a make prove or something...

I added the makefile syntax.

I like that Makefile setup! Very nice and clean. I'll add similar Makefiles for the other SAW labs.


adds toolbar to richtext editor

Pulls editor area (editor/preview/toolbar) out from Edit.js into it's own component.

👀

@alanshaw I'm gonna take a look at fixing the conflict in here and will renaming the thing while I'm  in there.


Admin Router: Make AR send proper User-Agent header

High Level Description
All requests originating from the Admin Router must have the User-Agent header properly set. As part of this PR the /internal/acs/api/v1/ endpoint has been moved into the shared master.conf file. This way we deduplicate the code a bit and keep the change DRY without changing the behaviour of AR.
Related Issues

https://jira.mesosphere.com/browse/DCOS_OSS-1445 Admin Router: AR should set unique user agent while doing requests to backend IAM/Mesos/Marathon

Related PRs
EE PR: https://github.com/mesosphere/dcos-enterprise/pull/1541

@mesosphere-mergebot label ready for review

Would you mind noting where the dcos-metadata/dcos-version.json additions are coming from?

@gpaul @jgehrcke I have created a Jira for @karlkfi to investigate why the API docs check missed the changes from https://github.com/dcos/dcos/pull/1866

test_vip failure noted in https://jira.mesosphere.com/browse/DCOS_OSS-1463

mergebot/enterprise/build-status/aggregate cannot go green because there is no mergebot-maintained connection between the upstream and downstream PR (why didn't we do that, Pawel?).

Before this can receive a ship-it label we must have confidence in https://github.com/mesosphere/dcos-enterprise/pull/1541.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee

@mesosphere-mergebot bump-ee


patch: add updated photo

Change-type: patch
Signed-off-by: Alan Boris<EMAIL_ADDRESS>

@balena-ci I self-certify!


[Prometheus-Sample-App] Updating Prometheus Sample App and Adding features LabelCount, DatapointCount, Norm Dist, and K8s cluster config

Added following features:

Configurable constant labels generation
Configurable multiple datapoints generation
Introduced normal distribution for summary metric values.
Created example deployment file to simulate multiple host/endpoints scrapping environment for OTEL Collector using Prometheus Sample App
Updated Readme to run cluster on Kubernetes and on Amazon EKS.

Issue: 49
cc @alolita @Aneurysm9

Merged


CopyList position on mouse cursor scales incorrectly

on my machine only

Issue occurs when desktop scaling isn't 100%. If you change it and change it back, (ie: 125% (broken) -> 100% (fixed) -> 125% (stays fixed)
We may need to do an initial check for desktop scaling settings, or disable DPI scaling on the app itself.

I'm unable to duplicate this now. Once we see this happen again, we can address it


INFO screen in the LCD display not showing after disabling Lightning capabilities

Hi all,
I have a Raspiblitz v1.7.1 running on a Raspberry Pi 4.
The extra services I enabled are JoinMarket and Electrum server.
Last week I disabled Lightning, since I was not making use of it. After that, the screen showing in the LCD is a blue one, instead of the typical INFO screen.

If I connect to my node via SSH and click on the Info option, I can in fact see the info screen in my laptop, but not in the LCD display:

I restarted a couple of times and the problem persists.
Also, I tried this:
sudo init 3
sudo init 5
restart

But no luck either.
Any help will be much appreciated.
Thanks!

Not sure if this is fixed in v1.7.2 should test.

I tested again on my machine for - it looks OK so far for 1.8.0. Closing this issue for now - but please reopen if you spot the problem again.