Code confusion  of filter  chr arm

Hi,
I  have  read  the  code file  shiny_utils.R and   RNAseqCNV_wrappper.R. But  I  do  not  understand  why filter   21p   in  get_arm_metr  function , and  filter(arm != "p" | !chr %in% c(13, 14, 15, 21))   befoe  metr_dipl().
Thanks.

Hi,
I am really glad you decided to look into RNAseqCNV and thank you for your insightful question.
21p does not carry too many genes. Since our gene filtering steps have additionally excluded a number of them, the copy number prediction was not as reliable as we would have liked it to be, so we excluded it from the analysis.
That said, it would be possible to include it in the analysis (perhaps through an additional parameter?).
Regards,
Jan

Hi,
Thanks  for   prompt answer.
If   we  edit the script and  do  not   filter  the 21p  and  the chr 13  14  15,    the  the model_dip  in   randomForest:::predict.randomForest(model_dip , ...)   need  to  be modeled again ?
In  addition, is  the  data  used for  randomForest  modeling  the  40  samples   mentioned on the home page (The in-build standard contains gene expression data from 40 ALL samples without large-scale CNVs ) ?
Thanks .

The edit should work - it is worth a try. If it crashes, let me know. That said, there might too few genes left for a robust prediction of CNV by the model and the figure might be more challenging to interpret. In addition, the 21p arm was not included in the model training data. All in all, I would expect worse accuracy.

No, the model was trained on a larger dataset, which included hundreds of instances of large-scale CNVs. If you would be interested in the method more in-depth, we have recently published a paper on RNAseqCNV - https://www.nature.com/articles/s41375-022-01547-8

If you have any other questions, do not hesitate to write again.


Migrate logs to structured logging

What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:

We have many more locations where structured logging isn't employed and I'd prefer to fix them all in one go instead of biting off such small chunks each time.
@eatjeff Would you like to migrate the other calls, too?

/remove-lifecycle rotten


Add comments, how can openai respond to Chinese?

Add comments, how can openai respond to Chinese?
thanks!

Translate the commands to chinese:

This is the way in this extension to OpenAI can respond in another languages.

thanks!


InfluxDB can't store field with different type under the same measure…

Description:
For the components like media_players the possible states:
STATE_IDLE, STATE_OFF,  STATE_PAUSED, STATE_PLAYING, STATE_UNKNOWN
The helper function state_as_number can't convert STATE_IDLE, STATE_PAUSED, STATE_PLAYING to numeric value:
if state.state in (STATE_ON, STATE_LOCKED, STATE_ABOVE_HORIZON,
                   STATE_OPEN):
    return 1
elif state.state in (STATE_OFF, STATE_UNLOCKED, STATE_UNKNOWN,
                     STATE_BELOW_HORIZON, STATE_CLOSED):
    return 0

return float(state.state)

It is not clear how to classify STATE_IDLE as 0 or 1.
When media player state is  IDLE, PAUSED or PLAYING the user would like to have this information stored in the InfluxDB.
My suggestion is that instead of using state_as_number function rather trying to convert to float directly. If state is a number represented by string or is already a number it will be stored as a number, if state is string it will raise a ValueError and will be stored as a string into InfluxDB.
With the current implementation the STATE_OFF is converted to number 0 but other states are strings. InfluxDB does not allow to store the same field with different type under single measurement.
I haven't updated the test and it fails now but first wanted to discuss if this change makes any sense.
Related issue (if applicable): fixes #
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#
Example entry for configuration.yaml (if applicable):

Checklist:
If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If code communicates with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

…ment

We should try to convert as many states to numbers, especially if they are strings. Just trying to cast random strings to floats is not going to help.

The current implementation throws an exception:
ERROR:homeassistant.components.influxdb:Error saving event "[{'tags': {'entity_id': 'lazienka', 'domain': 'media_player'}, 'measurement': 'media_player.lazienka', 'time': datetime.datetime(2016, 6, 7, 20, 1, 24, 594564, tzinfo=<UTC>), 'fields': {'value': 0}}]" to InfluxDB Traceback (most recent call last):   File "/Users/klstanie/PycharmProjects/github.com/kstaniek/home-assistant/homeassistant/components/influxdb.py", line 103, in influx_event_listener     influx.write_points(json_body)   File "/Users/klstanie/.virtualenvs/home-assistant/lib/python3.5/site-packages/influxdb/client.py", line 391, in write_points     tags=tags)   File "/Users/klstanie/.virtualenvs/home-assistant/lib/python3.5/site-packages/influxdb/client.py", line 436, in _write_points     expected_response_code=204   File "/Users/klstanie/.virtualenvs/home-assistant/lib/python3.5/site-packages/influxdb/client.py", line 278, in write     headers=headers   File "/Users/klstanie/.virtualenvs/home-assistant/lib/python3.5/site-packages/influxdb/client.py", line 248, in request     raise InfluxDBClientError(response.content, response.status_code) influxdb.exceptions.InfluxDBClientError: 400: {"error":"field type conflict: input field \"value\" on measurement \"media_player.lazienka\" is type int64, already exists as type string"}
Only STATE_OFF and STATE_UNKNOWN for media players are converted into the numbers. When the initial state is converted to number the measurement is created with numeric type for value. All the other states are not converted to the number causing the exception.
I was thinking of extending the state set in state_as_number to cover the media player states but converting into simple 1 or 0 will lose the information about the exact state of media player, i.e is in IDLE mode or PLAYING.
Any suggestions who it could be solved other way?


Artefact remains if beginButton / endButton is used and a button is hidden in ResponsivePopover via XML, not happens in case of implementing via JS

OpenUI5 version: 1.98.0
I'm playing with Sample: Shell Bar with Product Switch (Experimental) controller.
Wherever a product switch should have a close button or not depends on a device type and is implemented in a controller:
if (Device.system.phone) {
	oPopover.setEndButton(new Button({text: "Close", type: "Emphasized", press: this.fnClose.bind(this)}));
}

Everything is OK.
Now, I want to implement the same behaviour but in a declarative way (via the XML-template with the expression binding):
<endButton>

	<Button
		text = "Close"
		type = "Emphasized"
		press = ".fnClose"
		visible = "{device>/system/phone}" />

</endButton>

It works too, but an artefact remains if a device type is a desktop.
On a phone:

On a desktop:

On a desktop there should not be such an empty strip:

I assume, the reason for such issue is an endButton wrapper, which remains unhidden. If I don't use endButton, there is no such empty stip but then a button is aligned to the left and not to the right.
Unfortunately, endButton has no visible property or at least I applied it in a wrong way.
How to get rid of this empty strip in case of a declarative definition?

Hello @,
Thank you for sharing this finding. I've created an internal incident<PHONE_NUMBER> . The status of the issue will be updated here in GitHub.
Regards,
Iliana

Hi @pubmikeb ,
I see your point. Unfortunately, there are no conditional tags in XML views. In this case, it's entirely application logic. It is a valid scenario to have a footer button for desktop, for example. That's why it's implemented conditionally in the controller.
Basically, you could have an XML View with the Button inside, but in the controller, you'd need to implement the reverse logic- to remove the button where it's not needed. The workaround with the visible property won't work as the aggregations often need some more "support" layouting.
Cheers


Add functional style regex pattern-matching

GenericNPC reported this on 2013-04-07T04:25:10Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=9895
CC List

bearophile_hugs
dmitry.olsh (@DmitryOlshansky)

Description
Based on Scala's pattern-matching with regular expressions syntax, I created the function std.regex.regexSwitch(formerly switchRegex). It's main use-case is for reading textual input fields that can be in multiple formats:

    enum WeekDay
    {
        Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday
    }

    struct Schedule
    {
        WeekDay day;
        int hour;
    }

    assert(equal(
                [
                    Schedule(WeekDay.Sunday, 1),
                    Schedule(WeekDay.Monday, 14),
                    Schedule(WeekDay.Tuesday, 3),
                    Schedule(WeekDay.Wednesday, 4),
                    Schedule(WeekDay.Thursday, 17),
                    Schedule(WeekDay.Friday, 6),
                ],
                [
                    "Sunday 1AM",
                    "Monday 2PM",
                    "Tuesday 3",
                    "4AM Wednesday",
                    "5PM Thursday",
                    "6 Friday",
                ].map!(regexSwitch!(
                        `(\w+) (\d+)AM`, (WeekDay day, int hour) => Schedule(day, hour % 12),
                        `(\w+) (\d+)PM`, (WeekDay day, int hour) => Schedule(day, hour % 12 + 12),
                        `(\w+) (\d+)`, (WeekDay day, int hour) => Schedule(day, hour),
                        `(\d+)AM (\w+)`, (int hour, WeekDay day) => Schedule(day, hour % 12),
                        `(\d+)PM (\w+)`, (int hour, WeekDay day) => Schedule(day, hour % 12 + 12),
                        `(\d+) (\w+)`, (int hour, WeekDay day) => Schedule(day, hour),
                        ))()));

See https://github.com/D-Programming-Language/phobos/pull/1241

dmitry.olsh (@DmitryOlshansky) commented on 2013-04-08T13:33:32Z
(In reply to comment #5)
> (In reply to comment #3)
> > For the moment we are (sort of) at the stage 2.
>
> Oh, sorry. Had no idea about needing the `pull` keyword, or auto-closing when
> the request is pulled. My previous contribution to Phobos was done entirely on
> GitHub...
>
> Just to be clear - the rule is still to use a single commit message for the
> contribution, using git's history rewriting functionality to make fixes, right?

Yup, all the same. Plus this new idea of issue pre new stuff too.

I've seen cases where it the hook doesn't do closing. Could be a glitch or that an issue was mentioned but not as "fix issue xyz" but rather simply "issue xyz".

GenericNPC commented on 2013-04-08T10:22:37Z
(In reply to comment #1)
> Another way to do this is with the unapply() of Issue 596

Yes, it looks like it is possible to do this with unapply(which should be opUnapply to be compatible with D's naming conventions). However:

1) In the GitHub pull request, Dmitry Olshansky suggests to combine all patterns to a single, big pattern, to improve performance. I have no idea how to do it, but it might be done in the future, and it can't be done with the switch+opUnapply version.

2) switch in D is a statement, not an expressions - it does not return a value.

3) Scala's unapply for regular expressions does not convert data types. I don't know if it can be done in D's version - it's too early to tell, seeing that opUnapply has not been implemented yet.

4) opUnapply is not in D yet - it's still an enhancement suggestion. We don't know what complications we are gonna have when we try to use it for regular expressions.

dmitry.olsh (@DmitryOlshansky) commented on 2013-04-08T13:03:10Z
Only to add:
5) We can always add better abstraction if/when we are confident it's actualy better.

P.S. Doing minor correction - I'm reopening this untill the pull gets in. Let's do it pedantically in order ;)

@IdanArye

What we've unofficially all came to is a 3 step procedure:
1. Report
2. Pull posted there and keyword _pull_ is set.
3. Commits get posted automatically by post-commit hook, then somebody takes time to check and close it.

For the moment we are (sort of) at the stage 2.

bearophile_hugs commented on 2013-04-07T04:43:15Z
Another way to do this is with the unapply() of Issue 596

GenericNPC commented on 2013-04-08T13:10:03Z
(In reply to comment #3)
> Only to add:
> 5) We can always add better abstraction if/when we are confident it's actualy
> better.
>
> P.S. Doing minor correction - I'm reopening this untill the pull gets in. Let's
> do it pedantically in order ;)
>
> @IdanArye
>
> What we've unofficially all came to is a 3 step procedure:
> 1. Report
> 2. Pull posted there and keyword _pull_ is set.
> 3. Commits get posted automatically by post-commit hook, then somebody takes
> time to check and close it.
>
> For the moment we are (sort of) at the stage 2.

Oh, sorry. Had no idea about needing the `pull` keyword, or auto-closing when the request is pulled. My previous contribution to Phobos was done entirely on GitHub...

Just to be clear - the rule is still to use a single commit message for the contribution, using git's history rewriting functionality to make fixes, right?

dmitry.olsh (@DmitryOlshansky) commented on 2013-04-08T13:03:43Z
https://github.com/D-Programming-Language/phobos/pull/1241

dmitry.olsh (@DmitryOlshansky) commented on 2016-04-06T13:18:16Z
The key building block has arrived:

https://github.com/D-Programming-Language/phobos/pull/4158

GenericNPC commented on 2013-04-08T14:30:50Z
(In reply to comment #6)
> (In reply to comment #5)
> > (In reply to comment #3)
> > > For the moment we are (sort of) at the stage 2.
> >
> > Oh, sorry. Had no idea about needing the `pull` keyword, or auto-closing when
> > the request is pulled. My previous contribution to Phobos was done entirely on
> > GitHub...
> >
> > Just to be clear - the rule is still to use a single commit message for the
> > contribution, using git's history rewriting functionality to make fixes, right?
>
> Yup, all the same. Plus this new idea of issue pre new stuff too.
>
> I've seen cases where it the hook doesn't do closing. Could be a glitch or that
> an issue was mentioned but not as "fix issue xyz" but rather simply "issue
> xyz".

I used the GitHub syntax and wrote "fix #9895". I've changed it now to "fix issue 9895" to avoid problems...


go: 1.17.7 -> 1.17.8

Signed-off-by: Julien Pivotto<EMAIL_ADDRESS>
Motivation for this change
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Duplicate of #162671 (already on staging)

As this is a security fix, I would have expected it to go straight to unstable.

As this is a security fix, I would have expected it to go straight to unstable.

No, not every security fix to goes to master. Depends on severity and number of rebuilds.


Firestore Reactive Sample

Add a sample for demonstrating Firestore reactive Spring Data usage.
Fixes #1908.

@dzou Can we also add integration tests like for other samples? I think there is some testing support in Spring for Web Flux.

@dzou Can we also add integration tests like for other samples? I think there is some testing support in Spring for Web Flux.

Will do!

@dzou Can we also add integration tests like for other samples? I think there is some testing support in Spring for Web Flux.

Done. Added integration tests.


Aider crashes after some time of use

Issue
I have a couple of time crashed aider after some time of use. Can't recall what the previous error was but I suspect it was the same as this time. Here are the last three lines from the console:
site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)
Version and model info
Aider 0.31.1
Models: gpt-4-1106-preview with udiff edit format, weak model gpt-3.5-turbo
Running Aider in VSCode terminal on WSL2/Ubuntu 22.04.4 LTS.

Thanks for trying aider and filing this issue. I've added httpx.RemoteProtocolError to the list of errors that aider automatically tries to recover from by retrying.  The change is available in the main branch. You can get it by installing the latest version from github:
python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git

If you have a chance to try it, let me know if it works better for you.

I'm going to close this issue for now, but feel free to add a comment here and I will re-open or file a new issue any time.


CustomTextLayout/CustomTextRenderer performance tweaks

I cribbed most of the CustomTextLayout and CustomTextRenderer code from samples. One of these samples was written by @fwdr in the first place. He dropped by on email and said that there are a lot of things that could be done in this space to optimize the performance and workload for the Terminal specifically.
So this represents his e-mail based commitment to come back around to this and show us the right way to do it.

We're pretty sure we did this with a series of enhancements to the Dx Engine over the course of 2020 & 2021. Also, the new atlas renderer seems like it totally obsoletes the need for this one 😄


[Feature Request] Allow the debugger to ignore specific raised exceptions

Given that Python uses iterators basically everywhere and that normal iterator behavior throws a StopIteration exception to signal reaching the end of the iteration, it would be rather useful to be able to specify a set of exceptions to ignore during debugging when breaking on Raised Exceptions is enabled. Even just the ability to ignore StopIteration exceptions would be incredibly useful.
VS Code: Version 1.33.1
ms-python.python extension: Version 2019.4.2

Duplicate of #850


python310Packages.sasmodels: 1.0.5 -> 1.0.6

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.sasmodels is: "Library of small angle scattering models"
meta.homepage for python310Packages.sasmodels is: "http://sasview.org"
meta.changelog for python310Packages.sasmodels is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.0.6 with grep in /nix/store/wfm4f9dmh513gh1z73cng4psyi4gyzw2-python3.10-sasmodels-1.0.6

found 1.0.6 in filename of file in /nix/store/wfm4f9dmh513gh1z73cng4psyi4gyzw2-python3.10-sasmodels-1.0.6

Rebuild report (if merged into master) (click to expand)

6 total rebuild path(s)

3 package rebuild(s)

3 x86_64-linux rebuild(s)
3 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
python310Packages.sasmodels
python39Packages.sasmodels
sasview

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wfm4f9dmh513gh1z73cng4psyi4gyzw2-python3.10-sasmodels-1.0.6 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.sasmodels https://github.com/r-ryantm/nixpkgs/archive/7ec5abb10d6cdb168ccb0a059ad4e0ff726367ec.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wfm4f9dmh513gh1z73cng4psyi4gyzw2-python3.10-sasmodels-1.0.6
ls -la /nix/store/wfm4f9dmh513gh1z73cng4psyi4gyzw2-python3.10-sasmodels-1.0.6/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

python310Packages.sasmodels
python39Packages.sasmodels
sasview

Maintainer pings
cc @rprospero for testing.

Result of nixpkgs-review pr 164831 run on x86_64-linux 1

1 package failed to build:

sasview

2 packages built:

python310Packages.sasmodels
python39Packages.sasmodels

Result of nixpkgs-review pr 164831 run on x86_64-linux 1

3 packages built:

python310Packages.sasmodels
python39Packages.sasmodels
sasview


typo line 65 of node-syslog.js

Hi,
on line 65 of node-syslog.js you still have a typo saying "faculty" instead of "facility", resulting in messages being sent as "kern.emerg", popping up all over in the console.
64: this.connect(function (e) {

65:            var pri = '<' + ((that.options.faculty * 8) + severity) + '>'; // Message priority

65:           var pri = '<' + ((that.options.facility * 8) + severity) + '>'; // Message priority
66:            var entry = pri + [

Looks like the author didn't gave much of a :hankey: about this anymore. Fork ahead..


Re-use library versions from config

In model-verifier there is a JUnit Pioneer version set in-place. We want to reuse the version values from the config repository.
We should also review other modules for similar issues.

Done for base, core-java and web. The gcloud-java and jdbc-storage are already okay.


change(deps): upgrade zcash_script and zcash dependencies

Motivation
See https://github.com/ZcashFoundation/zebra/issues/4767
To do that we must update zcash_script and related dependencies.

Specifications

Designs

Solution
Update zcash_script which allows us to upgrade the zcash dependencies.
Updates zcash_proofs to upstream again and not our fork, after the Sprout params downloading code was merged.
Closes https://github.com/ZcashFoundation/zebra/issues/4767

Review
This PR is blocked on:

https://github.com/ZcashFoundation/zcash_script/pull/37 being accepted, a release being made, and changing this PR to point to it
zcash_proofs being released so that we don't depend on a git commit, which will allow us to remove duplicated dependencies (optional; we may skip this if we want) This may take a while so let's not wait for it

Reviewer Checklist

[ ] Code implements Specs and Designs
[ ] Tests for Expected Behaviour
[ ] Tests for Errors

Follow Up Work

I updated Cargo.lock to resolve merge conflicts with main.


Update Favicon and website title

Hey @VaibhavArora314, I want to work on this issue, kindly assign this to.


account list: add '--refresh' to sycn up the latest subscriptions from server

Fix #919

[x] The PR has modified HISTORY.rst with an appropriate description of the change (see Modifying change log).

Command Guidelines

[x] Each command and parameter has a meaningful description.
[x] Each new command has a test.

(see Authoring Command Modules)

Codecov Report

Merging #4091 into master will increase coverage by <.01%.
The diff coverage is 79.03%.

@@            Coverage Diff             @@
##           master    #4091      +/-   ##
==========================================
+ Coverage   70.49%   70.49%   +<.01%
==========================================
  Files         437      437
  Lines       28086    28132      +46
  Branches     4300     4314      +14
==========================================
+ Hits        19800    19833      +33
- Misses       7004     7017      +13
  Partials     1282     1282

Impacted Files
Coverage Δ

...ofile/azure/cli/command_modules/profile/_params.py
68.42% <100%> (+1.75%)
:arrow_up:

...rofile/azure/cli/command_modules/profile/custom.py
37.07% <60%> (-0.86%)
:arrow_down:

src/azure-cli-core/azure/cli/core/_profile.py
86.44% <80.35%> (-0.96%)
:arrow_down:

...li-cloud/azure/cli/command_modules/cloud/custom.py
13.79% <0%> (ø)
:arrow_up:

...nent/azure/cli/command_modules/component/custom.py
16.23% <0%> (ø)
:arrow_up:

...dback/azure/cli/command_modules/feedback/custom.py
31.25% <0%> (ø)
:arrow_up:

src/azure-cli-core/azure/cli/core/util.py
70.66% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3c613f...8ef9ee8. Read the comment docs.


RGBLED Missing active_high param

I've been asked about common anode RGBLEDs which work active low, and noticed that RGBLED doesn't have an active_high parameter, which would fix this.

Okay, this is me showing my utter electrical ignorance again and not realizing that of course PWM devices could be wired active-low. Doh! Should have this fixed up in a bit (I'll add an active_high parameter to PWMOutputDevice, which PWMLED will inherit then add it to RGBLED too)


[release-v1.9] Use MAXFILLUP scheduler

This is a missing patch on 1.9, see 1.8 branch:
https://github.com/openshift-knative/eventing-kafka-broker/blob/454166cc4fdfe038639589d078dd7b39f9721779/control-plane/pkg/reconciler/consumergroup/controller.go#L245

/cc @maschmid

/lgtm


Support for poetry

I love your software but I do miss pyproject.toml files in the list here:
https://mybinder.readthedocs.io/en/latest/using/config_files.html#config-files
I think I will try to build my own docker image for now.
HNY

I would also love support for pyproject.toml! I'd add that it's not just used by poetry, but is now the "standard practice", per the python packaging authority and is tool-agnostic.
(I'm currently switching from using setup.py to pyproject.toml for a package, and an environment is created and launched, but it contains none of my dependencies, so I'm assuming pip install . wasn't run.)

I'd love support for pyproject.toml as well. Are there any plans if/when this might happen?


Warning regarding cookies required for lucky

The warning regarding cookies required for "Lucky!" seems to ignore the CPS of the purchase. I postponed the purchase of "Kitten engineer" until the warning in the tooltip of the upgrade disappeared. Nevertheless, I ended far below the optimal bank balance for Lucky. I suspect the 40% CPS boost of the Kitten upgrade were not considered in the calculation.

It's a setting in CM...

Oh great to hear and sorry for the inconvenience. Found it!

Nah, it's fine, I get it all the time, lol.  Honestly, the default way is more efficient, but I can see the other way.


This is a test issue

Testing. Having problems with my repo. Testing

I received the test. Did you receive it?


SDP-845: Wallet provider management

What
This PR creates a Wallet Provider management page.
Known limitations
Contact info and Available countries info not included due to lack of necessary BE data

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

The page is crashing for me.

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

Something went wrong with PR preview build please check

Something went wrong with PR preview build please check

Something went wrong with PR preview build please check

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/

stellar-disbursement-platform-frontend-preview is available here:http://sdp-frontend-pr14.previews.kube001.services.stellar-ops.com/


setup.cfg and install_requires

Package dependencies are often put in the install_requires field of setup.cfg like:
https://github.com/eight04/ptt-article-parser/blob/de6639fbd0797e7c2fe9df853ed82cbeeed49bb4/setup.cfg#L30-L32
Is it possible to sync the version number with install_requires after pipm install/uninstall? For example:
# requirements.txt
a==0.1.0
b==0.2.0

# setup.cfg
a~=0.1.0
b~=0.2.0

After running command pipm update a
# requirements.txt
a==0.2.1
b==0.2.0

# setup.cfg
a~=0.2.1
b~=0.2.0

It is possible to implement such feature. Currently pipm relies on the pip's ability to manipulate requirements files. If you are interested in adding this feature, Have a look at update command. This can be used as the config parser.

I made a tool which uses setup.cfg to save production dependency:
https://pypi.org/project/vpip/
It is built on top of py3's venv, so it doesn't work with py2.


Make braille glyphs with connecting boxes, not dots.

Noting the fact that blind people do not read braille on computer screens..
We conclude that the braille glyphs with connecting blocks are more suitable for terminal applications.

Dot braille is less useful.

Suggest you revise agave to feature box-braille.

Hey clort, thanks for the issue. Personally, I agree, I do like using glyphs in gaming and graphical contexts, but recently in https://github.com/blobject/agave/issues/45 I decided to abide by "ADA Accessibility Guidelines".
I think the way to go would be to provide these as variants (https://github.com/blobject/agave/issues/72) and maybe default to the variant that would see the most use. I've also seen braille used to show quasi-graphical graphs (pretty tops, audio visualisation, etc.), so I do like the dotted variant as well.


[SPARK-44719][SQL] Fix NoClassDefFoundError when using Hive UDF

What changes were proposed in this pull request?
This PR changes jackson-mapper-asl's scope from test to ${hive.deps.scope}.
Why are the changes needed?
Fix NoClassDefFoundError when using Hive UDF:
spark-sql (default)> add jar /Users/yumwang/Downloads/HiveUDFs-1.0-SNAPSHOT.jar;
Time taken: 0.413 seconds
spark-sql (default)> CREATE TEMPORARY FUNCTION long_to_ip as 'net.petrabarus.hiveudfs.LongToIP';
Time taken: 0.038 seconds
spark-sql (default)> SELECT long_to_ip(2130706433L) FROM range(10);
23/08/08 20:17:58 ERROR SparkSQLDriver: Failed in [SELECT long_to_ip(2130706433L) FROM range(10)]
java.lang.NoClassDefFoundError: org/codehaus/jackson/map/type/TypeFactory
	at org.apache.hadoop.hive.ql.udf.UDFJson.<clinit>(UDFJson.java:64)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
...

Does this PR introduce any user-facing change?
No.
How was this patch tested?
manual test.

thanks, merged to master

In addition to jackson-mapper-asl, jackson-core-asl is also recovered. So, this is a revert of SPARK-43225 in terms of the dependency.

This PR changes jackson-mapper-asl's scope from test to ${hive.deps.scope}.


:art: Add support for CHARSET parameter

Without CHARSET=UTF-8 I can't support special characters currently.

Hello @iceydee,
This pull request has been rejected for the following reasons:

The charset for vCards  is  UTF-8. Any value other than "UTF-8" in a MIME charset parameter is invalid. And so it follows that all vCards are expected to be in UTF-8 and therefore there is no need for this to be repeated within the vCard itself. Infact, RFC 6350 (the RFC) specifically removes the CHARSET parameter in version 4.0 vCards (See Appendix A2 of the RFC). This library strictly complies with RFC 6350 and its extensions.
AnyParameter (despite the name) is not for any parameter. It is defined in the RFC as either an IANA token or an X-name. The charset parameter you propose to introduce does not fall within either of those.
The changes you have made to AnyParameter have not been accompanied by complementary changes to the tests and type declarations of AnyParameter.
There is no bug in the build script.

I make the following suggestions in relation to your troubles with special characters:

The problem is probably the parser in whatever test device your using. If it only seems to work when you use the CHARSET parameter (which was used in older vCard versions), then it does not support version 4.0 vCards. This is a strictly version 4.0 library. So, either change libraries or change test devices 🙃.

Just stumbled over this, because obviously MacOS' Contact app can't, in the year 2023, understand vCard v4.0.
So I will just patch my local version of this lib with this patch (thanks @iceydee) and be happy.
Standards are only useful if someone actually use them. Its so frustrating.


🐛 Bug Report: Using gitlabRepoPush or gitlabMergeRequest actions resulting in 400 invalid request of files already existing when using create option to repo

📜 Description
We are fetching our template to ./outputs where we think . is a path like tmp//outputs and when we debug wait we can validate the files are being fetched.
In our sourcePath for gitlab we are using ./outputs
The actions are creating the branch and giving 400 on when going to commit the files when using commitAction of 'create'.
All online examples show something similar to 'src' or './path/to/changes' so it is hard to know for sure but all debugging leaves us to believe we are doing it right and it has access to create the branch and we've got it to create groups so token seems ok
👍 Expected behavior
commit net new files that don't currently exist in the repo into the repo using commitAction of create
👎 Actual Behavior with Screenshots
400 error invalid request please make sure committed files aren't there
👟 Reproduction steps
running template
📃 Provide the context for the Bug.
No response
🖥 Your Environment
No response
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
No, I don't have time to work on this right now

I think that this could help with this issue https://github.com/backstage/backstage/pull/25548

I would suggest that the better solution is to remove commitAction, instead of adding auto to the list like #25548. The solution of relying on Repositories.tree is brilliant and can be applied by default for both publish:gitlab:merge-request and gitlab:repo:push (which is not covered in that PR).
As a template user, we likely want the same thing: Always be able to push a collection of files to a branch in a repo. This means:

If the branch exists, let user choose to create a new commit on top, or drop what we have and create a new one from the latest main branch. These can even be different scaffolder actions instead of via input configuration.
If files existed, always update it; else, always create it. Since scaffolder actions are stateless (until ctx.checkpoint somehow can help remember previous creations), there is not a use for delete.

My hunch is that during the original design of this scaffolder action, the gitbeaker's Commits.create method was not well-understood. This resulted in this configuration flowing outward to become a setting for this scaffolder action.
Instead of further extending that misdirection, let's help to redirect it to do the right thing. Else things get more contrived. E.g. people will not understand why we need getFileAction in the future, and take some time to see that it's a hack.
I can implement for us by following @Yghore's code if there isn't any one available, but I would like to seek greenlight to deprecate the commitAction configuration first. Also whether there's a need to even set it to deprecate, considering likely no one is using update and delete by itself imo... 😂

@Teo-ShaoWei, feel free to make some improvements to the action. Deprecating the commitAction sounds sane to be honest.
Just want to also make you aware of https://github.com/backstage/backstage/issues/22244 that we are considering, but maybe they could be entirely different actions instead to support all providers that don't use specific API's, just use native git instead.

@benjdlambert ooooh it's TIL for me, thanks for letting me know about #22244. Am loving it! ❤ I have a similar thought that we need more design leadership as I was writing the above. I'm glad to know we think alike here.
Once the design has been ironed out, I'll be interested in contributing to #22244 with the community (as long as I can get work hours on it =P). isomorphic-git's commit implementation makes more sense too. Other than processing a local folder as a whole (and deciding this create/update/delete mess for me), it also allows the author to be one of our human users, and Backstage to be a committer with a GPG key, which I think is the correct representation of a user-triggered but Backstage-scaffolded commit.


fix-#473: Fixed the hovering issue

Summary
Briefly describe the purpose of this PR.
Description
Explain in detail what this PR is all about. This should include the problem you're solving, the approach taken, and any technical details that reviewers need to know.
Images
Include any relevant images or diagrams that can help reviewers visualize the changes, if applicable
Issue(s) Addressed
Enter the issue number of the bug(s) that this PR fixes

Template should be strictly Closes <issue_number>
Example: Closes #1

Prerequisites

[ ] Have you followed all the CONTRIBUTING GUIDELINES?

Hey @krishnaacharyaa
I wanted to check in regarding the status of the PR. If there are any issues or concerns preventing it from being merged, please let me know, and I'd be happy to address them.

How is this fixing hovering issue? i see you have added rounded corners in the top @chaitya-titan


[Zeppelin 914]- Apply new mechanism to IgniteInterpreter

What is this PR for?
This handles replacing the registration of interpreter with static block by the interpreter-setting.json file
What type of PR is it?
[| Improvement |
Todos
Sub-Task
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-914
How should this be tested?
There shouldn't be any warning like below on starting the server
INFO 2016-09-29 00:25:46,247 - Bootstrapping Ignite Interpreter WARN 2016-09-29 00:25:46,250 - Static initialization is deprecated for interpreter Ignite, You should change it to use interpreter-setting.json in your jar or interpreter/{interpreter}/interpreter-setting.json INFO 2016-09-29 00:25:46,250 - Inclass=org.apache.zeppelin.ignite
And ensure that the Ignite related paragraphs run without any error
Screenshots (if appropriate)
Questions:
Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

Reopening for travis build

Reopening for Travis build.

Reopening for TRavis

@meenakshisekar The failures of this PR is related to actual behavior, not flaky. Can you take a look into the result of the test?

@meenakshisekar thank you for contribution! Could you rebase on the latest master please?
Also, please do not hesitate to let us know in case you need any help with this one, in order to get it merge!

@meenakshisekar Could you rebase this PR?

Hello!
I would like to help with ZEPPELIN-804 subtasks and make it possible to complete refactoring
If you don't mind I will rebase this PR and take a look at #1077

Ok sure please go ahead
On Wed, Dec 28, 2016 at 3:29 PM, DrIgor<EMAIL_ADDRESS>wrote:

Hello!
I would like to help with ZEPPELIN-804
https://issues.apache.org/jira/browse/ZEPPELIN-804 subtasks and make it
possible to complete refactoring
If you don't mind I will rebase this PR and take a look at #1077
https://github.com/apache/zeppelin/pull/1077
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/apache/zeppelin/pull/1561#issuecomment-269454686, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AKjsi4s-lIESQAOnzEj0YICFLs17cr66ks5rMjMagaJpZM4Kf7Ua
.

--
Thanks & Regards,
Meenakshi.

New pull request #1819
I rebased it on the latest master and fixed failing test

@meenakshisekar can you please close this PR?


Classically keep track of non-empty holes for the fox in the hole example

As the game progresses, it becomes increasingly slow (up to several seconds for games of 8 moves) to play. Partly, this is because of the game re-computing the probability distribution for the state at every move in order to determine a 'source' from which the fox can move.
This PR will circumvent that by keeping track of the empty holes classically.
Important: The real issue is an underlying one in the way this sampling happens. After pop()'s, ancillary qubits are added to the quantum world, which are subsequently also being sampled (I think?). That may not be the desired behaviour.

Here is an example of the behaviour, and a simple timing profile (over the entire length of the game)

Hi.
In my understanding, it is planned that a similar kind of optimization will be implemented in the Unitary library itself. If this is the case I would not add this change to the game, as it makes it more complex, although it was planned as a simple example for using the library. Also, it uses some kind of "meta"-knowledge about the underlying quantum operations (i.e. after a q-move which qbit will/will not have guaranteedly zero coeff/probability). Maybe, this is trivial for us, but a game-developer user of the library without quantum knowledge may not know such details.
If any of you does not agree, let's have a discussion about this on the next Tuesday meeting. I am open to change my mind.
Máté

You raise two important points (doing this in unitary & this making use of quantum knowledge this way). As far as I am concerned this should probably settle the issue in favor of not merging this. It was an attempt at speeding this up temporarily, though as I noted the real issue is a deeper one. Let's put our resources on fixing that instead.


can we have mutiple channels?

Please add ability to give it a list of channels and it can sync all files in the channels with/from special name directories.

not easy but I'll try for it later.


Use go-git functionalities for common git operations

Part of https://github.com/aws-controllers-k8s/community/issues/835
This patch updates all the helper functions to use go-git library
instead of having to call os/exec.Command/os/exec.CombinedOutput in
order to clone repository, fetch tags... These changes brings more
dependencies to our go.mod but in exchange it simplifies the way we
interact with git repositories and improves errors handling and
readability.
Future changes will also benefit from this patch, especially
conversation webhook generator that will need to checkout multiple
version tags.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Needs ATTRIBUTIONS.md update
/hold

/unhold

/approve
I am sure you tested this locally for some controller. Good to add the output in PR since there are no new unit tests.

Adding unit tests
/hold

Looks like it's very complex to unit test these util functions, and it's almost equivalent of testing the go-git library. @vijtrip2 and I agreed on leaving the PR as is and just run some local tests.
On my local machine
# First run (no local aws-sdk-go cache)
time ACK_GENERATE_CACHE_DIR=/tmp/gitest AWS_SDK_GO_VERSION=v1.38.60 make build-ack-generate build-controller SERVICE=ecr
building ack-generate ... ok.
Copying common custom resource definitions into ecr
Building Kubernetes API objects for ecr
Generating deepcopy code for ecr
Generating custom resource definitions for ecr
Building service controller for ecr
Generating RBAC manifests for ecr
Running gofmt against generated code for ecr
ACK_GENERATE_CACHE_DIR=/tmp/gitest AWS_SDK_GO_VERSION=v1.38.60 make     54,08s user 3,20s system 50% cpu 1:53,98 total

# Second run
time ACK_GENERATE_CACHE_DIR=/tmp/gitest AWS_SDK_GO_VERSION=v1.38.60 make build-ack-generate build-controller SERVICE=ecr
building ack-generate ... ok.
Copying common custom resource definitions into ecr
Building Kubernetes API objects for ecr
Generating deepcopy code for ecr
Generating custom resource definitions for ecr
Building service controller for ecr
Generating RBAC manifests for ecr
Running gofmt against generated code for ecr
ACK_GENERATE_CACHE_DIR=/tmp/gitest AWS_SDK_GO_VERSION=v1.38.60 make     8,04s user 1,27s system 176% cpu 5,264 total

/unhold

/lgtm


gnatsd-server delivers duplicate messages

We are experiencing issues where gnatsd will keep resending the same message for extremely long periods of time. Our gnatsd server is NATS server version 0.9.4.
From time to time we see that gnatsd sends the same data over and over. It seems more frequent when traffic is burstable and slow consumers are detected.
This is not a queued subscriber. When this scenario is detected our timestamps we use to measure latency just don't make sense. We take a timestamp and place in message when we receive data and use our subscriber to check with its timestamp what the latency generally is. There are periods where we see this number go into well over 48 hours, sometimes into the 100's of hours.
At the time we detect this we see that the actually message is the same message being repeated over and over and over. It appears the entire system does not just halt and send repeats as other messages have realistic timestamps.
Also it should be noted that our monitoring is constantly running. When we see our reports of over 48hours latency we did not slowly grow to 48hours but all of a sudden we have packets being delivered with extremely old timestamps. Meaning our latency will be 400 ms or so and then all of a sudden spike where some messages are being delivered over and over again from the past.
Has anyone experienced this type of issue? I would expect in a fire-forget system that when a new consumer does subscribe that a message from over 48h ago would not be delivered over and over.
Also at the same time I see the duplicated message, i see other timestamps that are reasonable and under 300 ms.
In our environment we use both java and go clients.
github.com/nats-io/go-nats:6b6bf392d34d01f57cc563ae123f00c13778bd57
In this scenario to rule out client though; when I seen this behavior I created a new subscription and was delivered the same 48h old message. Again there is no queue subscriber in play.
Thanks.

That is clearly not possible since gnatsd does not keep any message in memory. When a message comes in, it is sent directly to matching subscribers. The longest it can stay in the socket buffer is 2 seconds, which is the write deadline set by the server, after which the consumer's client connection is closed if data was not drained from the socket.
You may have an application that sends those messages, but not the server. As for the weird timestamps, make sure that for some reason this machine does not have its clock changed.

thank you for your quick response. i am continuing to investigate as we are battling latency issues.


Change VS to MinGW for CRAN

This PR is to help the release of CRAN package of LightGBM.
It removes Visual Studio requirement on Windows. User may still install LightGBM with Visual Studio from GitHub using Laurae2/lgbdl.
MinGW should be the default for the following reasons in Windows:

R expects MinGW compilation
Rtools, mandatory to start compiling packages, uses MinGW / gcc
Rtools is most likely installed than Visual Studio
CRAN does not use Visual Studio
Precompiled DLLs for CRAN are last resort only

Updates:

MinGW instead of Visual Studio by default
R README.md documentation to reflect changes

See issue #629.

@Laurae2 Did you test the build with MinGW ?
Does it use the mingw in Rtools, or need to install a standalone MinGW ?

@guolinke It seems to work now.
It does not need a separate MinGW, by default it should use Rtools.

@guolinke In Windows, cmake with Rtools requires to run twice because Rtools has sh.exe which cmake does not like (must be run twice to bypass the error). If running twice and using another MinGW without Rtools, it won't cause an error because cmake will tell a warning that it has already configured LightGBM for MinGW.

@Laurae2
I am thinking can we add a parameter named build_cran to https://github.com/Microsoft/LightGBM/blob/master/R-package/build_package.R ?
if build_cran is true, we change the default compiler to mingw in windows.

@guolinke We already have use_mingw in install.libs.R, we may rename it to build_cran to specify building with minimum requirements (cmake / Rtools).
But in any case, Rtools must be the default choice and VS an optional choice (not the other way around, it makes it unpractical especially for automated installations and future CRAN release).

@Laurae2 Thanks. I think we can test the compiler by using cmake --help. And using visual studio if it is installed, otherwise using the rtools.
BTW, does these cran test machine have the cmake ?

@guolinke CRAN test machines should have cmake because there are packages requiring cmake.
For instance, this CRAN package requires cmake: https://cran.r-project.org/web/packages/zstdr/index.html

@guolinke Build passing on my VS+MinGW Windows machine (VS selection) and MinGW Windows machine (VS failed, auto fallback to MinGW) now.

but getting this error now on my MinGW-only machine:
Error in eval(ei, envir) : Cannot find lib_lightgbm.dll

lib_path should change if vs build failed

@Laurae2 Thanks, can you also update the documents  ?

Success on both machines, using R 3.4.0:
VS + MinGW machine:
> devtools::install_github("Laurae2/LightGBM@patch-2", subdir = "R-package")
(...)
** building package indices
** testing if installed package can be loaded
Warning: package 'R6' was built under R version 3.4.1
* DONE (lightgbm)
Warning message:
GitHub repo contains submodules, may not function as expected!

MinGW-only machine:
> devtools::install_github("Laurae2/LightGBM@patch-2", subdir = "R-package")
(...)
** building package indices
** testing if installed package can be loaded
Warning: package 'R6' was built under R version 3.4.1
* DONE (lightgbm)
Warning message:
GitHub repo contains submodules, may not function as expected!

@guolinke I'll update docs now to change the default to VS and auto fallback to MinGW for Windows.

Thanks very much. I think now I can merge it.


The json data source in license-list-data is missing the isFsfLibre attribute

Compare: https://raw.githubusercontent.com/spdx/license-list-data/master/rdfnt/licenses.nt to https://raw.githubusercontent.com/spdx/license-list-data/master/json/licenses.json.
Take a license that's consider libre such as Intel, https://spdx.org/licenses/.
Note that the data is missing.
The NuGet client uses the json format as a source of truth for licenses, but now that's generating data based on an incomplete data source.
https://github.com/NuGet/NuGet.Client/pull/4022
The change was made in https://github.com/spdx/license-list-data/commit/b8d6af45ad2fcfed61bb85a8ad068aa4a77eadf9.

@nkolev92 I see missing isFsfLibre flag from the licenses.json file.  Is that the only data you see missing?
Just a note - the licenses.json file only contains a summary of the information (e.g. it does not include the license text).
To get the full set of details, reference the license json file in the details folder - for example https://github.com/spdx/license-list-data/blob/master/json/details/Intel.json
I checked and the fsfLibre is present there.
I will look into adding the fsfLibre to the licenses file, however, since we do that for other similar flags (e.g. isOsiApproved).

Thanks for the quick response!

I see missing isFsfLibre flag from the licenses.json file. Is that the only data you see missing?

Yes. We use isFsfLibre, isOsiApproved and isDeprecatedLicenseId.

Just a note - the licenses.json file only contains a summary of the information (e.g. it does not include the license text).

Yep, client side we just need the available license ids and their status.
Server side, we use the full license text, but we have not run into issues there afaik.

I will look into adding the fsfLibre to the licenses file, however, since we do that for other similar flags (e.g. isOsiApproved).

Perfect! That's exactly what we neeed.
Thanks!

I created a pull request for the Java library.
@nkolev92 Attached is the resultant licenses.json file with the additional fields.  If you could review and let me know if this resolves the issue.
licenses.json.txt

I created a pull request for the Java library.
@nkolev92 Attached is the resultant licenses.json file with the additional fields. If you could review and let me know if this resolves the issue.
licenses.json.txt

That works great. The data is just as expected.


A support for Unreal Engine 5.1 is planned?

I reviewed the source code, and it has undergone significant changes, therefore the 5.0 plugin does not function with 5.1.

Same here. It would be great to have a 5.1 version.

I was already wondering why my tracker is not being tracked ingame. I can confirm that the tracker is not getting picked up in 5.1 without any error messages.


Vagrant up timeout while waiting for machine to boot.

I have followed the instructions at http://docs.drupalvm.com/en/latest/deployment/local-codebase/ as best I can. I am getting the following timeout error with vagrant up. I've attached all the messages. I've also attached the config.yml with my changes.
I'm sure I've made a mistake somewhere since I'm still learning. What did I do wrong?
config.zip
C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/batch_action.
rb:82:in `block (2 levels) in run'
INFO interface: error: Timed out while waiting for the machine to boot. This
means that
Vagrant was unable to communicate with the guest machine within
the configured ("config.vm.boot_timeout" value) time period.
PS C.docx

Update. I ran "vagrant up" again and did not get the message.

@ColumbiaBlooms - Sometimes... Vagrant can just be strange :)


Lifecycle Nodes

What type of PR is this? (check all applicable)
Feature
Description / Documentation
Lots of lines changed but fundamentally its a pretty simple change. Lifecycle nodes just allow other nodes to activate and deactivate it. The LifecycleNode provides several method that need to be overridden by classes that inherit it, these are on_configure, on_activate, on_deactivate, on_shutdown, on_cleanup and on_error. These correspond with states in the LifecycleNode state machine, they are called when the state is 'transitioned' into, which is initiated by the node's manager. In these state callbacks, we simply activate/deactivate publishers and subscribers, rendering the node enabled or disabled. In our case, the trackdrive_handler_node is the manager of three lifecycle nodes, a pure pursuit node, a reactive controller and a path planner. A convenience class called LifecycleServiceClient has been created to simplify the managing of LifecycleNodes.
To Test / Run

Run the sim
Run particle pursuit
Run reactive controller
Run ordered mid spline
Run trackdrive handler

The reactive controller should perform one lap, then it will be deactivated, and the trackdrive handler will activate the pure pursuit and ordered mid spline nodes. After 10 laps all nodes (except trackdrive handler) should be shutdown.
Usability concerns or breaking changes?
Shouldn't be. We will see.

Unrelated to the PR content, but I like the "To test" section of the PR - would be worth adding to the template

Unrelated to the PR content, but I like the "To test" section of the PR - would be worth adding to the template

Run instructions are nice too

Had a thought today
What if we set these up as a lifecycle and a 'standard' node pairing. that way the standard can start running when you want to debug at certain points - not managed by the AS state either.
The example I was envisioning was:
-I'm testing pure pursuit, simulating us completing the discovery lap, but I'm not running my trackdrive handler cause i'm just testing stuff.
This way we can run all nodes in a managed way if we are simulating a race or actually running end-to-end

Naming suggestion for mid-spline planner, differentiate between node and lifecycle as prefix (like pursuit)


Implement text-matching command

Provides a way to regex-search the text inside a range.
Handles blocknodes (treated as newlines for searching purposes) and invisible spaces (ignored).
This is a heavily modified port of the match-text command from the now-defunct plugin-rework branch, for those that remember that.

closes https://binnenland.atlassian.net/browse/GN-3214


Generating Polyominoes

Hello snowmanam2,
I have forked your project and started a new branch where I am trying to modify the code so it could generate the 2D polyominoes but I am struggling with something, probably the rotation part.
I have changed lut to
const uint8_t point_rotations_lut[][2] = { 	{0,1}, 	{0,3}, 	{2,3}, 	{2,1} }
but the numbers I get do not follow the enumeration from Kevin Gong
Could you develop a bit on that. There may be a mistake at another place in the code though.
I also wonder what to change to make it work with higher dimensions.
Thank you,

Hi Wagyx,
The point_rotations_lut has a few more intricacies to it - especially if you aren't generating all rotations. Looking at the logic of "generator_create_rotations" in generator.c, note how the computed rotations depend on the dimensions of the seed polycube. Still, this is just a guess at the cause in your case because there are a number of other factors. To test if this is the only factor, you can temporarily replace the dimension checking code and directly compute all rotations.
Note each row of the point_rotations_lut should always include each dimensional component in some order - either as the base component (X,Y,Z = 0,1,2 in 3D) or its reflection (X',Y',Z' = 3,4,5 in 3D). The 3D case is ordered such that the first component / reflection corresponds to the chosen computed rotations in "generator_create_rotations". I further narrowed the 3D case with the second component meaning something if all 3 dimensions were different, though I don't recall it helped performance as much.
Assuming base components (X,Y = 0,1 in 2D) and reflections (X',Y' = 2,3 in 2D), I think the point_rotations_lut for the 2D case should be:
{0,1},  // X
{2,3},
{1,2},  // Y
{3,0}

My suggestion would be to start debugging back at the "generator_generate" function. Construct the point list key programmatically in a test main.c file and read out the output points of the "rkeys" to see if it matches your expectations. This is where I started when building the project. The result should return duplicate polycubes, but the duplicates should always appear in the same orientation.
Other considerations:

The point storage format will almost certainly need to change for 4D and onward. I found point comparison to be a critical factor in performance, so a single comparison of integers is better than multiple compares. In the case of 2D, I think you probably want to compute something under N=256 but probably more than the N=30 limit built into the current 3D code. So, probably uint16 for 2D packed as 2 8 bit components, and maybe uint32 or uint64 for higher dimensions?
The next issue you might find involving the point storage method is in the methods for determining point adjacency in filtering the polycubes for output (key.c neighbor and connected methods). Perhaps there are better ways to do this, but the current code is written using the raw point integers themselves as indices to an array of uint8 positions (the so-called "spacemap"). These positions can then be checked against neighbors by using a lookup table of offsets. For the 2D case, you might want to expand the size of the "spacemap" slightly to cover all possible values of a uint16 (65536). For 4D and above, you might want to find another method. I feared expanding it too much because of memory consumption and the possibility of frequent cache misses when trying to read back the data.
If you do change the data format beyond the original uint16 bounds, you might need to play around with how to sort the points efficiently. I used sorting networks for the 3D case because I found qsort was slower, but perhaps you could change the data type or change the sorting algorithm. Also, the key_compare method will need to change accordingly to compare the correct number of bytes per point.
I can only imagine trying to wrap my head around constructing the "point_rotations_lut" for higher dimensions. I reasoned the 3D version and the 2D version above by hand, but I'm not sure how one would approach higher dimensions.

Thank you for your detailed answer.
I have managed to fix my code. It computes all the rotations for now but it works.
I have made a branch on your forked repo here
I have also made an online viewer similar to the polycube viewer. Here is the repo.
The 4D case will need a lot more thoughts and considerations for sure.


adding (?) general way of accessing building and storey in bim_controller?

To my current understanding the only way to get to a building or storey name is with an existing object.
If the goal is to know/change the building of given element, that is convenient and straight forward. 🙂
element_id = 12341234
get_building_of_element = bc.get_building(element_id)
get_storey_of_element = bc.get_storey(element_id)
bc.set_building_and_storey([element_id], "other_building", "G04")

However if we want to know all current buildings and their stories with heights in the current model,
it seems like there is no general way to access it without iterating through all elements?
Would be great if it was possible to directly ask the model for existing buildings and their stories:
from collections import defaultdict

# potentially accessing all buildings in model
building_names = bc.get_buildings() # fictional
# yields list of str, building names:
# ["demo_project", "sample_project"]

# potentially accessing stories for all buildings in model
stories_per_building = defaultdict(dict)
for building_name in building_names:
    storey_names = bc.get_building_stories(building_name) # fictional
    for storey_name in storey_names:
        storey_height = bc.get_storey_height()
        storey_info = {storey_name: {"elevation": storey_height }
        stories_per_building[building_name].update(storey_info )

@fbpyr
new functions in version 29
bim_controller.get_all_buildings() -> List[str]
bim_controller.get_all_storeys(building_name: str) -> List[str]

🎉  Awesome!!
Looking forward to v29! 🙂


setPressRef() doesn't work, getting negative ppm

Prerequisites
Can you reproduce the problem reliably?
yes
Did you check current release notes for known issues?
Where can I find those?
If this is not the latest release, have you checked newer releases?
 If there is no release, give us the commit for the code you used.
2.0.0
Target:
STM32F103 BluePill, PAS CO2 Miniboard
Host OS and Version:
Debian GNU/Linux 11
Compiler:
gcc/10.3.1-2.3

Hi @schumar,
The output is reporting communication error. Can you share your wiring diagram? Are you using i2c? Are you using pull-up resistors?

Mmm, looks good. Are the logic levels of the I2C ports working at 3.3V or 5V for this board?

3.3V; I've lowered the I2C-frequency to 100kHz.

It looks like the Miniboard doesn't pull SDA fully to GND? (it's been a while since I bit-banged I2C, so maybe this is how it's supposed to look like):

Does it work with 100KHz?
Definitively, that does not look very neat... Is it properly grounded with the osci?
It would be good to know the sensor version, which can be obtained by running the serial-device-id example. But it won´t likely work with this board given the i2c issue.
Would you have by chance a xmc2go eval kit to check on that platform? Or an Arduino Uno?

Does it work with 100KHz?

no, same behaviour

Definitively, that does not look very neat... Is it properly grounded with the osci?

Yes, double-checked just now

It would be good to know the sensor version, which can be obtained by running the serial-device-id example. But it won´t likely work with this board given the i2c issue.

This works: (tried at 100kHz)
product id  : 2
revision id : 15

Would you have by chance a xmc2go eval kit to check on that platform? Or an Arduino Uno?

XMC 2Go, Arduino Nano, even a TTGO ESP8266 -- I'll give them a try.

 I commented out the setPressRef() calls, just in case you are wondering about the missing pressure reference error: 1

And the same with the XMC 2Go?
We will need for some time to reproduce the error by our application engineers. We will get back to you as soon as possible.

Hi @schumar,
Did you have the chance to try with the XMC2Go?
There is a chance that the sensor is damaged, since negative values in the concentration could be due to the 12V supply not connected or not working.

Hi @jaenrig-ifx , sorry, didn't have a lot of time in the last few days.
I have rebuilt the whole circuit (with the BluePill) on a new bread-board, and the SDA-at-1V issue went away (guess some grounding issue at the sensor side?).
Same behavior as before though (negative ppms).
I then switched to the XMC2Go, and that wasn't able to communicate with the sensor at all. I saw it sending via I2C, but the sensor just didn't respond.
Switched back to BluePill, and it worked again (with negative ppms).
I then came to the same conclusion as you -- sensor might be broken. Tried another one, instantly got sane values (around 700ppm). Didn't have time to switch back to the XMC2Go, but as it now works using the BluePill, and the XMC2Go isn't a good fit for the sensor anyway (it doesn't bring out V_USB to a dedicated pin, but I need those sweet 5V to power a boost-converter, so I can get the 12V required), I'll stick with the BluePill.
Thanks for your help! I'll ping you tomorrow via WebEx, in case someone wants to have a look at the broken sensor (and yes, I assume that it was perfectly fine before I somehow managed to destroy it.)


Isolate secrets to each module

Original feature request:

Right now the in-memory secret store is shared across a whole dagger session, which includes the original client caller and all the modules it invokes (including transitively).
Instead secrets should be passable as inputs/outputs of Functions but only accessible that explicit way, not accessible via the "ambient secret store".

To achieve this isolation, we keep the same global secret store, but ensure that each secret has a "scope" - this scope split the global secret store into lots of smaller stores.
Note, this is really quite a tricky puzzle, and the requirements become more fiddly as you lay them out:

Secrets across different sessions/modules should be allowed the same name
Secrets shouldn't affect caching (any more than they do today)
Direct access to the secret store using the "feature" where they can be pulled directly using the DockerBuild API without explicit passing
What should the "lifetime" of a "module-scoped ambient secret store" be? If I create a secret in a Foo function call, but then try and access it in Bar, is it still there or is it only valid per-call? If it's still there, then does it matter which module called me - and does that make for weird dependency state?

We can tick all these boxes by doing the prefixing trick - we also need to explicitly handle the DockerBuild frontend case, so we protect against reading directly from there. The only question really, is what prefix to use. A couple ideas I had:

Client ID - but this is very bad for caching, so probably not
Hash of the secret ID - this doesn't work, since there is no real parent (beyond the root query object)
Unique ID of the current module (git hash + subpath for git modules, contenthash for a directory module) - quite good, maybe work to calculate, but also not an option for generic privileged nesting

Could avoid some weird lifetime issues if we extended this and also included the unique IDs of all the parents as well (so A->X and B->X calls would have two separately scoped X secret stores)

Generate a unique buildkit secret for every use of a secret and base the prefix off a hash of everything prior to the use - also potentially possible, but now we need to make sure we don't accidentally cause a buildkit edge merge here!
...something else I'm not thinking of?

TODO:

[x] Prefix secrets with the client ID that created them (which isolates them). Update all secret access logic with this.
[x] Intercept dockerfile builds, and rewrite secrets with the client prefix.

This is needed, since frontends have the ability to directly access secrets from the secret store (via ExecOp/etc). We can use a custom gateway/frontend to protect against this.

[ ] Use a "stable" ID for prefixing (instead of client IDs). Every time we change a secret ID we invalidate cache, so we have to avoid doing this.

[x] Ensure this is tested as well

I haven't looked through this in depth yet, but I just did a quick test and found that the grpc client metadata ends up plumbed all the way through to where ExecOp asks for secrets (here). So if you call engine.ClientMetadataFromContext in our GetSecret implementation (here) you can actually access the ClientID (without any changes that would break caching).
I think this offers a simpler route to implementing isolation based on the client ID since each module gets its own even within the context of a session. Just wondering if you tried out that route too.
Just leaving this as a quick drive-by thought since I hit a timebox, will resume taking a look again later.

I haven't looked through this in depth yet, but I just did a quick test and found that the grpc client metadata ends up plumbed all the way through to where ExecOp asks for secrets (here). So if you call engine.ClientMetadataFromContext in our GetSecret implementation (here) you can actually access the ClientID (without any changes that would break caching).
I think this offers a simpler route to implementing isolation based on the client ID since each module gets its own even within the context of a session. Just wondering if you tried out that route too.

Hm just one concern with this - if two modules call this at the same time, we'd edge merge, and only get one call to the ExecOp. So theoretically, a module without access to the secret, could get access to the result of a computation that required the secret (though it wouldn't actually leak the secret).
The only way to avoid this would be to make sure we actually have different cache keys (e.g. by actually changing the secret access IDs).

Hm just one concern with this - if two modules call this at the same time, we'd edge merge, and only get one call to the ExecOp. So theoretically, a module without access to the secret, could get access to the result of a computation that required the secret (though it wouldn't actually leak the secret).

I'm presuming we were going to update to use a stable id as prefix rather than client id, and at that point we'd have the same edge merge problem, right?
I was thinking through the cases where this would be undesirable and realized that practically speaking, the "edge-merge" case is actually the same as re-use of a cached exec op. E.g. all the cases where it would be undesirable for an edge merge to happen can also happen if exec-ops from completely different sessions get de-duped due to local/remote cache hits.
So I think this actually gets at a larger long-standing and pre-existing issue. Probably still be worth addressing as part of our changes here though.
The only thing I can think of to completely squash this problem in all cases (edge merge, cache hits) would be to change the underlying secret ID (i.e. the one that is provided to LLB and used in GetSecret calls to the attachable) to be a hash of the secret value.

This would work conceptually because the main problem with edge merges/cache hits is that a malicious client could set a secret to an invalid value, but if the underlying ID is actually a product of the value, then the only way to get an edge-merge/cache-hit would be if the malicious client actually knew that value (at which point its game over anyways)
We'd ideally want to do it in such a way that we never actually pass that hash around to module code and instead just rewrite it internally right before submitting LLB to be solved (i.e. with OpDAG).

So trying to put everything together, we could:

Store secrets in the secret store with this metadata attached to each entry:

name - the user-specified name
client ID - the client ID that has access to this secret
plaintext
hash - the hash of the plaintext

When an exec op is submitted, update secret IDs to be the hash rather than name (using the secret store to know the mapping, we can append an extra method to our implementation to support this in our own code pretty easily)
GetSecret is updated to:

check the client ID from the context
find the secret with the provided hash (the secret ID) that is available to that client ID

WDYT @jedevc? I'm not opposed to any solution that works and handles all the cases; the above just feels simplest and most robust to me (if the theory works out in practice), but that's obviously subjective.

I'm presuming we were going to update to use a stable id as prefix rather than client id, and at that point we'd have the same edge merge problem, right?

Yes, we need a stable id. I was actually thinking that we could pick a "module id" as a stable id:

For a git module, this would be hash(commit SHA + subpath)
For a local module, this would be hash(blob ID + subpath)

For the top-level module, we could try and keep this more constant (by using hash(path), instead), so that as you develop your module, you don't keep getting different secret IDs.

This is stable, should be pretty easy to propagate everywhere, but doesn't really provide isolation between generic nested sessions (though I'm not necessarily sure this is an issue, this is more obscure, and also expectations of security are much lower).
Also, what's really cool about this is that we allow caches/edge-merges for secrets only inside the context of that module - other modules get different buildkit secret ids, so they can't collide.

We'd ideally want to do it in such a way that we never actually pass that hash around to module code

This is potentially a pain :cry: Without this, we don't have a way to distinguish secrets from each other - and we can't even rely on session id here. If I return a Secret from my module (the result of looking it up using secret), and then it gets into another module, then secret gets called in the context of the consumer module. So we need some identifying info in the secret itself (see accessor in the current PR).

to be a hash of the secret value.

Ah, nit-pick, but this then changes secret value changes to always invalidate cache - it's a tiny issue, but if possible I'd like to preserve the current behavior.

I'm not opposed to any solution that works and handles all the cases

I think I'm going to try the module "ID" as the secret prefix - it ticks all the required cases, and also addresses the things I mentioned above. Even though it is a bit more work, it shouldn't be that much trickier.
I don't think I'm far away from getting this PR into a ready state, should be done this week, just need one more proper focus session.

Ah, nit-pick, but this then changes secret value changes to always invalidate cache - it's a tiny issue, but if possible I'd like to preserve the current behavior.

Yeah I think this is important. Just keeping in mind that people might be using shortlived secrets (e.g. AWS access keys that expire after an hour), I think we'll definitely want to avoid cache busting on them. Which unfortunately makes things very difficult.

Think I've got most of the tests passing now (:crossed_fingers:). Should be ready for a more proper reiview :tada:

I know this is what we've landed on in the past, but I'm not sure I agree with it anymore. It means that you can provide invalid credentials that happen to have the same secret name and the operation will "succeed" due to caching. Something about that feels deeply wrong.

Yeah the issue is we need to be able to distinguish between "I manually changed my secret and want that action to affect external state" and "my credentials are super secure and rotate all the time, but I would still like to enjoy caching." To me it's pretty bad if we end up in a state that doesn't allow frequent cred rotation and caching to coexist. But yeah this probably gets into higher level discussions re: state, deployments, and secret management in general.

Please double check my reasoning here, but I think there may still be a backdoor available via loadSecretFromID. Namely, even though the ID format and accessor are "opaque", it's not that hard for something malicious to derive those (i.e. figure out the accessor for another unrelated module in the same session) and construct an ID manually that's provided to loadSecretFromID and thus grant access to calling plaintext and/or using the secret in other API calls.

Ok, so - I don't think there's an issue directly with loadSecretFromID - we could construct an ID manually that creates a secret, but still the secret will always be constructed from a call to secret. There is an issue with calling secret though, you can just guess the accessor and then grab a secret.
I had a rather nice shower-thought about how to resolve this (have pushed it in the last commit). Essentially, we use the plaintext-digest and put that into the ID - if you use an accessor, you must specify this plaintextDigest as well - when loading a secret we check that the value works. Providing this value essentially ensures that you are accessing an "officially" minted secret and that you have a real one.
This has a few fun properties:

Secrets minted using setSecret/setSecretFile set plaintextDigest + accessor, so when they're passed between modules, these are set, and so can be accessed easily.
If a malicious module attempts to mint a secret manually, it can't just guess the accessor, it also needs to work out what the digest of the secret is - at that point, it needs to know the full secret, which defeats the whole purpose of the attempt.
This is only a schema-level thing - the buildkit secret IDs are still entirely based on the accessor, so caching is unaffected.
This essentially splits out the two properties from this patch, instead of trying to merge them together:

Buildkit ID for uniqueness (based on module source ID)
Proof of secret ownership

However, one note - is it ok that the plaintext secret digest is part of the dagql graph? We take a lot of pains to keep the plaintext out of the graph, so I wonder if we need to do some other stuff to keep the plaintext secret digest out (but we can't just mark it as sensitive, otherwise the client never actually gets access to it, which is kind of the whole point).
If we do have concerns about that, we could potentially do some other magic as well, and combine the plaintext digest with the client ID (which is fine to use here, since this doesn't affect the buildkit caching), and track the client ID that set each secret in the secret store (which lets us verify).

Essentially, we use the plaintext-digest and put that into the ID - if you use an accessor, you must specify this plaintextDigest as well - when loading a secret we check that the plaintext matches. Providing this value essentially ensures that you are accessing an "officially" minted secret and that you have a real one.

This does affect caching though, specifically of module functions themselves that accept secrets as args. The values of all args are part of the cache key for the exec-op underlying the function call. If the ID changes, that invalidates the execution of the function call.

This is somewhat more obscure atm since we invalidate cache of functions once per session, but it can still matter today (i.e. if you call the same function multiple times from within a session) and will matter enormously once we add support for cache control of function execution.

To be clear, I'm not opposed to this since like I said before I now think it makes sense to always invalidate cache based on the value of the secret. It's also how the CLI currently sets the secret name anyways too (code). I think it's essentially a step closer to what I suggested here https://github.com/dagger/dagger/pull/6601#issuecomment-1953131129

Also echoing what I said previously about the fact that while I agree it's unfortunate that this means short-lived tokens invalidate cache, that A) feels closer to correct behavior even if not performance optimal and B) can be addressed in the future via "secrets providers" (which in general feels like the better way of using short lived creds anyways)

The thing I'd want to avoid is just using the ClientID as you mentioned towards then end of the comment since that would break our plans to make functions fully cacheable across sessions in the near-ish future.

This does affect caching though, specifically of module functions themselves that accept secrets as args.

Agh, sorry, I did understand this at some point, and then it slipped my mind.

I now think it makes sense to always invalidate cache based on the value of the secret.

I don't have super strong opinions on the idea of this, but. If the secret hash is sensitive, then we want to avoid it leaking anywhere outside of the current machine:

Into telemetry
Into logs
And into cache (and magicache)

Since magicache will cache the inputs to functions, I'd really rather not include any hashed version of the password in there - any attacker with access to the cache (just like @vito's point above about dagviz/telemetry), can then perform an offline bruteforce attack - we then rely on the entropy of the secret to provide all the security, which just doesn't feel right. I can't see a way to be comfortable with potentially uploading secret hashes to caching services (it feels like a security incident just waiting to happen).

I think the only approach left is to carefully track exactly how secrets are passed between modules, and then when accessing secrets, ensure that that module was passed that secret (either by return from a module that had access, or by argument called by a module that had access).
This feels tricky though, since it also requires accounting for secrets that are then stored in custom objects.

Given the time before the release, I'm tempted to simply go back to just the accessor - even though it can be leaked by knowing the ID (I wrote a test for this and pushed it, we can enable it later).
Even though it's possible for a malicious module to potentially extract secrets from other running modules, I'm not actually convinced this is the highest priority - if I can run code on a machine, I can definitely think of lots of other nasty things to do - especially since we have privileged nesting/execs.
As-is, this PR at least prevents secrets from accidentally colliding over each other, so I think is still an improvement to take for v0.10.0 - there's clearly a lot of future work here.


[Search] Initial feature-flagged implementation

Description
Add offline-capable search matching the UI design behind a "search" feature flag.
We've opted to try lunr as an offline search library with reasonably wide language support. elasticlunr is another option that shares the same language support.
The plan is to integrate lunr or similar providing a basic search API via a hook and then add the search UI from the design, perhaps omitting more involved features in this first pass.
Remaining work
Must

[x] Add French content and then update lunr initialization to respect language. Deal with code splitting etc.
[ ] Prune common coding terms from the English stop words and see whether that helps. E.g. currently can't search for if/while/else
[x] Review keyboard navigation

Hover highlights as part of this
Add a href to the links for tabbing (serialise router state), add an onClick to the thing as a whole

[x] Translatable strings - also need to consider fallback
[ ] Improve extracts. Show more context? Show fewer extracts if there are many matches? Don't truncate words? Try to equalise the text length shown?
[x] Tracking of things you've clicked through to (with the lighter highlight)
[ ] Explore wildcard queries of other means of matching API terms from prefixes.

Deferrable?

Decide whether we do 100+ results or exact number, if so what are the full rules.

The performance is fine for the moment. Revisit when we remove some English stop words.

Add narrower search button when the sidebar is narrower

Rolling in with general tablet work.

Filter by type

Raised #481

Recent searches

Raised #482

Highlight terms after you click through (not sure I want to do this at all)

I just don't think we should do this.

Design changes/issues

The search box that's initially visible on larger screens is odd. Is it an input? If so we'll need to move the focus when you click on it. Is it a button that looks like an input? If so that's very confusing. Perhaps it should just be a button as per the smaller width design?

For now I've matched the design but I'm unsure about it.

We want an Explore / Reference split as shown in the design (though with Explore/Reference not the text used in the design).

We might want to make the toolkit and topic clearer as the text is muted in the design.

Padding is a bit odd in the results pane. Fine in our implementation.

Conclusion

The serialization of the Sanity "portable text" for indexing can be handled with something like this:
https://www.sanity.io/schemas/portable-text-to-plain-text-cc845843

At the moment we're mostly escaping away the solr query syntax. We could instead programatically construct a search, e.g. split on whitespace and create a term query, perhaps with a trailing wildcard for each phrase (query.term(term, { wildcard: lunr.Query.wildcard.TRAILING })).
The trailing wildcard might be useful for reference. Hard to say without trying it.

At the moment we're mostly escaping away the solr query syntax. We could instead programatically construct a search, e.g. split on whitespace and create a term query, perhaps with a trailing wildcard for each phrase (query.term(term, { wildcard: lunr.Query.wildcard.TRAILING })).
The trailing wildcard might be useful for reference. Hard to say without trying it.

A significant barrier to using this is that it doesn't seem that the wildcard matches the empty string. So display* doesn't match display. I think perhaps we need to take a step back and think about another approach (perhaps in the tokenising stage).


add dims for chunk and test_zygote

Trying to fix https://github.com/FluxML/Flux.jl/issues/1841 using @mcabbott's suggestion, but Zygote has trouble differentiating through Iterators.partition, I get
julia> test_zygote(chunk, rand(10), 3)
test_rrule: chunk on Vector{Float64},Int64: Error During Test at /home/carlo/.julia/packages/ChainRulesTestUtils/XI7i2/src/testers.jl:195
  Got exception outside of a @test
  MethodError: no method matching size(::Base.Iterators.PartitionIterator{Base.OneTo{Int64}})
  Closest candidates are:
    size(::Union{LinearAlgebra.QR, LinearAlgebra.QRCompactWY, LinearAlgebra.QRPivoted}) at ~/julia/julia-1.7.1/share/julia/stdlib/v1.7/LinearAlgebra/src/qr.jl:567
    size(::Union{LinearAlgebra.QR, LinearAlgebra.QRCompactWY, LinearAlgebra.QRPivoted}, ::Integer) at ~/julia/julia-1.7.1/share/julia/stdlib/v1.7/LinearAlgebra/src/qr.jl:566
    size(::Union{LinearAlgebra.Cholesky, LinearAlgebra.CholeskyPivoted}) at ~/julia/julia-1.7.1/share/julia/stdlib/v1.7/LinearAlgebra/src/cholesky.jl:494
    ...
  Stacktrace:
    [1] axes
      @ ./abstractarray.jl:95 [inlined]
    [2] _tryaxes(x::Base.Iterators.PartitionIterator{Base.OneTo{Int64}})
      @ Zygote ~/.julia/packages/Zygote/FPUm3/src/lib/array.jl:184
    [3] map
      @ ./tuple.jl:221 [inlined]
    [4] ∇map(cx::Zygote.Context, f::MLUtils.var"#70#71"{Int64, Vector{Float64}}, args::Base.Iterators.PartitionIterator{Base.OneTo{Int64}})
      @ Zygote ~/.julia/packages/Zygote/FPUm3/src/lib/array.jl:199
    [5] _pullback(cx::Zygote.Context, #unused#::typeof(collect), g::Base.Generator{Base.Iterators.PartitionIterator{Base.OneTo{Int64}}, MLUtils.var"#70#71"{Int64, Vector{Float64}}})
      @ Zygote ~/.julia/packages/Zygote/FPUm3/src/lib/array.jl:244
    [6] _pullback
      @ ~/.julia/dev/MLUtils/src/utils.jl:165 [inlined]
    [7] _pullback(::Zygote.Context, ::MLUtils.var"##chunk#69", ::Int64, ::typeof(chunk), ::Vector{Float64}, ::Int64)
      @ Zygote ~/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0
    [8] _pullback
      @ ~/.julia/dev/MLUtils/src/utils.jl:164 [inlined]
    [9] _pullback(::Zygote.Context, ::typeof(chunk), ::Vector{Float64}, ::Int64)
      @ Zygote ~/.julia/packages/Zygote/FPUm3/src/compiler/interface2.jl:0

As a workaround I can define a custom partition function I guess.
cc @theabhirath @darsnack

You could hide it in a function, since it has only indices, or try to write the gradient for CR.
But here, I think it's worth writing the gradient for this whole function. Much like the rule for eachcol(x), it can allocate just one similar(x), not one per slice.

The test facility comes from https://github.com/JuliaDiff/ChainRulesTestUtils.jl/issues/211

Something like
_partition(x...) = Iterators.partition(x...)
@non_differentiable _partition(x...)

doesn't work because actually the problem is not in differentiating partition but in ∇map  over PartitionIterator

The rrule needed should resemble this
https://github.com/JuliaDiff/ChainRules.jl/blob/8108a77a96af5d4b0c460aac393e44f8943f3c5e/src/rulesets/Base/indexing.jl#L77

Implemented the rrule, now it is much better
julia> x = rand(100, 200);

julia> @btime gradient(x -> sum(chunk(x, 10)[1]), $x);
  19.878 μs (197 allocations: 166.16 KiB)
``

@mcabbott looks good?

Codecov Report

Merging #47 (80658db) into main (ea45061) will decrease coverage by 0.11%.
The diff coverage is 85.29%.

@@            Coverage Diff             @@
##             main      #47      +/-   ##
==========================================
- Coverage   88.91%   88.80%   -0.12%
==========================================
  Files          13       13
  Lines         379      411      +32
==========================================
+ Hits          337      365      +28
- Misses         42       46       +4

Impacted Files
Coverage Δ

src/utils.jl
93.75% <85.29%> (-3.13%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea45061...80658db. Read the comment docs.

For some reason, github doesn't let me ask for your review @mcabbott ... is this good now?


New sample - Multi-Person Mail To Link

Q
A

New sample?
yes

What's in this Pull Request?
Added new column formatting sample which demonstrates how to add mailto: link against each person in multiple selection person column.

@thechriskent @tecchan1107 Can you please review this PR?

@PopWarner @VesaJuvonen @thechriskent Can you please review this PR?

Hi @ganesh-sanap. Thank you so much for your contribution and your patience. We are getting caught up on the PRs and will be processing yours and the remaining PRs as soon as we are able.
Keep up the amazing work and thank you again. 💜🧡

Nice sample @ganesh-sanap! Thank you for the contribution and thank you again for your patience! 🧡💜👏👏🤩

Great stuff again @ganesh-sanap. One request for future samples, if you could include one sample per Pull Request, that would be awesome. Looks like we had three samples inside this one PR, it will help ensure nothing gets missed. Thank you again! 💜🧡


mimalloc: assertion failed in mi_segment_span_free_coalesce

I'm using mimalloc v2.0.9. I'm getting an assertion error when I do the following:

Allocate a large block of memory from thread 1 (i.e., requiring an MI_SEGMENT_HUGE).
Thread 1 exits, abandoning the segment.
The block of memory is freed from thread 2.
Thread 2 exits

#include <assert.h>
#include <pthread.h>
#include <mimalloc.h>

static char* ptr;

static void* thread1_allocate(void *_unused)
{
  ptr = mi_malloc(22085632);
  return NULL;
}

static void* thread2_free(void *_unused)
{
  assert(ptr);
  mi_free(ptr);
  ptr = NULL;
  return NULL;
}

int main(int argc, char** argv) {
  pthread_t thread1, thread2;

  pthread_create(&thread1, NULL, &thread1_allocate, NULL);
  pthread_join(thread1, NULL);

  pthread_create(&thread2, NULL, &thread2_free, NULL);
  pthread_join(thread2, NULL);

  return 0;
}

mimalloc: assertion failed: at "/raid/sgross/mimalloc/src/segment.c":635, mi_segment_span_free_coalesce
  assertion: "segment->used == 1"
Aborted

Thanks Sam! I studied it and the assertion is indeed too strict (so no bug). It is because for v2.0.9 I made huge segments behave more regularly and no longer as abandoned "orphans" such that heap walking works correctly now for huge segments/blocks. To still release memory early for huge block frees we now decommit them directly on free but the block and segment itself is reclaimed later on like regular segments -- this lead to the now failing assertion but it is actually correct behavior now.


[BUG]: Corner Case reusing type incorrectly

Using the example code below, the tests will fail on successive runs of the test with different input values. The issue appears to be a combination of [ClassDataSource<>] and [NotInParallel].
It appears that using [ClassDataSource<>] does not actually use multiple instances of the class under test. When combined with [NotInParallel], then the Dispose() becomes noticeable by later tests as the class itself has been disposed already and then reused.
Example code:
public abstract class TestBase : IDisposable
{
	private bool _disposed;

	public void CheckDisposed()
	{
		if (_disposed)
			throw new InvalidOperationException("Already disposed");
	}

	public void Dispose()
	{
		_disposed = true;
	}
}

[ClassDataSource<Dummy>]
[NotInParallel]
public sealed class Tests(Dummy dummy) : TestBase
{
	[Test]
	[MethodDataSource(nameof(GetValues))]
	public void DoTest(int value)
	{
		CheckDisposed();
	}

	public static IEnumerable<int> GetValues() => [1, 2, 3];
}

public sealed record Dummy;

Fix in v0.1.1099

Thank you!


Zoom in so i can zoom out

Codecov Report

Merging #226 into master will decrease coverage by 0.03%.
The diff coverage is 75.00%.

@@            Coverage Diff             @@
##           master     #226      +/-   ##
==========================================
- Coverage   90.51%   90.48%   -0.04%
==========================================
  Files          91       91
  Lines        3985     3992       +7
  Branches      874      876       +2
==========================================
+ Hits         3607     3612       +5
- Misses        375      377       +2
  Partials        3        3

Impacted Files
Coverage Δ

src/component/NGLComponent.tsx
92.32% <75.00%> (-0.40%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb527ba...6b8c51b. Read the comment docs.