[Q] Difference between kernel-headers and kernel-headers-system-docker?

RancherOS Version: (ros os version)
v0.7.1
Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
VirtualBox 5.x
Ok, related to my erroneous evaluation of the kernel eco system in relation to Rancher, I'm having a hard time distinguishing the difference between these two packages.
Docs indicate ambiguous and obvious:

[...] the only reason for different services is due to the fact that the storage places for System Docker and Docker are different. [...]

But HOW are they different? Shouldn't the volume mounts take care of any needs since we're practically only dealing with filesystem overlays?
Also, it appears kernel-headers-system-docker does not actually supply the headers as it indicates.
More info here: https://github.com/rancher/os/issues/1463#issuecomment-267386384

at this point, we're seeing the 2 as historical, and would remove the system one. I don't know why one works differently to the other - as they both use the same image and yaml, but that's something to investigate (wanna make a PR?)

I need to investigate more building my own RancherOS image; I'd love to help where I can, but time is limited :D
I am also trying to get my team to document their findings better for contribution.
We're discussing completely customizing our kernel to include everything in it rather than modules; so that might come sooner than later.
Also, I want to take a look at supplying a kernel through config that's post-build; doing a custom build just for a kernel isn't ideal imo.

We should talk before you get too far into it - we're re-doing much of how the kernel and packaging works in the next months

I'm down to talk; right now we're sufficing with compiling modules against the kernel headers but we will need to swap kernels frequently for testing in the near future.
I looked around quickly but couldn't find the init or grub settings (or /boot partition) to update grub for a new kernel, but I didn't really dig deep.

yeah - 0.8.0 we're ditching grub and using syslinux so that should make that end easier too

turns out there's a really good reason to only use system-docker for these - otherwise its hard to bring back your zfs user-docker ... so https://github.com/rancher/os-services/pull/94


specify location of ‘empty’ method

Commit message:

An instance method is more flexible than a static method. It facilitates
> Identity([1, 2, 3]).empty()
Identity([])

since Identity#empty can dispatch to the inner value's empty method.

I find empty most useful when an instance is not available, so I would still want a way to obtain an empty value statically.

That's not where I started, but it's where I ended up in the long discussion in #88.  I'm afraid this is one of those square peg/round hole situations that are bound to come up when you try to merge static and dynamic typing.  I don't know that there is a good solution.

I'd probably go a step further and prefer the empty instance to be a plain property value rather than a nullary method.

This was proposed in #82. This would mean that the Identity type could not satisfy Monoid, I believe. Does this bother you, @scott-christopher?

The typical implementation of Const.of requires an empty value of the inner type, which makes it difficult to wire up if an instance is required.
Const.of = (_) => Const(OtherType.empty())

Hmm, I see the problem.
It seems we have four options:

Require an instance method only, making the Const type sad. :cry:
Require a static method only, making the Identity type sad. :cry:
Continue to support both, but state that the instance method should be used if both are present.
Do nothing.

Does anyone object to the third option? It's an improvement on the status quo, and shouldn't make anyone sad.

3 sounds reasonable
On Tue, 13 Sep 2016, 13:05 David Chambers<EMAIL_ADDRESS>wrote:

I'd probably go a step further and prefer the empty instance to be a
plain property value rather than a nullary method.
This was proposed in #82
https://github.com/fantasyland/fantasy-land/pull/82. This would mean
that the Identity type could not satisfy Monoid, I believe. Does this
bother you, @scott-christopher https://github.com/scott-christopher?
The typical implementation of Const.of requires an empty value of the
inner type, which makes it difficult to wire up if an instance is required.
Const.of = (_) => Const(OtherType.empty())
Hmm, I see the problem.
It seems we have four options:

Require an instance method only, making the Const type sad. 😢
Require a static method only, making the Identity type sad. 😢
Continue to support both, but state that the instance method should
be used if both are present.
Do nothing.

Does anyone object to the third option? It's an improvement on the status
quo, and shouldn't make anyone sad.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fantasyland/fantasy-land/pull/162#issuecomment-246660316,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACcaGGBkqQPm6KlceCvUJw_jqb34nMeuks5qppGPgaJpZM4J7gYW
.

I'll open a new pull request shortly.


Multiple fixes

The commit messages should explain, but right now the gem is unusable. Please consider these fixes and bump the gem version.
Thanks

@guitsaru can you take a look?


Split packages

I'm going to create two new packages, Kledex.Bus and Kledex.Domain so that unwanted dependencies are removed.

It should be Kledex.Bus and Kledex.Core (or Abstractions)

I have opted for the default implementation of interfaces for the time being.


Fix auth

Fix Authentication
What user problem are we solving?
Signing In With Github didn't work on Firefox or Safari.
What solution does this PR provide?
This provides a temporary fix of signing in with popup instead of redirects. The popup method is ok, but there is also the chance where some setting in a user's browser prevents the popup from appearing.
My guess for why signing in via redirects didn't work is that right now all of firebase's signinwithredirect methods utilize third-party cookies, which are gradually being deprecated.
There seems a growing standard for browsers to use FedCM for identity federation instead: https://github.com/fedidcg/FedCM/blob/main/explainer.md. FedCM however seems really new (I asked ChatGPT and it had no idea what it is), and doesn't have support for all of the major browsers yet (mainly firefox and safari), but it seems like those two also plan to support FedCM in the future.
This is what firebase suggests as fixes to this problem in the meantime: https://firebase.google.com/docs/auth/web/redirect-best-practices.
I chose the signInWithPopup option in the website above since it was the simplest, but for production we could try out the reverse proxy option using nginx if we want to keep redirects for prod.
Testing Methodology
Signing in with github and google works now in firefox and safari and chrome.
Any other considerations

what abt chrome? @dwu359

@dwu359 screen recording of fix?

Yes it worked on chrome, and it still works on chrome. Idt a screen recording is necessary, but I can provide one if you guys have trouble testing it.

@dwu359 have you verified that the user can stay signed in if we sign in, close and reopen the page? is that functionality affected since the local storage update stuff was removed

Yes that still happens, the only difference is that a loading screen doesn't appear for the split second between the page loading and firebase loading auth. The user shouldn't notice for most cases.

@dwu359 wait on @farisdurrani to review


ionic serve doesn't work

As the title suggests..

here problem with redux 4.0.1. Any solution ?
ERROR in node_modules/@angular-redux/store/lib/src/components/ng-redux.d.ts(10,31): error TS2420: Class 'NgRedux<RootState>' incorrectly implements interface 'ObservableStore<RootState>'.   Property '[Symbol.observable]' is missing in type 'NgRedux<RootState>'. node_modules/@redux-saga/core/types/effects.d.ts(17,8): error TS2307: Cannot find module '@redux-saga/types'. node_modules/@redux-saga/core/types/index.d.ts(3,86): error TS2307: Cannot find module '@redux-saga/types'. node_modules/@redux-saga/core/types/index.d.ts(170,52): error TS2344: Type 'S' does not satisfy the constraint '(...args: any[]) => any'. node_modules/@redux-saga/core/types/index.d.ts(189,23): error TS2344: Type 'S' does not satisfy the constraint '(...args: any[]) => any'. node_modules/redux-persist/src/index.d.ts(69,46): error TS2307: Cannot find module 'react'. node_modules/redux-persist/src/index.d.ts(91,38): error TS2304: Cannot find name 'React'. src/services/network.service.ts(20,18): error TS2551: Property 'onchange' does not exist on type 'Network'. Did you mean 'onChange'?

Hi, I've updated the project to Ionic 5 and Angular 9. I've also updated other dependencies too. Can you try again with this updated project? Thank you.


feat(behavior_path_planner): handle duplicated road turn signal

Description
In this PR, I changed turn signal decider to enable it to handle duplicated trajectories. Results can be seen from the following video.
https://user-images.githubusercontent.com/43805014/197686197-302df1d6-8da9-4ee7-a4c0-14d74130b920.mp4

Related links

Tests performed

Notes for reviewers

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@purewater0901 Could you add more explain on what you are trying to solve?
I don't really get what you mean by "duplicate" trajectory, and it would be also nice to explain how it was behaving before this PR is merged.

@mitsudome-r Thank you for your comment. I updated PR title and description. In addition, I upload a video, which shows the result without this PR


fix: save experiment audit result as object

Please ensure your pull request adheres to the following guidelines:

[ ] make sure to link the related issues in this description
[ ] when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes

Related Issues
Thanks for contributing!

:tada: This PR is included in version 1.8.16 :tada:
The release is available on:

v1.8.16
GitHub release

Your semantic-release bot :package::rocket:


Evernote SDK Timeout

Hi, I have a problem
When I am trying to upload note with attachment around 10 Mb, I receive the following exception:
java.io.InterruptedIOException: timeout (java.net.SocketException: Socket is closed)
I am using "createNoteAsync" method with onSuccess() and onException() handlers. My account is Premium. When we are using Evernote iOS SDK, everything goes fine.
What can be the problem?
Thanks in advance and best regards

Hi,
Can you help me with the above issue? I don't know what to do - Exception details don't provide much information. Message is just "timeout". Files of smaller size are uploaded normally. Can I configure the timeout value?
BR,
Alexey


Improve error handling in abacus-eval

Coverage decreased (-0.008%) to 91.296% when pulling d63b15bbdefab1d13426d3de2cb1714259847ee9 on SAP:improve-abacus-eval into f02280be7a53704fe7904e3c1151aef1229e5315 on cloudfoundry-incubator:master.

Coverage decreased (-0.008%) to 91.296% when pulling d63b15bbdefab1d13426d3de2cb1714259847ee9 on SAP:improve-abacus-eval into f02280be7a53704fe7904e3c1151aef1229e5315 on cloudfoundry-incubator:master.


Expose API to set log level in embedder

[build_iwasm ($CLASSIC_INTERP_BUILD_OPTIONS, -DWAMR_BUILD_LIB_PTHREAD=1, macos-latest, darwin)](https://github.com/bytecodealliance/wasm-micro-runtime/actions/runs/7220553955/job/19673804322)
GitHub Actions has encountered an internal error when running your job.

I'm amending the commit to restart the job.


More gui stuff

Codecov Report
Base: 72.15% // Head: 66.96% // Decreases project coverage by -5.19% :warning:

Coverage data is based on head (0625c59) compared to base (77a32bf).
Patch coverage: 67.15% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #283      +/-   ##
============================================
- Coverage     72.15%   66.96%   -5.20%
- Complexity      399      735     +336
============================================
  Files            70      119      +49
  Lines          1232     2552    +1320
  Branches        125      288     +163
============================================
+ Hits            889     1709     +820
- Misses          311      760     +449
- Partials         32       83      +51

Impacted Files
Coverage Δ

src/main/java/seedu/address/MainApp.java
0.00% <0.00%> (ø)

...main/java/seedu/address/commons/core/Messages.java
0.00% <ø> (ø)

...u/address/logic/commands/AddCommissionCommand.java
0.00% <0.00%> (ø)

...ain/java/seedu/address/logic/commands/Command.java
100.00% <ø> (ø)

...va/seedu/address/logic/commands/CommandResult.java
94.44% <0.00%> (+6.20%)
:arrow_up:

...java/seedu/address/logic/commands/ExitCommand.java
100.00% <ø> (ø)

...java/seedu/address/logic/commands/HelpCommand.java
100.00% <ø> (ø)

...dress/logic/parser/AddCommissionCommandParser.java
0.00% <0.00%> (ø)

.../seedu/address/logic/parser/FindCommandParser.java
100.00% <ø> (ø)

...ress/logic/parser/OpenCommissionCommandParser.java
0.00% <0.00%> (ø)

... and 85 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Playlists

Description
Added the ability to use YouTube playlists, as closely as possible mirroring the Video api, and making minimal changes to the pre-existing Video infrastructure.
Issues to fix
Playlists, #102
No commands are added at this stage, I can do that too, if you'd like. But internally, it (should) all be finished.
Testing
The tests (some) failed, I suspect this is due to issue #104

I think we should not use the Video struct for playlist entries. This would allow to pass incomplete videos zu the Client. Why not creating a PlaylistEntry struct?

I think we should not use the Video struct for playlist entries. This would allow to pass incomplete videos zu the Client. Why not creating a PlaylistEntry struct?

I was going to, but decided against it, as it'd be a needless abstraction. Instead I included the Loaded field, as shown. Its use is also shown in the example I added in example_test.go.
There are three states: NotLoaded, WeaklyLoaded, and FullyLoaded. WeaklyLoaded indicates it was loaded from a playlist (or by some other method in the future) and that not all the information is there, in this case we know that the metadata required to download is not there. So one must fully load it, which is now possible by a function I created and exposed.

Note: to my knowledge I had removed all underscored variables in one of the commits previously, I can't find any now. If I somehow missed one please direct me to it :) I'm not sure why I even did that, god knows what language I was using in the past few days...

@clr1107

Note: to my knowledge I had removed all underscored variables in one of the commits previously, I can't find any now. If I somehow missed one please direct me to it :) I'm not sure why I even did that, god knows what language I was using in the past few days...

like TestYoutube_extractPlaylistID in playlist_test.go ?

@clr1107

Note: to my knowledge I had removed all underscored variables in one of the commits previously, I can't find any now. If I somehow missed one please direct me to it :) I'm not sure why I even did that, god knows what language I was using in the past few days...

like TestYoutube_extractPlaylistID in playlist_test.go ?

@weeee9  There's also response_data.go and itag_test.go. File names are fine iirc, especially for testing the convention is that it's package_test.go or smth. As for the method names in tests those are auto-generated by gotests; IntelliJ does it as does VS Code :)

Sorry, is this getting merged?

Sorry, is this getting merged?

hi @clr1107  , thank for PR.
There's still conflict, could you solve it first?  We will review new changes soon.

some tests are failed because now youtube has both single and double quote for embedded body in different videos.
I will fix the test by new PR. Please ignore the failed tests in this PR.
@corny  , I see @clr1107  have already done  some requested changes, still some changes to finish?

This would be awesome, im not sure what still has to be done for it to be merged, but this would make this api 10/10

any update here?   the PR  has been opened for a long time and codebase has a lot of change. If no update here,  I suggest to close this PR and open new one instead of working on this one.

any update here?   the PR  has been opened for a long time and codebase has a lot of change. If no update here,  I suggest to close this PR and open new one instead of working on this one.

any update here?   the PR  has been opened for a long time and codebase has a lot of change. If no update here,  I suggest to close this PR and open new one instead of working on this one.

No updates, I've had no time sorry. It was ready to merge a long time ago but I suppose it just never was. I no longer have time to recreate it, so I can't redo it.
Thanks.

any update here?   the PR  has been opened for a long time and codebase has a lot of change. If no update here,  I suggest to close this PR and open new one instead of working on this one.

No updates, I've had no time sorry. It was ready to merge a long time ago but I suppose it just never was. I no longer have time to recreate it, so I can't redo it.
Thanks.


deprecation warnings loading current grafana board into the latest grafana

we should take a look at the warnings, it's probably graphs or similar that we need to update.

There are the panels we need to migrate:
Validator

Attestation Performance
Attestation Inclusion Distance (slots)
Included Blocks

Beacon Chain

Trailing Distance
Participation Rate
Peer Client Software

System

Memory Usage
GC Time

Queues, Executors and Caches

Batch Signature Verification Queue Size
Signature Verification Batch Size
Signature Verification Batch Size

Panels have been updated. Will run a sanity check before a release.


Add optional cache layer to help with (jquery.instagramFeed) issue #25. Default state is not caching.

Greets Team InstagramFeed,
I've added a cache layer to help deal with this error message:
InstagramFeed: It looks like YOUR network has been temporary banned because of too many requests. See https://github.com/jsanahuja/jquery.instagramFeed/issues/25

This adds 3 new config options:
   // option.name: default_value
   cache_use: false,
   cache_for: "30:min",
   cache_prefix: "igf_",

Docs & version number updated as per CONTRIBUTING.md.
An aside: I ran into a small snafu with the version number. There's a mismatch between src/
InstagramFeed.js (@1.4.0) and package.json (@1.5.4). I ended up going with 1.6.0 since this is a backwards compatible API change.
Let me know if you'd like anything changed and thank you SO much for an Instagram option that isn’t painful!
-Dan

One more thing team InstagramFeed. If you're cool with adding this PR — and after fixing anything you guys want me to — I'll go ahead and add it to jquery.instagramFeed also.

As it is stated in the CONTRIBUTING.md file:

When contributing to this repository, please first discuss the change you wish to make via issue, email, or any other method with the owners of this repository before making a change.

I am sorry but the implementation of the client-side catching should follow what was described in #49 (the same done in https://github.com/jsanahuja/jquery.instagramFeed/pull/97)

I'm not sure what you’re asking for here @jsanahuja. #49 seems to imply that queries should be made through the IG graphql url instead of how you guys are doing it now; scraping returned html.
I'm also confused about what you mean by:

Should be implemented as it was done in https://github.com/jsanahuja/jquery.instagramFeed/pull/97/files

as the url you referenced show code that isn’t  present in the current version of InstagramFeed. Should I have started the PR from a different branch?
My thoughts on the "complex" time specification come down to the idea of making the API easy on the end user. I agree that 3 variables could be reduced but I didn’t want to be presumptuous by giving the end user no way of turning the caching off, nor assuming that one caching time would fit all.
I agree with what you said in #97:

I don't really know, depends on how often does people publish on their Instagram accounts.

... the cache time needs to be up to the end user.
I also agree that the default caching state should be "ON"
What would you recommend as a solution, from an end user perspective, that would allow:

Easy to "think about" cache time periods
Cache on by default

Perhaps cache_for: false would turn off caching?

I'm not sure what you’re asking for here @jsanahuja. #49 seems to imply that queries should be made through the IG graphql url instead of how you guys are doing it now; scraping returned html.
I'm also confused about what you mean by:

Should be implemented as it was done in https://github.com/jsanahuja/jquery.instagramFeed/pull/97/files

as the url you referenced show code that isn’t present in the current version of InstagramFeed. Should I have started the PR from a different branch?
My thoughts on the "complex" time specification come down to the idea of making the API easy on the end user. I agree that 3 variables could be reduced but I didn’t want to be presumptuous by giving the end user no way of turning the caching off, nor assuming that one caching time would fit all.
I agree with what you said in #97:

I don't really know, depends on how often does people publish on their Instagram accounts.

... the cache time needs to be up to the end user.
I also agree that the default caching state should be "ON"
What would you recommend as a solution, from an end user perspective, that would allow:

Easy to "think about" cache time periods
Cache on by default

Perhaps cache_for: false would turn off caching?

jquery.instagramFeed is the jQuery version of this repository. We do the same on both repositories so the cache thing should be implemented as it was done there in https://github.com/jsanahuja/jquery.instagramFeed/pull/97/files. You can use different ways but the end user API (and defaults) must be the same
Thank you :)

I am closing this as this has been implemented in #54

Thanks for considering the change @jsanahuja. Glad you guys already had caching in the pipe!
I noticed that you guys rolled out #54 with 2.0.2 and that it's live on packagist so, again, thanks for getting caching implemented with the new cache_time var.

it's live on packagist

I don't know who mantains that. We only publish it in the Github NPM Registry (more info here) and CDNJS


test

Thank you for submitting a pull request! Please make sure you have read the contribution guidelines before proceeding.

This PR implements or fixes... (explain your changes)
…
This PR closes the following issues... (if applicable)
…
Does this PR require the Docs to be updated?
…
Does this PR require new tests?
…
This branch been tested on... (click all that apply / add new items)
Browsers:

[ ] Chrome version
[ ] Firefox version
[ ] Safari version
[ ] IE / Edge version
[ ] iOS Browser version
[ ] Android Browser version

Codecov Report

Merging #1 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #1   +/-   ##
=======================================
  Coverage   78.34%   78.34%
=======================================
  Files          32       32
  Lines        1339     1339
  Branches      215      215
=======================================
  Hits         1049     1049
  Misses        242      242
  Partials       48       48

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f8c4e1c...c2409ac. Read the comment docs.


abci: Precommits added to the LastCommit set don't have vote extensions verified

Bug Report
Setup
CometBFT version
v0.38 and main
What happened?
Invalid vote extensions are included in the ExtendedCommitInfo offered to the ABCI application via PrepareProposal.
What did you expect to happen?
ExtendedCommitInfo only includes valid vote extensions.
Moreover, the LastCommit field of a block should only contain Precommits whose extensions were accepted by the application.
How to reproduce it
A validator receives a Precommit from the previous height with an invalid vote extension during the NewHeightRoundStep.
How to fix it
There are a few options:

do not fix

instead, inform applications that PrepareProposal may receive non-validated extensions
the downsides  are

2/3N extensions may be verified twice by the proposer, once in VerifyVoteExtension and once in PrepareProposal.
the LastCommit field of a block may contain information of Precommits whose vote extensions are invalid.

verify the extensions of Precommits received while waiting for timeout_commit

the downsides are

the application has already moved on to the next height and will now be required to validate extensions from the previous height.
makes the spec more complex.

a simple and correct option here is to reject extensions for the previous height, however this would impact in the reward distribution (undo what timeout_commit is supposed to enable).

simply rejecting old extensions could impact in reward distribution (undo what timeout-commit is supposed to allow.

Could you clarify this? Are we worried that we won't reward validators that included invalid vote extension? And don't we drop those votes anyway during consensus (i.e. before we enter the commit timeout phase where we wait for extra precommits)?

The simplistic implementation of VerifyVoteExtension will be to reject all vote extensions with height previous to the current height. This will make the point of having this additional waiting period useless.

Ok, i never thought we would do that. The options says "verify the extensions of Precommits received while ..." and not "reject....". So I wouldn't count it under "Cons" for that option.

We had a meeting today to decide how to address this issue (attendees @cason, @lasarojc, @sergio-mena).
First let us describe the options we considered:

Option 1a. Clarify the spec: "Some extensions delivered in PrepareProposal may not be verified (and explain why), but more than 2/3 will always have been verified". Suggestion for app developers: "use your code for VerifyVoteExtension also as part of PrepareProposal"
Option 1b. Modify ABCI structs (and internal consensus structs) to distinguish unverified votes/extensions from verified votes/extensions when calling PrepareProposal.
Option 2. When a precommit arrives late (after its the block has already been delivered to the application) we nevertheless call VerifyVoteExtension on it. The app will observe a call to VerifyVoteExtension for the previous height, but still has the opportunity to verify the extension. In this option, all vote extensions delivered in PrepareProposal have been verified by the app
Option 3. Refactor the way timeout_commit is implemented so that the block is delivered only after timeout_commit expires. So all precommits that will be part of LastCommit in the following block could trigger VerifyVoteExtension for their extensions having the app always receiving the call for the present height.

These are the decisions we made:

First is it obvious Option 3 implies massive changes in the consensus logic, and it is not certain if there will be negative implications performance-wise
We agree Option 2 is still too risky to backport to v0.38.x: it implies a noticeable change in behavior, and many Apps have already implemented against current behavior described in the spec's grammar. Changing it in a point release in v0.38.x may lead to unexpected negative consequences for those apps that may have already undergone some testing phases.
Therefore, we agree we need to go for Option 1.

We agree that we believe that Option 1b can be done and backported to v0.38.x without disruption to chains doing point upgrades to v0.38.x.
But also agreed it's far from trivial to implement
Decisions:

we will implement Option 1b only when users show the need for this
we will go for Option 1a for the time being

For the long term:

not clear whether we would like to implement Option 2, Option 3, or even stay with Option 1
so, we agree this should be part of the ABCI 3.0 discussions

Action items

[x] Update the issue with this text
[ ] Implement option 1a (fix the spec) -- this should close this issue
[ ] Contact the teams affected
[ ] Create a new issuse, explaining options 2 and 3, and hook it to the ABCI 3.0 list in the tracking issue

In my opinion, changing the spec to match the behavior after the fact (option 3) is also a breaking change just as much as making a change in code (option 2) would be. The reason I say this is that developing a vote extension based oracle around 0.38 did in fact require us to make changes to our proposal handlers in order to compensate for the bug or "change in spec", however you want to refer to it.
The expectation that the application has when using vote extensions is that every VE passed to the prepare proposal handler has passed a VerifyVoteExtension call. In order to compensate for this expectation no longer holding, we have to verify VEs based on state in the previous block. This is also something we need to do in Process Proposal and in general is not an easy thing to do (this might be made easier in the SDK as well as Comet, I'm not sure).
If the new expectation for applications is to run validation on previous block VEs and by extension previous block state, then it should be easier to reference and access that state in modules. As is, we think we're going to rely on nodes having EvidenceParams.MaxAgeNumBlocks set to non-zero in order to access previous state--something which feels brittle overall.

Thanks for the feedback, @Eric-Warehime
Given that both the current behavior as well as option 2 require validation using previous state, do you feel that option 3, in spite of delaying block execution, might be a better solution?

I share your concern over option 3 from a performance perspective. It seems like at worst it could universally increase block times by timeout_commit which would probably scare most chains away from ever using vote extensions.
Option 1b seems best from an end user perspective. If I have a quorum of VEs that I know are valid and some additional VEs which have no guarantees of validity, I can choose as an application to use only the pre-validated VEs or attempt to validate the additional VEs after the fact.
That being said, my Comet knowledge isn't great enough to know all the hurdles implementing this in a backwards compatible way entails.


Autodiscovery v0.8.0

Hello,
I'm facing that auto discovery does not work properly.
As I understand with autodiscovery off I anyway get metrics for all of the database, so my catch is that  if a new database is created then no postgres exporter pod restart need.

Could you please clarify what is auto discovery for?

I have tried setting --disable-default-metrics --disable-settings-metrics --auto-discover-databases and the database to "", all this doesn't make the exporter work I still get errors (less though, but still) #314
903 error(s) occurred:

collected metric "pg_postmaster_start_time_seconds" { label:<name:"server" value:"postgres-stage-postgresql:5432" > gauge:<value:1.576833383e+09 > } was collected before with the same name and label values
collected metric "pg_database_size" { label:<name:"datname" value:"postgres" > label:<name:"server" value:"postgres-stage-postgresql:5432" > gauge:<value:7.852703e+06 > } was collected before with the same name and label values
collected metric "pg_database_size" { label:<name:"datname" value:"identityservice" > label:<name:"server" value:"postgres-stage-postgresql:5432" > gauge:<value:8.532639e+06 > } was collected before with the same name and label values

Hi!
Maybe you use the old version queries.yaml. For some queries, you need to add field "master", Look into new file queries.yaml in the repository.

Thank you @sfalkon, this is a great catch. Seems to work  great https://github.com/helm/charts/pull/19749

@dennybaa can you close this?

@yashbhutwala Sure no problem


Resolve error on dependency install with yarn

Hello,
I have an issue while trying to use the vue3-datatable library.
I followed the installations guide in the documentation but when it comes to import the datatable component and use it inside my vue3 app i encounter this error :
error  Resolve error: Cannot find module<EMAIL_ADDRESS>    at createEsmNotFoundErr (node:internal/modules/cjs/loader:960:15)
    at finalizeEsmResolution (node:internal/modules/cjs/loader:953:15)
    at resolveExports (node:internal/modules/cjs/loader:482:14)

The error tell me that it is searching the .umd.js file right ? The problem is that i can't find this file inside the node_modules folder of this library.
I've tried to uninstall and reinstall the module, still the same.
Do you have any idea on how i can solve this issue ?
Thanks
_I hope i gave enough details, if not, dont hesitate to ask me more about it _

Hi,
I think can you please try below solution and let me know its works for you or not.
Try setting your "moduleResolution" strategy to "node" which isn't the default on "module": "umd".
check moduleResolution setting in tsconfig file.

It was already set to 'node'. Can it be the node version of the project ? Its currently on 16.16
Or maybe the vue version ? Im not using the latest one of vue 3

Hi,
I am not sure about this package will work on vue 2.
Could you please provide zip of sample project so i can check.


Update go.mod

Move from go 1.17 to go1.18
Ran `go mod tidy`

Signed-off-by: Swati Sehgal<EMAIL_ADDRESS>

I would prefer to be aligned with u/s in this repo

This change is not needed for now, we will wait for kubernetes-sigs/scheduler-plugins to get upto speed and resync the repo at a later stage.


CI: K8sKafkaPolicyTest Kafka Policy Tests KafkaPolicies

CI failure
Test output:
/home/jenkins/workspace/Ginkgo-CI-Tests-Pipeline/k8s-1.10-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:420
Failed to produce to empire-hq on topic empire-announce
Expected
    <*errors.errorString | 0xc0005030e0>: {
        s: "ExecKafkaPodCmd: command 'kubectl exec -n default empire-hq-6f98fd55c7-k6gmh sh -- -c \"echo 'Happy 40th Birthday to General Tagge' | ./kafka-produce.sh --topic empire-announce\"' failed Exitcode: 1 \nStdout:\n \t \nStderr:\n \t [2019-08-05 02:52:18,875] WARN Removing server kafka-service:9092 from bootstrap.servers as DNS resolution failed for kafka-service (org.apache.kafka.clients.ClientUtils)\n\t org.apache.kafka.common.KafkaException: Failed to construct kafka producer\n\t \tat org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:416)\n\t \tat org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:288)\n\t \tat kafka.producer.NewShinyProducer.<init>(BaseProducer.scala:40)\n\t \tat kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:48)\n\t \tat kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)\n\t Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers\n\t \tat org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:64)\n\t \tat org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:373)\n\t \t... 4 more\n\t command terminated with exit code 1\n\t \n",
    }
to be nil
/home/jenkins/workspace/Ginkgo-CI-Tests-Pipeline/k8s-1.10-gopath/src/github.com/cilium/cilium/test/k8sT/KafkaPolicies.go:236

Seems like Kafka broker failure:
2019-08-05T02:51:08.856470289Z /usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2019-08-05T02:51:08.85655577Z   'Supervisord is running as root and it is searching '
2019-08-05T02:51:08.912524557Z 2019-08-05 02:51:08,912 CRIT Supervisor running as root (no user in config file)
2019-08-05T02:51:08.912619025Z 2019-08-05 02:51:08,912 WARN Included extra file "/etc/supervisor/conf.d/kafkaproxy.conf" during parsing
2019-08-05T02:51:08.912654614Z 2019-08-05 02:51:08,912 WARN Included extra file "/etc/supervisor/conf.d/zookeeper.conf" during parsing
2019-08-05T02:51:08.912661395Z 2019-08-05 02:51:08,912 WARN Included extra file
"/etc/supervisor/conf.d/kafka.conf" during parsing
2019-08-05T02:51:08.956783785Z 2019-08-05 02:51:08,956 INFO RPC interface 'supervisor' initialized
2019-08-05T02:51:08.957008041Z 2019-08-05 02:51:08,956 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2019-08-05T02:51:08.957293828Z 2019-08-05 02:51:08,957 INFO supervisord started with pid 1
2019-08-05T02:51:09.964836635Z 2019-08-05 02:51:09,962 INFO spawned: 'zookeeper' with pid 8
2019-08-05T02:51:09.964855433Z 2019-08-05 02:51:09,964 INFO spawned: 'kafka_proxy' with pid 9
2019-08-05T02:51:09.965850897Z 2019-08-05 02:51:09,965 INFO spawned: 'kafka' with pid 10
2019-08-05T02:51:09.971813905Z 2019-08-05 02:51:09,971 INFO exited: kafka_proxy (exit status 2; not expected)
2019-08-05T02:51:11.051749593Z 2019-08-05 02:51:11,051 INFO success: zookeeper entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-08-05T02:51:11.053020839Z 2019-08-05 02:51:11,052 INFO spawned: 'kafka_proxy' with pid 259
2019-08-05T02:51:11.053344759Z 2019-08-05 02:51:11,053 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-08-05T02:51:11.05985448Z 2019-08-05 02:51:11,059 INFO exited: kafka_proxy (exit status 2; not expected)
2019-08-05T02:51:13.09070723Z 2019-08-05 02:51:13,088 INFO spawned: 'kafka_proxy' with pid 272
2019-08-05T02:51:13.101894211Z 2019-08-05 02:51:13,101 INFO exited: kafka_proxy (exit status 2; not expected)
2019-08-05T02:51:16.618403922Z 2019-08-05 02:51:16,618 INFO spawned: 'kafka_proxy' with pid 304
2019-08-05T02:51:16.622548689Z 2019-08-05 02:51:16,622 INFO exited: kafka_proxy (exit status 2; not expected)
2019-08-05T02:51:17.624170015Z 2019-08-05 02:51:17,623 INFO gave up: kafka_proxy entered FATAL state, too many start retries too quickly

5289e4d7_K8sKafkaPolicyTest_Kafka_Policy_Tests_KafkaPolicies.zip
https://jenkins.cilium.io/job/Ginkgo-CI-Tests-Pipeline/7970/testReport/junit/Suite-k8s-1/10/K8sKafkaPolicyTest_Kafka_Policy_Tests_KafkaPolicies/

This may have been fixed by https://github.com/cilium/cilium/issues/8779 - can we close if this does not occur again @nebril ?

We are no longer seeing this in the CI.


Create separate --v1alpha3=true prow pipeline

We need to run v1alpha3 tests in a separate pipeline since they are not ready to gate commits.

Is this for pilot e2e ?

Correct. We are not testing v1alpha3 in prow at all at the moment, and it
needs to prove to be reliable to become pre-submit.
On Fri, Mar 16, 2018 at 10:55 AM Sebastien Vas<EMAIL_ADDRESS>wrote:

Is this for pilot e2e ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/4270#issuecomment-373794732, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJGIxou85vFoWAue0Z6LEr3vdMgqfn8-ks5te_xzgaJpZM4SrLn3
.


To-Do:

[x] Add white point
[ ] Add chromatic adaptation conversions
[ ] Add white point to color spaces

Looks like a good start :+1: Just a few things I thought about:

The white point macro seems simple enough to not really be necessary. Will it become more complex?
The Rust conventions says that type names that are abbreviations should be camel cased as one word (COW -> Cow). Maybe we should do the same with the various variants of WP.
It's probably a good idea to make the white_point module public to not clutter the crate root too much.

The white point macro seems simple enough to not really be necessary. Will it become more complex?

Not really. Its just that I have to add a few more and right now its not clear what the correct x,y values are. I''l remove it in the end when I add the remaining white points.

The Rust conventions says that type names that are abbreviations should be camel cased as one word (COW -> Cow). Maybe we should do the same with the various variants of WP.

D65Fov10 is already camel case. Are you talking about something else? Oh and if you can think of a better name, let me know :)

It's probably a good idea to make the white_point module public to not clutter the crate root too much.

Ok

Not really. Its just that I have to add a few more and right now its not clear what the correct x,y values are. I''l remove it in the end when I add the remaining white points.

Ok, I was just checking. I have been thinking about phasing out some macros, in general, but that's a separate topic.

D65Fov10 is already camel case. Are you talking about something else?

Those are correctly cased. I'm talking about the type parameters, like WP -> Wp, SWP -> Swp, and so on.

Oh and if you can think of a better name, let me know :)

I'll see what I can come up with, but you have most likely witnessed how bad I am at naming things. The name "Palette" is a masterpiece among my creations ;) Speaking of which, you are always welcome to suggest alternate names for things I contribute with.

Those are correctly cased. I'm talking about the type parameters, like WP -> Wp, SWP -> Swp, and so on.

Yeah, those need to be changed. I thought trait bounds were all caps, but they need to be camel case too.

They are temporary types, so it's the same rule for them as well, but they are usually only one letter long, so it's easy to forget :)

I have implemeted the white point for lab and changed the into iterator from
pub trait IntoColor<T>: Sized{

to
pub trait IntoColor<WP = D65, T = f32>: Sized
    where T: Float,
     WP: WhitePoint<T>
{

I get an error  for  converting xyz to rgb
        let rgb: Rgb<T> = xyz.into_rgb();
        Self::from_rgb(rgb)

error
src/hsl.rs:63:31: 63:39 error: unable to infer enough type information about `_`; type annotations or generic parameter binding required [E0282]
src/hsl.rs:63         let rgb: Rgb<T> = xyz.into_rgb();
                                            ^~~~~~~~

This works
        let rgb = <Rgb<T>  as FromColor<WP, T>>::from_xyz(xyz);
        <Self as FromColor<WP, T>>::from_rgb(rgb)

Is there any other altenative?

Is the white point of xyz known?

For now I am not implementing the White Point for Xyz and Yxy. I'll implement it for everything else and only then add it in for Yxy and Xyz if it makes sense.
I've been using the convert as notation for now. Let's see how it shapes up.

Well, it can't possibly know what white is if it's not specified, so that's most likely the reason :)

Tests still fail due to color enum. I will update it next.

Why are you renaming almost every color to *Space?

Why are you renaming almost every color to *Space?

I was trying to keep the color spaces compatible with earlier versions and I have defined the original colors to use the default white point
pub type Lab<T> = LabSpace<D65, T>;
pub type Laba<T> = LabSpaceAlpha<D65, T>;

I am expecting this to work as usual before and after the white point change
   let lab1 = Lab::new(0.5, 0.5, 0.5)

   let lab2 = Lab {l: 0.5, a: 0.5, b:0.5}

Is it unnecessary since we are setting the defaults in the definition?
   pub struct LabSpace<Wp = D65, T = f32>

Oh, ok. Well, most users should not be affected by the change, unless most users uses an other float type than f32. Adding the extra white point shouldn't be a huge step from changing the precision, and they can always make their own custom aliases when they need to.
I see the point and I understand that it may become tedious in some situations, but the fact that users can make their own aliases makes me think that it's probably not such a big deal, after all.

I'll revert to using the short names. Having the defaults pretty much makes it unnecessary.

We can do something about it later if it turns out to be a problem. There may even be a better solution then. Who knows?
You have probably noticed, but type aliases and the documentation don't really play nice together. You can't see any implementations for aliases, which is why I made the links for the alpha aliases. You can write impl MyAlias { ... } and it works fine, but it won't show up anywhere in the documentation. It's a bit annoying and only somewhat related to this, so I'll stop there :)

Tests still fail for Srgb, Gamma rgb and color enum conversion.

I have mostly skimmed through the changes, but I think it looks good so far. The only things I can really comment on at the moment is that I'm leaning towards making the white point fields public, to allow Rgb { red: ..., green: ... blue: ..., _wp: PhantomData } style construction, and that I'm quite sure Xyz and Yxy should have a white point parameter.
You can't know what XYZ for white is if you haven't specified a white point, and the requirement of <xyz as  as FromColor<WP, T>>::... is an effect of that problem. We can't just throw away the reference to how the values are supposed to be interpreted. It would actually make more sense to do this with RGB, since we know that rgb(1, 1, 1) will always represent white, but the perception of an XYZ value differs depending on the reference white.

The project builds, but tests fails due to private _wp field.

The only things I can really comment on at the moment is that I'm leaning towards making the white point fields public

I was just thinking the same thing. I can't think of a downside to this. Is wp a good enough name or should I make it white_point (instead os _wp).

I'm quite sure Xyz and Yxy should have a white point parameter.

I am not convinced that Xyz and Yxy need a white point.
Consider the white point (0.31271,0.32902, 1.0). This represents the D65 Illuminant for 2 degree observer. If we define the white point for this Yxy, then all following will be the same. The white point is essentially meaningless here.
Yxy<D65,_> == Yxy<D50,_> == Yxy<A,_> == (0.31271,0.32902, 1.0)

But the same is not the case for the other colors.
Rgb<D65,_> != Rgb<D50,_> != Rgb<A,_>

Also, The conversions Color -> Xyz will depend only on the source white_point and Xyz -> Color will only depend on the target white point and we should not run into any issues in the conversions.

I was just thinking the same thing. I can't think of a downside to this. Is wp a good enough name or should I make it white_point (instead os _wp).

white_point looks nicer to me.

then all following will be the same. The white point is essentially meaningless here

We must not mix up perceptive and numeric equality. The Yxy values will have the same energy values, but they will look different under different illuminants, while the Rgb values will look the same, but will have different energy levels under different illuminants.
Take this snowy house, for example. The snow looks mostly white because the colors are adapted to sunlight, but it's blue in the shadows, where it's only illuminated by the sky. We could adapt the colors to make the shadows look white/gray, but the rest would then look yellow. Both the sunlit and shadowy snow is still "white", though, since the snow will reflect more or less equal amounts of every wavelength.

I'll continue my example with some code and numbers. Let's pretend that the sky itself is a D50 illuminant and that the sun is a D65 illuminant. Then the shadowy snow would be Rgb::<D50, _>::new(1.0, 1.0, 1.0), which looks like purest white if our eyes adapt to the D50 light, and the sunlit snow would be Rgb::<D65, _>::new(1.0, 1.0, 1.0). The camera in the photo is adapted to the sunlight, or D65 in this example, so we have to translate the color of the shadowy snow to D65 to know how it will look in the photo:
let snow_shadow: Rgb<D50, _> = Rgb::new(1.0, 1.0, 1.0);
let snow_shadow_xyz: Xyz = snow_shadow.into();

//This is no longer numerically the same as Rgb::new(1.0, 1.0, 1.0),
//meaning that it will look different.
let snow_shadow_photo: Rgb<D65, _> = snow_shadow_xyz.into();

Let's say that we want to know if the shadowy snow reflects the same amount of light as the sunlit snow, and therefore should look the same under the same light. That's where we have to do a chromatic adaption:
let snow_shadow: Rgb<D50, _> = Rgb::new(1.0, 1.0, 1.0);
let snow_shadow_xyz: Xyz = snow_rgb.into();

//Adapt from D50 to D65, or move snow from the shadow into the sunlight
let snow_photo_xyz: Xyz = snow_shadow_xyz.adapt::<D50, D65>();

//This should now be equal to Rgb::new(1.0, 1.0, 1.0)
let snow_photo: Rgb<D65, _> = snow_photo_xyz.into();

The white point does matter to XYZ, but not in the same way as to RGB. It's still important for knowing how the signals are perceived, while RGB needs it to know what the signals should be. This is the first example again, but with a white point aware Xyz:
let snow_rgb: Rgb<D50, _> = Rgb::new(1.0, 1.0, 1.0);
let snow_xyz: Xyz<D50, _> = snow_rgb.into();

//Adapt our brain to the sunlight, so the same numbers, but different white point.
let snow_shadow_photo_xyz: Xyz<D65, _> = snow_xyz.with_white_point();

//This is no longer numerically the same as Rgb::new(1.0, 1.0, 1.0),
//meaning that it will look different.
let snow_shadow_photo: Rgb<D65, _> = snow_shadow_photo_xyz.into();

...and the second example:
let snow_shadow: Rgb<D50, _> = Rgb::new(1.0, 1.0, 1.0);
let snow_shadow_xyz: Xyz<D50, _> = snow_rgb.into();

//Adapt from D50 to D65, or move snow from the shadow into the sunlight
let snow_photo_xyz: Xyz<D65, _> = snow_shadow_xyz.adapt();

//This should now be equal to Rgb::new(1.0, 1.0, 1.0)
let snow_photo: Rgb<D65, _> = snow_photo_xyz.into();

It's still possible to do the same calculations without the white point on Xyz, but you have to keep track if it yourself. Letting Xyz keep track of it instead will require an explicit transition between white points, but there's not the same risk of mixing things up. I feel like it just completes the whole model.

Yeah, Xyz also needs an illuminant. For chromatic adaptation needing to do xyz_src to xyz_dst pretty makes this necessary. I'll make the change.
Right now the Color conversions do not take chromatic adaptation into account. They only convert from and into the same white point.
What I am concerned about needing white point annotation in conversions. For example, rgb -> hsb should just work assuming the same white point without having to specify it. White Point annotation is only necessary when chormatic adpatation is needed.
// assumes default D65
let rgb = Rgb::new(1.0, 0.0,0.0);

// Should automatically use D65 white point
let hsl = rgb.into_hsl()
let lab = rgb.into_lab() // or even rgb.into()

// Instead of Manually having to specify whtie point
let hsl: Lab<D65,_> = rgb.into();
let lab: Lab<D65,_> = rgb.into();

Can we use the existing FromColor and IntoColor to make this happen?
Should I add this in this PR or in a new PR?

just run into a usablility issue. This code fails
let lab = Lab::new(0.4623,-0.517,0.499);
let lab = Lab {l: 0.4623, a: -0.517, b: 0.499, white_point: PhantomData };

with the following error even though the default (D65 and f32) are defined for the lab space.
 unable to infer enough type information about `_`; type annotations or generic parameter binding required

This works
    let lab = Lab::<D65,f64> {l: 0.4623, a: -0.517, b: 0.499, white_point: PhantomData };

Can we do anything to infer the default white points?

Right now the Color conversions do not take chromatic adaptation into account. They only convert from and into the same white point.

I think that's good. Chromatic adaption can be more destructive than normal conversion, so I think it should be done explicitly.

Can we use the existing FromColor and IntoColor to make this happen?
Should I add this in this PR or in a new PR?

Those conversions should just work. The white point becomes a part of the type, so it should be restricted in the same way as the float type. I wrote them out in my example to make sure it was clear, but it should only be necessary if something other than D65 is used, just as with the float type.

Can we do anything to infer the default white points?

I don't know. It my depend on the context. Is it just the struct syntax? Is it only when the type isn't specified anywhere else?

Those conversions should just work. The white point becomes a part of the type, so it should be restricted in the same way as the float type

So FromColor<Wp, T> becomes FromColor<Swp, Dwp, T> and same with the IntoColor  trait.

I don't know. It my depend on the context. Is it just the struct syntax? Is it only when the type isn't specified anywhere else?

You can check the last commit for all the places I has to explicitly specify the type. It's pretty much everywhere.

So FromColor<Wp, T> becomes FromColor<Swp, Dwp, T> and same with the IntoColor trait.

That's not what I meant. I think it's good that chromatic adaption is explicit and not a part of FromColor or IntoColor. It fits better as its own trait, in my opinion.

You can check the last commit for all the places I has to explicitly specify the type. It's pretty much everywhere.

Oh, no! It looks like we have stumbled upon an unfinished feature. See this playpen: http://is.gd/qFU2o5
I'm not sure what to do here... This problem will also be present in the planned Rgb reform.

How is HashMap default hasher impelmentation working then?

Looks like new is only implemented for HashMap<K, V, RandomState>: https://doc.rust-lang.org/src/std/collections/hash/map.rs.html#506
We could do something similar: http://is.gd/AIkZ09 But the situation is different, because the white point is never passed to a function.

We can use something like this http://is.gd/do9sEJ

Hmm... All of these are weird in different ways.

The alias makes new available without any fuss, but it's an alias, so we have to deal with naming in a good way. We don't want to "punish" anyone with too long names.
Only implementing new for D65, without passing the Wp to with_wp, is convenient for the default case, but may be surprising and inconvenient for the general case.
Only implementing new for D65, and passing the Wp to with_wp, has the same pros and cons, but with the need to pass an unused parameter instead of specifying a type parameter. Not sure if better or worse, to be honest.

It looks like these two also works:
let rgb = Rgb::<D65>::with_wp(1.0, 0.0, 0.0);
let rgb: Rgb<D65> = Rgb::with_wp(1.0, 0.0, 0.0);

That is, only one type parameter is actually necessary.

Went with option 2, for now. Need to add chromatic adaptation

Not sure why tests are failing. They work fine on both stable and nightly on my computer.

Looks like png is broken. You probably haven't cargo updated.

Looks like it's caused by the bitflags crate. I submitted an issue.

This is mostly done from my end. Just need to make sure the tests pass once the png crate is fixed and any other documentation/ clean up required.

Also let me know if any of the trait parameters in Chromatic Adaptation/ White Points need to be associated types. I am not 100% clear when to use associated types, so I just make everything trait parameters

Great! I'll take a closer look in a moment. I haven't gotten any response from the png people, which is a bit surprising. It's widely used, so we can't be the only ones affected. I'm actually considering fixing it, myself, if I can.

I am not 100% clear when to use associated types, so I just made everything trait parameters

My rule of thumb is to use associated types when the trait is only supposed to be implemented once per type. Like in Mix. A color should only be mixable using one type of mix factor, while From uses a type parameter because you may want to convert from more than one type.

I have looked through it and commented on a few things, but it's otherwise very good. Let's hope the png issues are resolved soon.

I was thinking a bit more about the chromatic adaption trait. I don't know how practical it is to merge it with the IntoColor trait, but I thought that there could possibly be a From and Into variant of it:
pub trait AdaptFrom<Swp, Dwp, T, C> where
    Swp: WhitePoint,
    Dwp: WhitePoint,
    T: Float,
{
    fn adapt_from(color: C) -> Self;
    fn adapt_from_using(color: C, matrix: ...) -> Self;
}

impl<S, D, Swp, Dwp, T> AdaptFrom<Swp, Dwp, T, S> for D where
    S: IntoColor<Swp, T>,
    D: FromColor<Dwp, T>,
    Swp: WhitePoint,
    Dwp: WhitePoint,
    T: Float,
{
    fn adapt_from(color: C) -> D {
        //...
    }

    fn adapt_from_using(color: C, matrix: ...) -> D {
        //...
    }
}

pub trait AdaptInto<Swp, Dwp, T, C> where
    Swp: WhitePoint,
    Dwp: WhitePoint,
    T: Float,
{
    fn adapt_into(self) -> C;
    fn adapt_into_using(self, matrix: ...) -> C;
}

impl<S, D, Swp, Dwp, T> AdaptInto<Swp, Dwp, T, D> for S where
    S: IntoColor<Swp, T>,
    D: AdaptFrom<Swp, Dwp, T, Src>,
    Swp: WhitePoint,
    Dwp: WhitePoint,
    T: Float,
{
    fn adapt_into(self) -> C {
        C::adapt_from(self)
    }

    fn adapt_into_using(self, matrix: ...) -> C {
        C::adapt_from_using(self, matrix)
    }
}

I'm obviously trying to mimic From and Into here, but I think they are quite good. You can use them as both Rgb::<D50>::adapt_from(color) and color.adapt_into(), and they are useful for different situations. adapt_from can, for example, be used like this: maybe_color.map(Rgb::<D50>::adapt_from) or even  maybe_color.map(AdaptFrom::adapt_from), instead of (at best) maybe_color.map(|c| c.adapt_into()).
What do you think about that? The names are, of course, up for discussion.

FromAdapt and IntoAdapt is good. I will add it in.
Can we merge adpatation and from and into color?
With impl specialization we could have merged Adaptation with from and into traits
Have the usual fast conversions when white points are the same
pub trait FromColor<Wp = D65, T = f32>: Sized
    where T: Float,
        Wp: WhitePoint<T>
{
   from_xyz()
  from_xyz_using() // should be same as from_xyz, no adaptation
}

Have the chromatic adaptation variant when different
pub trait FromColor<Swp, Dwp, T >: Sized
    where T: Float,
        Swp: WhitePoint<T>,
        Dwp: WhitePoint<T>
{
   from_xyz()
   from_xyz_using()

I wonder how much effort it would be using macro's?
One issue I can think is that now we can support chromatic adaptation for user defined white points (will be useful for icc profiles), macro's will only implement it for the whitepoints defined in the library.

I feel like it's probably best to keep them apart at the moment. It will allow implementations that are independent from From/IntoColor, and I expect chromatic adaption to be significantly less common than conversion within the same white point, so it's good if the usual conversion has as little overhead as possible. I would also really like to keep the support for user defined white points.

Everything should build now if you update image to 0.7. Turns out they were using an old version of png.

Adapt from is currently defined as
///Converts color from one reference white point to another via chromatic adaptation (Swp -> Dwp)
pub trait AdaptFrom<S, Mat, Swp, Dwp, T>: Sized
    where T: Float,
          Swp: WhitePoint<T>,
          Dwp: WhitePoint<T>,
          Mat: TransformMatrix<Swp, Dwp, T>,
          S: IntoColor<Swp, T>,
{
    ///Convert the source color to the destination color using the bradford method
    fn adapt_from(color: S) -> Self;
    ///Convert the source color to the destination color using the specified method
    fn adapt_from_using(color: S, method: Mat) -> Self;
}

This works
        let input_a = Xyz::<A, f32>::with_wp(0.315756, 0.162732, 0.015905);
        let computed_vonkries: Xyz<C, f32> = Xyz::adapt_from_using(input_a, Method::VonKries);

But this does not
        let computed_bradford: Xyz<C,f32> = Xyz::adapt_from(input_a);
       // errors with unable to infer enough type information about `_`; type annotations or generic parameter binding required

It is failing to infer information for TransformationMatrix
        let computed_bradford: Xyz<C, f32> = <Xyz<_,_> as AdaptFrom<_,_,_,_,_>>::adapt_from(input_a);

// Erros with

src/chromatic_adaptation.rs:264:71: 264:72 error: unable to infer enough type information about `_`; type annotations or generic parameter binding required [E0282]
src/chromatic_adaptation.rs:264         let computed_bradford: Xyz<C, f32> = <Xyz<_,_> as AdaptFrom<_,_,_,_,_>>::adapt_from(input_a);

Any way to make  let computed_bradford: Xyz<C,f32> = Xyz::adapt_from(input_a);  work directly?

Changed to adapt to
///Converts color from one reference white point to another via chromatic adaptation (Swp -> Dwp)
pub trait AdaptFrom<S, Swp, Dwp, T>: Sized
    where T: Float,
          Swp: WhitePoint<T>,
          Dwp: WhitePoint<T>,
          S: IntoColor<Swp, T>,
{
    ///Convert the source color to the destination color using the bradford method
    fn adapt_from(color: S) -> Self;
    ///Convert the source color to the destination color using the specified method
    fn adapt_from_using<Mat: TransformMatrix<Swp, Dwp, T>>(color: S, method: Mat) -> Self;
}

Now both these work
        let computed_bradford: Xyz<C,f32> = Xyz::adapt_from(input_a);
        let computed_vonkries: Xyz<C, f32> = Xyz::adapt_from_using(input_a, Method::VonKries);

I think that's the way to go.

Are you ok with leaving the RGB conversion as is for now? That's the only thing remaining for now.

That will leave it in an incorrect state, which I'm not super comfortable with... That link I posted before has an algorithm/formula for converting from RGB to XYZ, which looks like it's more or less like this:

convert RGB chromaticity coordinates from Yxy::new(1.0, x, y) to XYZ,
put the calculated values in a matrix,
multiply the matrix and the white point XYZ to get some constants,
put the constants and the previously calculated XYZ coordinates values in a new matrix,
multiply the matrix and the RGB values.

We have all the facilities for that. The tricky part is, however, to invert it in an efficient way and I'm not sure how hard that is. Maybe there are some shortcuts. Anyway, do you think you could get something, with hard coded sRGB primaries, that gives the correct results? I would rather have that, and then optimize, than something that doesn't work as expected. I mean, it has to be done eventually, and I think it will simplify the RGB reform testing if it's already correct. What do you say?

I will, of course, help as much as I can, and there's no hurry. I'm currently looking for a good matrix inversion algorithm.

Can we merge this into a temporary branch for now, add in the rgb changes and the then merge it with the master then? I guess in my head the chromatic adaptation and rgb stuff are separate :)
I just want to make sure the changes so far are ok, before working on the rgb stuff. The formula's needs to invert matrices , so might need a external library for that and also I am currently not sure how the rgb spaces interact with the conversion stuff.
Let me know if this works or else I will add in the conversion for Srgb now.

I would say that it's most related to the abstraction of the white point, since it's one of the components in the conversion formula. I think we can pull this off by hard coding the sRGB primaries, since that's almost what we have done already.
I think that it sounds like a good idea, in any case, to pull in a linear algebra library. These things are tedious enough as it is, so why make it worse? I don't know which library is the best, but I have used cgmath before, and it has served me well. I have checked and it can invert 3x3 matrices without problem. It's also based on the num crate (as far as I can tell), so we don't have to change our reliance on Float. nalgebra is also popular, but it looks like it's a bit spotty when it comes to the matrix inversion. There should otherwise not be any significant differences, as far as I can tell.
I understand that this PR is becoming quite big now, so let's take it one step at the time. Let's begin by checking and polishing what's already done, to make sure it's all good, and integrate the linear algebra library. And then we can decide what to do with the XYZ <-> RGB and RGB <-> Luma (yes, that's also affected) conversion.
So, let's put that problem aside, for now, and focus on what's already here. I'll take another look at the new stuff, but there shouldn't be an awful lot to fix now.

I think that was all. Nothing really big to change, and mostly typos and documentation.

All changes done

Here is the inverse for 3x3 matrix
fn invert_array(a: [f64; 9]) -> [f64; 9] {
    let d0 = a[4] * a[8] - a[5] * a[7];
    let d1 = a[3] * a[8] - a[5] * a[6];
    let d2 = a[3] * a[7] - a[4] * a[6];
    let det =  a[0] * d0 - a[1] * d1 + a[2] * d2;
    if !det.is_normal() {
        return [0.0 ; 9]
    }
    let d3 = a[1] * a[8] - a[2] * a[7];
    let d4 = a[0] * a[8] - a[2] * a[6];
    let d5 = a[0] * a[7] - a[1] * a[6];
    let d6 = a[1] * a[5] - a[2] * a[4];
    let d7 = a[0] * a[5] - a[2] * a[3];
    let d8 = a[0] * a[4] - a[1] * a[3];

    [d0/det, -d3/det, d6/det, -d1/det, d4/det, -d7/det, d2/det, -d5/det, d8/det]
}

Is ii better to use an external library?

What happens when the inverse does not exist? do we return black color, original color?

I can't figure out if this is simpler or if it's better to add an external library.

We don't have to worry about correctness if we add an external library.

What happens when the inverse does not exist? do we return black color, original color?

Good question. What does it even mean for it to not have an inverse? Can it happen with sane coordinates? Is it even ok to panic or should it return an Option instead of assuming?

Option or result does not get us anything ,we will have to change the entire from and into color to use the option types (it will be odd to only have it for the rgb colors).
Most rgb color spaces should not have an issues. Inverse should be present. This will only be an issue with user defined rgb varaints / custom icc profiles etc.
So we either panic or return black color.
Trying to think of ways to make the black color work. May add an error log message. Maybe we can make is_valid for custom color spaces and return a Option/Result in the constructor of the color spaces.
Panic will be simpler though and even the std library does it for things like divide by 0.

I've looked at both cgmath and nalgrbra. Both perform similary to the [T; 9]. cgmath uses a struct of vec's and nalgebra uses a struct with 9 elements, but both perform roughly the same.
cgamth uses S of type BaseFloat with additional trait bounds apart from float, and nalgrbra has not trait bounds, so nalgebra is a better option.

Option or result does not get us anything ,we will have to change the entire from and into color to use the option types (it will be odd to only have it for the rgb colors).

 Ignore that idea :P

Panic will be simpler though and even the std library does it for things like divide by 0.

I'm starting to think that too. I can't think of a practical reason to have such a set of primaries and expect conversion to work, so we might as well.

cgamth uses S of type BaseFloat with additional trait bounds apart from float, and nalgrbra has not trait bounds, so nalgebra is a better option.

I just realized that they don't blanket implement the traits, so it will interfere with how we do it. The only thing I'm worried about regarding nalgebra is that Mat3 doesn't implement Inv. It's implemented for rotation matrices, but I'm not sure if that's enough. They may ignore scale and translation.

he only thing I'm worried about regarding nalgebra is that Mat3 doesn't implement Inv. It's implemented for rotation matrices, but I'm not sure if that's enough. They may ignore scale and translation.

Not sure I understand. There is an inv() for Mat3

Inv is implemented is here https://github.com/sebcrozet/nalgebra/blob/6fee70bd19a42454ce896f48f1b4b034565262f3/src/structs/spec/mat.rs#L64

Oh, it its! The documentation must be outdated or something. Now there's just that BaseNum requirement...

...looks like it's the same situation as with cgmath. It's custom and not blanket implemented.

This seems to work
use num::Float;
use nalgebra;
use nalgebra::Inv;

pub fn test<T: Float>(a1: T, b1:T, c1:T, d1: T) -> nalgebra::Mat2<T> {
    nalgebra::Mat2::new(a1,b1,c1,d1)
}

pub fn run() {
    let out = test(2.0_f32, 3.0, 4.0, 5.0);
    let inv = out.inv();
    println!("{:?}",inv );
}

cgmath errors with BaseFloat not implemented
pub fn test_cgamth<T: Float>(a1: T, b1:T, c1:T, d1: T) -> cgmath::Matrix2<T> {
    cgmath::Matrix2::new(a1,b1,c1,d1)
}

pub fn run() {
    let out = test(2.0_f32, 3.0, 4.0, 5.0);
    let inv = out.inv();
    println!("{:?}",inv );
}

So i guess we are going with nalgebra

But that doesn't restrict where it's important. This is closer to our situation:
use num::Float;
use nalgebra;
use nalgebra::Inv;

pub fn test<T: Float>(a1: T, b1:T, c1:T, d1: T) -> Option<nalgebra::Mat2<T>> {
    nalgebra::Mat2::new(a1,b1,c1,d1).inv()
}

pub fn run() {
    let out = test(2.0_f32, 3.0, 4.0, 5.0);
    println!("{:?}",out );
}

yeah, checked at the wrong place. blas has too much overhead I think and I don't see any other alternative. I'll add the custom function above with a few tests.

I found vecmath, which looked promising until I saw that they have their own special float crate. I can't find anything else that's not dynamic.

Do you see any issues with aliasing [T; 9] as Mat9?
///A 9 element array representing a 3x3 matrix
pub type Mat9<T> = [T;9];

pub struct ConeResponseMatrices<T: Float> {
    ///3x3 matrix for the cone response domains
    pub ma: Mat9<T>,
    ///3x3 matrix for the inverse of the cone response domains
    pub inv_ma: Mat9<T>,
}

I would rather call it Mat3 or Mat3x3, but otherwise, no. Go for it. It communicates intention better than plain [T; 9].

I have implemented Rgb <-> Xyz conversion

No test for Srgb D65 to D50 conversion. I could not find the transformation matrix for this.
Is there a way to compare WP == D65. I have used a work around to compare the xyz components
What needs to be done with the luma?

The XYZ of the primaries should be the same, no matter the white point, so that implementation of RgbPrimaries should be generic over Wp. There shouldn't be a need to check what the white point is.
Luma is basically XYZ without XZ, so it's affected in the same way. Maybe it would be easiest to go RGB -> XYZ -> Luma. We can still go Luma -> RGB, though.
Oh, and there's no need to prefix paths in use with ::. They are global by default.

It would be good with some tests, as well. Something like this should do:
let a: Rgb<D65> = Rgb::new(1.0, 1.0, 1.0);
let b: Rgb<D50> = a.adapt_into();
assert_relative_eq!((a.red, a.green, a.blue), (b.red, b.green, b.blue));

and maybe a couple more for other colors than white.

I am surprised that test actually passes. I was expecting them to be different.
Here is a check on http://www.brucelindbloom.com/index.html?Eqn_DIlluminant.html
use D65 white point and srggb space convert rgb(1,1,1)  to xyz (0.95, 1, 1.08). Now change white point to D50 with same srgb space and convert xyz to rgb which gives (0.92, 1.0, 1.14), which was what I was expecting.

I don't have Java in my browser :P Sounds like strange results, anyway. It just hit me that RGB can jump between white points, since it describes the transition from primaries to white, but I'm not sure we can make such an optimization and still keep the blanket implementation of AdaptFrom.

I meant this link http://www.brucelindbloom.com/ColorCalculator.html. You'll see the same on easyrgb.com

I'm not sure where the chromatic adaption happens in there... It affects the values, but I can't determine how. I'll dig in the source and see if it's in the JavaScript.

Oh, it looks like it's adapting between reference white and the white point of the RGB model. That's good to know...

So there is still an issue with the chromatic adaptation test for Rgb<D65> to Rgb<D50> . They still show up a 1 for all the values

So here is the test
    #[test]
    fn d65_to_d50() {
        let a: Rgb<D65> = Rgb::new(1.0, 1.0, 1.0);
        let x: Xyz<D65> = a.into_xyz();
        let x1: Xyz<D50> = x.adapt_into();
        let y: Rgb<D50> = x1.into_rgb();
        let b: Rgb<D50> = a.adapt_into();
        println!("");
        println!("RGB D65 {:?}", a);
        println!("XYZ D65 {:?}", x);
        println!("XYZ D50 {:?}", x1);
        println!("RGB D50 {:?}", y);
        println!("RGB D50 {:?}", b);
        assert_relative_eq!(a.red, b.red);
        assert_relative_eq!(a.green, b.green);
        assert_relative_eq!(a.blue, b.blue);
    }

Here is the output
RGB D65 Rgb { red: 1, green: 1, blue: 1, white_point: PhantomData }
XYZ D65 Xyz { x: 0.95047003, y: 1, z: 1.08883, white_point: PhantomData }
XYZ D50 Xyz { x: 0.96422017, y: 1.0000001, z: 0.82521, white_point: PhantomData }
RGB D50 Rgb { red: 1.0000005, green: 1.0000001, blue: 0.9999999, white_point: PhantomData }
RGB D50 Rgb { red: 1.0000005, green: 1.0000001, blue: 0.9999999, white_point: PhantomData }

I think we are missing something in the Rgb-Xyz converison

Thinking about all of this is a bit mind bending, but the way I understand it is that XYZ and xyY are the only spaces where the values are actually different for different white points. The other spaces are based on perception, meaning that RGB(1.0, 1.0, 1.0) is white, no matter what energy levels or optical nerve signals that corresponds to. It still looks white. Adapting from Rgb<D65> to Rgb<D50> is therefore practically pointless, because white is white, and white is always RGB(1.0, 1.0, 1.0).

Lindbloom's color calculator gives the same XYZ values (using sRGB, which is D65, and adapting to D50). I tried to use the easyrgb calculator as well, but it gave me an error for D50 RGB -> XYZ (probably due to too precise validation), but going XYZ -> D50 RGB gives roughly (1, 1, 1). I think all is well.

It should be (1,1,1) in both cases. http://www.brucelindbloom.com/ColorCalculator.html shows the gamma corrected values and we are looking at the linear values. So that expalins the difference in the values seen.
Also from the link below, there are 2 types of white points ICC D65 and ASTM D65 (same for all other D series). www.brucelindbloom.com uses ASTM, ICC profiles use the ICC values. There are slight differences between them. I've used the ASTM values.
More information on this http://ninedegreesbelow.com/photography/srgb-color-space-to-profile.html

Also from the link below, there are 2 types of white points ICC D65 and ASTM D65 (same for all other D series). www.brucelindbloom.com uses ASTM, ICC profiles use the ICC values. There are slight differences between them. I've used the ASTM values.

It never ends, does it? :confounded: We should make sure we use the correct one for sRGB, and maybe provide the others as well. Maybe group them in modules. I don't know. At least make sure sRGB conforms to its standard for this PR, and maybe improve the rest later. We must draw the line somewhere.

Does everything look good. As as far as I can see I have made all the recommended changes.

I don't know. At least make sure sRGB conforms to its standard for this PR, and maybe improve the rest later. We must draw the line somewhere.

There are very small differences between, like in the 4th decimal place 0..3271 and 0.3272. We should be ok.

I suppose it's fine, then. I hope nobody thinks this is suitable for super high precision life-on-the-stake applications :wink:
I'm still checking some of the new stuff, but I'm quite sure there's nothing more. Just that typo.

I can't find the d65_to_d50 test. Did you skip it?

Yup, not sure if it makes sense

It does, as a regression test, to check that nothing odd is going on. That, and the typo fix, should be all.

Let me merge all commit and take another look

Ok, just tell me when you feel ready. I'll be busy with all sorts of other things, so no hurry.

Fixed up a few typos and changed the docs for chromatic adaptation, Xyz( referenced only D65 ) and Yxy.
Should be good to go!

Nice! I'll, however, have to postpone my final review until tomorrow.

I found a couple of typos that I couldn't ignore, so I took the liberty to point out a couple of strange indentations as well. It's otherwise great and I would have merged it straight away.

Done!

Great! Thank you! It's exciting how this library has grown from the original idea of just being a simple color calculations library. It's still a color calculations library, and it's still about as simple to use, but it has so much more power. And more is coming.
@homu r+

:pushpin: Commit 2ed7fd2 has been approved by Ogeon

:zap: Test exempted - status


Typescript Definitions

I would like to use this library with my React / Typescript project. Could you add Typescript definitions?

Hello @ideoclickVanessa,
At this point we do not have enough workforce to add the TypeScript definitions, but we can help you if you are interested in contributing to the project.
You can open a PR so that we can discuss the development of this feature. (https://github.com/nossas/slate-editor#contributing)

That's ok. We ended up picking Quill over Slate partly because it is written in Typescript and thus already has the definitions.


install-photobooth: Added choice to create a service instead of the cron job for virtual-device cam

Prerequisites checklist

[x] I have read the contributing guidelines.

What is the purpose of this pull request?

[ ] Documentation update
[x] Bug fix
[ ] New feature
[ ] Other, please explain:

What changes did you make? (Give an overview)
I've had some issues with the new modprobe/cronjob setup. Modprobe didn't create the virtual webcam on RPi4.
As the cronjob is configured to run as user www-data, it could not execute modprobe to create the virtual webcam.
I'm not sure why the service setup was purged, but in order to get things working, i've created a new service, which starts the CameraControl daemon using the new cameracontrol.py.
Another problem I've had with the virtual webcam was, an error which I've got when ffmpeg was restarted:
[video4linux2,v4l2 @ 0x55d2beacf0] ioctl(VIDIOC_G_FMT): Invalid argument
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:0 --

After re-crating the virtual webcam using rmmod and modprobe the camera was working again. So I've implemented the --forceRecreateCam command line argument, which removes the webcam kernel module and reloads them. This option requires cameracontrol.py to be executed as root.
Is there anything you'd like reviewers to focus on?
This update only affects installations where the user selects virtual webcam persistence using a systemd service. It is no revert to the legacy service which used the legacy cameracontrol.py. The created service starts the CameraControl daemon from the new cameracontrol.py. The kernel modules are reloaded at each service start, which also shouldn't lead to any issues.
I've also set the check for the v4l2-ctl --list-devices command to false, as the execution of it exits with an error because it cannot open /dev/video0:
$ v4l2-ctl --list-devices
bcm2835-codec-decode (platform:bcm2835-codec):
	/dev/video10
	/dev/video11
	/dev/video12
	/dev/video18
	/dev/video31
	/dev/media1

rpivid (platform:rpivid):
	/dev/video19
	/dev/media0

Gphoto2 Webcam (platform:v4l2loopback-000):
	/dev/video9

Cannot open device /dev/video0, exiting.

As this is not the virtual webcam, this is no real issue.

Hey @Benni0 and thanks for your contribution!
Could you fix the formatting as mentioned via pre-commit hook linter?

Hi @andi34,
I have to thank you for this awesome project.
Sure, I can apply the formatting as recommended by the linter, but I think it is better as it is now.
Line 52 uses single quotes like they are used in line 68, which I haven't modified.
Line 698 is not the longest line in the file and uses the same format as the line before.
If I change the quotes, it would make sense to change all single quotes to double quotes.
What is your opinion on the linter recommendations?

Some single quotes are needed because we're also using double quotes in same context.
To be true, i am always running the pre-commit hook automatically on commit and let it do it's job how we've defined it.
If you don't change the configuration but changing everything to single quotes it'll complain about that

Sorry, seems I've missed this newline.


possible to reconsider non-commercial license for the pretrained weights?

This comment indicates that considerable compute resources were needed to train NeuralGCM: https://github.com/google-research/neuralgcm/issues/101#issue-2426040472
This puts the value of neuralgcm out of reach for many groups who would like to try, fine-tune, or use the pretrained weights. It seems like their only route is to duplicate effort and train a new model from scratch rather than fine-tune or use the original pretrained weights.
It would be great if companies (big and small), nonprofits and partnerships between the two could use the pretrained model weights without worrying about the "noncommercial" part of the license.

We are still figuring out our full licensing strategy around NeuralGCM, but certainly we are very eager to support non-profit and non-profit adjacent use-cases. Please do reach out directly to me at<EMAIL_ADDRESS>with details and I can put you in touch with our partnerships team.

We have relicensed NeuralGCM weights on CC BY-SA, which removes the non-commercial restriction.


add script to create ios build

where are we setting the app version?

@robertavram App version is set by a separate command:
README.md:
To bump and sync config.xml, package.json and bower.json semantic versions:
$ gulp bump --patch
$ gulp bump --minor
$ gulp bump --major
$ gulp bump --setversion=2.1.0


HTTP Error

Fixes #2837

@wetneb Please review 😄

@urvashigupta7 This is great!

However, this doesn't tell the user the right message.  Typically that would come from the server itself.  I think it might be better in the message to the user to just return the Status Code, Message | JSON | HTML as the Exception message?
{"code":401,"request":"13a9aff6-7eea-437c-a8e9-590e98cf8e43","type":null,"message":"Unauthorized","responseHeaders":null}

You can see an example of this sort of handling we do in other places like here:
https://github.com/OpenRefine/OpenRefine/blob/49fd21759c81cfccd08fce8b6ddf20a427bbf2a0/main/src/com/google/refine/operations/column/ColumnAdditionByFetchingURLsOperation.java#L434

I have made the changes. And yes @wetneb I would like to write test also.

Great! To write a test I recommend you use okhttp3.mockwebserver.MockWebServer as done in this test: https://github.com/OpenRefine/OpenRefine/blob/749704518c330c6f96ec3e784c45b8c055d5c6cd/main/tests/server/src/com/google/refine/operations/column/ColumnAdditionByFetchingURLsOperationTests.java#L198-L202
You should add your test as a new method of ImportingUtilitiesTest.
Let me know if you get stuck.

@tfmorris okay.. I will make the changes

@wetneb Could you please summaries the steps to create the test. If I am not wrong MockWebServer will be used to return a 401 error code. But, I am not getting how urlimporting request needs to be send. Do I need to call this method? - https://github.com/OpenRefine/OpenRefine/blob/3a6fc90f02559944cacbbe926b19d363db892086/main/src/com/google/refine/importing/ImportingUtilities.java#L176

Yes this is the method that you will need to test. I would recommend to use Mockito to generate mocks for the arguments it expects, and set up the methods of these mocks to return the appropriate values to trigger the code path you have been working on. In particular, you can mock the HttpRequest object, such that you do not have to actually send any real HTTP request.
If you are not familiar with Mockito, I recommend this tutorial: https://www.baeldung.com/mockito-series

Hi @wetneb. I am having trouble with sending Httpservletrequest object to the method. Actually, the request's content-type is multipart/form-data.
Could you please tell me how to send the data.

@urvashigupta7 please update this PR with what you have written so far, otherwise it is hard to help you without seeing the code you are working on.

@wetneb I have pushed the code. Actually I need to send download URL with the request object. Sending URL with getParameter will not work as it need is multipart/form-data

Nice, thank you! It is definitely not an easy method to test - this area of our code would really benefit a good refactoring.
I can see two approaches:

go deeper down the mocking route: use your debugger to find out how the HttpRequest is read by the file parsing method (upload.parseRequest(request)) and update your mocking accordingly
refactor the retrieveContentFromPostRequest by introducing a new method for the specific case of URL download (which could be worth doing for other cases: clipboard, file upload…). Your new method would directly take the supplied URL as parameter, and would be much easier to test.

Also, have you had a look at @tfmorris's suggestions above?
Keep up the great work!

Congratulations and thank you for this PR, welcome to the team!

Thank you @wetneb 😄

Good work! That's a challenging test for your first PR.

Thank you so much! @tfmorris


Solution added of Stone Game III in c++

Pull Request Template
What have you Changed(must)
Added C++ code for 1406 Stone Game III
Write here what you have changed in the codebase
Issue no.(must) - #1072
Pr will be close and marked as spam. If issue number not found or issue was assigned to someone else.
Marking as spam can block your account from HacktoberFest.
Self Check(Tick After Making pull Request)

[x] This issue was assigned to me if not it will be marked as spam.
[x] My file is in the proper folder
[x] I am following clean code and Documentation.
[x] I have added the title and what program will do first in the file
[x] I have named the file as (Problem #). (Problem Title(in Camel Case)).(extension)
ex. 2. AddToNumbers.py

README - How to Contribute

changed the title to be more appropriate.


I need v1.6.4.Can you post this version?

Hi
1.6.5 dont work
I need v1.6.4.Can you post this version?

You want https://github.com/Royalgamer06/SteamWebIntegration/blob/033b792252381f5825fb74ec4de6c27f0a0b1d2d/Steam Web Integration.user.js ?
I'm sure V1.6.4 won't work.

shows only one game
https://ibb.co/iLLyDG

and 1.6.3 will work? Can it try?


Need -RequirePrivacy $true when creating global SMB mappings or else you get a permissions error

Following https://github.com/moby/moby/issues/37863 I finally discovered that you need -RequirePrivacy $true when creating global SMB mappings.  Without this, the share gets created fine but if you map to this within a container you get a nasty access denied error:
C:\Program Files\Docker\docker.exe: Error response from daemon: hcsshim::CreateComputeSystem 9c6b8850915c7b9982f43ddbdf70418d4fc8b9f0e90338e588dfee98a5b8775a: Access is denied.
So,
$secpassword = ConvertTo-SecureString 'MyPassword' -AsPlainText -Force
$creds = New-Object System.Management.Automation.PSCredential("server\myShare", $secpassword)
New-SmbGlobalMapping -RemotePath \server\myShare -Credential $creds -LocalPath Y: -Persistent $true -RequirePrivacy $true
Running this docker container now works, does not throw errror, and I can see the files under C:\test.
docker run --rm -it -v Y::C:\test --name demo mcr.microsoft.com/dotnet/core/runtime:2.2 cmd.exe

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2b8233b4-c7d6-111a-9bb1-5dae2da7a96c
Version Independent ID: b4674f08-a39e-2b28-291f-0331744d66df
Content: Persistent Storage in Containers
Content Source: virtualization/windowscontainers/manage-containers/persistent-storage.md
GitHub Login: @cwilhit

Ran into this problem where the container would intermittently throw access denied errors during the lifetime of the container. Running icacls.exe G:\FolderName /grant 'Everyone:(OI)(CI)(F)' on the host made the folder available again in the container but only for about 15 minutes. I've just kept the container running for an hour now with the -RequirePrivacy $trueswitch on the New-SMBGlobalMapping command and it hasn't thrown an access denied error yet.
Microsoft documentation here https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/persistent-storage needs to be updated to use the switch as well.

We hugely appreciate your contribution, feedback, and support to our technical documentation.
We’ve phased out GitHub Issues on this repository as the feedback mechanism for content and have replaced it with a new feedback system. For more information see, Content User Feedback. As part of this rollout, we are migrating your issue where our team will use your comments to improve the documentation experience. Unfortunately, this does mean you won't be able to access this issue after the migration is complete.
All issues will be migrated by Thursday 29th August 2024.
Once again, thank you for your valuable feedback, we hope you’ll continue to contribute using our new feedback system.


energy (dpu_probe.h)

Hey there, There's a line for enabling the measurement of energy here. I wonder if this file is embedded in this project? Because I think this file wasn't part of the UPMEM toolkit.

May I ask how you solved it? Because I can't find this file either.

I asked UPMEM helpdesk about dpu_probe.h and they said it is only available in certain UPMEM HW version


Change how we install glide to work with homebew's build

make install put the glide binary in /user/local/bin and tried to change permissions, which was causing permission errors. Relying on the Go bin/ seems to work around that adequately.

Merge like you're awkwardly flailing in a public forum! 😊


[notebook] NameError: name 'get_cmap' is not defined

When I run the following in a Jupyter Notebook
from pygamma_agreement import Continuum
from pyannote.core import Segment

continuum = Continuum()
continuum.add("Annotator1", Segment(2.5, 4.3), "Maureen")
continuum.add("Annotator2", Segment(2.3, 4.5), "Maureen")

continuum

...I get the following error
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/pygamma_agreement/notebook.py in <module>
    320 try:
--> 321     notebook = Notebook()
    322 except NameError:

~/.local/lib/python3.8/site-packages/pygamma_agreement/notebook.py in __init__(self)
     63         self._style_generator: Iterator[LabelStyle] = iter([])
---> 64         self.reset()
     65

~/.local/lib/python3.8/site-packages/pygamma_agreement/notebook.py in reset(self)
     69
---> 70         cm = get_cmap('Set1')
     71         colors = [cm(1. * i / 8) for i in range(9)]

NameError: name 'get_cmap' is not defined

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
~/.local/lib/python3.8/site-packages/IPython/core/formatters.py in __call__(self, obj)
    343             method = get_real_method(obj, self.print_method)
    344             if method is not None:
--> 345                 return method()
    346             return None
    347         else:

~/.local/lib/python3.8/site-packages/pygamma_agreement/continuum.py in _repr_png_(self)
    904         """
    905
--> 906         from .notebook import repr_continuum
    907         return repr_continuum(self)
    908

~/.local/lib/python3.8/site-packages/pygamma_agreement/notebook.py in <module>
    321     notebook = Notebook()
    322 except NameError:
--> 323     raise ImportError()
    324
    325

ImportError:

Hey, thanks for the issue :)
Did you install matplotlib? (Either directly, or by running pip install pygamma-agreement[notebook])

Yes, as we speak I just realized I had not installed it. But thanks for the quick feedback!

No problem, this might be helpful to others (and to me as it's making more aware of what might be missing from the documentation).

I would like to propose better handling for this error


Search Box to search for courses, streams and subtitles

Motivation and Context
This Change solves the problem of finding lectures in a large list of lectures. It also enhances the search for subtitles or streams in courses.
Closes  #1337.
Description
The change adds a search bar at the top of every page where you can search for courses or contents of a course, depending if you're on a course page or on the home page.
Steps for Testing
You can't test this change on the Testserver system.
You have to set up your own local instance of TUM-Live for testing and also have to setup your own meilisearch service.
To start the meilisearch service, you can run the following command:
docker run -it --rm \
    -p 7700:7700 \
    -e MEILI_MASTER_KEY='MASTER_KEY'\
    -v $(pwd)/meili_data:/meili_data \
    getmeili/meilisearch:v1.9 \
    meilisearch --env="development"

If the data isn't automatically exported to meilisearch, you can run this job from the Maintenance Tab in the Admin panel.
You can enter your search query in the search bar on every page or you can press 'Enter' while the search bar is focused to see the full search page with filters etc.

From my point of view, there is nothing left to review.
There are a few threads (like https://github.com/TUM-Dev/gocast/pull/1380#discussion_r1804770695) which are not quite resolved, but from my point nothing blocks merging.

From my point of view, there is nothing left to review.

@CommanderStorm Thanks for your review and suggestions, it was much appreciated! :)


Added a margin between book title and setting button

#1775
Description
Addition of right margin to book title div to add some gap between the book title and the settings button.

[x] I have performed a self-review of my own code.

@viveknegi84 can you share a screenshot how it is looking on your end now ?

@P4NK4J Above is the preview on my end.


ui-select inside zf-tabs

It works fine outside zf-tabs
but we have empty list inside them.
prefix $parent. doesn't help.

this issue with be addressed with https://github.com/base-apps/angular-base-apps/issues/24


Refactor makefile script

Fix shellcheck issues
Fix some shell style inconsistencies.
Split out some functions.

Signed-off-by: Ben Kochie<EMAIL_ADDRESS>

@roidelapluie Any other comments?

LGTM :)


How to get the processed_trans.npy file

IEMOCAP_02_to_four_category.ipynb looks for a file called procesed_trans.npy in part 07-A, cell 42:
data = np.load('../data/processed/IEMOCAP/processed_trans.npy')

Apparently, it contains 128-dimensional features for every data point. But I could not find any part of the code that generates this .npy file. The closest I found was generating a precessed_tran.csv file in IEMOCAP_00_extract_label_transcription.ipynb. This file generates the session ids and the raw text transcriptions as strings. How can I convert these raw values to the 128-dimensional features in the .npy file?

Please refer to the following notebook that contains the process to generate "processed_trans.npy."
https://github.com/david-yoon/multimodal-speech-emotion/blob/master/preprocessing/IEMOCAP_NLP_01_Transcription_to_Index.ipynb

Thanks


[tutorial] For in loop

Description of the tutorial
For in loop tutorial
Do you want to be assigned to work on this?
(mark [x])

[x] YES
[ ] NO

Optional: Any additional info you want to add
None

@Parth-ops You're assigned now.


Fixes to glance provider to handle exceptional conditions.

Don't raise FnF if there is an image locally.
Don't download non-active images.
Perform atomic downloads.

Is this logic covered by tests?
I would add on the Glance plugin documentation the fact that old images don't get removed automatically but it's the User's responsibility.
We could also add the deletion ourselves. An optional configuration flag in the plugin's config where, if set, it deletes old images if older than X (X is the config value in seconds/hours/days/centuries).

@noxdafox I wouldn't add local deletion capabilities at all. These are image providers, not image managers. Each instance of the provider should take care of placing a requested image where the client expects it and nothing more. Cleanup is the client's responsibility, for this and any other providers as the provider cannot know how or by whom the image is being used or when.
I'll update the providers' documentation in general to explicitly note that providers DO NOT clean after themselves in any case.

Conflicts on the branch.

@dangonzo Could you be more specific?