TF website tutorial "Save and load" fails on Google Colab for entire models

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):  NO
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Google Colab on cloud (run in Safari browser on Mac OSX 10.14.6)
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): NA
TensorFlow version (use command below): TF 2.3.0, Git version v2.3.0-0-gb36436b087
Python version: Appears to be 3.6
Bazel version (if compiling from source): NA
GCC/Compiler version (if compiling from source): NA
CUDA/cuDNN version: ??
GPU model and memory: ??

Describe the current behavior
The TensorFlow tutorial page https://www.tensorflow.org/tutorials/keras/save_and_load fails even when run in Google Colab, your cloud environment. Where it fails exactly is towards the bottom, in the section of the tutorial describing how to save and load entire models with model.save() and load_model(). I find that the restored models (in either SavedModel or HDF5 format) fail to reproduce the original model accuracies, given exactly the provided code. Instead I get around 8-9% accuracies.
I get the same problem running the downloaded Jupyter notebook on my own laptop (Mac OSX 10.14.6, TF 2.4.0-dev20200727, Git v1.12.1-37595-g9f2e1a7246, TF from pip3, Python 3.8.5).
Describe the expected behavior
The expected behavior is for the saved and reloaded models to have exactly the same accuracies on the same test data.
Standalone code to reproduce the issue
I've attached a small notebook that replicates the problem on my laptop.
minimal.ipynb.txt
This was modeled after a similar tutorial (https://machinelearningmastery.com/save-load-keras-deep-learning-models/ under "Save Model Weights and Architecture Together"). That tutorial's code runs fine with the given example data on my laptop (the saved and reloaded model has identical accuracy to the original), yet the very similar code in the attached notebook using the MNIST data and slightly different model does not. I'm not sure what this means. Honestly I'm pretty new to Keras. I'm not sure if this is a bug in the Keras code, or just an error in the tutorial. But again, it doesn't work with the provided tutorial in the Google Colab environment, suggesting it's not an error on my part.
FWIW, I do notice that the reloaded SavedModel model and the reloaded HDF5 model, when saved from the same original model (as in my example notebook), both give the identical bad accuracy.
Other info / logs
Not sure if relevant, but these warning appear the first time model.save() is used (from Google Colab):
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
INFO:tensorflow:Assets written to: saved_model/my_model/assets
On my laptop, I get the same warnings (with a different path to tracking.py: /Users/lilley/Library/Python/3.8/lib/python/site-packages/tensorflow/python/training/tracking/tracking.py).
The example run given on the tutorial page suggests that this functionality was originally working in TF 2.2.0.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
INFO:tensorflow:Assets written to: resnet_model_combine.model/assets

How can I get rid of this?? How to update?

I have the same issue please help

I have tried in colab in TF nightly version and was able to reproduce the issue.Please, find the gist here.Thanks!

I think the issue I reported some days ago is exactly the same: https://github.com/keras-team/keras/issues/14181

@ErikEngerd Thank you for directing me to your issue. I'm a bit new with Github and with TF/Keras. I was afraid I submitted my issue in the wrong forum, which appears to be the case, so perhaps the people with the power to address this issue will never see this thread (or perhaps ignore it).
In my issue report, I specifically emphasized that I was using their unedited example code in Google Colab, which would seem to rule out issues in my coding and my environment. I see in your thread that you also pointed this out -- thank you! Hopefully the issue will be addressed soon.

@jasonlilley Thanks for reporting this issue. I agree this is an issue with TF2.3 and tf-nightly. It was working well with TF2.2 so this is a regression issue. Thanks!

This is a bug with using the sparse categorical accuracy. For now, please compile the model with metrics='sparse_categorical_accuracy' instead of just 'accuracy'.

@jasonlilley I can confirm that this is working as expected when I followed @k-w-w suggestion. Here is the gist for reference. Thanks!

@k-w-w Thank you for replying. I was also able to confirm that the model compilation suggestion worked in my notebooks on my laptop. I suggest that someone change the tutorial page with the fix.

This issue has been fixed in commit 9d8947

I get the same Error when attempting to save with tf '2.3.1'.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.
2020-10-11 19:19:17.739772: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:tensorflow:From /home/bsvogler/.local/lib/python3.8/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
This property should not be used in TensorFlow 2.0, as updates are applied automatically.


deps/yara/patch: patch YARA surces to fix bug in v4.0.1

Bug being fixed: https://github.com/VirusTotal/yara/pull/1289/files
This needs to be removed once the fix gets to RetDec.

run tc tests


lookup martian.proxy: no such host

If I start proxy.exe, I cannot access the certificate:
PS C:\> curl -v -x <IP_ADDRESS>:8080 martian.proxy/authority.cer
*   Trying <IP_ADDRESS>:8080...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080 (#0)
> GET http://martian.proxy/authority.cer HTTP/1.1
> Host: martian.proxy
> User-Agent: curl/7.78.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 502 Bad Gateway
< Warning: 199 "martian" "dial tcp: lookup martian.proxy: no such host" "Sat, 25 Dec 2021 13:37:40 GMT"
< Content-Length: 0
<
* Connection #0 to host <IP_ADDRESS> left intact

https://github.com/google/martian#intercepting-https-requests-and-responses

OK running like this:
proxy -generate-ca-cert

I can get the cert with cURL:
curl -x <IP_ADDRESS>:8080 martian.proxy/authority.cer

but for some reason, the same request does not work with Android Studio:

This site can't be reached
martian.proxy's server IP address could not be found.
DNS_PROBE_FINISHED_NXDOMAIN

Android Chrome seems to be weird in that the hostname needs to be a valid TLD. So these all fail:
martian.proxy
martian.test
martian.example
martian.localhost

but this works:
martian.com

Working command:
proxy -generate-ca-cert -api martian.com

https://datatracker.ietf.org/doc/html/rfc2606


Grammar fix

Coverage remained the same at 96.803% when pulling 47587ff5c87e87c9315c7a17a6eb58458e186dd7 on coryhouse:patch-2 into 6c46f57f6f4a3c3dc91f98cc9a7da3f379ae6a19 on yannickcr:master.

Would you please merge #566, #568, #569, #570, all into the same PR? There's no need to even have these be separate commits.

These were all performed via a mobile device on github.com so there was no easy way to merge.

So to answer your question, no, I won't go through the re-work of merging these into one PR.

@coryhouse for future reference, even in that scenario you can continue editing files, in your fork, on the same branch, and it will add commits to the existing PR.
I'll manually squash these.

Nice! News to me. Thanks for the tip.


export jason function not working - nothing happens

Hello,
i am migrating to another computer and wanted to use the export function of tankerkoenig.0 but when i click on the button. nothings happens !
tankerkoenig version 2.0.5
nodejs v12.20.0
npm 6.14.8

Hello,
can not reproduce this issue. Does it work in other adapters settings?

Hello,
can not reproduce this issue. Does it work in other adapters settings?


anyways to speed up the tests

i don't have too many test. but the runner takes up to 2 minutes.
just wondering if anyone has any tips to speed up running the tests

@thejacer87 2 mins seam fine, some complex CI tests might take hours, depending on the underlying hardware.
You can speed up your tests using mix of cache and artifacts routines in gitlab runners.
Please read more on caching in gitlab-runner here


Ability to open poe trade with selected options

I'm really enjoying the tool, the only option missing is opening the browser to POE trade or the official trade site after getting price predictions. Would you be interested in implementing that?

Nevermind, I was just not aware clicking the currency icon was doing exactly that. Found an older issue explaining that.

Glad to found it by yourself. There is also a wiki entry about this:
https://github.com/Kyusung4698/PoE-Overlay/wiki/Evaluate#figures


Adjust minimal config and fix sync committee computation

A couple of issues that revealed one by one 😅

Issue 1: sync committee tests are really slow.
Proposed solution:
Set minimal config's EPOCHS_PER_SYNC_COMMITTEE_PERIOD from 256 to 8.

Issue 2: minimal config's SYNC_COMMITTEE_SIZE is equal to num_validators (=64) in the test suite
That is, with minimal config, every time we call get_sync_committee_indices, we always get all active validator indices.
Proposed solution:
To enrich the test cases (especially, test_process_sync_committee.py::test_invalid_signature_previous_committee needs it), set minimal config's SYNC_COMMITTEE_SIZE from 64 to 32.

Issue 3: get_sync_committee_indices may return duplicate indices
test_process_sync_committee.py::test_invalid_signature_previous_committee should have failed due to Issue 2. However, it didn't fail because get_sync_committee_indices returned duplicate indices and made the signature verification invalid anyway.
#### Proposed solution:
Ensure that get_sync_committee_indices won't return duplicate indices.
Update: it's an intentional design for minimizing the complexity!
So this PR only rewrites the test cases.

Issue 4: active_validator_count < SYNC_COMMITTEE_SIZE in tests
In test suite (not in mainnet), we may see active_validator_count < SYNC_COMMITTEE_SIZE. The Vectors of length SYNC_COMMITTEE_SIZE will have to be padded.
#### Proposed solution:
In https://github.com/ethereum/eth2.0-specs/pull/2130#issuecomment-727687495, @vbuterin suggested "I guess another option would be to have them be vectors and pad the pubkeys and bits with zeroes to represent that there's nothing there". In this PR:
- Pad G1_POINT_AT_INFINITYs to SyncCommittee.pubkeys and SyncCommittee.pubkey_aggregates.
- Pad Falses to BeaconBlockBody.sync_committee_bits
Update: ditto. it's an intentional design for minimizing the complexity!
So this PR only rewrites the test cases.
/cc @JustinDrake @vbuterin, does the above solution look like the correct conclusion of #2130? Are you still considering using Bitlist and List?

get_sync_committee_indices may return duplicate indices

This is by design! :) Maybe we should add a note explaining that this is by design so that this point is not repeatedly brought up.

Are you still considering using Bitlist and List?

Nope :)

get_sync_committee_indices may return duplicate indices

This is by design! :) Maybe we should add a note explaining that this is by design so that this point is not repeatedly brought up.

Are you still considering using Bitlist and List?

Nope :)

@JustinDrake

This is by design!

Oh interesting! Do you mean, assume that SYNC_COMMITTEE_SIZE = 4, get_sync_committee_indices returning [0, 1, 2, 2] is allowed? That is, if index 0 and 2 participate but index 1 doesn't, should the sync_committee_bits be (1) [True, False, True, True] or (2) unset the duplicate one: [True, False, True, False]?
For (1), it seems that we would double reward the duplicate indices in
https://github.com/ethereum/eth2.0-specs/blob/dev/specs/lightclient/beacon-chain.md#sync-committee-processing
For (2), it seems fine as we don't have inactivity penalties for the sync committee. Although it's still a weird presentation to me. 😅 I'd say a note is certainly needed. 👍
Thanks for the response. I think I can change test_process_sync_committee.py::test_invalid_signature_previous_committe test vector gen to be independent of issue 3.

@JustinDrake

This is by design!

Oh interesting! Do you mean, assume that SYNC_COMMITTEE_SIZE = 4, get_sync_committee_indices returning [0, 1, 2, 2] is allowed? That is, if index 0 and 2 participate but index 1 doesn't, should the sync_committee_bits be (1) [True, False, True, True] or (2) unset the duplicate one: [True, False, True, False]?
For (1), it seems that we would double reward the duplicate indices in
https://github.com/ethereum/eth2.0-specs/blob/dev/specs/lightclient/beacon-chain.md#sync-committee-processing
For (2), it seems fine as we don't have inactivity penalties for the sync committee. Although it's still a weird presentation to me. 😅 I'd say a note is certainly needed. 👍
Thanks for the response. I think I can change test_process_sync_committee.py::test_invalid_signature_previous_committe test vector gen to be independent of issue 3.

Set minimal config's EPOCHS_PER_SYNC_COMMITTEE_PERIOD from 256 to 8.

this seems fine to me! i agree the tests do take a while.

set minimal config's SYNC_COMMITTEE_SIZE from 64 to 32.

It seems the test was correct as written, although it is not obvious why and in fact breaks when we shorten the EPOCHS_PER_SYNC_COMMITTEE_PERIOD. the main problem (as you point out) is that active_validator_count == SYNC_COMMITTEE_SIZE so that the eligible set of validators across sync committee periods is the same. in the current case, we change the balances enough across multiples of EPOCHS_PER_SYNC_COMMITTEE_PERIOD ( at 256) that we get different subsets of the validator pool. And balances do not change enough at EPOCHS_PER_SYNC_COMMITTEE_PERIOD == 8 to make a difference. So the test fails when we shorten the period.
I asked @JustinDrake for his rationale on why we allow duplicates and in short it minimizes complexity both for the protocol and also client implementations. So that seems worth keeping the possibility of duplicates.
I think we can just make these two config changes and the tests or state transition logic do not need to change.

Set minimal config's EPOCHS_PER_SYNC_COMMITTEE_PERIOD from 256 to 8.

this seems fine to me! i agree the tests do take a while.

set minimal config's SYNC_COMMITTEE_SIZE from 64 to 32.

It seems the test was correct as written, although it is not obvious why and in fact breaks when we shorten the EPOCHS_PER_SYNC_COMMITTEE_PERIOD. the main problem (as you point out) is that active_validator_count == SYNC_COMMITTEE_SIZE so that the eligible set of validators across sync committee periods is the same. in the current case, we change the balances enough across multiples of EPOCHS_PER_SYNC_COMMITTEE_PERIOD ( at 256) that we get different subsets of the validator pool. And balances do not change enough at EPOCHS_PER_SYNC_COMMITTEE_PERIOD == 8 to make a difference. So the test fails when we shorten the period.
I asked @JustinDrake for his rationale on why we allow duplicates and in short it minimizes complexity both for the protocol and also client implementations. So that seems worth keeping the possibility of duplicates.
I think we can just make these two config changes and the tests or state transition logic do not need to change.

ah, you responded above while i was composing my previous answer; to chime in here:

Do you mean, assume that SYNC_COMMITTEE_SIZE = 4, get_sync_committee_indices returning [0, 1, 2, 2] is allowed? That is, if index 0 and 2 participate but index 1 doesn't, should the sync_committee_bits be (1) [True, False, True, True] or (2) unset the duplicate one: [True, False, True, False]?

right [0, 1, 2, 2] is admissible and the sync_committee_bits refer to the committee itself, not the broader validator set; so (1) corresponds to validator index 0 and validator index 2 participating but validator index 1 not participating

ah, you responded above while i was composing my previous answer; to chime in here:

Do you mean, assume that SYNC_COMMITTEE_SIZE = 4, get_sync_committee_indices returning [0, 1, 2, 2] is allowed? That is, if index 0 and 2 participate but index 1 doesn't, should the sync_committee_bits be (1) [True, False, True, True] or (2) unset the duplicate one: [True, False, True, False]?

right [0, 1, 2, 2] is admissible and the sync_committee_bits refer to the committee itself, not the broader validator set; so (1) corresponds to validator index 0 and validator index 2 participating but validator index 1 not participating

Thanks @ralexstokes! 👍

It seems the test was correct as written, although it is not obvious why and in fact breaks when we shorten the EPOCHS_PER_SYNC_COMMITTEE_PERIOD.

It seems that somehow EPOCHS_PER_SYNC_COMMITTEE_PERIOD = 256 resulted in a seed without duplicate indices and EPOCHS_PER_SYNC_COMMITTEE_PERIOD = 8 resulted in a seed with duplicate indices in some test cases. 😆

right [0, 1, 2, 2] is admissible and the sync_committee_bits refer to the committee itself, not the broader validator set; so (1) corresponds to validator index 0 and validator index 2 participating but validator index 1 not participating

It's clever to benefit from pubkey aggregation in this way! I want to double check if it's fine to double reward the duplicate indices. The block proposer would choose to set all duplicate indices' bits to True in sync_committee_bits for increasing their own reward amount, so it's more likely we will see [True, False, True, True] in the above case.
It should be a rare case with a tiny incentive in mainnet, just want to ensure that we are all aware of it. 👀
p.s. double-rewarding would break test_process_sync_committee.py::test_sync_committee_rewards assumption.

it seems that we can just shrink the minimal configuration constants and drop the changes to "pad" the sync committee

Sounds good to me! By default, I'll revert the last commit + make tests pass the mainnet config tomorrow.

Thanks @ralexstokes! 👍

It seems the test was correct as written, although it is not obvious why and in fact breaks when we shorten the EPOCHS_PER_SYNC_COMMITTEE_PERIOD.

It seems that somehow EPOCHS_PER_SYNC_COMMITTEE_PERIOD = 256 resulted in a seed without duplicate indices and EPOCHS_PER_SYNC_COMMITTEE_PERIOD = 8 resulted in a seed with duplicate indices in some test cases. 😆

right [0, 1, 2, 2] is admissible and the sync_committee_bits refer to the committee itself, not the broader validator set; so (1) corresponds to validator index 0 and validator index 2 participating but validator index 1 not participating

It's clever to benefit from pubkey aggregation in this way! I want to double check if it's fine to double reward the duplicate indices. The block proposer would choose to set all duplicate indices' bits to True in sync_committee_bits for increasing their own reward amount, so it's more likely we will see [True, False, True, True] in the above case.
It should be a rare case with a tiny incentive in mainnet, just want to ensure that we are all aware of it. 👀
p.s. double-rewarding would break test_process_sync_committee.py::test_sync_committee_rewards assumption.

it seems that we can just shrink the minimal configuration constants and drop the changes to "pad" the sync committee

Sounds good to me! By default, I'll revert the last commit + make tests pass the mainnet config tomorrow.

It should be a rare case with a tiny incentive in mainnet, just want to ensure that we are all aware of it. 👀

Yep, it's fine. Duplicate pubkeys is also incentive aligned since every key in the sync committee is independently chosen.

It should be a rare case with a tiny incentive in mainnet, just want to ensure that we are all aware of it. 👀

Yep, it's fine. Duplicate pubkeys is also incentive aligned since every key in the sync committee is independently chosen.

p.s. double-rewarding would break test_process_sync_committee.py::test_sync_committee_rewards assumption.

hmm this is true! i'm happy to fix that test now although it should show as a test failure if we hit it in which case i'd suggest we defer fixing it until then -- lmk what you think is best :)

p.s. double-rewarding would break test_process_sync_committee.py::test_sync_committee_rewards assumption.

hmm this is true! i'm happy to fix that test now although it should show as a test failure if we hit it in which case i'd suggest we defer fixing it until then -- lmk what you think is best :)

It probably makes sense to craft a test specifically to have a double participant in the committee. This is a case that could easily be overlooked by client devs and be ripe for consensus failure

It probably makes sense to craft a test specifically to have a double participant in the committee. This is a case that could easily be overlooked by client devs and be ripe for consensus failure

It probably makes sense to craft a test specifically to have a double participant in the committee.

Given that tests are often designed with a small number of validators it may also make sense to craft a test with no double participant as well :)

It probably makes sense to craft a test specifically to have a double participant in the committee.

Given that tests are often designed with a small number of validators it may also make sense to craft a test with no double participant as well :)

@hwwhww i added the patch to keep trying committees (by varying the seed) until we get or do not get duplicates and this gives us the flexibility to not have to rely on config settings. feel free to just close that PR if you don't think its a meaningful improvement

@hwwhww i added the patch to keep trying committees (by varying the seed) until we get or do not get duplicates and this gives us the flexibility to not have to rely on config settings. feel free to just close that PR if you don't think its a meaningful improvement

@ralexstokes
I think #2189 looks good for dealing with the edge case! That is, we can the @with_configs decorators for efficiency. 🙂

@ralexstokes
I think #2189 looks good for dealing with the edge case! That is, we can the @with_configs decorators for efficiency. 🙂

@hwwhww ah yes you are right! we definitely need to keep the config selectors...
i believe #2189 is ready to merge into this one. feel free to do so if it looks good to you :)

@hwwhww ah yes you are right! we definitely need to keep the config selectors...
i believe #2189 is ready to merge into this one. feel free to do so if it looks good to you :)


When do I need to change fallback_on_stride in deformable conv?

fallback_on_stride is used as follows:
if not self.with_dcn or fallback_on_stride:
But fallback_on_stride in config is always False
So,fallback_on_stride means?

fallback_on_stride means fallback to the normal convolution if stride > 1.

Why we will fallback to the normal convolution if stride > 1?

@djangogo This is just an option and turned off by default.


Get NullReferenceException when running kestrel for a while

# dnx . kestrel
Started

Unhandled Exception: System.NullReferenceException:
Object reference not set to an instance of an object.
at Microsoft.AspNet.Server.Kestrel.Networking.UvShutdownReq.UvShutdownCb(IntPtr ptr, Int32 status)

# dnx . kestrel
Started

#63

I have actually gotten a similar issue

/cc @davidfowl do you guys have someone fixing this already. This is quite a bad blocking issue. @shhsu Do you have a potential fix for this?

It sounds like the period of time that the async shutdown is "active" needs to be guarded by a gchandle as well. Same as the uvwritereq - handle created when operation begins and freed in the callback.

The reproduction of this problem is simple.

Build and run HelloMvc app with coreclr on linux (I used Ubuntu)
Hit the index page with some load test (I used WCat)
Kestrel should crash pretty fast (10~20 seconds)

Note: I seem to be getting similar trouble with Kestrel on coreclr on Windows as well. After hitting HelloMvc on Kestrel on Windows the dnx process would become unresponsive. I will be debugging it.


This project really needs unit tests

I really love this project, and am using it heavily for a few projects... but its starting to feel more then a little unstable. Perhaps Unit tests via karma and mocha would go a long way to stabilize the project

@mattapperson Thanks for this issue - and I totally agree. Please see #968 and https://github.com/callemall/material-ui/issues/134#issuecomment-116089846

@mattapperson Sorry for the typo in my last comment - see #396 as well.

No worries :) thanks!


Messages don't log correct execution duration for steps with status: "failed"

Current behavior
Using messages system as a source of data for reports. Messages don't correctly log how much time scenarion is being executed if first step fails. In messages this time is saves as 0.

Desired behavior
Messages correctly log time of execution for steps ans scenarios with status "FAILED"
Test code to reproduce

https://stackblitz.com/edit/angular-orraiy?file=cypress.config.ts
Versions

Cypress version: ^12.3.0"
Preprocessor version: ^15.1.2
Node version: v16.17.1

Checklist

[x] I've read the FAQ.
[x] I've read Instructions for logging issues.
[x] I'm not using<EMAIL_ADDRESS>(package name has changed and it is no longer the most recent version, see #689).

This has been fixed with v17.2.0.


update-clamav-task

RE: https://redhat-internal.slack.com/archives/C04PZ7H0VA8/p1720604130651199

/approve


Adding android gradle plugin as an extra property of project template

React Native version:
0.58.8
Steps To Reproduce
This is an enhancement (or feature request)
Describe what you expected to happen:
having an extra property in ext that specify android gradle plugin version, so libraries developer can read and set the main project android gradle plugin version to their library and so whole project build with the same gradle plugin. this is an important and useful property when users using CI server with code repository for building their product. for example, I use a docker with previous installed all necessary resources for building an android app in my CI server, but some of libraries I used have different gradle plugin version, so my docker has to download the different version of gradle plugins to build a final product.
I mean something like this:
I wanted to submit a pull request but I'm not expert in gradle build system and not familiar with this repository and I couldn't find android gradle template file, So I suggest my changes in here
buildscript {
    ext {
        buildToolsVersion = "28.0.3"
        minSdkVersion = 16
        compileSdkVersion = 28
        targetSdkVersion = 28
        supportLibVersion = "28.0.0"
        gradlePluginVersion = "3.4.1" <----- add this property to react native android template
    }
    dependencies {
        classpath 'com.android.tools.build:gradle:$ext.gradlePluginVersion'
    }
}

By adding the property other library developers can do something like this in their library's gradle file:
def safeExtGet(prop, fallback) {
    rootProject.ext.has(prop) ? rootProject.ext.get(prop) : fallback
}

buildscript {
  repositories {
    ...
  }

  dependencies {
    classpath 'com.android.tools.build:gradle:${safeExtGet('gradlePluginVersion', '3.4.0')}'
  }
}

I don't familiar with gradle syntax, so it might be wrong in the syntax of what I wrote, I just want to suggest my idea.
Also, it would be great if make a standard mechanism that libraries could read version of gradle wrapper version of main project from gradle-wrapper.properties , because I have seen some libraries that have gradle-wrapper.properties file in their directory. for example, you can take a look at react-native-keychain library. I have no idea right now how can we add this feature, but I think it's worth discussing about it

It looks like not a RN issue, please file an issue to third-party modules you use. Thank you

@dulmandakh
I understand this is not an issue but I didn't know where I must suggest my opinion.
Actually, I suggested a convention, third-party libraries must know how the key gradlePluginVersion and its value will be defined in the main project's build gradle, else some developers assume that it has been defined like below:
gradlePluginVersion = "3.4.1"
and some others may assume like below:
gradlePluginVersion = "com.android.tools.build:gradle:3.4.1"
Also, there is no convention on the name of the key. what is better: ( gradlePluginVersion, gradlePlugin, ...).
Making a unique convention from react-native community seems a better solution than the libraries developers use their own convention.


APPS: Fix behavior of print_name() on out==NULL and result type of dump_cert_text()

This fixes spurious error queue entries like
007E7E0D01000000:error:100C0102:BIO routines:BIO_puts:passed a null parameter:../openssl/crypto/bio/bio_lib.c:406:
007E7E0D01000000:error:100C0102:BIO routines:BIO_puts:passed a null parameter:../openssl/crypto/bio/bio_lib.c:406:

mentioned in https://github.com/openssl/openssl/issues/16300#issue-967625058.
These are due to the IMHO unfortunate design decison that BIO_puts(NULL, ...), takes a NULL BIO input as an error,
while most/all other BIO print functions simply are no-ops in this case.

24 hours has passed since 'approval: done' was set, but this PR has failing CI tests.  Once the tests pass it will get moved to 'approval: ready to merge' automatically, alternatively please review and set the label manually.

Merged - thanks.


Pass username around login calls

When a user connects their account to Spotify the page is reloaded meaning a user's username will be lost. I need to find a way around this problem.

I found a way around it. The node.js backend now contains a queue of users logging in. A user's username is added to the queue when login begins. Then when the callback is called the username is popped out of the queue and sent back to the frontend as a url param.

While this solution works currently, I am unsure how it will behave when multiple users are logging in at once.


[Bug Report] SD bit in mstatus doesn't update when FS=11

Hi,
After runing instruction csrs mstatus, a0(a0=0x00007800) in rsd, mstatus is set to 0x00007800, not the expected value 0x80007800.
According to the privilege specification of RISC-V, if FS=11,  SD should be set to 1.

So I wonder is this a bug in rsd?

Your point seems correct, and we forgot to include the FS and XS bits in the floating point extension implementation. We will implement the feature.
I'm sorry for my late response. I would like to mention that I work at a university in Japan, and currently, we are in one of the busiest periods of the year due to many important events, such as the thesis defense and some important examinations. This has significantly limited my time to check everything...

Currently, I do not have enough time to fix this issue. To ensure it is not forgotten, I have reopened it. Once the issue is resolved, I will close it again. Thank you!


Improve Ability to Run Application under Robo Test

Is your feature request related to a problem? Please describe.
We'd like to have a "chaos monkey" test that is able to exercise different parts of the app UI.  The problem is that the onboarding flow's onboarding "test" will likely block the robo tester from actually moving forward.
Describe the solution you'd like
Either:

Detect when running under a robo test with the isUserAMonkey API and bypass the onboarding test.  I don't know if isUserAMonkey is the right API to check, or whether we should use FirebaseTestLabUtil. isFirebaseTestLab
Add a robo test script that can fill in the test correctly in order to move forward.

I think the check is better than the robo script, because the check can also work for Google Play prelaunch reports easily.
I think we can inject this check somewhere in the BackupViewModel or the MainActivity.
Additional context
Robo tests will be run both by our continuous integration scripts, as well as by Google Play prelaunch reports.
The isUserAMonkey option is probably easier for forks, as they won't have to manually supply the robo script to Google.  The disadvantage is that a small amount of test coverage is lost, although we already use instrumentation tests to extensively cover the onboarding flow.
It should be possible to test this locally.  Android emulator and adb have a monkeyrunner tool (its different from the robot test, but close enough).

Hi @ccjernigan, do we want the monkey test runner skips also the Restore Wallet screen (beside the Create Wallet screen)? As the runner probably won't get through this part of the app too.

Yes, that's a good idea.
Although that one might require us to re-think the requirements of this issue.  That one is hard to skip (compared to the test).  It would probably mean needing to generate a random seed phrase because otherwise it wouldn't know how to bypass.

Yes, that's a good idea.
Although that one might require us to re-think the requirements of this issue. That one is hard to skip (compared to the test). It would probably mean needing to generate a random seed phrase because otherwise it wouldn't know how to bypass.

Does it really need to be a random seed? Or can we provide just some mock seed like we do in our existing tests? Thank you.


Please add a license file

I'd like to add this to awesome-zsh-plugins, but it doesn't have a license.
Please add a license - if you don't have one in mind,  I recommend choosealicense.com to help choose an appropriate one.

Thank you, I just did add a MIT License file!

Thanks!


Use >5.7 kernel on Github Actions

So far we are using Github Actions to run tests and checkers.
The problem is that all ubuntu-latest Github Actions have quite an old kernel, too old for us. If we want to have integration tests in CI, we need to be able to run them on kernel newer than 5.7.
I noticed that libbpf developers are somehow able to use their custom kernel and they have a repo with CI config:
https://github.com/libbpf/ci
https://github.com/libbpf/libbpf/tree/master/.github
I have no idea how that works, didn't dig into that yet. Figuring that out would be the part of the task.

Examples of jobs running: https://github.com/libbpf/libbpf/runs/4229553591?check_suite_focus=true


battery (voltage) should be in volts not millivolts - enhancement

dict/measures.js
field: battery,
label: Battery,
icon: battery-bluetooth-variant,
unit: V,
accuracy: 3,
node_modules/node-ruuvitag/dataformats/5.js
robject.battery = (    (powerInfo >>> 5) + 1600   ) /1000.;
node_modules/node-ruuvitag/dataformats/3.js
robject.battery = battery/1000.;

Superseded by #18 remove battery voltage entirely.

I close this issue in favour of #18.


Getting nil omniauth response hash

After my callback phase I'm getting just a nil'ed out hash in my controller callback action.
 info=#<OmniAuth::AuthHash::InfoHash email=nil first_name=nil gender=nil image=nil last_name=nil name=nil nickname=nil phone=nil urls=#<OmniAuth::AuthHash website=nil>>
Things seem to be "working" other than this - no errors or anything. The omniauth hash does have an id_token and a token but for some reason that doesn't seem to be translating into any real data. Why might this be?
Here's my configuration:
provider :my_provider,
:name                    => "my_provider",
:scope                   => [:openid, :eportal],
:response_type           => :code,
:client_signing_alg      => :RS256,
:client_x509_signing_key => "-----BEGIN CERTIFICATE-----.....-----END CERTIFICATE-----",
:discovery               => true,
:issuer                  => <issuer_uri>,
:client_options          =>
{
:port         => 443,
:scheme       => "https",
:host         => ,
:identifier   => ,
:redirect_uri => <my_redirect_uri>,
:secret       => <my_secret>
}
Am I missing something that is preventing it from decrypting the id_token to put the data into the omniauth info hash? There seem to be so many configuration options but I haven't been able to find in the documentation the right combination of settings to make this work.

@jjbohn @cmrd-senya @rdingwell Any idea?

have you tried it with the email and profile scopes added in

I originally had email in there, but either one of those I get an invalid_scope error.

Hi,
Which scope values does your provider support?
Generally it follows the specification, but the value returned by the scope value depends on the provider.
http://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims

There was a mixup, and they just added the :email scope. However, when I add that in, I get the same thing, a user info hash full of nil's.

Could you check the extra.raw_info ? There is raw data in it.
https://github.com/jjbohn/omniauth-openid-connect/blob/master/lib/omniauth/strategies/openid_connect.rb#L62-L64

That is empty as well.

If so, it seems that empty data is returned from the provider.
ummm, Does the provider support scope values such as email, profile?

Just email
On Nov 11, 2016 6:10 PM, "Tomoya Usami"<EMAIL_ADDRESS>wrote:

If so, it seems that empty data is returned from the provider.
ummm, Does the provider support scope values such as email, profile?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/jjbohn/omniauth-openid-connect/issues/66#issuecomment-260076804,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEz9ikMpgluNV81wqiY6oPHqNTEXZFWnks5q9PXKgaJpZM4KwFvx
.

Could you get the response you requested with curl command?
$ curl -H "Authorization: Bearer ${YOUR_ACCESS_TOKEN}" ${USERINFO_ENDOPONT}

Ok, I'm told that I need to take my id_token, split it by ".", decode each element of the array (3 elements) into Base64, and then decrypt the 3rd element. The first element should contain data indicating Type of Token=JWT, Algorithm=RSA256, a thumbprint, and keyId, while it looks like the 2nd element contains the data. Is this what the gem is doing (i.e., does sound correct)?

Verified that the 2nd element looks like it has username, email, etc. the data that I'm looking for. However, when userinfo! is called on the AccessToken in OpenIDConnect, the get call returns an empty hash. I verified that it's setting the access token as the bearer authorization code. I suspect this could is an issue on the provider side?

Do you mean that id_token returned by the provider contains information such as username, email address, etc.?

I suspect this could is an issue on the provider side?

I think so too.

Verified it was an issue on the provider end, thanks for the help.


VSCode extension: If possible, avoid using "+ L" in keybindings

Validations

[X] I believe this is a way to improve. I'll try to join the Continue Discord for questions
[X] I'm not able to find an open issue that requests the same enhancement

Problem
You seems to have switched over from "+ M" to "+ L" for your default keybindings in the VSCode plugin. That whole set of keymaps is basically not an option for anyone using vim bindings. It also interferes with a few other plugins like LaTeX. In general, I'd suggest just staying away from the home row.
Solution
I tend to remap things to use " ... + ' " (single quote) as that tends not to be used. Another option would be to use a chord.

+1

I'm in the same boat, also it seems that the inline hints don't reference the set vscode bindings, so after you've updated the bindings to taste the inline hints will still tell you to use the old keybinds.


Pattern match failure for newtypes

Hi again Adam,
While trying to compile this:

module Main where
import Prelude
newtype I = I Int    -- this fails with:
-- fay: user error
-- (Pattern match failure in do expression at src/Fay/Compiler/Decl.hs:101:7-20)
-- data I = I Int    -- this compiles
test :: I -> Fay ()
test x = do
let
I i = x
return ()module Main where
import Prelude
newtype I = I Int    -- this fails with:
-- fay: user error
-- (Pattern match failure in do expression at src/Fay/Compiler/Decl.hs:101:7-20)
-- data I = I Int    -- this compiles
test :: I -> Fay ()
test x = do
let
I i = x
return ()

I get a compiler error.  A workaround is using data instead of newtype.  Thought you should know.
Best wishes,
Henry Laxen

Messed up the formatting, sorry.
module Main where
import Prelude

newtype I = I Int    -- this fails with:
                     -- fay: user error
                     -- (Pattern match failure in do expression at src/Fay/Compiler/Decl.hs:101:7-20)

-- data I = I Int    -- this compiles

test :: I -> Fay ()
test x = do
  let
    I i = x
  return ()

Very hacky fix, but it works! Released as fay-<IP_ADDRESS>
Thanks for the report!


Feature Request: Add autocomplete to the folder selector in the bookmarklet

I have many folders and it is sometimes slow to add a new feed to a folder because I'm bad at reading a list of words. Perhaps use one of these controls:
http://harvesthq.github.com/chosen/

Excellent suggestion. Just for kicks, did you know I built this: http://documentcloud.github.com/visualsearch/

I did, but it felt rude to ask you to add faceted search to Newsblur. Even though I want it.

I want faceted search as well. And I'll obviously use VisualSearch.js for it. After social, I predict way more social features than private features will be called for. I'd love to build search, it's just a ton of work. Maybe I could make it premium only and only index active premium feeds, but that's still a few hundred thousand feeds and millions of stories.

What about just search on starred stories to start? I would be pretty psyched if I could just find that awesome story about llamas* I starred six months ago.
*I would love to read an awesome story about llamas, but I haven't yet.


Base group margins on $main-padding-tb

The margins between group label and groups should be proportional to $main-padding-tb for consistency. (Probably just $main-padding-tb/2).

Too vague. Scss is in need of some major refactoring though.


[BUG] mne.transforms.compute_volume_registration error for CT-MR registration

Hmmm so this version that was the last one I had working before it was abstracted into a function gives the correct affine that is hard coded in the tutorial
import os.path as op

import numpy as np
import nibabel as nib

import mne

misc_path = mne.datasets.misc.data_path()
T1 = nib.load(op.join(misc_path, 'seeg', 'sample_seeg', 'mri', 'T1.mgz'))
CT = nib.load(op.join(misc_path, 'seeg', 'sample_seeg_CT.mgz'))

mri_to = T1.get_fdata().copy()
mri_to /= mri_to.max()
affine_from = T1.affine
ct_from = CT.get_fdata().copy()
ct_from /= ct_from.max()
affine_to = CT.affine

# first optimize the translation on the zoomed images using
# ``factors`` which looks at the image at different scales
trans_affine = affine_registration(
    moving=ct_from,
    static=mri_to,
    moving_affine=affine_from,
    static_affine=affine_to,
    nbins=32,
    metric='MI',
    pipeline=[center_of_mass, translation])[1]

CT_translated = resample(moving=CT.get_fdata(),
                         static=T1.get_fdata(),
                         moving_affine=CT.affine,
                         static_affine=T1.affine,
                         between_affine=trans_affine)

# Now, fine-tune the registration
reg_affine = affine_registration(
    moving=CT_translated.get_fdata(),
    static=T1.get_fdata(),
    moving_affine=CT_translated.affine,
    static_affine=T1.affine,
    nbins=32,
    metric='MI',
    pipeline=[rigid])[1]

print(np.dot(trans_affine, reg_affine))

gives
array([
    [0.99270756, -0.03243313, 0.11610254, -133.094156],
    [0.04374389, 0.99439665, -0.09623816, -97.58320673],
    [-0.11233068, 0.10061512, 0.98856381, -84.45551601],
    [0., 0., 0., 1.]])

But when I run what I thought was the exact same version with the version of transforms.py in main:
reg_affine, _ = mne.transforms.compute_volume_registration(
    CT_orig, T1, pipeline='rigids')

I get
array([[   0.92903491,    0.32371046,    0.17918057, -168.69835435],
       [  -0.32354052,    0.94570636,   -0.03100004,  -46.79813635],
       [  -0.17948724,   -0.02917206,    0.98332768,  -57.91772517],
       [   0.        ,    0.        ,    0.        ,    1.        ]])

This is a bit perplexing as by design they are supposed to do the exact same thing.

Also I when I checked going step-by-step, the translation affine was correct but then the rigid must be off


Using collapses outside of collapsibles shortcode

I believe bootstrap allows the use of collapses outside of an accordion-style set of collapses. If you do this using Bootstrap 3 Shortcodes, PHP will throw errors. If you changed the function bs_collapse to reflect the following changes, [collapse] would work fine with or without the surrounding [collapsibles] shortcode.
Line 1321 (added test for global):
$parent = isset( $GLOBALS['collapsibles_count'] ) ?
    'custom-collapse-' . $GLOBALS['collapsibles_count'] : null;

Line 1330 (removed data-parent="" wrapper from %4):
%6$s

Line 1340 (add ternary to see if $parent was null, if not, added data wrapper back in):
$parent !== null ? ' data-parent="#' . $parent . '""' : '',

This should be all set! Thank you for this report.

No worries, thanks for the plugin!


release/ci: Fix release workflow automation

This triggers the creation of a new release branch on minor release
ie releasing 1.28.0 will automatically create the release/v1.28 branch with the mobile ci removed and the initial dev commit added
it also removes the currently not working wf to reopen branches, i will follow up to get that working
it also updates relevant release docs by removing obsolete sections

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

/lgtm deps


Individual PageTransitionAnimation for each PersistentBottomNavBarItem

Hey, this might be an oversight from me, but is there any way to specify an individual PageTransitionAnimation for each PersistentBottomNavBarItem?

I'm not entirely sure what you want to achieve:
If you want to change the slide animation that happens every time you change the tab (if you have that enabled) -- that is not possible
If you want to change which animation is used when pushing a new screen, you have to specify that parameter for each push individually (and not on the PersistentBottomNavBarItem). You can so that by calling pushNewScreen like so:
pushNewScreen(
  context,
  screen: ...,
  pageTransitionAnimation: set your transition here
)

that is not possible

and there's no workaround or something? only thing i could think of is replacing the persistentTabItem with a button, which then pushes a nee screen

Actually, a simple button wouldn't be too bad. Is there a way to do this or do I have to make a custom bottom navbar for this?

Ok, if your solution works feel free to post it here for others :)


LAB1: Implement function that produces Fibonacci sequence

Fixes #205
Review of colleague's PR #
Changes proposed in this PR:

Implements Fibonacci sequence for positive number inputs
Added unit tests for edge cases

Codecov Report

Merging #152 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #152   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      2    +1
  Lines           2     15   +13
=====================================
+ Hits            2     15   +13

Impacted Files
Coverage Δ

01_fib/ovlords/main.go
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7984f47...4813395. Read the comment docs.

This looks much more like idiomatic Go code now. Thanks.

Thanks for the suggestions.


How can I stream from a plugged-in usb camera on Mac?

Hi I have a USB cam (Microsoft LifeCam) plugged in Mac which works well in Photo Booth. I wonder how can I stream it to Kinesis? I tried several method (looking at _sample_app.cpp, stream to rtsp from VLC and then stream from rtsp) without much luck. I wonder what's the recommended way here?
(/dev/video* doesn't exist, vlc rtp stream also don't let me set IP address, I'm not sure if I got the rtsp url wrong there.)
Thanks!

@voidcenter you don't need to produce an rtsp stream in order to stream to KVS. The USB camera can be used directly. For that purpose you can use GStreamer plugin and use gst-launch command or try using the sample application:
Usage: AWS_ACCESS_KEY_ID=YourAccessKeyId AWS_SECRET_ACCESS_KEY=YourSecretAccessKey ./kinesis_video_gstreamer_sample_app <my_stream_name> -w <width> -h <height> -f <framerate> -b <bitrateInKBPS>
https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/master/install-instructions-macos.md
Let us know whether you are able to run this OK.

hi @MushMal , thanks for your reply. when I ran kinesis_video_gstreamer_sample_app it would always use my built-in cam on the laptop instead of using the plugged in cam. I have two cameras on the laptop, wonder if there is any way I can specify which camera it should stream from?  (sorry if i missed it, but from the command you give I don't see where it specify the streaming source.)

Hi @voidcenter, can you replace the this line in the sample app https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp/blob/135e04361f68636d82594497ad12426b93d7bd32/kinesis-video-gst-demo/kinesis_video_gstreamer_sample_app.cpp#L683 with
source = gst_element_factory_make("avfvideosrc", "source");
g_object_set(G_OBJECT (source), "device-index", 1, NULL);

Also, if you have built libgstkvssink.so in the kinesis-video-native-build folder, you can quickly test things out by doing
GST_PLUGIN_PATH=$PWD downloads/local/bin/gst-launch-1.0 avfvideosrc device-index=1 ! videoconvert ! video/x-raw,format=I420,width=640,height=480,framerate=30/1 ! vtenc_h264_hw allow-frame-reordering=FALSE realtime=TRUE max-keyframe-interval=45 bitrate=500 ! h264parse ! video/x-h264,stream-format=avc,alignment=au,profile=baseline ! kvssink stream-name=YourStreamName storage-size=128 access-key="access-key" secret-key="secret-key"

Hi @chehefen , cool, will try that!  thanks!

Cool @chehefen , those work wonderfully. Thanks!


[REQUEST] OB error - Microsoft.AspNetCore.Server.Kestrel[22]

warn: Microsoft.AspNetCore.Server.Kestrel[22]
As of "05/24/2021 07:16:11 +00:00", the heartbeat has been running for "00:00:01.7538024" which is longer than "00:00:01". This could be caused by thread pool starvation.

When does it happen?
Does it happen consistently?
I see it's just a warning, does it affect the usability of the program?
Have you modified your values according to this guide? https://discourse.openbullet.dev/t/making-your-openbullet-2-instance-more-stable/2111

not really sure couldn't remember when or what i was doing when i saw it
not really
no
I did before but that was days ago since then all values are by default "1000"

Since it's just a warning and cannot be consistently reproduced, I'm going to close this for the time being. Reopen if you manage to give me a way to reproduce it and if it breaks the program, otherwise I cannot help with it.


Fix Mummy and Parental Bond Interaction

From issue: https://github.com/Zarel/Pokemon-Showdown/issues/2367#issuecomment-381371924
Added Parental Bond check in Mummy to maintain the power modification

Hm, this works, but I think in the future I'd want a more flexible way to deal with this.

With the current modifier move.hasParentalBond I can put the check into the multihit script instead and remove it from both ParentalBond and Mummy's code. Will try writing and testing that right now. I think I can change it to move.multihitType if it makes more sense.

Sure.

Should be good for review now.

Hm, I think I like the "just put it in Mummy" trick the most. Doing it correctly is going to cause a lot more unrelated problems.

Sorry for putting a major refactor under you and making you rebase, but it was easier to show than tell what I meant by multihitType

I rebased for you. Thanks!


Added flags to cmakelists in test project to treat all warnings as errors

Refer to #1798
Changes on test/CMakeLists.txt to always use -Wall, -Werror on GCC/Clang and the /WX, /W4 flags on MSVC.
Fixed build issue for GCC
Fixed build issues for Clang

Coverage remained the same at 100.0% when pulling 1a23a9fd4a84d5083f37e151152e856021a04088 on jmoore91:feature/build_tests_with_warnings_as_errors into bde57124187c5732c026ffb9357c0491344c45e7 on nlohmann:develop.

Keep Alive.
What is the  current state? Is it ready for merging?

I restarted the failing CIs. Let's see if they succeed now. Unless the build is green, we cannot merge.

Can you please update to the latest develop branch - there have been several fixes with respect to the CI.


Update SignalR version to 1.7.0

Issue describing the changes in this PR
resolves #803
resolves #347
Pull request checklist

[x] My changes do not require documentation changes

[ ] Otherwise: Documentation issue linked to PR

[ ] My changes should not be added to the release notes for the next release

[x] Otherwise: I've added my notes to release_notes.md

[x] My changes do not need to be backported to a previous version

[ ] Otherwise: Backport tracked by issue/PR #issue_or_pr

[x] I have added all required tests (Unit tests, E2E tests) No code changes

/check-enforcer evaluate


Lecture 2: update to BN example

I have updated the I(G) example according to piazza post 65.
https://ceesu.github.io/pgm-spring-2019/notes/lecture-02/

Looks great, thanks!


powerline: 2.1->2.1.4

Also required me to fix cflags on gyp. Together, these commits make powerline work with tmux 2.0.
Tested on darwin.

couldn't test it for pypy (since it would take a day to build it) but works for other versions.

tnx!


Query parameters support

Hi,
it seems the query parameters are not supported associated to a response. Example :
`
import requests
from py_fake_server import FakeServer
server = FakeServer(host="localhost", port=8081)
server.start()
server.on_("get", "/hello?toto=test").response(status=200, body="Hello, World!", content_type="text/plain")
response = requests.get(server.base_uri + "/hello")
assert server.was_requested("get", "/hello?toto=test").exactly_once().check()
`
The result :
AssertionError: Expect that server was requested with [GET] http://localhost:8081/hello?toto=test. 1 times.
But server was requested 0 times.
Did I miss something in the doc or is this function really missing?
Thanks,
Jean-Christophe

Hi!
You need to specify endpoint without query parameters and check them using with_query_params method. Something like this:
server = FakeServer(host="localhost", port=8081)
server.start()
server.on_("get", "/hello").response(status=200, body="Hello, World!", content_type="text/plain")

response = requests.get(server.base_uri + "/hello?toto=test")

assert (server.was_requested("get", "/hello")
    .exactly_once()
    .for_the_first_time()
    .with_query_params({"toto": "test"})
    .check())

Right now it's not possible to create endpoint (response) for request with specific query parameter. If you want to send "Hello, World!" only for the request with "?toto=test", you need to extend _on method here: https://github.com/Telichkin/py_fake_server/blob/cfc1786dc8c29d7418e842bbb819b5780d10dfa6/py_fake_server/server.py#L68


nix: make review-shell.nix code static

Previously, the whole shell.nix was created at runtime using string substitution and other crimes that made it hard to work on the expression itself.
This patch turns the string substitutions into Nix function arguments to an otherwise static Nix expression and only generates the list of attrs to be built dynamically.

@mergify queue

Thanks!


pulsar_ml_cache_used_size Prometheus metric is negative (Pulsar 2.6.1)

Describe the bug
The prometheus metric pulsar_ml_cache_used_size exposed by Broker is negative. I would presume cache sizes should be 0 or above?
To Reproduce

Launch Pulsar on Kubernetes
Inspect metrics endpoint of Broker

$  curl -s http://platform-pulsar-broker:8080/metrics/ | grep  pulsar_ml_cache_used_size
# TYPE pulsar_ml_cache_used_size gauge
pulsar_ml_cache_used_size{cluster="platform-pulsar"} -312<PHONE_NUMBER>848
$  curl -s http://platform-pulsar-broker:8080/metrics/ | grep  pulsar_ml_cache_used_size
# TYPE pulsar_ml_cache_used_size gauge
pulsar_ml_cache_used_size{cluster="platform-pulsar"} -829<PHONE_NUMBER>674
$  curl -s http://platform-pulsar-broker:8080/metrics/ | grep  pulsar_ml_cache_used_size
# TYPE pulsar_ml_cache_used_size gauge
pulsar_ml_cache_used_size{cluster="platform-pulsar"} -308<PHONE_NUMBER>131
$  curl -s http://platform-pulsar-broker:8080/metrics/ | grep  pulsar_ml_cache_used_size
# TYPE pulsar_ml_cache_used_size gauge
pulsar_ml_cache_used_size{cluster="platform-pulsar"} -1039<PHONE_NUMBER>946

Expected behavior
Cache sizes are zero or above
Screenshots

Additional context

Pulsar 2.6.1
Kafkaesque Helm Chart 1.0.24
Probably has not bearing but exposeTopicLevelMetricsInPrometheus is set to false as we have a lot of topics

The cache_used_size is get from here which is stored in EntryCacheManager , the cacheSize is decrement here, and the only place I find can possibly make it negative is here, when clearing a cache we're not exactly returning entry cache size before entries removal instead of the size of entries removed.
I'll have a patch for it.
But IMO this metrics is not very accurate as when it's calculated using Bytebuf.readableByte, and when we remove entry from cache after we consuming it, the readableByte will be smaller than before, which means we're not properly reducing the size of cached entries.

Thanks @MarvinCai for the patch.  For your last comment does this mean there should be an improvement ticket filed as I think understanding the size of this cache is super useful for performance/scaling decisions but I could be wrong. CC: @sijie @addisonj


Some directives add extra new line on each format.

When printing an if directive, a new line is being added each time.
Original Code
namespace Namespace
{
#pragma
    public class ClassName { }
}

Next Format
namespace Namespace
{

#pragma
    public class ClassName { }
}

I addressed this by removing a new line if it exists before a literal line. This may affect https://github.com/belav/csharpier/issues/22


$ npm run deploy not working

I followed the instruction and it worked perfectly up to $ npm start; however, I keep on getting the following error messages when I run $ npm run deploy. I tried to figure out but it was beyond me. If anyone can give me advice, that would be great.
$ npm run deploy

> @ deploy /Users/masa/projects/react/react_trivia_2
> node tools/deploy.gh --release

[21:44:38] Starting 'deploy'...
ReferenceError: regeneratorRuntime is not defined
    at Function.open (/Users/masa/projects/react/react_trivia_2/node_modules/git-repository/lib/Repository.js:198:14)
    at Promise (/Users/masa/projects/react/react_trivia_2/tools/deploy.gh.js:26:19)
    at module.exports.task (/Users/masa/projects/react/react_trivia_2/tools/deploy.gh.js:23:39)
    at task (/Users/masa/projects/react/react_trivia_2/tools/task.js:18:55)
    at Object.<anonymous> (/Users/masa/projects/react/react_trivia_2/tools/deploy.gh.js:23:18)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)

I encountered this issue as well.
Look to issue #93 for the solution.
Add require('babel-polyfill'); into your deploy script.

@reecefenwick Thank you for your advice! It worked.


tweaked Disease to use SNOMED CT generics

Simplified disease names to match generic naming from SNOMED-CT

@bradhead will this be the format for compound vaccinations?  Feels like if it was CSV format might keep it a bit more standardized for parsing and then it's the responsibility of the web client to display the data logically versus the API to dictate its presentation.


fix: Use UUID as a precondition when calling the eviction API

Fixes #N/A
Description
This change ensures that when we attempt to evict a pod, we don't accidentally evict a pod with a namespaced name that was recently re-created and shifted onto another node. This is particularly critical for StatefulSets, where pods can be re-created quickly with the same name.
How was this change tested?
make presubmit
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 39 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 80.593%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
7932

Relevant Lines:
9842

💛  - Coveralls


Incorrect GRE scans in sub-NYU1

According to @evaalonsoortiz, the GRE scans in sub-NYU1 are from another project. The GRE scans in sub-NYU1 on the OpenNeuro database must thus be replaced with the correct scans from this project.

Upon closer inspection, it seems all MtSinai subjects except for sub-NYU4 have the wrong GRE scans.
I found the DICOMs for the correct GRE scans, but I am encountering an issue performing the conversion to NIFTI. Running dcm2niix with the same arguments that I have used with success for GRE scans for other subjects does not properly separate the DICOMs into multiple NIFTI files (one for each channel). Instead, it outputs a single NIFTI file that contains all the channels in the fourth dimension.
The following is the output from dcm2niix when I attempt the conversion of these scans:

> dcm2niix -z y -f %s_%p -m o -o "./out" .\gre_2D_2mmIso_RefV450_uncombined_ND\
Chris Rorden's dcm2niiX version v1.0.20240202  (JP2:OpenJPEG) (JP-LS:CharLS) MSC1938  (64-bit Windows)
Found 2001 DICOM file(s)
Convert 44 DICOM as ./out\29_gre_2D_2mmIso_RefV450_uncombined (144x192x44x1)
Warning: Images sorted by instance number [0020,0013](1..44), but AcquisitionTime [0008,0032] suggests a different order (134747..134747)
Warning: Unable to determine slice direction: please check whether slices are flipped
Convert 946 DICOM as ./out\30_gre_2D_2mmIso_RefV450_uncombined_ph (144x192x43x22)
Warning: Instance Number (0020,0013) order is not spatial.
Warning: Interslice distance varies in this volume (incompatible with NIfTI format).
Warning: Missing images? Expected 43 images, but instance number (0020,0013) ranges from 44 to 1
Convert 43 DICOM as ./out\31_gre_2D_2mmIso_RefV450_uncombined_ph (144x192x43x1)
Warning: Images sorted by instance number [0020,0013](1..44), but AcquisitionTime [0008,0032] suggests a different order (134747..134747)
Warning: Unable to determine slice direction: please check whether slices are flipped
Convert 968 DICOM as ./out\28_gre_2D_2mmIso_RefV450_uncombined (144x192x44x22)
Conversion required 7.609000 seconds.

The 22 scans in the 4th dimension in (144x192x44x22) correspond to the 22 channels that we want to obtain as separate files.

I found the tool fslsplit which allowed me to split the scan into 22 different channel scans as desired, but it seems I am unable to open any of them in fsleyes, which indicates something may be wrong with them. I get the error numpy.linalg.LinAlgError: Singular matrix. The relevant data (the original scan and the split scans) can be found here: https://drive.google.com/file/d/1LHpLRVmdiWZCXBZFYsO5erYpgfxFoTg-/view?usp=sharing
I will continue to look into this.

The Singular matrix error seems to be the same error @evaalonsoortiz gets in issue #61 (see the screenshot of the error she gets when attempting to open the file in fsleyes): https://github.com/spinal-cord-7t/coil-qc-code/issues/61#issue-2273813610. The dcm2niix output for both conversions (the DICOMs from #61 and from this issue) includes the warning Warning: Unable to determine slice direction: please check whether slices are flipped.
Maybe there is a way to flip the slice directions in the DICOMs before using dcm2niix? Does anyone know if this is something that has been done in the past to mitigate an issue like this?

@evaalonsoortiz @Kyota-exe
Sorry for doing this at such an inopportune time. I am working from the data I have, so part of this might be not valid.
My "gre_2D_2mmIso_RefV450_uncombined_ND" folder has over 900 files. This is because it did not only contain the uncombined MAG files, but everything.
So my solution to this was:
1, Open the folder itself in a DICOm viewer that can handle multiple series (Osirix, Horos, MicroDICOM, etc). For this specific folder, this was four series, of the following sizes:
44 slices, Magnitude (A)
44X22 slices, Magnitude (B)
44 slices, phase (C)
44X22 slices, Magnitude (D)
I deleted everything apart from B. I was left with 44x22, that is 968 files, corresponding to 44 slices of the GRE scan and 22 channels of the coil.
2, I converted it using dcm2niix (Most recent version, Ubunutu)
3, I got the "Singular matrix" error referenced above.
3b, I managed to open the nifti file in MRICrom and ITK SNAP (windows, because they use qform to visualise)
4, I opened the header using fslinfo and fslhd, and noticed that qform and sform were not the same. This is the ususal cause of "singular matrix" errors
5, I ran fslorient -copyqform2sform B.nii
6, FSLeyes could now open the file correctly.

Do you have the DICOMS for all 3 subjects? Could you do this for all of them? If so, then @Kyota-exe could upload the proper nifits to the data repo.

I only have data for myself, Laura and Spinoza. If you point me at Rob's data, I can download it.
With our without Distortion Correction?

Example without fixing headers here:
https://drive.google.com/drive/folders/13q_HCEvhgC3Op_WjDeiKyU25QUXozsib?usp=drive_link
Let me know if this is sufficient, and what edits you'd like if its not

The slices seem to have some intensity variation across them:

My best guess is that here, we see the issue where the slice and coil
dimensions are populated correctly, in the sense that there is no mixing
dimensions, there is still a mix in ordering according to channels.
So while slice 1 might show slice1,coil1, slice 10 might show slice10,coil
5.
Sadly, the anonymisation used by the university stripped a lot of useful
info.
Basically this is as good as it gets
On Thu, Dec 19, 2024, 20:08 evaalonsoortiz @.***> wrote:

The slices seem to have some intensity variation across them:
Screenshot.2024-12-19.at.2.07.22.PM.png (view on web)
https://github.com/user-attachments/assets/3fb47f2e-392b-4095-9f53-6a2b83b7dca9
—
Reply to this email directly, view it on GitHub
https://github.com/spinal-cord-7t/coil-qc-code/issues/37#issuecomment-2555581031,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ATDM6U5SYMT2HV6W76SVHXL2GMKRTAVCNFSM6AAAAABT5NCYYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJVGU4DCMBTGE
.
You are receiving this because you commented.Message ID:
@.***>

I'll give a try with explicitly sorting by dicoms tags.
On Thu, Dec 19, 2024, 20:13 Papp Dániel @.***> wrote:

My best guess is that here, we see the issue where the slice and coil
dimensions are populated correctly, in the sense that there is no mixing
dimensions, there is still a mix in ordering according to channels.
So while slice 1 might show slice1,coil1, slice 10 might show slice10,coil
5.
Sadly, the anonymisation used by the university stripped a lot of useful
info.
Basically this is as good as it gets
On Thu, Dec 19, 2024, 20:08 evaalonsoortiz @.***>
wrote:

The slices seem to have some intensity variation across them:
Screenshot.2024-12-19.at.2.07.22.PM.png (view on web)
https://github.com/user-attachments/assets/3fb47f2e-392b-4095-9f53-6a2b83b7dca9
—
Reply to this email directly, view it on GitHub
https://github.com/spinal-cord-7t/coil-qc-code/issues/37#issuecomment-2555581031,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ATDM6U5SYMT2HV6W76SVHXL2GMKRTAVCNFSM6AAAAABT5NCYYGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJVGU4DCMBTGE
.
You are receiving this because you commented.Message ID:
@.***>

@evaalonsoortiz
OK, I have looked into all the tags of the data (including hidden/proprietary ones).
I have read the VB17 DICOM conformance statements.
The in-vivo data is simply missing required DICOM tags. There should be a tag to indicate what coil element the data has been acquired with, which is tag (0051,100F). The anonymization procedure has scrubbed this tag from the in-vivo data (I have access to myself and Laura), but not from Spinoza.
So the only tag that tells any converter anything is "Instancenumber" and the like, which shows that slice X has 22 instances associated with it, but it has no way to sort which of those 22 instances reflects which coil.
In summary, the linked file is as good as it gets.

Thanks for your efforts Daniel!


usleep(500);

Hey, is there a technical reason for this line?
usleep(500);

It was a workaround related to a bug of an old version of Android Studio.
It seems solved for the latest Android Studio.
I've removed these lines.


Serialization and cancellable effects

There are some really nice things about Redux that I'd love to see how Effector matches
In Redux, you can persist and later reload the entire state of the application:

Serialize the entire store as JSON (this is very easy when you only have a single store)
Persist the JSON-string with localStorage
Later reload localStorage - and have your application look exactly like it did earlier

Actions (in Redux) can be serialized/sent between different clients
This can be really useful when designing real-time applications. One client could broadcast an action to all the other clients.
Complex side-effect logic with redux-saga or rxjs for redux
Redux has extension libraries that allow for doing things like cancelling async functions to avoid race conditions.

How does effector match up on these points?

For localStorage there are at least two libraries to work with:
https://www.npmjs.com/package/effector-persist
https://github.com/yumauri/effector-storage
or you could link store with LS via watchers like
$someStore.watch((actualValue) => localStorage.setItem('myTestKey', actualValue)
Also effector provides you domains(essential thing for testing and SSR/SSG) - namespaces, in other words you could create every application store under that domain and later on serialize it or hydrate
Side-effect are managed via effects, example with cancelation

More examples to go: https://github.com/zerobias/effector#more-examples


[SPARK-11962] Add optional getAs functions to Row

getAsOpt[T] functions have been added to Row and GeneicRowWithSchema to get the values present in a row object optionally. Corresponding tests have also been added.

Can one of the admins verify this patch?

I am still learning to work with Git. Looks like I have a couple commits which are not mine in there. However a compare did not show any code other than mine, so I filed a pull-request. Please let me know if I messed something up and need to resubmit the request.

I just found out this would fail style checks.


Ambassador pods crash when frequently creating/deleting mappings

Describe the bug
As part of our testing, one of the tests does the following:

There is one single TLSContext, using a wildcard certificate.
Infinite loop

50 mappings are created (1sec delay between each operation)
Then, the 50 mappings are deleted (1sec delay between each operation)

During the test, there is no traffic for these mappings. There are other mappings receiving traffic (few requests, minimal load).

With Ambasador 1.12.1, after some minutes (3 to 10 in our tests), Ambassador pods die. This test passed with Ambassador 1.11.1.
Additional context
When the pod fails, Ambassador (loglevel=trace) has logged this error:
[...]
time="2021-03-18 14:57:12" level=debug msg="Taking K8sSecret snapshot.SecretRef{Namespace:\"kumori\", Name:\"cluster-core-cert-server-wildcard-test-kumori-cloud-cert\"}" func=github.com/datawire/ambassador/cmd/entrypoint.ReconcileSecrets file="github.com/datawire/ambassador/cmd/entrypoint/secrets.go:140" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="Taking K8sSecret snapshot.SecretRef{Namespace:\"kumori\", Name:\"kumori-examples-cert-server-wildcard-vera-kumori-cloud-cert\"}" func=github.com/datawire/ambassador/cmd/entrypoint.ReconcileSecrets file="github.com/datawire/ambassador/cmd/entrypoint/secrets.go:140" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="Taking K8sSecret snapshot.SecretRef{Namespace:\"kumori\", Name:\"wildcard-default\"}" func=github.com/datawire/ambassador/cmd/entrypoint.ReconcileSecrets file="github.com/datawire/ambassador/cmd/entrypoint/secrets.go:140" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: KubernetesEndpointResolver resolver endpoint-alertmanager is active (CRD)" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).saveResolver" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:201" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: KubernetesEndpointResolver resolver endpoint-grafana is active (CRD)" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).saveResolver" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:201" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: KubernetesEndpointResolver resolver endpoint-prometheus is active (CRD)" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).saveResolver" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:201" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: KubernetesEndpointResolver resolver kumori-default is active (CRD)" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).saveResolver" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:201" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: endpoint resolver endpoint exists by default" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).reconcileEndpointWatches" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:116" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: endpoint resolver kubernetes-endpoint exists by default" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).reconcileEndpointWatches" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:116" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: TCPMapping kt-082051-eedebd49-19031-tcpmapping-linked uses the default resolver (CRD)" func="github.com/datawire/ambassador/cmd/entrypoint.(*endpointRoutingInfo).checkTCPMapping" file="github.com/datawire/ambassador/cmd/entrypoint/endpoints.go:232" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: --------" func=github.com/datawire/ambassador/cmd/entrypoint.watcherLoop file="github.com/datawire/ambassador/cmd/entrypoint/watcher.go:192" CMD=entrypoint PID=1 THREAD=/watcher
time="2021-03-18 14:57:12" level=debug msg="WATCHER: --------" func=github.com/datawire/ambassador/cmd/entrypoint.watcherLoop file="github.com/datawire/ambassador/cmd/entrypoint/watcher.go:192" CMD=entrypoint PID=1 THREAD=/watcher
panic: reflect: slice index out of range [recovered]
	panic: reflect: slice index out of range

goroutine 1798 [running]:
encoding/json.(*encodeState).marshal.func1(0xc00108bd88)
	encoding/json/encode.go:326 +0x85
panic(0x1b16920, 0x20bc530)
	runtime/panic.go:969 +0x175
reflect.Value.Index(0x1ab7440, 0xc00073f938, 0x197, 0x7, 0x1e50100, 0xc0018bcf30, 0x196)
	reflect/value.go:961 +0x1f2
encoding/json.arrayEncoder.encode(0xc000e5dd70, 0xc000f5c080, 0x1ab7440, 0xc00073f938, 0x197, 0x100)
	encoding/json/encode.go:886 +0x8d
encoding/json.sliceEncoder.encode(0xc000e5dd90, 0xc000f5c080, 0x1ab7440, 0xc00073f938, 0x197, 0x100)
	encoding/json/encode.go:860 +0x8f
encoding/json.structEncoder.encode(0xc0002f8600, 0x15, 0x21, 0xc000ace660, 0xc000f5c080, 0x1e4d9e0, 0xc00073f8c0, 0x199, 0xc000f00100)
	encoding/json/encode.go:759 +0x2ab
encoding/json.ptrEncoder.encode(0xc000ace690, 0xc000f5c080, 0x1bbd420, 0xc001a41388, 0x196, 0x100)
	encoding/json/encode.go:915 +0x125
encoding/json.structEncoder.encode(0xc00079c480, 0x5, 0x8, 0xc000ace9c0, 0xc000f5c080, 0x1d441c0, 0xc001a41380, 0x199, 0x1b90100)
	encoding/json/encode.go:759 +0x2ab
encoding/json.ptrEncoder.encode(0xc000ace9f0, 0xc000f5c080, 0x1b7e480, 0xc001a41380, 0x16, 0x1b70100)
	encoding/json/encode.go:915 +0x125
encoding/json.(*encodeState).reflectValue(0xc000f5c080, 0x1b7e480, 0xc001a41380, 0x16, 0xc001080100)
	encoding/json/encode.go:358 +0x82
encoding/json.(*encodeState).marshal(0xc000f5c080, 0x1b7e480, 0xc001a41380, 0x7f0d6b490100, 0x0, 0x0)
	encoding/json/encode.go:330 +0xf4
encoding/json.Marshal(0x1b7e480, 0xc001a41380, 0xc0008ce510, 0xc001a41301, 0x0, 0x0, 0xc00108be58)
	encoding/json/encode.go:161 +0x52
encoding/json.MarshalIndent(0x1b7e480, 0xc001a41380, 0x0, 0x0, 0x1e898fa, 0x2, 0xc000b80ed0, 0xc00118ef00, 0xc0012f4f88, 0xc00108bf60, ...)
	encoding/json/encode.go:176 +0x3f
github.com/datawire/ambassador/cmd/entrypoint.(*SnapshotHolder).Notify(0xc0008ce510, 0x21346c0, 0xc000b80ed0, 0xc000729540, 0xc00118f6e0, 0xc000ba59d0)
	github.com/datawire/ambassador/cmd/entrypoint/watcher.go:430 +0x1ad
github.com/datawire/ambassador/cmd/entrypoint.watcherLoop.func1(0xc000c25980, 0x21346c0, 0xc000b80ed0, 0xc000729540, 0xc00118f6e0, 0xc000ba59d0)
	github.com/datawire/ambassador/cmd/entrypoint/watcher.go:184 +0x85
created by github.com/datawire/ambassador/cmd/entrypoint.watcherLoop
	github.com/datawire/ambassador/cmd/entrypoint/watcher.go:180 +0x1f1

Used mappings:
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
  name: mymapping-i // i = [1,2,....,100]
  namespace: kumori
spec:
  add_request_headers: {}
  host: mytest-i.mycompany.com // i = [1,2,....,100]
  load_balancer:
    policy: round_robin
  precedence: 1
  prefix: /.*.*
  prefix_regex: true
  resolver: kumori-default
  rewrite: ""
  service: myservice.kumori:5000 // The same service for all the mappings
  timeout_ms: 3600000
  use_websocket: false

Ambassador Daemonset:
apiVersion: apps/v1
kind: DaemonSet
metadata:
  annotations:
    deprecated.daemonset.template.generation: "7"
  name: ambassador
  namespace: kumori
spec:
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      service: ambassador
  template:
    metadata:
      annotations:
        consul.hashicorp.com/connect-inject: "false"
        kubectl.kubernetes.io/restartedAt: "2021-03-18T12:44:12+01:00"
        sidecar.istio.io/inject: "false"
      creationTimestamp: null
      labels:
        app.kubernetes.io/managed-by: getambassador.io
        service: ambassador
        type: ingress
    spec:
      containers:
      - env:
        - name: AMBASSADOR_SINGLE_NAMESPACE
        - name: AMBASSADOR_DRAIN_TIME
          value: "600"
        - name: AMBASSADOR_FAST_RECONFIGURE
          value: "true"
        - name: AES_LOG_LEVEL
          value: info
        - name: AMBASSADOR_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: HOST_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.hostIP
        image: docker.io/datawire/ambassador:1.12.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /ambassador/v0/check_alive
            port: 8877
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        name: ambassador
        ports:
        - containerPort: 8443
          hostPort: 8443
          name: https
          protocol: TCP
        - containerPort: 8877
          hostPort: 8080
          name: admin
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /ambassador/v0/check_ready
            port: 8877
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 3
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: "1"
            memory: 800Mi
          requests:
            cpu: 200m
            memory: 300Mi
        securityContext:
          allowPrivilegeEscalation: false
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /tmp/ambassador-pod-info
          name: ambassador-pod-info
      dnsPolicy: ClusterFirstWithHostNet
      nodeSelector:
        kumori/ingress: "true"
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        runAsUser: 8888
      serviceAccount: ambassador
      serviceAccountName: ambassador
      terminationGracePeriodSeconds: 30
      volumes:
      - downwardAPI:
          defaultMode: 420
          items:
          - fieldRef:
              apiVersion: v1
              fieldPath: metadata.labels
            path: labels
        name: ambassador-pod-info
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 1
    type: RollingUpdate

Has no one else detected this problem?

Hello,
I have repeated the test using docker.io/datawiredev/ambassador:8f0a64551e11, and it works fine  :-)

@rhs, do you know if this correction will be included in the next release?
If so, will it be in the short term?
Thank you very much!

Hello @rhs,
I have repeated the test using Ambassador v1.13.1, and it is working fine.
So... v1.13.1 includes the correction?
Thx.

We were informed via Slack (by Cindy Mullins) that the fix was included en v1.13.0. We tried to reproduce the issue using 1.13 and the issue seems to be solved.


Flaky test: PrivDebugGetStateRootOnchainGroupAcceptanceTest

April 16 failure:
org.hyperledger.besu.tests.acceptance.privacy.PrivDebugGetStateRootOnchainGroupAcceptanceTest > blockParamShouldBeApplied FAILED
and
org.hyperledger.besu.tests.acceptance.privacy.PrivDebugGetStateRootOnchainGroupAcceptanceTest > nodesInGroupShouldHaveSameStateRoot FAILED
Have just failed on a Circle CI build
https://app.circleci.com/pipelines/github/hyperledger/besu/8277/workflows/81e00426-bd7d-418d-b158-cf9f0ff1d809/jobs/41843
Failed again yesterday:
https://app.circleci.com/pipelines/github/hyperledger/besu/8453/workflows/44ac2e16-9568-4c80-8edf-d301eeee1b14/jobs/43042
The failure is because the tx receipt is not populated
Expecting:
  <TransactionReceipt{transactionHash='null', transactionIndex='0x0', blockHash='0x3cf5c7abf7c052974f3479e1831a4731a1ac47c4af5fa49457aae37b9e9fc3e2', blockNumber='0x29', cumulativeGasUsed='null', gasUsed='null', contractAddress='null', root='null', status='null', from='null', to='null', logs=null, logsBloom='0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', revertReason='null'}>
to be equal to:
  <TransactionReceipt{transactionHash='null', transactionIndex='null', blockHash='null', blockNumber='null', cumulativeGasUsed='null', gasUsed='null', contractAddress='null', root='null', status='null', from='null', to='null', logs=null, logsBloom='null', revertReason='null'}>
when recursively comparing field by field, but found the following difference:

field/property 'from' differ:
- actual value   : "0xfe3b557e8fb62b89f4916b721be55ceb828dbd73"
- expected value : "0x91240f5b6994c7ed80f9f94b1aa847880ad3b150"

closed as duplicate of https://github.com/hyperledger/besu/issues/1909


Widget to grab the Uptime Status

Is it a duplicate question?
No feature requests seem like they have this feature described
Is your feature request related to a problem? Please describe.
I have a landing page which has a nice overview of the information from other sources, but I would like just a small section dedicated to Uptime Kuma this way I can quickly check that it is all good and if I see any errors go into the actual Web UI
Describe the solution you'd like
An endpoint or widget in which I can add it to my other website and it would render the Uptime Kuma summary on the left side (see attached photo)
Describe alternatives you've considered
One alternative I was thinking was on my landing page load to load a screenshot of that specific div taken from Puppeteer and load that on my landing page.
Additional context
It would be nice just to have it hit http://my-uptime-kuma/widget and return HTML that would generate the left side summary (see attached)

<iframe src="http://<your_domain>/list" width="500" height="600"></iframe>

maybe a workaround for this.

So navigating to http://<IP_ADDRESS>:3001/list nothing is displayed. Where do you go to find the valid endpoint urls for Uptime Kuma? It is the same background color as the rest of the style but nothing is rendered, almost like a default page for any endpoint that is invalid. Is there another location for it to be?

So navigating to http://<IP_ADDRESS>:3001/list nothing is displayed. Where do you go to find the valid endpoint urls for Uptime Kuma? It is the same background color as the rest of the style but nothing is rendered, almost like a default page for any endpoint that is invalid. Is there another location for it to be?

Update to latest 1.3.2 and /list will be available

http://my-uptime-kuma/dashboard/list or now you can make status pages http://my-uptime-kuma/status/my-status-page

@97WaterPolo
Could you please check if this issue is still an active issue?
This seems like it is resolved.

Closing as resolved.


Use id in format rule check if available

If an element being checked with a format rule has an "id" key, use that as the identifier, otherwise use whatever was passed as id-or-line-number (which will presumably be the line number at that point).
Pivotal bug: 105713556

Looks Great To Me.


Publishing of messages not working

I don't Homie responding to ANY MQTT messages, which could be cause by

Incorrect MQTT topic being used by me. For the demo on https://github.com/marvinroger/homie-esp8266/blob/master/docs/2.-Getting-started.md, I understand the topic should be devices/003/light/on and the message should be true (assuming the device ID was set to 003 in the config).
Some issue in Homie, preventing it from getting the MQTT messages
Probably lots of other reasons too.

Also, after reporting the system messages to MQTT, the device sends a couple of "devices/003/$signal" messages, then goes offline with a "/devices/003/$online" = false message..
Would be good to rule out option 1. Can someone confirm that topic and message is correct, for the demo code referenced?
Using an ESP201 Sweet peas ESP8266 device, with latest PlatformIO (2.8.6) and Homie (1.3.0).

You should use topic devices/003/light/on/set => true/false

Yes, to receive messages you need to send them on devices/003/light/on/set..
The "/devices/003/$online" = false is not normal, it means the device is disconnected from the broker... I'll investigate.

By the way, what is your MQTT broker?

@dholmen, ah that can explain it. Will test that topic. I read the docs so many times... but missed that final "set"
@marvinroger: Mosquitto 1.4.2-4 running on a Synology NAS.

@mountaindude read https://github.com/marvinroger/homie :)

@marvinroger: Promise - I did read that (excellent) page a ton of times, just that it was 1.30 am in the morning... should have known better. lol

Btw, the issue with the device going offline could be due to the way the serial port works on this particular ESP8266 unit. Will try powering it from just a cell phone charger rather than a computer USB port.

@mountaindude I can't reproduce this behavior... Was it a power issue?

Closing this, feel free to reopen.


WIP: Update readme with more instructions on openwhisk-catalog

How to install openwhisk-catalog
How to run the test cases

I closed it because it has been merged with PR #44.


usm: http2: tls: Clean entry from tls_http2_iterations before flipping the tuple

What does this PR do?
Fixed an edge case for cleanup of of tls_http2_iterations map

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


DateTime Max Value Does Not Roundtrip through .Pack and .Unpack

There is a bug where DateTime max value in UTC does not roundtrip through .Pack and .Unpack
DateTime epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
DateTime date = new DateTime(3155378975999999999L, DateTimeKind.Utc);

var serializer = MessagePackSerializer.Get<DateTime>();

using (MemoryStream ms = new MemoryStream())
{
    serializer.Pack(ms, date);

    var test = ms.ToArray();

    ms.Position = 0;

    long diff = (long)date.Subtract(epoch).TotalMilliseconds;
    //=253402300800000 which cannot be added back to epoch successfully
    //*ERROR*: epoch.AddMilliseconds(diff);

    //*ERROR*: The unpacked value is not expected type. The added or subtracted value results in an un-representable DateTime.
    DateTime result = serializer.Unpack(ms);
}

I closed this issue because 3 months gone since it was fixed.


Add temporary schema for session local objects

Summary
A schema that's dropped at the end of a session.
This is useful for things like CREATE TEMP TABLE ... which would allow us to hold table in-memory for sessions.
Specifications

Rationale

Impact

Tasks

Technically we have this. Currently only holds tables, but could be extended to hold other objects in the future. Issues should be opened for specific objects as needed.


[ENH] Averagin techniques in their own module

Describe the feature or idea you want to propose
There should be a proper module for time series averaging instead of being under clustering module because its not only related to clustering
Describe your proposed solution
a propoer averagin module in root
Describe alternatives you've considered, if relevant
No response
Additional context
No response

so we would need a plan for this. Would we have fit and predict? Is it really a form of CollectionTransformer? Could have supervised I'm guessing?
I think it might be good to start off thinking what applications there are for averaging and what methods we would consider.
Applications:
finding centroids for k-means
nearest centroid classification
dimensionality reduction as a pre processing transformer
I cant really think of an algorithms any other than means and barycenter averaging? Post links here

Actually its very usefull in data augmentation as well, in this case you just need a module that takes your data and generates the average(s)
https://germain-forestier.info/publis/aaltd2018.pdf
https://germain-forestier.info/publis/icdm2017.pdf
It also can be used as a metric for choosing the source/target datasets for transfer learning
https://germain-forestier.info/publis/bigdata2018.pdf

I think it would have fit and predict yes, fit would generate the weights if the user chooses the option of weights (first and second paper i sent above) where these weights are generated from the data
and predict would simply give back the average(s)

decided not to do this @hadifawaz1999

It was decided to keep the module as is, though should be added to API

see #1537


User UI interaction without login for a long time

Bug Report
Steps to reproduce the behavior：
If you don't log in, go directly to the menu page, click on all kinds of connections and there is no response. You will only swipe to the first menu, and you will not jump to the login page directly, and the user name will be displayed in the upper right corner

Delete the local cache data, and then jump to the login page


Use basic auth in metadata check for console authentication

Streamlines the authentication process for the Console API. By doing an initial authentication via headers/metadata, the Console SDK code generators can properly emit the Authenticate method by reading the BasicAuth security option in Console Swagger doc.
Closes #440

There is a change I needed to make on the frontend. Will reopen shortly.

Ready for review.

Putting a reminder here to myself to think about how this change to the console API will affect any current GUIs that our clients have built on top of the Console API.

Closing this PR because @zyro and I talked through a way to modify the code generator that makes a lot of sense.


Fix breadcrumb links and improve their titles

While moving the breadcrumb generation into a template tag, I failed to notice that the links generated therein are actually wrong.
Here's a fix that also improves titles: Previously breadcrumbs for pages that existed but didn't have a title set did get an empty string and thus an empty entry in the breadcrumb list. Now they'll be given the slug-part.

Coverage increased (+0.2%) to 75.154% when pulling 6cc74d795420a806d5bf64570c8ea2199afdabcb on wagnerflo:breadcrumbs into b7db696075ceebb5676be61f44e2d806cc472255 on mgaitan:master.

Coverage increased (+0.2%) to 75.154% when pulling 6cc74d795420a806d5bf64570c8ea2199afdabcb on wagnerflo:breadcrumbs into b7db696075ceebb5676be61f44e2d806cc472255 on mgaitan:master.

Actually: What the breadcrumb code is doing, is making design decisions (what text to display for the link) in the code instead of the templates.
Please hold of merging this one. I'll amend the pull request and fix this in a (IMHO) better way.

Coverage increased (+0.8%) to 75.771% when pulling 1c064c7b27fdc6fe72a56163354d7e894b0d7e1f on wagnerflo:breadcrumbs into b7db696075ceebb5676be61f44e2d806cc472255 on mgaitan:master.


Fix compatibility issues

fixes #14
I edited the code, so the value will only be accessed if it exists. That means, that in future we could write all states for all models of the Piko MP Plus and the adapter will only update/create states which it really founds in the XML. All other states will be ignored as they would like they have no value.

I also updated the StatesMapper.ts with the states for single string inverters. (dcpower, dccurrent, etc.) because it only had states for a double string inverter (dcpower1, dcpower2, dccurrent1, dccurrent2). Now it gets all states.

The state "acvoltage" was listed twice in the array in StatesMapper.ts .

I added a states for the yields section (yields->totalWork) to watch the total work of the inverter.

I hope i can help with that PR, if u need anything let me know!
Great job with the adapter!
Greetings

Hi ekoschmi,
cool that you contribute something to the adapter. I wrote the adapter for a friend and have no idea what there is for inverter and what is important :D.
Would you write something about your changes in the README.md in the changelog under WORK IN PROGRESS.
Then I would take your changes in the master branch and create a new release.
PS: I have stored an xml example of the PIKO 3.0-2 MP plus in test/test_xml_files. I changed some data, because I don't know if they should be published. Maybe you could also add an example of your inverter for integration tests.

Hey @Doggi  - there u go!

at fa3bb37 i fixed an issue i got today. ioBroker runs on a raspberry which is connected via wifi. Every now and then the adapter gets a timeout error while requesting. But after that timeout it continues working without problems. So i edited the main.ts to reset that fail counter if a request is succesful. I think that it makes sense if u got an error but the next time it works, to reset that counter. Whats do u think?

Hey @ekoschmi,
perfect.
Good idea, is accepted and is adopted.