README: setup instructions

:bug: The Bug
In the setup instructions in tests/README.md we tell the user to clone the repository using git clone git@github.com:hoodiehq/hoodie-account-server.git. But GitHub itself recommends the https:// protocol instead of the git@ which is SSH (see which remote should I use).
:dart: The Goal
Replace the git clone git@github.com:hoodiehq/hoodie-account-server.git with https://github.com/hoodiehq/hoodie-account-server.git in the tests/README.md
:clipboard: Step by Step
If this is your first, welcome :tada: :smile: Here is a great tutorial on how to send a pull request

[ ] claim this issue (comment below)
[ ] Clone the repository to your computer
[ ] Edit the tests/README.md file.
[ ] Commit the change docs(tests): fixed git clone URL
[ ] Start a Pull Request.  Mention closes #138 in the description of the pull request
[ ] Done :+1: Comment below to ask for a review :)

Ping us in the Hoodie Chat or on Twitter if you have any questions :)

hey can I work on this issue?

oh yes absolutely, all yours @huzaifafaruqui :)

Thanks @huzaifafaruqui, that was perfect :ok_hand: welcome to Hoodie! Are you on twitter? We like giving new contributors a shout out, if you don’t mind?
Let me know if you want to keep contributing, we're happy to help you find something. Best is to join our Hoodie chat. You can also send me a direct message in there, I’m @gregor.
Have a fantastic weekend!


SQLNormalizedCache reactivity differs from LruNormalizedCache with watchers and prettifyDump

Summary

Issue 1: Reactivity between two caches

I'm experiencing issues with reactivity when using LruNormalizedCache when compared against SQLNormalizedCache.
When using just SQLNormalizedCache, all watchers are called correctly when data is returned from the server after mutations and queries (just like the web and iOS clients). The interfaces all update correctly and the app launches using only the SQL cached data (indicating there is data in cache — see Issue 2).
However, when I use LruNormalizedCache, the watchers are not called when they should be in response to mutations (and are with SQLNormalizedCache). Any insight on how I can get the same cache reactivity behavior from the two would be super helpful!

Issue 2: Direct cache access & prettifyDump

When using just SQLNormalizedCache, I want to directly access a query and add data to that query (pagination of a list using infinite scroll). However, when I try to access the cache directly using apolloClient.apolloStore.read(targetQuery).execute() where targetQuery was executed at app launch and should be in cache, I get an ApolloException: Failed to perform store operation.
When trying to dump the cache, it logs nothing and logging apolloStore.normalizedCache().dump() yields
{class com.apollographql.apollo.cache.normalized.OptimisticNormalizedCache={}, class com.apollographql.apollo.cache.normalized.sql.SqlNormalizedCache={}}
Even when I query using NETWORK_ONLY, then in the onResponse, I call the same exact query using CACHE_ONLY, I get a null response. This happens for both SQLNormalizedCache and LruNormalizedCache.
This is even more confusing considering the app launches correctly with no internet connection using only the SQLNormalizedCache.
Conclusion
Any insight on how to get the apolloStore.read(targetQuery) to return data and/or getting the normalizedCache().dump() to work would be greatly appreciated. Otherwise, insight on how to get the LruNormalizedCache to react the same way that the SQL cache does would also work.
Version
build.gradle (:app)
apollo {
    generateKotlinModels = true
    customTypeMapping = [ 'JSON' : "org.json.JSONObject" ]
}

dependencies {
    def apolloVersion = "2.4.0"
    implementation "com.apollographql.apollo:apollo-runtime:$apolloVersion"
    implementation "com.apollographql.apollo:apollo-coroutines-support:$apolloVersion"
    implementation "com.apollographql.apollo:apollo-normalized-cache:$apolloVersion"
    implementation "com.apollographql.apollo:apollo-android-support:$apolloVersion"
    implementation "com.apollographql.apollo:apollo-normalized-cache-sqlite:$apolloVersion"
}

Apollo Cache Config
    private val resolver: CacheKeyResolver = object : CacheKeyResolver() {
        override fun fromFieldRecordSet(field: ResponseField, recordSet: Map<String, Any>): CacheKey {
            val id = recordSet["id"] as? String
            if (id != null) return CacheKey.from(id)
            return CacheKey.NO_KEY
        }

        override fun fromFieldArguments(field: ResponseField, variables: Operation.Variables): CacheKey {
            val id = field.resolveArgument("id", variables) as? String
            if (id != null) return CacheKey.from(id)
            return CacheKey.NO_KEY
        }
    }

private val cacheFactory = LruNormalizedCacheFactory(EvictionPolicy.builder().maxSizeBytes(10 * 1024 * 1024).build())
private val sqlCacheFactory = SqlNormalizedCacheFactory(context, "apollo.db")

apolloClient = ApolloClient.builder()
            .serverUrl(serverUrl)
            .logger(LocalLogger())
            .okHttpClient(okHttpClient)
//          .normalizedCache(cacheFactory, resolver) // Swap between two cache factories. This one is not reactive.
            .normalizedCache(sqlCacheFactory, resolver)
            .addCustomTypeAdapter(CustomType.JSON, jsonCustomTypeAdapter)
            .defaultHttpCachePolicy(HttpCachePolicy.CACHE_FIRST)
            .build()

Hi! Thanks for reaching out! Indeed, the normalized cache is an area that requires a lot of polishing/rework. It's pretty high in the todo list. Until then, things you can try:

What is the size of the data you are downloading? Your LruNormalizedCache is configured for 10MB so there's a chance some items are evicted, which might explain why the cache is not reactive anymore.
SqlNormalizedCache does not implement dump which explains why you aren't seeing anything. I created a follow up issue there: https://github.com/apollographql/apollo-android/issues/2696
What you can try for debugging is to chain a Lru cache to a SQL cache:

LruNormalizedCacheFactory(EvictionPolicy.NO_EVICTION).chain(SqlNormalizedCacheFactory(this, "apollo.db"));

This way you'll be able to call dump and have all your cache (since there's no eviction)

Other things to look for are cache misses due to missing ids even though it doesn't seem like it is the case here.

For the reactivity/.read not working issues, if you can upload a reproducer project somewhere, that'd help a ton.
Hope this helps,
Martin

Thank you for your quick response, just wanted to update you before I close the issue.

Chaining the caches does not work for the watchers or cache reads to work. I think it's the issue u described that LruNormalizedCache is not ready for production watchers. (They all work on SQL cache). The size of the data stored in cache is a maximum of 1MB in my cases.
Thanks for adding #2696 will be useful when it's implemented.
The read issues disappeared when I reinstalled the app from scratch on the 2 test devices and simulators which is super weird (It was happening on all devices I was testing on but then I created a new simulator and it worked).  Potentially it was a change to the db schema for SQLight?

Looking forward for the updates to the normalized cache!


Stream architecture rework p.1

(only UDP H264 is supported now)

Please be sure to remove all development and lost code around, I should review the PR later today.

Okay, I intentionally left some lost code in src/stream/stream.rs so you can better grasp to where its going to expand in the next parts. After your review I can execute the fixup.

We can continue as it's and get more improvements merged later


[Feature Request] Keras ops.gradient function

Hello,
I was wondering whether it would be possible to provide a keras ops grad function based on e.g. the jax design.
This would be helpful to implement energy/force models or PiNNs in keras 3.
I remember that the first keras initially had a backend gradient function.
I already tried and it worked for tensorflow and torch but for Jax models the gradient output did not work in training for me.
Is this something you would be interested in? Do you see potential problems with jax?
Thanks!

This is not straightforward in a multi-backend setting. We could make it work for Keras graphs (Functional models) but not for arbitrary backend-level computation.
Why not just use backend specific APIs like tf.GradientTape or jax.grad?

Okay. Do you mean something linke this (see below).
I just wanted to leave it in case somebody is looking for a similar issue.
I tested and it seems to work for all backends.
If you do not see an obvious problem with keras API you can close this issue.
import keras as ks
import numpy as np
from keras.backend import backend

model = ks.models.Sequential()
model.add(ks.layers.Dense(32, activation="swish"))
model.add(ks.layers.Dense(32, activation="swish"))
model.add(ks.layers.Dense(1, activation="linear"))

class FirstDeriv(ks.models.Model):

    def __init__(self, m):
        super(FirstDeriv, self).__init__()
        self.m = m

    def call(self, inputs, **kwargs):

        if backend() == "torch":
            import torch
            with torch.enable_grad():
                inputs.requires_grad = True
                out = self.m.call(inputs, **kwargs)
                out_sum = torch.sum(out, dim=0)[0]
                out_grad = torch.unsqueeze(torch.autograd.grad(out_sum, inputs, create_graph=True)[0], dim=-1)
        elif backend() == "tensorflow":
            import tensorflow as tf
            with tf.GradientTape(persistent=True) as tape:
                tape.watch(inputs)
                out = self.m.call(inputs, **kwargs)
                out_sum = tf.reduce_sum(out, axis=0, keepdims=False)[0]
            out_grad = tf.expand_dims(tape.gradient(out_sum, inputs), axis=-1)
        elif backend() == "jax":
            import jax
            import jax.numpy as jnp
            out = self.m.call(inputs, **kwargs)
            def fn_out(inputs):
                out_call = self.m.call(inputs, **kwargs)
                out_sum = jnp.sum(out_call, axis=0)[0]
                return out_sum
            grad_fn = jax.grad(fn_out)
            out_grad = grad_fn(inputs)

        else:
            raise NotImplementedError()

        return out, out_grad

X = np.random.normal(2, 3, size=(20,1))
y = [np.square(X), 2*X]

model_grad = FirstDeriv(model)
model_grad.predict(X)
model_grad.compile(optimizer=ks.optimizers.Adam(), loss=["mean_absolute_error", "mean_absolute_error"])
model_grad.fit(X, y, epochs=1000, verbose=2)

Thanks for the code snippet. Closing as there are no plans to add a backend-agnostic gradient API at this time.


v1.2

Features:

[x] Add new patient information to database.
[x] Delete patient information from the database
[x] Schedule an appointment for the patient
[x] Cancel an appointment
[x] View all upcoming appointments displayed in a concise and accessible format

Good job on completing our first milestone! ❤ @xuelinglow @limrik @ThatGirlJam @dayn-1


How can i do some AJAX requests by selenium in spiderflow?

I don't know how to build AJAX requests by dragging and dropping
thank for your help!

U need to inject the support component for selenium, and then u can use supported fucntions.
U can get some guide from here:
https://www.spiderflow.org/intro.html
Benjamin Zhou
@.***

------------------ Original ------------------
From: @.>;
Send time: Friday, Feb 10, 2023 8:38 PM
@.>;
@.***>;
Subject:  [GUANQUNZHOU/Spider-Flow] How can i do some AJAX requests by selenium in spiderflow? (Issue #1)
I don't know how to build AJAX requests by dragging and dropping
thank you for your help!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Fix links to book-html and github-pages

book-html and github-pages are now in the publish sub folder
@choldgraf , what would you think of adding a Edit on Github button as we see on some Sphinx documentations?

perfect, thanks very much for catching this!!!


show_doc and # exports module don't play nice

Using nbdev  0.2.18.
I added a module to an #exports to route code somewhere else. Now nbdev_build_docs fails with
An error occurred while executing the following cell:
------------------
show_doc(BayesianCNN, default_cls_level=2)
------------------

(which is a cell that is created by the html conversion code for the cell whose target module I've changed)
I hope this provides sufficient information to reproduce it.
(BayesianCNN is a class that I defined in that cell as example, and then I thought: maybe I should collect this model in its own module to avoid copy-pasting the code over my notebooks.)
Thanks,
Andreas

There is nothing we can do to help without:

seeing the whole stack trace
seeing the notebook that creates the problem.

We are not magicians :)

nbdev_build_lib
Converted 01_batchbald.ipynb.
Converted 02_joint_entropy_computation.ipynb.
Converted 03_consistent_mc_dropout.ipynb.
Converted 04_active_learning.ipynb.
Converted 05_repeated_mnist.ipynb.
Converted 06_example_experiment.ipynb.
Converted 07_black_box_training.ipynb.
Converted A1_some_models.ipynb.
Converted index.ipynb.
touch batchbald
nbdev_build_docs
converting: /home/blackhc/PycharmProjects/bald-ical/03_consistent_mc_dropout.ipynb
An error occurred while executing the following cell:
------------------
show_doc(BayesianCNN, default_cls_level=2)
------------------

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-5-0dff4c94e4b1> in <module>
----> 1 show_doc(BayesianCNN, default_cls_level=2)

NameError: name 'BayesianCNN' is not defined
NameError: name 'BayesianCNN' is not defined

Traceback (most recent call last):
  File "/home/blackhc/anaconda3/envs/active_learning/bin/nbdev_build_docs", line 8, in <module>
    sys.exit(nbdev_build_docs())
  File "/home/blackhc/anaconda3/envs/active_learning/lib/python3.7/site-packages/fastscript/core.py", line 73, in _f
    func(**args.__dict__)
  File "/home/blackhc/anaconda3/envs/active_learning/lib/python3.7/site-packages/nbdev/cli.py", line 101, in nbdev_build_docs
    notebook2html(fname=fname, force_all=force_all, n_workers=n_workers)
  File "/home/blackhc/anaconda3/envs/active_learning/lib/python3.7/site-packages/nbdev/export2html.py", line 522, in notebook2html
    raise Exception(msg + '\n'.join([f.name for p,f in zip(passed,files) if not p]))
Exception: Conversion failed on the following:
03_consistent_mc_dropout.ipynb
Makefile:13: recipe for target 'docs' failed
make: *** [docs] Error 1

That's all I got in the console.
I'll see if I can create a very basic repo to reproduce this. It seems like quite a generic bug.
PS: And wow that was quick reply & fast turn-around from your side 🔥

Ok, the error would indicate BayesianCNN has not been exported properly. Double-check it's indeed present in the modules and that the module has been installed at least with a dev install so that python can find it.

Have a look at https://github.com/BlackHC/reproduce_nbdev_154/ (and the repo) which allows on to reproduce the issue. Rename 02_core.ipynb to 00_core.ipynb or vice-versa
It's the order of notebook. If you try to export to a notebook that is run "later" than the current notebook, nbdev_build_lib will silently fail to export the cells and nbdev_build_docs will crash.
(However, if that notebook is new and the module does not exist yet, nbdev_build_lib will fail with an error message about the module not existing yet.)
I think it's not clear that the order of notebook execution matters so much. At least the error messages might need improvement.
Furthermore, I wanted to use the feature to write some cells into an appendix for later re-use, which is not really possible this way? (Or is there a way to change the order of notebooks in the nav bar?)

It's the order of notebook. If you try to export to a notebook that is run "later" than the current notebook, nbdev_build_lib will silently fail to export.

Oh, that's not very nice, we should fix that. Yes the order of execution matters, especially for cells exported in other notebooks.

Or is there a way to change the order of notebooks in the nav bar?

You can write your custom sidebar by setting the corresponding attribute to true in your settings.ini and editing the file sidebar.json in docs/ (see the fastai2 project for an example of custom sidebar.json)

I think having order requirements is reasonable. Obviously, if it was possible to be more flexible, that would be great, but otherwise catching this case with a good error message could go 80% of the way.
My user story is that I have the main notebooks and introduce an example model (that I keep reusing) in one of the notebooks to show how to plug things together. I wanted to export that model into an appendix notebook to reuse this later without having to copy & paste the code again.

Think I have a fix for this https://github.com/fastai/nbdev/pull/165 - as long as your converting all nbs, you can export to a notebook that is run "later"
hope this helps (o:

Yup, #165 should fix this.

Thank you!


Make each sheep item be able to drop down their information rather than just display

Each sheep item added should have their name and a drop down toggle that, when pressed, will drop down with all the pertinate information.

I will claim this issue.


[WIP] Implementation of Map API

I'm not particularly skilled at implementation especially the more delicate and fine tuned tools available, additionally maps are fairly spread around which makes this extremely difficult for me to implement.
This is currently just the ground work and it will not build due to other missing API's, the implementation of MixinMapData is complete except for updating which I don't know how to fix/complete. Additionally it uses @Overwrite to perform most of the work because I don't know how to use @Redirect and @Inject.
If one of @SpongePowered/developers that is more skilled could walk me through this that would be fairly beneficial since I would like to focus more on implementation.

This is long out of date and will be changed on SpongeCommon anyway.


Missing key detail

Windows 10 on ARM64 does not support x86 kernel-mode drivers.
It also does not support x86 user-mode drivers.
For clarity, it would be best if this article lists both, currently it only mentions kernel-mode drivers.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8da0acb6-3b62-3c4d-a40b-fadb3b438e8d
Version Independent ID: 19a5ad0a-ee47-1834-694f-ee89882fa78d
Content: Building ARM64 Drivers with the WDK - Windows drivers
Content Source: windows-driver-docs-pr/develop/building-arm64-drivers.md
Product: windows-hardware
Technology: develop
GitHub Login: @tedhudek
Microsoft Alias: tedhudek

Hi @embermctillhawk thanks for the feedback.  Great point - I just made the fix -- should appear on the live site within 10 minutes or so.
Thanks again for caring about the docs :)


Validate that policy config schemas are compatible with react-jsonschema-form

The 3scale UI uses react-jsonschema-form to build forms from the policy config schemas.
We need to add some kind of validation to make sure that apicast policy config schemas can be correctly rendered by that library. We have some limitations. For example, using an array of objects with oneOf does not work correctly: https://github.com/mozilla-services/react-jsonschema-form#q-does-rjsf-support-oneof-anyof-multiple-types-in-an-array-etc
In particular, the JSON schema of the rate limit policy is affected by this problem. The limiters and the error_settings properties in the schema are arrays that are not compatible with the react-jsonschema-form library.
There's a web tool to verify that a JSON schema can be rendered by the library: https://mozilla-services.github.io/react-jsonschema-form/
\cc @y-tabata

I started a WIP branch to modify the rate limit schema and make it compatible with the react-jsonschema-form library:  https://github.com/3scale/apicast/tree/adapt-rate-limit-policy-schema-to-react-jsonschema
I still need to modify the code and the tests to adapt them to the new schema.

The schema of the rate-limit policy was modified in https://github.com/3scale/apicast/pull/696
The issue of introducing an automated way of validating the schemas to make sure they're compatible with react-jsonschema-form remains open.


How to show current file on other branch

In vim-fugitive I can do :Gedit develop:%. The % expands to the current file path and the command will open that file path but from the specified branch (here develop) in my editor. How do I achieve the same in GitLens? I can compare my directory and branch and then navigate through the explorer view but that's a bit clunky. I really just want the current file on the current branch and (next to it, on the side), the same file on a different branch.

From the command palette, you can use the GitLens: Open Changes with Branch or Tag... — compares the active file to the tip of the selected branch. The command is also on the right-click menu of items in the Source Control sidebar.
Or you can use GitLens: Open Changes with Revision... — compares the active file with whatever revision you choose from the quick pick. The command is also on the right-click menu of items in the Explorer & Source Control sidebar. and as the alt (⌥) command of the Open Changes with Previous Revision command in the editor toolbar

Hope that helps!

This always opens a diff rather than the file though

yes magit has the same feature in emacs. i can pick a file and say "open me that file read-only as it is currently in branch X" and you get to pick which branch. I was hoping this could available through a command in vs code or vscode-gitlens but right now I'm under the impression it's not possible as-is.

Ah, so to just open a readonly version of a specific revision use, Open Revision... — opens a version of the active file with whatever revision you choose from the quick pick.

That doesn't let me select a branch though ;) although it's close since I can pick the most recent commit of that branch

Yeah that is the equivalent. But I'll add a new command to open the revision from a branch or tag shortcut like the comparison one.

:+1:

@cideM if you think this was helpful, consider adding a reward to this issue using speedoss.com to support the issue solver for their work :)


Reforming dynamic lists in SSZ

Background reading: https://github.com/ethereum/eth2.0-specs/issues/1115
From my experience implementing SSZ partials (https://github.com/ethereum/eth2.0-specs/blob/ssz-impl-rework/test_libs/pyspec/eth2spec/utils/ssz/ssz_partials.py), I've come to the conclusion that the fact that paths do not correspond to specific generalized indices (as generalized indices depend on the depth of a tree which for dynamic-sized lists is dynamic) leads to a large amount of complexity in the SSZ partials implementation. append and pop and the complexities around rebalancing lists around powers of 2 are particularly problematic.
My proposed solution to this is as follows. From the perspective of SSZ hashing, we remove lists, so the only data types are (i) base types, (ii) fixed-size vectors, (iii) containers (we could also add unions by hashing a Union[A, B... Z] identically to Container(a=Vector[A, 1], b=Vector[B, 1] ... z=Vector[Z, 1])).
All existing lists in the spec (validator list in the state, transaction lists in blocks...) get converted to fixed-size lists of some size (for lists in blocks, we know the maximums already; for the validator set we can pick a very generous limit, eg. 2**40). Hence, from the perspective of hashing, all data types become constant-sized types, and so the generalized index that a given path (eg. state -> state.validator_registry[123514].pubkey) corresponds to is always the same value.
However, in the SSZ vector data type, we now add a flag, "serialization type". To start off, the serialization types are FULL and DYNAMIC; a third one to be added later is SPARSE. A FULL vector is serialized like vectors are today; a DYNAMIC vector is serialized by serializing the items in the vector up until the highest nonzero item the same way that a list is serialized today. This makes DYNAMIC vectors essentially equivalent to current lists in terms of serialization, except that a maximum length must be specified. A SPARSE vector would be serialized by prepending the list of items with a (item, position) table of what all of the nonzero items are. Current vectors would become vectors with a FULL serialization type, current lists would become vectors with a DYNAMIC or possibly SPARSE serialization type (if DYNAMIC is used, then it may make sense to add a validation rule in those cases that verifies that there is no non-empty object that appears after any empty object in the list).

I would like to make an alternative proposal: Replace list by a type List[A, max_length], which is still a dynamic list type but with maximum length max_length.

Merkleization would be like a Vector but with the length mixed in (in other words: it is the hash tree root where on the left is a static vector filled with zeroes and on the right there is the length as a uint64)
Serialization is just like a List at the moment

The reason why I would propose this is that I think it is preferable that the type is aware of its current dynamic length. The code for this is much easier to write, reason about, and check statically and dynamically. It is also consistent with the other changes we are making to SSZ (like unions and bitlists) which go into the direction of externalising typing work into SSZ and making it work "naturally" as you would expect from a modern programming language.

Merkleization would be like a Vector but with the length mixed in (in other words: it is the hash tree root where on the left is a static vector filled with zeroes and on the right there is the length as a uint64)

I can see how this can be valuable. But there is one question: do we want length mixed in as an item at the top, or as an element of the list? Huffman theory would say that mixing length in at the top is only optimal if half of all queries to the list are queries of its length, but that seems implausibly high. It would also say that reserving the first element of the array to represent length is only optimal if a length query is only as common as a query to a randomly selected element, which seems implausibly low. Which of these two extremes is less bad? The former is better from the PoV of not changing things as much as possible, the latter is better from the PoV of simplicity.

Huffman theory would say that mixing length in at the top is only optimal if half of all queries to the list are queries of its length, but that seems implausibly high.

I'm not sure, but if we assume that Merkle proofs typically access one element, and every proof hast to come with a proof of the length of the list, then actually half the accesses are to the length.
(We could skip this if we use zero-checking logic instead of checking the length, but this might need nasty special cases for lists where zero is a valid value. Not what I would prefer from a simplicity point of view)

and every proof hast to come with a proof of the length of the list

Ah, I suppose if we want our list get proofs to return what list[i] would actually return, which is IndexError if the index is out of bounds, then you're right. OK, I can see the rationale for mixing in at the top as in the status quo.

See if we can describe the new extended SSZ better/more minimally (or at least my idea of it, all just an idea):

Any collection is defined as <header><body>
<body> is always defined as the concatenation of the serialized elements in the collection.
Bitfields require the <body> to be exactly bit-length bits, left-padded to the first multiple of 8.
BigInts are the same as bitfields, but interpreted as little-endian (TBD) integer.
<header> knows 5 different encodings/hashings:

Known, header: `` (zero bytes), root: merkle(pad_right(elements, M))
Body size and elements location are known on compile time.

fixed-size element containers
fixed-size element full vectors

Dynamic, root: H(len, merkle(pad_right(elements, M)))

Implicit,  header: `` (zero bytes)
Body size known from context (decoders use read bound)

fixed-size element dynamic vectors (a "list")
Bitfield (new)

Explicit: <offset>*

variable-size element containers
variable-size element full vectors
dynamic vectors (a "list")

Sparse, root: merkle(pad_sparse(elements, indices, M)):
Indirectly: header: <<index><offset>>*:
- Sparse variable-size element vector (new)
Directly: header: <index>*
- Sparse fixed-size element vector (new)
Selected: header: <index>, root: H(index, H(element)):

Option

* indicates zero or more elements, concatenated.
<index>, <offset> and <length> are always 4 byte little endian integers (i.e. uint32).
Offsets always count the bytes up to (but not including) the element.
M is defined as the power of 2 defined as collection limit.
merkle chunkifies (partition elements into 32 byte chunks) the input if it is of a basic type or a bitfield.
merkle computes the binary-merkle-root of the given elements.
pad_right(elements, M) pads the elements with zeroes to the given power of 2. **
pad_sparse pads the gaps and ends to match the given power of 2. **
**: Optimized away within merkleization in real-world implementations.

note on Sparse-Indirectly: we can do <index>*<offset>* too, if we want to merkle just part of the header efficiently for inclusion proofs constructed in limited contexts (contracts?).
note on <body>: encoders for bodies can be remixed with different header types to implement all the types easily:

read by offsets
read by stride (fixed increment per element)
read by element size (variable increment per element).

I'm very confused by your post... at least it does not feel like a simplification to me ;)
Have we defined sparse objects yet?
Also the bitfields is not how it works (we pad them on the left)

Dynamic, root: H(len, merkle(pad_right(elements, M)))
Mixing in the length is always on the right

Have we defined sparse objects yet?

No, but I posted this serialization earlier in the chat, as it is consistent with offsets, and simple. But improvements are very welcome. (also see note on ordering of indices)

Also the bitfields is not how it works (we pad them on the left)

See edits, that is what I did previously. But that's not consistent with the current verify-bitfield behavior, which is like a little endian int. Also, a big little endian integer sounds more consistent too me. But considered big-endian for formatting/reading reasons (prefer it too, as there's no "gap" in the bits when you align a 9 bit integer in bytes)

Mixing in the length is always on the right

Whoops, wrote it a bit quick, but you get the idea

But that's not consistent with the current verify-bitfield behavior, which is like a little endian int.
Oh yes, forgot it's little endian. Then it should be on the right.

I may add information here to help others who just arrive.
What is an SSZ Partial?
An SSZ partial is an object that can stand in for an SSZ object in any function, but which only contains some of the elements in the SSZ object. It also contains Merkle proofs that prove that the values included in the SSZ partial actually are the values from the original SSZ object; this can be verified by computing the Merkle root of the SSZ partial and verifying that it matches the root of the original SSZ object.
Source: https://github.com/ethereum/research/tree/master/ssz_research/partials

Addressed in #1180


Fix protocol constants

https://github.com/MinaProtocol/mina/pull/14776#issuecomment-1939536688 pointed out that the protocol constants to_input function is missing the grace_period_slots field that was added in that PR. We should fix this. Luckily, this issue doesn't introduce a vulnerability in the snark from my understanding, but would cause the circuit to fail to compile if we were ever requesting the protocol constants as a private input.

Fixed in https://github.com/MinaProtocol/mina/pull/15690.


Move importers

Resolves #67. Resolves #55?

@marySalvi pointed out that the request_body serializer for the Create Dataset endpoint was confusing because some fields will always be non-applicable. Since we give the user the option to specify a dataset with either an upload or an importer, there are two separate potential request_body serializers we can allow (see 26ff616).
However, we can't offer two potential request_body serializers on the same endpoint. Should we recombine these to one (and allow the serializer to always have some non-applicable fields) or should we split the Create Dataset endpoint into two (one for import and one for upload)? Or is there some way to implement multiple request_body serializers that I haven't found?

Another idea: If we split the create endpoint in two, we would have an "upload dataset" endpoint and an "import dataset" endpoint. The "import dataset" endpoint would accept most of the normal fields plus some extra information that is not saved to the Dataset model, such as the importer to use and the arguments to pass it. Information relating to the import is only ever used in this endpoint, so why save it to the model once we've used it? Since we decided to remove the source_uri field from the Dataset, why not remove the importer field and extension field, too?

For both this PR and #70, The seven migration files we had have been squashed into one. To migrate your db, do the following in your django container:
python manage.py migrate core 0001

Then switch branches, then:
python manage.py migrate

This only needs to be done once.


Copy from build window

Although I can select text, it does not seem possible to copy text output from the build window. It is not possible to bring up the context menu, and Cmd-C does not get the text onto the clipboard.
Is this fixable?

Ugh totally not this extension's fault. They way Atom rebinds keys is strange. Still learning.

If anyone else runs into this, I found the real cause was with emacs-core-keys.

Thanks for taking the time to report this!


Box-shadow: none doesn't work.

Right now, there doesn't seem to be any elegant way to declare box-shadow: none. You have to either use unsafe("boxShadow", "none") or boxShadows([`shadow("none")]). The latter feels hacky and isn't really any safer than the "unsafe" method.
I'm not sure how this would easily coexist with the existing boxShadow function, though. Maybe there could be a special-case boxShadowNone function?
I'm also not sure how many other properties are affected by this same issue. Some similar ones have more elegant workarounds. Example: outline(`none) is invalid, but outlineStyle(`none) accomplishes the same result and is fine. Box-shadow, unfortunately, can't be broken into smaller properties (AFAIK).
Edit: After posting this, it occurred to me that perhaps a straightforward way of enabling this functionality is to allow boxShadows to accept  `none in addition to  `shadow.

shadow types need some refactoring. They prevent to abstract the type 'rule'.

Resolved with PR #153
I'm using a phantom type to dissociate a box or a text shadow definition.
Now, box and text shadows are rules and that allows to create these css rules:
open Css;

boxShadow(Shadow.box(~x=px(1), ~y=px(2), red));
boxShadows([Shadow.box(yellow), Shadow.box(red)]);
boxShadow(none);
boxShadow(inherit_);
boxShadow(initial);
boxShadow(unset);
important(boxShadow(none));

textShadow(Shadow.text(~blur=vh(1.), red));
textShadows([Shadow.text(yellow), Shadow.text(red)]);
etc...

This is a breaking change of course (I don't see how to do differently) and the syntax is a little bit more verbose for single statement:
boxShadow(~x=px(1), ~y=px(2), red)
vs
boxShadow(Shadow.box(~x=px(1), ~y=px(2), red));

It also resolve issue #147

Fixed in 10.0.0


Specify units for temperature

eddystone-tlm:

Beacon temperature is the temperature sensed by the beacon and expressed in a signed
8.8 fixed-point notation. If not supported the value should be zeroed.

What are the units? Degrees C? Degrees K?
If it is the former, it is ambiguous, because the "not supported" value (0) is also a valid value for an outdoor beacon.

One thing that would work is degrees C, but with "not supported" represented by -128 (0x8000).

yes, celsius. agree that 0 isn't a great default. thanks for catching this.


cmd/gofmt: better error message when gofmt is out of date

It's not uncommon to run an old gofmt that may not be up-to-date with the latest syntactic changes (most recently, alias type declarations). There may be future (Go 2) changes where this becomes more acute.
When gofmt reports a syntax error, we may want to compare the gofmt (build time) version with the current "go version" and possibly report significant version discrepancies with the error ("please update gofmt").
Reminder issue to look into this.

The issue for importers has been addressed (#25856). We're probably not going to make major syntactic changes towards Go 2 in 1.12, so postponing for next release.


Fkt2

Summary Change Description
The main purpose for creating this PR is for us to evaluate class based views (CBV), in addition to PR87 created by Rob. So I am not requesting a formal review.
Affected Issue Numbers

This could affect almost all views, as well as related urls and templates if we decided to use function based views (FBV).

Also introduced a few base.html (e.g. base_t1.html and base_t3.html ) to evaluate webpage Navbars (combing sidebar and topbar, or topbar only).

use both{% url  %}  and get_absolute_url in templates for evaluation purpose.

Code Review Notes
Rob documented his comments on CBV and FBV(PR87). I learned a bit more about CBV in the last few days and Googled CBV. A few people mentioned it's a pain to learn CBV at the beginning but switched to it after having a better understanding - a shortcut is to review Django CBV source code.
My suggestion: reading CBV methods and source code before deciding to use CBV or FBV (may use both?). For Pagination feature, it's a single line with CBV. Ordering a list is also easy with CBV, though I haven't added any code yet.

a good website for reviewing CBV methods and source code:
https://ccbv.co.uk/projects/Django/3.1/django.views.generic.list/ListView/

Use CBV to display all records, and  details for a record for each of the following model classes:
Animal, Compound, MSRun, PeakData, PeakGroup, Protocol, Sample, Study.

I learned how to use get_queryset with ListView,  in order to retrieve all or a subset of object list (at least with one filtering condition). The details are described below using Animal/Sample as examples:

example:
Models: Animal and Sample

DataRepo/views.py:
Animal :
AnimalListView and AnimalDetailView
a few lines of code (same code for most models, just changing model name)
Sample:
SampleDetailView ( similar to AnimalDetailView)
SampleListView:
add <10 lines of code for get_queryset, retrieving all samples, or samples for an animal while using the same template (sample_list.html).

DataRepo/urls.py:
mapping out the paths for the above views with and without filtering string

animal_list.html:
display all records for Animal model class with customized field names and order

animal_detail.html: display the data for a specific animal, customizing layout of the filed info (break into two sections) and link to sample list of the animal.

sample_list.html:
display all samples without filtering string in URL, but can display the sample list for an animal (based on the animal id in URL query string).

Notes:
Same strategy was used for MSRun/Peakgroup and PeakGroup/PeakData

Ran Superlinter locally. Except a few errors for DataRepo/urls.py,  most of errors are associated with downloaded files for css and templates. We may not need them after finalizing our design, but it is helpful to have them as examples at this stage.

used base_t1.html for all templates except using base_t3.html for compound list. Found it's handy to have a sidebar for browsing data.

Models didn't include latest update, but had enough data for PeakGroup/PeakData.

Checklist

[ ] All issue requirements satisfied (or no linked issues)
[ ] Linting passes.
[ ] Migrations created & committed (or no model changes)
[ ] Tests implemented (or no code changes)
[ ] All tests pass

Obsolete


Fix luma → cmyk conversion

Previously, the luma → cmyk conversion produced incorrect results.
This new conversion matches the values gained from luma → rgba → cmyk.
This PR also fixes a bug where repr(luma(0%)) printed rgb(0%, 0%, 0%, 0%).

I think @reknih should check this for sure.

Okay, I'm very sorry, I read the diff in reverse. We actually intentionally produce a mix of pigments for grays here such that full black becomes very rich black (sometimes called Photoshop black) instead of dark gray. Most likely, RGB to CMYK conversion should be fixed instead to align with this behavior. More on this here: https://www.bittbox.com/culture/photoshop-101-true-black-cmyk
We removed this commit from main. Sorry for the confusion.

Oh, that makes sense; I'll probably check later today on how this can be solved without introducing problems like these.

Okay, I'm very sorry, I read the diff in reverse. We actually intentionally produce a mix of pigments for grays here such that full black becomes very rich black (sometimes called Photoshop black) instead of dark gray.

I knew this code predated #2171 and that there was a good reason for it but I couldn't remember it.


can't auto bind this in class

I'm use Decorator autobind in front of class, but it displays undefined when the method of the class is fired.

@drdevelop please create a failure unit test


.has() api method

Am I missing something or there is no .has() api method? (similar to Map)
What I need is to distinguish between situation when there's no key in cache (has would return false) or there is a key in cache, but it's value is undefined. If I use .get(key) for this - both will return undefined.

You're not missing anything, there is no .has() method implemented yet :)
I'm busy on a paid project at the moment so won't be able to implement this right now but would happily accept a PR to Keyv + official storage adapters implementing this.

If the key expired should it return true or false? In other words, is .has() testing whether the key was set and hasn't been removed yet or whether it was set and is still fresh? Because if it's the second then you don't even need to implement it into the underlying storage adapters, in keyv's .has() you just .get() the key and return false if it's undefined.

@MySidesTheyAreGone I'm afraid that wouldn't work, if the key is fresh or stale if the value was set as undefined it'll always return undefined.
@kirillgroshkov What is your use case for this btw? This is something that's possible to implement but it seems like maybe it could be suboptimal Keyv usage in your app, e.g checking for null or undefined, true or false etc.
I Definitely agree a .has() method would be useful but just interested why you need to differentiate between explicitly setting undefined and checking it doesn't exist.
undefined as a value support was only really added for Map() compatibility, it's not something I was expecting people to store as a useful value.

Note to self, we could make undefined an illegal value to set().
Then .has() could be implemented as:
has: val => (this.get(val) !== unefined);

Deviats from the Map spec slightly as we can't set undefined, but maybe that is ok.

Closing this as it is being tracked on #36

Agree with this

@ririko5834 @dusansimic @kirillgroshkov @alphmth - we are getting really close to supporting this and it is looking like this will be out and live next week.


How to use in clustered node server (PM2)?

I'm struggling trying to implement OAuth2 in a pm2 clustered environment.  The example provided stores the new OAuthClient in a global variable.  This is great if everyone is connecting to that one server instance.  In production that is almost never the case.
I am going to assume that if process 1 and process 2 create wholly new OAuthClients, then access tokens returned to 2 will invalidate tokens in 1.  Is that correct?
If so,  I need a way to reconstitute an OAuthClient from data stored in a database.   I imagine the flow will look like this:

On connect request, check global variable is populated.
If not, grab the access and refresh tokens and expiry from the database
Recreate OAuthClient and manually and enter data from (2).  HOW?
If access token has expired, call RefreshToken() etc. ...

I have the same issue. Have you figured out a solution?

Right now I just use a single server instance.  :-/  I don't anticipate it being that bad for now because the proportion of users actually hitting QBO is very small -- just those with accounting privileges.
Eventually I will revisit the clustered environment, and imagine that it will work by deeply serializing the oauthClient into a database or local json file, check if it is there, and reconstitute the oauthClient from that data if it is.  I don't have the cycles for that at this time.
It would be nice if the API had  toJSON() and fromJSON() methods that could attempt to instantiate the client in this way, or something similar.

I treat the refresh_token as a single-use token. When a new access_token is obtained, I store the new access_token and refresh_token in a central database. In a cloud server environment where multiple instances of my program might attempt to access QuickBooks simultaneously, a race condition can occur. To address this, I serialize access to the stored refresh_token using Postgres.
I store the tokens in a table row designated for each company_id and include a column that serves as a lock when refreshing the tokens. This ensures that only one process can refresh the tokens at any given time. While I have not extensively tested this solution in a production setting, it appeared to work effectively during the limited testing conducted before the project shifted away from QuickBooks.

@abisalehalliprasan Do you know a workaround for this?


Ignore timeout curl exceptions

Description
Throw Curl exceptions Unless if they are timeout related.
As far as I can see all exceptions generated are being forwarded to slack thus I ignored these specific exceptions from Curl.class.php
Another solution would be to change the hardcoded  5 seconds timeout limit on curl.class.php.
This bit of functionality is hard to test from my side with no slack access, but it should filter properly those exceptions on curl.
Fixes ##1023

Thanks for the PR @voidfire , we'll get this reviewed! Can we show you some appreciation for the help?

Apologies for the Slack logging. While this project is open source like everything at LBRY, this project is not as commonly developed by others and not as user-friendly. If you wanted to add support for multiple logging methods configurable by config.php, we'd be happy to accept that change.
I'm unclear on why timeouts should not except, as it's ambiguous what is expected when a cURL times out. If you're looking to differentiate between timeouts and other failures, how about adding a different named exception for the timeout case?

(I read the issue after commenting. I'm a jackass.)
I understand why now (some calls to log user visits timeout), but I think it would still be better to explicitly handle this case then simply ignore them.

@voidfire @tzarebczan I'm going to close this. Still happy to merge/fix this if we can instead handle it via a specific timeout exception that can be ignored in the cases it ought to be.


Smaller environment.yml?

Looks like the environment.yml might've been generated by freezing a global system environment, since it includes a ton of things including stuff like alsa-lib and a bunch of X11 stuff.
Could we get only the actual deps of the script? I tried to go through it manually but there's too many to try it one by one.

I'll list necessary deps and add a smaller one later.

Smaller environment.yml added.

Thanks.  was still unable to build the full env.
getting stuck with the appended =pyhd8ed1ab_1 (example) specifications
tried to lookup channels that may have the packages
HOWEVER the step by step POST ENV CREATE conda install sequence you added to README works in Windows


Optimize bundles during batch upload

User driven: https://discord.com/channels/905144809105260605/905144809105260608/1131376049305948240
The user wanted to upload this bundle using our batch upload page. However, this Bundle is of type collection, and isn't set up with proper references for ingestion.
This PR does two things to make the batch upload tool more useful:

On the client, it converts non-input bundles into transaction type bundles using our convertToTransactionBundle function
It augments the convertToTransactionBundle with a topological sort, to make sure that resources are created before they are referenced. This will help more gracefully handle mal-ordered bundles

I've tested this PR with the bundle above. This should open us up to utilize much more example data from FHIR implementation guides

coverage: 94.299% (-0.02%) from 94.317% when pulling ea7018a722b2b0668e37ba404db556f7849b9173 on rahul-bundle-topoligical-sort into b0a0636d3def08a6167741617fb407a12c55edbd on main.

@codyebberson ready for re-review


K8SPS-171 PS operator 0.3.0

Test name
Status

api-tests
passed

Test name
Status

api-tests
passed

Should we merge this or not since we skipped the support for VS in 0.3.0? Maybe leave in Draft status and wait for 0.4.0? (when we should probably remove 0.2.0 files from this)


Possible compatibility with SellYourStuff

this is more of a suggestion but I wondered if this mod could be more compatible with SellYourStuff, a mod that allows you to sell company equipment back to the company, I think the reason why it doesn't work is because the spawned equipment from Remnants has no assigned store bought value and thus it just sell back for 0.
Perhaps if you could edit a config and/or items just spawn with their store prices could fix it.

Thank you for the suggestion,
I can definitely see if I can make it work, normally it should be easy and like you said It will be a config value.
The current way on how the spawned equipment/remnant items scrap cost work, I just set the scrap value to zero.
This is just a decision I made for making this mod a bit more balanced.
If everything goes well, you will see this feature be added in the next update or the one after that.

thanks you!


Clean up

Clean up! making configuration simpler.

          3 Warnings

:warning:
Package.swift was updated

:warning:
Big PR

:warning:
Please include a CHANGELOG entry.
You can find it at CHANGELOG.md

          1 Message

:book:
Ahoy, @paulofaria!! Thanks for your contribution!

  Generated by :no_entry_sign: danger

Current coverage is 94.20% (diff: 100%)

No coverage report found for master at d5ec2fa.

Powered by Codecov. Last update d5ec2fa...bd73831


Jest 20 has problems reading config through --config CLI option

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
Since switching to jest 20, it doesn't apply my exclusion patterns from the collectCoverageFrom setting, whereas jest 29 used to do this correctly.
If the current behavior is a bug, please provide the steps to reproduce and either a repl.it demo through https://repl.it/languages/jest or a minimal repository on GitHub that we can yarn install and yarn test.
I'm using this config:
{
  "collectCoverageFrom": [
    "src/**/*.{js,jsx}",
    "!src/{client,server,shared}/*.{js,jsx}",
    "!src/{client,server,shared}/{components,data,services}/*/index.js",
    "!src/client/services/analytics/**",
    "!src/shared/services/raven/**"
  ]
}

For jest 19, this config would correctly exclude the ignored files from the coverage report, whereas jest 20 still includes them in the coverage report.
jest 19: https://coveralls.io/builds/11407983
jest 20: https://coveralls.io/builds/11453060
What is the expected behavior?
For jest to apply the exclusion patterns and ignore the matching files when reporting coverage.
Please provide your exact Jest configuration and mention your Jest, node, yarn/npm version and operating system.
Jest 20
yarn 0.24.4
node 7.10.0
macos sierra

After testing some more, it seems that jest 20 does not load my .jestrc config (used in combination with jest --config .jestrc --coverage. Which did work with jest 19.

Would you mind providing a repo with this failing case of reading Jest config this way?

Hey,
Using --config .jestrc didn't work like it used to in v19.x. However it worked with a js config file even without the --config flag

We're experiencing a very terse error in the lerna project when attempting to run jest 20.0.3 with a --config option (that previously worked in jest 19):
https://github.com/evocateur/lerna/tree/upgrade-jest
After cloning and checking out the branch:
yarn
yarn integration

What it yields (after the preintegration lifecycle script runs):
$ jest --config test/config/integration.json

error Command failed with exit code 1.

test/config/integration.json
{
  "bail": true,
  "roots": [
    "<rootDir>/test/integration"
  ],
  "testEnvironment": "node",
  "setupTestFrameworkScriptFile": "<rootDir>/test/integration/_setupTestFramework.js",
  "snapshotSerializers": [
    "<rootDir>/test/helpers/serializePlaceholders.js"
  ],
  "verbose": true
}

yarn jest -- --showConfig --config=test/config/integration.json doesn't even output the config, as the default without --config does successfully.

As a workaround for now you may want to set "rootDir": "../../" or so.

@thymikee Well, it started running, but now I have a ton of really scary Maximum call stack size exceeded and UnhandledPromiseRejectionWarning spamming my console (which notably still hasn't exited since I began running the tests with the updated rootDir two minutes ago...)
$ jest --config test/config/integration.json
 FAIL  test/integration/lerna-init.test.js
  ● lerna init › initializes empty directory

    RangeError: Maximum call stack size exceeded

      at _callee$ (test/integration/lerna-init.test.js:31:33)
      at tryCatch (node_modules/regenerator-runtime/runtime.js:64:40)
      at GeneratorFunctionPrototype.invoke [as _invoke] (node_modules/regenerator-runtime/runtime.js:355:22)
      at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (node_modules/regenerator-runtime/runtime.js:116:21)
      at step (test/integration/lerna-init.test.js:8:388)
      at test/integration/lerna-init.test.js:8:548

  ● lerna init › updates existing metadata

    RangeError: Maximum call stack size exceeded

      at _callee2$ (test/integration/lerna-init.test.js:42:33)
      at tryCatch (node_modules/regenerator-runtime/runtime.js:64:40)
      at GeneratorFunctionPrototype.invoke [as _invoke] (node_modules/regenerator-runtime/runtime.js:355:22)
      at GeneratorFunctionPrototype.prototype.(anonymous function) [as next] (node_modules/regenerator-runtime/runtime.js:116:21)
      at step (test/integration/lerna-init.test.js:8:388)
      at test/integration/lerna-init.test.js:8:548

  lerna init
    ✕ initializes empty directory (2344ms)
    ✕ updates existing metadata (476ms)
    ✓ removes VERSION file (2ms)

Snapshot Summary
 › 2 obsolete snapshots found, run with `yarn run integration -- -u` to remove them.

Test Suites: 1 failed, 1 of 7 total
Tests:       2 failed, 1 passed, 3 total
Snapshots:   3 passed, 3 total
Time:        4.812s
Ran all test suites.
error Command failed with exit code 1.
[~/github/lerna] (bump-jest)$ (node:54328) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): RangeError: Maximum call stack size exceeded
(node:54328) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): RangeError: Maximum call stack size exceeded
(node:54328) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 2)
(node:54328) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 1)
(node:54330) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): RangeError: Maximum call stack size exceeded
(node:54330) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 1)
(node:54327) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): RangeError: Maximum call stack size exceeded
(node:54327) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 1)

Err, it did exit, the UnhandledPromiseRejectionWarning stuff just emitted after the terminal reappeared.

And it works when config is set from package.json?

I can run my unit tests with only the jest stanza in package.json, no --config passed.
  "jest": {
    "collectCoverageFrom": [
      "src/**/*.js"
    ],
    "modulePathIgnorePatterns": [
      "<rootDir>/test/fixtures"
    ],
    "roots": [
      "<rootDir>/src",
      "<rootDir>/test"
    ],
    "setupFiles": [
      "<rootDir>/test/helpers/env.js"
    ],
    "testEnvironment": "node",
    "testMatch": [
      "**/test/*.js"
    ]
  },

Notably, the package.json config does not run the same tests.

Hm, maybe one of your libs are conflicting with regenerator-runtime?

The version of regenerator-runtime didn't change, though, nor did any of my babelrc config etc. The only variant in this equation is jest.

@ismay it's happening the same to me, if I run yarn test -- --showConfig (where test is jest --config .jestrc) it doesn't read .jestrc file at all...

@ismay @serginator exactly same here, I have renamed my .jestrc into jest.config.js and all working back again. I have just trying to move from 19.0.1 into 20.0.3

Mmm so you can load the default config file, but what if you need another
config file for CI purposes for example? That's why I'm using --config,
because I need to have two config files, .jestrc for development and
.jestrc-ci for Jenkins :(

@serginator oh but it is working with a custom config, it's just not picking up the .jestrc (IDK why)
My change is jest --config jest.config.json and it works (doing it from an npm test)

Then I guess I have to test it tomorrow, maybe if you name the file .jestrc
it doesn't read it, but if it is a .js it works

I was having a similar issue to where jest 20.0.4 would just quit with no feedback and no tests run when I used the --config option.  Running the same command through npm at least gave me Exit status 1, but that really didn't clear up what the problem was.
My jest.config.json isn't in my project root, but I use  within it.  I didn't have a rootDir option, though.  As soon as I added a rootDir entry with a path back to where jest is run from, it started working.  This wasn't needed in jest 19, and the lack of feedback turned it into a guessing game for me to try to figure out what the problem was.

Ok i've tested it and the problem looks like if the file doesn't have .json extension, it doesn't read it. I've renamed my config files to jest.config.json and jest-ci.config.json and they both work as expected. Thanks @pjnovas!

I also ran into issues with the --config option after upgrading, and this is what I have found:

The configuration file rootDir is no longer relative to the current working directory (in which jest is called), but instead relative to the config file location. This is a good change, but I haven't seen it documented anywhere.
These changes cause module resolution errors which are thrown here, but appear to get swallowed somewhere, causing Jest to simply exit with error code 1.

tl;dr: I had a jest.config.json file with rootDir: '' in a subdirectory. Changing it to rootDir: '..' makes the tests run again.

I'm having trouble too.

If my config is called jest.config.js and has modules.export, it works.
If it's jest.config.json with plain JSON content, it does not work.
.jestrc does not work.
Putting the config in the jest key in package.json works.

Running jest --showConfig correctly shows my root directory, but it just doesn't seem to want to use the .jestrc file. It's strange, but I've simply added the config to the package.json and it's no big deal. But it's a little confusing because I believe the two failing options I mentioned are supposed to work, right? (I'm new to Jest, so I'm not sure what the Jest 19 behavior was.)

Can you try with jest@test.

After installing jest@test, my npm list looks like this:
    ├─┬<EMAIL_ADDRESS>│ └─┬<EMAIL_ADDRESS>│   └──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>invalid
│ └──<EMAIL_ADDRESS>extraneous
npm ERR! invalid<EMAIL_ADDRESS>/Users/msikma/Projects/reisan/reisan-application/packages/reisan-backend-api/node_modules/jest
npm ERR! extraneous<EMAIL_ADDRESS>/Users/msikma/Projects/reisan/reisan-application/packages/reisan-backend-api/node_modules/jest-cli
npm ERR! extraneous<EMAIL_ADDRESS>/Users/msikma/Projects/reisan/reisan-application/packages/reisan-backend-api/node_modules/node-pre-gyp
npm ERR! extraneous<EMAIL_ADDRESS>/Users/msikma/Projects/reisan/reisan-application/packages/reisan-backend-api/node_modules/ajv-keywords/node_modules/ajv

Test results are identical to my previous comment.
I think it's fair if I give more information about my setup, just for completeness:

package.json (note: tests with .jestrc etc. were done with the jest section removed)

I'm running a Lerna multi-package project, which has its own package.json (with nothing in it except Lerna itself), but I ran the same tests with the package outside of the multi-package setup and the results were the same.

Only jest.config.js and jest key in package.json are automatically taken as your config. for .jestrc you need to specify --config flag. E.g. jest --config=.jestrc

My bad, as I mentioned before I was under the mistaken impression that this would work out of the box. I was wrong about that. No problem, sticking with the regular methods is fine.
More to the point, I'm not getting the same error when using --config as other people, with either jest@test or 20.0.4.

hate this bug to :ballot_box_with_check:

For those having silent failures, check that you're node version is > 6.

Observation:
When I'm running Jest and specifying a config file via the --config option and I try to the <rootDir> token in the transforms section, I get the following error:

I am able to use <rootDir> in the testMatch section.  However, in this caserootDir is replaced with the path where the config is located.
Jest version : 21.2.1

Closing as from what I understand the issue is fixed. Happy to reopen if anybody still has errors

I still got an issue! If you place config in .jestrc it could not find any test. However if you add .json extension to config file, it works.

Also having this issue with a config file with .js as the extension. Adding rootDir: '..', to the config as someone suggested above works.


Ask for more detail document about deploy application to children-clusters

This ideal is awesome and perfect.
Can you describe more detail about that the flow process from accept yaml to scheduling and deployment applications to children-clusters?
Such as, How to deploy Describtion to children-clusters?  How to distribute Subscription to Base?
Give me some pointers idea and related code function about these, I will learn it.
Thanks a lot.

@victory460 Thanks for your interest.
Here is a Chinese document that briefly talk about the application deployments.
And also you can learn these concepts from https://clusternet.io/zh-cn/docs/concepts/.

Give me some pointers idea and related code function about these, I will learn it.

You can find all the controllers https://github.com/clusternet/clusternet/tree/main/pkg/controllers/apps. These controllers will reconcile specific objects and populate new objects. This is quite similar with deployment, replicaset controllers in Kubernetes.

@victory460 Also you can try to search Wechat official account Clusternet, where you can receive some new articles and chart with me.

Thanks!

@victory460 Also you can try to search Wechat official account Clusternet, where you can receive some new articles and chart with me.


chore(drawer): upgrade to mdcw 2.x

related to #885

Codecov Report

Merging #891 into rc0.13.0 will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           rc0.13.0     #891   +/-   ##
=========================================
  Coverage     93.98%   93.98%
=========================================
  Files            85       85
  Lines          3576     3576
  Branches        560      560
=========================================
  Hits           3361     3361
  Misses           90       90
  Partials        125      125

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a83e71...369d884. Read the comment docs.

@lucasecdb I just checked https://github.com/material-components/material-components-web-react/issues/885...it looks like list has a breaking change. What is it that we need to fix?

I think it's more like a "usability" breaking change, the API is still the same, so I think we are good

Yes it looks like this was the PR:
https://github.com/material-components/material-components-web/pull/4568/files.
We are good!
On Tue, Jun 11, 2019 at 10:36 AM Lucas Cordeiro<EMAIL_ADDRESS>wrote:

I think it's more like a "usability" breaking change
https://github.com/material-components/material-components-web/blob/master/CHANGELOG.md#breaking-changes,
the API is still the same, so I think we are good
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/material-components/material-components-web-react/pull/891?email_source=notifications&email_token=AAENSIKWBSNE3BW3DNDL7LTPZ7PDBA5CNFSM4HPXIVT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXN5T2Q#issuecomment-500947434,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAENSILKOAYQGTCQ6GDV6J3PZ7PDBANCNFSM4HPXIVTQ
.

i just rebased with master, and it messed this PR up :(

oh let me try to fix it

can you update once more? I had to rebase again.

@lucasecdb pinging the PR :)

I think I lost some commits in between those force pushes :-(

Ah yeah :( Sorry I made this so complicated.

its ok, git is kinda hard when you are trying to rebase into a branch that has been updated


[filemanager] replace euidaccess() with opendir()

euidaccess implementation in glibc directly compares effective uid/gid to Unix permissions of file and does not take Posix ACLs into account.
Since we only need this check to display a meaningful error message, call opendir instead and check if it succeeds.

Tested on Xperia 10 II and works fine.


not working

dyld: Library not loaded: @loader_path/../../../../opt/theora/lib/libtheoraenc.1.dylib
Referenced from: /usr/local/opt/ffmpeg/lib/libavformat.60.dylib
Reason: image not found
zsh: abort      scrcpy
Please help, its not working for me.

How did you install it?

I'm meeting the same issue after trying to update my scrcpy to the latest version by running "brew install scrcpy" in terminal

Reinstalling brew, scrcpy and android tools solved it for me as the answers from here suggested: https://github.com/Genymobile/scrcpy/issues/3505

I used brew to install and i re-installed brew scrcpy and android tools too still same issue


[Feature][seaTunnel-Web] support schedule

Search before asking

[X] I had searched in the feature and found no similar feature requirement.

Description
Does the community currently have plans to support using scheduling directly inside the seatunnel web?
Usage Scenario
No response
Related issues
No response
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi, now the seatunnel support restapi and shell to submit job, maybe you can use other scheduler tool to scheduler the job.

之前有支持发布任务到dolphinscheduler吧？现在怎么不支持了？


kind: fix script var scope

Description
since the IMG var is also used without going through the if/branch in the main scripts few lines below
https://github.com/kubeflow/model-registry/blob/a12fd00d67b0e8c002cd5e021d8697a3fdb168da/scripts/deploy_on_kind.sh#L40-L41
fix variable scope by providing the default globally.
How Has This Been Tested?
 ./scripts/deploy_on_kind.sh

Merge criteria:

All the commits have been signed-off  (To pass the DCO check)

[x] The commits have meaningful messages; the author will squash them after approval or in case of manual merges will ask to merge with squash.
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work.
[x] Code changes follow the kubeflow contribution guidelines.

If you have UI changes

[ ] The developer has added tests or explained why testing cannot be added.
[ ] Included any necessary screenshots or gifs if it was a UI change.
[ ] Verify that UI/UX changes conform the UX guidelines for Kubeflow.

wdyt @Al-Pragliola ?

nice catch, /lgtm


SNOW-1359484 Allow write Snowpark pandas DataFrame and Series in sess…

…ion.write_pandas

Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

Fixes SNOW-1359484

Fill out the following pre-review checklist:

[ ] I am adding a new automated test(s) to verify correctness of my new code
[ ] I am adding new logging messages
[ ] I am adding a new telemetry message
[ ] I am adding new credentials
[ ] I am adding a new dependency

Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.

Allow user to use session.write_pandas to write Snowpark pandas objects:

It reuse and does not change the interface of write_pandas
it reuse to_snowflake to implement the write logic
copied main tests from test_pandas_to_df.py and use them to test writing Snowpark pandas DataFrame.

It would be nice to have a test that validates the naming conventions as well to make sure the quote_id code paths are working correctly.

I have integ tests to cover the case where quote_id is true or false, and the code is basically the same as the existing code: https://github.com/snowflakedb/snowpark-python/blob/cc3d2898d4170cc68ca8d59a7872e9c2b1ba7929/src/snowflake/snowpark/session.py#L2219

It's also surprising to me that new functionality would not require a CHANGELOG update.
The change log is added in pandas_changelog.md. In the next release, we will merge them together so this is not in changelog.md.


Fix Handlebars grammar

The Handlebars repository contains the following grammar files:

Handlebars.tmLanguage
Handlebars.JSON-tmLanguage
grammars/handlebars.json (symlink to Handlebars.JSON-tmLanguage)

We were processing the .tmLanguage and the .json file and complaining about a duplicate scope. This triggered a nil-handling bug (which is now fixed), but we also shouldn't complain in this case at all. Having the same grammar in multiple formats is OK.
/cc @arfon https://github.com/github/linguist/commit/6062d3b25cbe99b8660724250436eeb249483119#commitcomment-11249896

:zap: thanks @aroben


chmod of home directory : "permission denied" on overlayfs, works with aufs (OSX / DockerMachine)

(docker-machine version: 0.7.0; VirtualBox version: 5.0.16 r105871; OSX version: 10.11.4 (15E65))
A newly created user can't chmod their home directory if the storage-driver is overlayfs.  It works perfectly with aufs though.
This isn't restricted to home directories.  Users can not change permissions on the 'top-most' directory they own.
e.g. Given A / B / C  (with A owned by root, and B and C owned by someuser), someuser can change permissions on C, but not on B
Note that there are no host-mounted directories, volumes, etc.  This is with a default/baseline ubuntu container...
To test this
Overlay
Create the machine
paglierino:mahesh$ docker-machine create -d virtualbox --engine-storage-driver=overlay dev
Connect to the machine, and start up ubuntu
paglierino:mahesh$ docker-machine ssh default
docker@default:~$ docker run -it ubuntu /bin/bash

Add a user
root@61448047d92a:/# adduser --disabled-password --uid 501 --home /var/lib/postgresql --shell /bin/bash --gecos 'PostgreSQL administrator' postgres
Adding user `postgres' ...
Adding new group `postgres' (501) ...
Adding new user `postgres' (501) with group `postgres' ...
Creating home directory `/var/lib/postgresql' ...
Copying files from `/etc/skel' ...

Try to change permissions for that user, and fial
root@61448047d92a:/# su - postgres
postgres@61448047d92a:~$ pwd
/var/lib/postgresql
postgres@61448047d92a:~$ ls -ald .
drwxr-xr-x 2 postgres postgres 4096 Apr 16 20:16 .
postgres@61448047d92a:~$ chmod 700 .
chmod: changing permissions of '.': Operation not permitted
postgres@61448047d92a:~$ ls -ald .
drwxr-xr-x 2 postgres postgres 4096 Apr 16 20:16

aufs
Create the machine
paglierino:mahesh$ docker-machine create -d virtualbox --engine-storage-driver=aufs notdev
Connect to the machine, and start up ubuntu
paglierino:mahesh$ docker-machine ssh notdev
docker@default:~$ docker run -it ubuntu /bin/bash

Add a user
root@9ec0a58c827e:/# adduser --disabled-password --uid 501 --home /var/lib/postgresql --shell /bin/bash --gecos 'PostgreSQL administrator' postgres
Adding user `postgres' ...
Adding new group `postgres' (501) ...
Adding new user `postgres' (501) with group `postgres' ...
Creating home directory `/var/lib/postgresql' ...
Copying files from `/etc/skel' ...

Try to change permissions for that user, and profit
root@1f32cac40882:/# su - postgres
postgres@1f32cac40882:~$ ls -ald .
drwxr-xr-x 2 postgres postgres 4096 Apr 16 20:15 .
postgres@1f32cac40882:~$ pwd
/var/lib/postgresql
postgres@1f32cac40882:~$ chmod 700 .
postgres@1f32cac40882:~$ ls -ald .
drwx------ 2 postgres postgres 4096 Apr 16 20:15 .

Hi @dieswaytoofast, thanks for the issue and the detailed steps to reproduce.  I think it's an upstream Engine issue and not a Machine issue.  It seems that the result should be the same on any system running Docker.  Please re-open the ticket at https://github.com/docker/docker/issues/new and the maintainers there may be able to help you.

This works for me on overlay, on a host not created with machine (docker4mac), so I think something else is causing the failure. Can you strace the failing chmod?

@justincormack I get:
postgres@768f5591ab0c:~$ strace chmod 700 .
strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Operation not permitted
strace: test_ptrace_setoptions_for_all: unexpected exit status 1

Maybe I need more cap in the container?

Alright yeah here we go.  Running with --privileged allowed strace to actually work.
postgres@c38c69c41ae1:~$ strace chmod 700 .
execve("/bin/chmod", ["chmod", "700", "."], [/* 13 vars */]) = 0
brk(0)                                  = 0x1f9d000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8f18cd6000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=9703, ...}) = 0
mmap(NULL, 9703, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f8f18cd3000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\37\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1840928, ...}) = 0
mmap(NULL, 3949248, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f8f186f1000
mprotect(0x7f8f188ac000, 2093056, PROT_NONE) = 0
mmap(0x7f8f18aab000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1ba000) = 0x7f8f18aab000
mmap(0x7f8f18ab1000, 17088, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8f18ab1000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8f18cd2000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8f18cd0000
arch_prctl(ARCH_SET_FS, 0x7f8f18cd0740) = 0
mprotect(0x7f8f18aab000, 16384, PROT_READ) = 0
mprotect(0x60c000, 4096, PROT_READ)     = 0
mprotect(0x7f8f18cd8000, 4096, PROT_READ) = 0
munmap(0x7f8f18cd3000, 9703)            = 0
brk(0)                                  = 0x1f9d000
brk(0x1fbe000)                          = 0x1fbe000
umask(0)                                = 02
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
fchmodat(AT_FDCWD, ".", 0700)           = -1 EPERM (Operation not permitted)
open("/usr/lib/charset.alias", O_RDONLY|O_NOFOLLOW) = -1 ENOENT (No such file or directory)
write(2, "chmod: ", 7chmod: )                  = 7
write(2, "changing permissions of '.'", 27changing permissions of '.') = 27
write(2, ": Operation not permitted", 25: Operation not permitted) = 25
write(2, "\n", 1
)                       = 1
close(1)                                = 0
close(2)                                = 0
exit_group(1)                           = ?
+++ exited with 1 +++

Thats not very helpful. Which kernel is this?

@justincormack 4.1.19
Linux default 4.1.19-boot2docker #1 SMP Thu Apr 7 02:41:05 UTC 2016 x86_64 GNU/Linux

It does look very much like https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1555997 but in a different kernel version. I wonder if the bug got backported? You could test not in a container with this https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1555997/comments/3

(docker for mac seems fine, but it switched to 4.4 series kernel recently, used to be almost same kernel as boot2docker)

I'll try that minimal dupe and see if I get same results

Yup.  It's a 🐞 upstream
docker@default:~$ mkdir /tmp/lower /tmp/upper /tmp/work /tmp/merged
docker@default:~$ sudo mount -t overlay overlay -o lowerdir=/tmp/lower,upperdir=/tmp/upper,workdir=/tmp/work /tmp/merged
docker@default:~$ sudo touch /tmp/merged/testfile
docker@default:~$ sudo chown +x /tmp/merged/testfile
chown: unknown user +x
docker@default:~$ sudo chown $USER:$USER /tmp/merged/testfile
docker@default:~$ chmod +x /tmp/merged/testfile
chmod: /tmp/merged/testfile: Operation not permitted

@tianon FYI

Lovely -- think it's time we bumped boot2docker to the 4.4 series too? :smile:
(It's a longterm, just like 4.1.)

@tianon SGTM

Since we're working on this elsewhere, going to go ahead and close.  Thanks @dieswaytoofast  @justincormack @tianon  for your effort.


Fix types for printing command errors

High-level description
A change from the etcd code derived from ZK code was wrongly applied backj to the ZK code. This PR makes things good again. And simplifies the etcd code slightly too.
Corresponding DC/OS tickets (required)

D2IQ-66238 ZK backup script decodes command output incorrectly

Related tickets (optional)

D2IQ-ID JIRA title / short description.

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain.
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

@mesosphere-mergebot label Ready For Review

@mesosphere-mergebot request-review @wavesoft

@mesosphere-mergebot changelog-not-required unreleased code


Update for 2.0

app is replaced with a default 2.0.0-preview2 'dotnet new mvc'.
templates/aspnet-s2i-template.json is removed -> replaced by dotnet-example in s2i-dotnetcore repo
readme points to s2i-dotnetcore repo for instructions how to use this in OpenShift

@jerboaa I want your ok for removing the template and readme updates. Then we can merge this.
We can verify the app works ok once we have an s2i image. So no need to have a close look at the app changes.

You want to merge this into master, if I see this correctly. Perhaps once we nave an image we should be sure it builds.

Yes merge in master and then create a dotnet20 branch from this.
I'm ok with leaving this PR open until we can verify it builds with the s2i image.

dotnet new mvc claims it's using C# and F#.

You can select one using the lang option. The default is c#.
.net doesn't support using multiple languages in a single project.

@tmds I can confirm that this works with s2i and the new 2.0 images.


Value cannot be null. (Parameter 'clientSecret') when sending an activity from a bot

We are trying to send an activity directly to our bot using the /api/messages endpoint for our bot. We are doing this so that we can have 2 Twilio numbers associated with our bot.
First, we get an access token for the bot with the following code:
var authClient = new HttpClient();
var authRequest = new HttpRequestMessage
{
    Method = HttpMethod.Post,
    RequestUri = new Uri($"https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token"),
    Content = new StringContent($"grant_type=client_credentials&client_id={BOT_MIRCOSOFT_APP_ID}&client_secret={BOT_MICROSOFT_APP_PASSWORD}&scope=https%3A%2F%2Fapi.botframework.com%2F.default", Encoding.UTF8, "application/x-www-form-urlencoded"),
};
var authResponse = await authClient.SendAsync(authRequest).ConfigureAwait(false);

We then put this access token in the header of our api call to the bot and send the bot an activity through the /api/messages endpoint. The activity reaches the bot and everything works until we try to send an activity out from the bot. Our AdapterWithErrorHandler class catches an error with the message: Value cannot be null. (Parameter 'clientSecret'). We have searched thoroughly and could not find where this clientSecret variable is.
Here is the stack trace for the error:
   at Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential..ctor(String clientId, String clientSecret)
   at Microsoft.Bot.Connector.Authentication.MicrosoftAppCredentials.<BuildAuthenticator>b__14_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Microsoft.Bot.Connector.Authentication.AppCredentials.<BuildIAuthenticator>b__34_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.get_Value()
   at Microsoft.Bot.Connector.Authentication.AppCredentials.<GetTokenAsync>d__32.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Connector.Authentication.AppCredentials.<ProcessHttpRequestAsync>d__31.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Bot.Connector.Conversations.<ReplyToActivityWithHttpMessagesAsync>d__10.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Connector.ConversationsExtensions.<ReplyToActivityAsync>d__17.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at Microsoft.Bot.Builder.BotFrameworkAdapter.<SendActivitiesAsync>d__34.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Bot.Builder.TurnContext.<>c__DisplayClass25_0.<<SendActivitiesAsync>g__SendActivitiesThroughAdapter|1>d.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Builder.TurnContext.<SendActivityAsync>d__24.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at SampleBot.Bots.EchoBot.<OnMessageActivityAsync>d__0.MoveNext() in C:\Users\myuser\source\repos\SampleBot\SampleBot\Bots\EchoBot.cs:line 19
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Builder.ActivityHandler.<OnTurnAsync>d__0.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Builder.BotFrameworkAdapter.TenantIdWorkaroundForTeamsMiddleware.<OnTurnAsync>d__0.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Builder.MiddlewareSet.<ReceiveActivityWithStatusAsync>d__3.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Bot.Builder.BotAdapter.<RunPipelineAsync>d__18.MoveNext()

This error is keeping us from sending an activity out from our bot so any information is helpful.

@andrew-j-frank - Thank you for the report. We prefer that you ask how-to questions on Stack Overflow. Here's the blurb you should see when you raise a new issue in this repo:

🚨 The issue tracker is not for questions 🚨
If you have a question, please ask it on https://stackoverflow.com/questions/tagged/botframework

I will close this here so it can be handled on Stack Overflow: https://stackoverflow.com/questions/62880211/value-cannot-be-null-parameter-clientsecret-when-sending-an-activity-from-a


Pipeline Name is Ambiguous

Currently the Pipeline name that is used is ambiguous in its usage within Faros.

Pipeline Management within Faros App:

Current value is set here:
https://github.com/faros-ai/faros-cicd-github-action/blob/main/src/main.ts#L89
To me it makes more sense for it to be something like this:
https://github.com/faros-ai/faros-cicd-github-action/blob/main/src/main.ts#L76

Pipelines no longer exposed in the UI in this way


Large complex sheet may show no highlighting

Reproduction

row-highlighter version 1.1.2 March 15 2021
MacBook Pro (15-inch, 2018), 2.9 GHz 6-Core Intel Core i9, 32 GB 2400 MHz DDR4
MacOS 11.2.2
Firefox 86.0.1
Disabled uBlock Origin extension
Open https://docs.google.com/spreadsheets/d/1Rq3RCYELRs95yISdlrkcUP-rNG4kZN7k3APNlaKDe6I/edit#gid=0 example
OPTIONAL: Reload page using browser Reload button

Expected

See highlighted rows and columns

Actual

See normal color, white, without highlight colors

Workaround

Simplify the text in Column "A" to "hello" for all Rows (paste all)

Discussion

May be a timing issue
Is difficult to make a spreadsheet that does not show highlighting correctly. I believe the complexity of this example sheet is the reason it does not render correctly.

Thanks for the detailed report!
I have opened the example sheet but it is highlighted.
It may take some time to resolve this issue.
I will continue to investigate.

@nrichards
Sorry for the delay.
With the recent release of v2, I've updated the logic for highlighting, which should resolve the problem.
If you're still experiencing issues, please don't hesitate to create a new issue so I can investigate further.
I appreciate your feedback and thank you for bringing this to my attention!


Out of Direct Memory because of IdleStateHandler

Expected behavior
When I use netty as a server, I use IdleStateHandler to keep heartbeat for each client. When a lot of clients connected the server should works fine.
Actual behavior
When a lot of clients connected, the server raised OutOfDirectMemoryError.
Steps to reproduce

Create instance of IdleStateHandler to make readeridle time bigger than 0.
new IdleStateHandler(60, 0, 0, TimeUnit.SECONDS)
Connect a lot of clients to the server.

Minimal yet complete reproducer code (or URL to code)
Netty version
4.1.6 Final
JVM version (e.g. java -version)
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
OS version (e.g. uname -a)
Linux PW6J1216 3.0.101-63-default #1 SMP Tue Jun 23 16:02:31 UTC 2015 (4b89d0c) x86_64 x86_64 x86_64 GNU/Linux
######################################################
I found this issue is because of the state was not well protected in multi-thread enviroment.
The code io.netty.handler.timeout.IdleStateHandler.java(from version 4.1.6 to netty-all-5.0.0.Alpha2):
`
 private void initialize(ChannelHandlerContext ctx) {
    switch (state) {   // Mark A. Here can reach many times
    case 1:
    case 2:
        return;
    }

   state = 1;   // Mark B. Here can reach many times

    EventExecutor loop = ctx.executor();

    lastReadTime = lastWriteTime = System.nanoTime();
    if (readerIdleTimeNanos > 0) {
        readerIdleTimeout = loop.schedule(
                new ReaderIdleTimeoutTask(ctx),  // Mark C. Here can reach many times
                readerIdleTimeNanos, TimeUnit.NANOSECONDS);
    }
    if (writerIdleTimeNanos > 0) {
        writerIdleTimeout = loop.schedule(
                new WriterIdleTimeoutTask(ctx),  // Mark D. Here can reach many times
                writerIdleTimeNanos, TimeUnit.NANOSECONDS);
    }
    if (allIdleTimeNanos > 0) {
        allIdleTimeout = loop.schedule(
                new AllIdleTimeoutTask(ctx), // Mark F. Here can reach many times
                allIdleTimeNanos, TimeUnit.NANOSECONDS);
    }
}

`
When code executing in multi-thread ENV, many threads can reach the place of Mark A/Mark B at the same time(in both version of 4.1.6 and even the lastest version netty-all-5.0.0.Alpha2), so the place mark as Mark C/D/F can reach many times. I found code changed in netty-all-5.0.0.Alpha2, the member state changed from "private int state" to "private volatile int state", but it still not multi-thread safe. I prefer to change it to "private AtomicInteger state = new AtomicInteger(0)", here is my code:
`
public class IdleStateHandler extends ChannelDuplexHandler {
...
private AtomicInteger state = new AtomicInteger(0); // 0 - none, 1 - initialized, 2 - destroyed
...
private void initialize(ChannelHandlerContext ctx) {
if(state.compareAndSet(0, 1)) {
EventExecutor loop = ctx.executor();
        lastReadTime = lastWriteTime = System.nanoTime();
        if (readerIdleTimeNanos > 0) {
            readerIdleTimeout = loop.schedule(
                    new ReaderIdleTimeoutTask(ctx),
                    readerIdleTimeNanos, TimeUnit.NANOSECONDS);
        }
        if (writerIdleTimeNanos > 0) {
            writerIdleTimeout = loop.schedule(
                    new WriterIdleTimeoutTask(ctx),
                    writerIdleTimeNanos, TimeUnit.NANOSECONDS);
        }
        if (allIdleTimeNanos > 0) {
            allIdleTimeout = loop.schedule(
                    new AllIdleTimeoutTask(ctx),
                    allIdleTimeNanos, TimeUnit.NANOSECONDS);
        }
    }
}

private void destroy() {
    state.set(2);

    if (readerIdleTimeout != null) {
        readerIdleTimeout.cancel(false);
        readerIdleTimeout = null;
    }
    if (writerIdleTimeout != null) {
        writerIdleTimeout.cancel(false);
        writerIdleTimeout = null;
    }
    if (allIdleTimeout != null) {
        allIdleTimeout.cancel(false);
        allIdleTimeout = null;
    }
}

...
}`

@LiangKongRong HI, Do you use IdleStateHandler in shared mode or create new instance per channel ?

@LiangKongRong like @amizurov suggested this sounds more like a missusage. Can you you confirm how you use it ?

Thanks @amizurov , new instance per channel.
Thanks @normanmaurer .
We use IdleStateHandler when connecting request recieved. The code is something like ctx.channel().pipeline().addFirst(new IdleStateHandler(600, 0,0)) ; in the channelRead0() method of a sub class of SimpleChannelInboundHandler. When 160K clients connected, normally the instance number of
ReaderIdleTimeoutTask should be 160K because nubmer of IdleStateHandler is 160K, but I got 16000K ReaderIdleTimeoutTask. Finally when I fixed this with my code, ReaderIdleTimeoutTask drop to 160K.
Sorry, Out of direct memory was not because of this bug. I am still investingating for the root. I think it may be cause by to many direct memory pooled in the stack.

@LiangKongRong are you sure this issue is still present in 4.1.32.Final which is the latest version of netty that was released ?.
initialize(..) and destroy(...) should never be called concurrently and always from the same thread. So I don't see why you would need an atomic here.

@normanmaurer We add a new IdleStateHandler() in channelActive(ChannelHandlerContext ctx) method of  another handler, I am not sure if it was misused. I see most code add handlers in ChannelInitializer. I will check this in version 4.1.32.Final later. Thanks.

@normanmaurer I found the root cause of Out Of Direct Memory. We send BinaryWebSocketFrame by writeAndFlush() of a nio channel in my thread, and direct memory allocated in this thread. But when the message sent out, the direct memory cached inside the netty event loop thread. Direct memory exhaust when a lot of direct memory allocated in my thread and cached in netty event loop thread. I fixed my code to make writeAndFlush() executing inside event loop thread. But I think it will better if writeAndFlush() can do something to make it safer. Unspoken rules is not user friendly.

@amizurov if you want the read timeout to be per channel it cannot be shared because the handler has a state.

@johnou, sorry the message above to me ?

@normanmaurer If I correct understand that different between execution in eventLoop and in separate thread that we create WriteAndFlushTask.newInstance(next, m, promise); and reserved memory for this taskctx.pipeline.incrementPendingOutboundBytes(task.size); so this is can change channel writability, maybe this is root of problem ?

@amizurov I was just pointing out that if you want to d/c clients that stop writing to the socket a new IdleStateHandler is required per session to detect this.

@normanmaurer Merry Christmas. I am sorry, most of the instances of io.netty.util.concurrent.ScheduledFutureTask is unreachable, I use JVisualVM tool inside JDK before it cannot identify reachable/uncheachable state. I used another tool (Eclipse MAT), the number of reachable instances is normal. But I have another question: It seems schedule new task in IdleStateHandler by method schedule() instead for scheduleAtFixedRate() or scheduleWithFixedDelay() can make the timeout most precise, but as a result many many tasks created. In my application 40% instances are io.netty.util.concurrent.ScheduledFutureTask and io.netty.util.concurrent.PromiseTask$RunnableAdapter, I think it increase the precessure of GC. If application can except timeout not so precise, scheduleAtFixedRate() or scheduleWithFixedDelay() would be better choise.
Thanks everybody too. @amizurov @johnou @medusar

@LiangKongRong I am happy to review a PR ... Please provide one if you think you can improve it.