Oblitirate test flakiness

Fixes #683
Fixes #467
Also, #736 did a good job for us and:
Closes https://github.com/celestiaorg/celestia-node/issues/582
Closes https://github.com/celestiaorg/celestia-node/issues/608
Hopefully closes https://github.com/celestiaorg/celestia-node/issues/584
Hopefully closes https://github.com/celestiaorg/celestia-node/issues/579
Bonus: https://github.com/celestiaorg/celestia-node/issues/476
(it does not really fix this #476, but it fixes the flakeyness that was mixed up with #476, while #476 is another one that was not reproduced for a long time, and I assume it is now fixed)

Codecov Report

Merging #756 (5977b76) into main (2d50db4) will increase coverage by 0.17%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #756      +/-   ##
==========================================
+ Coverage   52.63%   52.81%   +0.17%
==========================================
  Files         116      116
  Lines        6573     6574       +1
==========================================
+ Hits         3460     3472      +12
+ Misses       2751     2742       -9
+ Partials      362      360       -2

Impacted Files
Coverage Δ

core/testing.go
83.14% <ø> (ø)

service/share/testing.go
100.00% <0.00%> (ø)

fraud/bad_encoding.go
34.44% <0.00%> (+3.33%)
:arrow_up:

das/daser.go
75.33% <0.00%> (+4.74%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2d50db4...5977b76. Read the comment docs.

From an additional run, one guy failed, fixing him

Ok, 3 more times, and it passes. Should be enough

Ok, I accidentally pushed runtime.GOMAXPROCS(1), which fixed heightSub on CI completely, but it is cheating

With some runtime tricks I was able tor reproduce and fix all the flakes, but this one is not and it makes me insane

ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)

maybe try increasing the timeout there? I think if we rely on timeouts we will always run into some flakines no?

The test does not fail because of the timeout.

maybe try increasing the timeout there? I think if we rely on timeouts we will always run into some flakines no?

I put global timeouts in general for tests with concurrency. If they deadlock for some flakey or other reason.

I think the heightSub test is fixed forever with the time sleep. However https://github.com/celestiaorg/celestia-node/issues/579 showed up again lol

I ran CI 6 times and on the seventh time, it failed with something I didn't see before.

https://github.com/celestiaorg/celestia-node/issues/759 But it is unrelated to this PR so I am merging this


Implement ANI atomic symmetry functions.

Three changes to the atomic symmetry functions are made:
New parameters for the symmetry functions
The radial symmetry function can take another sequence of parameter values (Rcs), that control the distance probed by the symmetry function. For the angular symmetry functions, a new ANI-specific function is introduced. It takes additional parameter values that let the function probe specific angular (thetas) and distance (Rcs) ranges.
Allow the number of parameter values to differ
Another change is to allow the number of supplied parameter values to be different for each parameter type. Previously, for say the angular symmetry function, the same number of eta, lambda, and zeta parameters would have to be supplied. Now, however, as many parameters can be supplied for each argument as desired, and the vector
output by the function will have one element for each combination of parameter values.
Make the atomic symmetry functions JITable when species is dynamically supplied
Previously, only the functions output by the higher-order functions radial_symmetry_functions and angular_symmetry_functions_ani could be JITed, but those output functions only took a set of coordinates as an input, since the atomic species types were already supplied to the higher-order function and thus "baked in". Now, however, the higher-order functions themselves can be JITed, so it is possible to have a JITed function that takes a both a list of species types and a set of coordinates as input.
This is done by calculating a mask based on the species types and applying it to a matrix of symmetry function values calculated across all the atoms, instead of using fancy indexing based on the species type.

Working on fixing the failing Travis CI checks.


stream/group settings: Improve error strings.

Use proper punctuation.
Make errors friendlier.

The check is failing because the files are not linted.

Fixed the linter error and marked to merge once CI passes.


ONLY AWS? KNATIVE MIGRATION POSSIBLE?

Hi all, I appreciate your effort in developing InfiniCache, and I wanted to kindly ask the authors what do they think about the migration of such serverless-based distributed in memory cache to a more portable environment such as Knative, which allows you to deploy serverless workloads using K8s. I am evaluating the possibility of implementing this idea and I would like to know your opinion if possible. Thanks in advance for any possible insight you can give me

Porting InfiniCache to open-source serverless platforms is definitely doable, with less hassles and glitches. One thing that makes such an implementation relatively easier is you essentially gain the "server" part, where a Knative function could run as a short-lived server that can accept and serve inbound connections. Whereas in Lambda-based InfiniCache, a significant portion of the code is for handling and walking around these Lambda-inherent limitations.
Another thing is the applications. InfiniCache sitting on public clouds could save money for the cloud tenants who opt to use the InfiniCache service.  We are also looking for application scenarios like this. Any comments or suggestions would be much appreciated towards this angle.

Hi @tddg and thanks for the answer. I understand what you are saying. I understand that components such as the client may be easily used as it is, but many other components and files import AWS specific libraries to interact with the cluster. Do you think that, the API calls to AWS which are contained in InfiniCache, can be substituted with some other API calls? Given that there is no documentation, can I ask you (given that you are the authors) which could be the best way to start understanding and then migrating such project in Knative? I have of course red the original paper, and I understand more or less the code structure (files, dirs etc) but I would like to replace all the API calls to AWS with Knative. Thanks

Dear @tddg and and other infinicache developers/authors. Thanks a lot for the usefull suggestions. I am really interested to start such migration, For this reason I am analyzing the implementation and I came up with few considerations and many doubts about what the specific packages do. First of all I would like to tell to you what I understood and I would like to kindly ask if would be possible for you to correct what I missunderstood and please complete some information if uncomplete.
Implementation Description
The implementation is written in Go and is composed by a module github.com/mason-leap-lab/infinicache which contains 10 packages. The project go.mod file shows that it uses 13 external libraries, of which 5 indirect.
The directly imported libraries are:

github.com/ScottMansfield/nanolog v0.2.0 - nanosecond scale logger
github.com/aws/aws-lambda-go v1.13.3 - tools to help Go developers develop AWS Lambda functions
github.com/aws/aws-sdk-go v1.28.10 - AWS SDK for the Go programming language
github.com/buraksezer/consistent v0.0.0-20191006190839-693edf70fd72 - provides a consistent hashing function
github.com/cespare/xxhash v1.1.0 - Go implementation of the 64-bit xxHash algorithm
github.com/cornelk/hashmap v1.0.1 - lock-free thread-safe HashMap
github.com/google/uuid v1.1.1 - generates and inspects UUIDs
github.com/klauspost/reedsolomon v1.9.3 - Reed-Solomon Erasure Coding in Go
github.com/seiflotfy/cuckoofilter v0.0.0-20200106165036-28deee3eabd7 - bloom filter that efficiently allows also deletion
github.com/mason-leap-lab/redeo v0.0.0-20200204234106-1e6f10c82f05 - for building redis-protocol compatible servers/services
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b - allows to create ANSI colored strings and codes
github.com/onsi/ginkgo v1.7.0 - testing
github.com/onsi/gomega v1.4.3 - matcher lib for ginko

The packages which compose this implementation of infinicache are:

global - contains procedures for global configuration (set ip address, ports, etc..) and is composed by the following files:

infinicache-master/proxy/global/global.go
infinicache-master/proxy/global/ip.go The global package does not depend on AWS

client contains the client protocol and is composed by the following files:

infinicache-master/client/client.go
infinicache-master/client/ec.go
infinicache-master/client/ecRedis.go
infinicache-master/client/log.go

Moreover the client implementation does not depend of AWS libraries.

server implements the proxy server and contains files:

infinicache-master/proxy/server/config.go - Lambda runtime config
infinicache-master/proxy/server/group.go - Group - manages GroupInstances of LambdaDeployments
infinicache-master/proxy/server/meta.go - represents an object to be stored in cache (called meta)
infinicache-master/proxy/server/metastore.go - a store of metas
infinicache-master/proxy/server/placer.go - implements a Clock LRU for object/metas removal
infinicache-master/proxy/server/proxy.go - the main file of the Proxy server implementation which initializes the lambda group and manages client operations. It uses other local packages to manage things such as infinicache/proxy/collector, infinicache/proxy/lambdastore
infinicache-master/proxy/server/scheduler.go - scheduler of the requests (I think, I have to !!!analyze better!!! how it is used and for what)

main contains files for deploying a lambda function in AWS, the handler for the function and the proxy entry point:

infinicache-master/deploy/deploy_function.go - deploys the function (Lambda) on AWS, hence uses github.com/aws/aws-sdk-go
infinicache-master/lambda/handler.go - handler file for Lamda Runtime, it is written for AWS Lambda runtime, hence imports AWS SDK, in particular github.com/aws/aws-lambda-go/lambda and and github.com/aws/aws-lambda-go/lambdacontext. Basically it handles the commands provenient from proxy.
infinicache-master/proxy/proxy.go - main handler file for proxy, it lunches/initializes the proxy server and all the components that it uses

collector (I have to investigate it better because it is not well understood why the lambda/collector is accessing AWS S3). The package contais the following files:

infinicache-master/lambda/collector/collector.go - updates DataEntries to S3, probably it saves data on the permanent storage if data is new?? ToDo: !!!analyze better!!! (uses github.com/aws/aws-sdk-go/)
infinicache-master/proxy/collector/collector.go - collects data from Lambdas?? ToDo: !!!analyze better!!!

storage used by lambda runtimes to store data chunks. Contains files:

infinicache-master/lambda/storage/storage.go - chunk storage for cache nodes

migrator implements the relay that allow to perform the backup protocol for maximizing data availability. Given that AWS Lambda does not inbound TCP/UDP connections, the authors of infinicache came out with a protocol wich overcomes this disadvantage. As described in the original paper this is highly related to AWS implementation. This package contains the following files:

infinicache-master/lambda/migrator/client.go
infinicache-master/lambda/migrator/intercept_reader.go
infinicache-master/lambda/migrator/storage_adapter.go
infinicache-master/migrator/forward_connection.go
infinicache-master/migrator/migrator.go

lifetime - Cit. "Each Lambda runtime is warmed up after every T_warm interval of time" the authors use a T_warm value of 1 minute as motivated by observations made in the original paper. This package should manage Functions lifespan. This package contains the following files:

infinicache-master/lambda/lifetime/lifetime.go
infinicache-master/lambda/lifetime/session.go
infinicache-master/lambda/lifetime/timeout.go - this file uses github.com/aws/aws-lambda-go/lambdacontext but just to choose some constants based on the type of instance of lambda in therms of CPU power, hence it can be easily replaced with some other api call that does the same

lambdastore: manages the pool of cache nodes and the connections lifecycles

infinicache-master/proxy/lambdastore/connection.go
infinicache-master/proxy/lambdastore/deployment.go
infinicache-master/proxy/lambdastore/instance.go
infinicache-master/proxy/lambdastore/meta.go

types contains types definitions used by the packages. contains the following files:

infinicache-master/common/types/types.go
infinicache-master/lambda/types/response.go
infinicache-master/lambda/types/types.go
infinicache-master/proxy/types/control.go
infinicache-master/proxy/types/request.go
infinicache-master/proxy/types/response.go
infinicache-master/proxy/types/types.go

Take into consideration that the above is just a draft which will be subject to modifications.
Until now this is my understanding of the implementation. Given that there is no documentation, I tryied to infer the above description by reading the code and matching it with the original paper. I honestly say that this is enough high level overview about how this implementation was written.
I want to kindly ask now some specific questions about the coontaining files:

Does the /lambda/collector collect data from S3 because of a MISS? What I mean is that the only reason I think a cache node would need a collector would be when the data is not cached, hence it has to be first fetched from the permanent storage.
And proxy/collector? Collects chunks from cache nodes and merges them?
What does in particular the proxy/server/scheduler do?

I anticipate thanks for any possible insight you could give me. Of course, given that I work for implementing infinicache concepts over an opensource serverless workload management framework such as Knative, we could collaborate by exchanging informations using this thread and I can offer my knowledge as well. Thanks and bests

Hi @neboduus. Thanks for your interest in InfiniCache!
I appreciate the detailed specification for each file in our repo from you. this is very helpful for others to understand the src code.
As for your question,

/lambda/collector is responsible for collecting some metrics such as latency breakdown in the lambda runtime side. This file is mainly focused on the performance evaluation part.
proxy/collector is also responsible for collecting metrics like latency, but it is on the proxy and client side.
proxy/server/scheduler controls the Group. And the Group controls the lambda cache nodes in this InfiniCache deployment.

Yes, I believe porting InfiniCache to open-source serverless platforms is workable. I am also very interested in how InfiniCache performs on an open-source serverless framework. Since the container start-up time in Knative may different from AWS Lambda. On the other side, you may not need to maintain the short Warm-up interval. But this is the idea just occurs to my mind, let's see what Infinicache will happen on Knative : )
Thanks for your work!

Dear all!
Thanks to you for the effort for designing such innovative distributed system. I editedd the description above and if you think it might be correct, you could add it to the readme file.
I would take advantage of this thread and ask you other things during my research if for you is ok.

I was evaluating the main limitation of Infinicache, which is dealing with small objects. I understand that such limitation comes from the fact that AWS charges Lambdas with a per-request approach, hence, the cache nodes will have to process more many requests for manage more many chunks and therefore the hosting costs will increase. But what if the cloud provider do not charge you with a per request approach? What if, for example you have your own cloud and you would like to use infinicache for caching objects which range from small to large? I am thinking that in this case the limitation do not apply anymore right?
I do not know if you know Knative, but it is a framework build uppon K8s which allows you to run serverless workloads. But what means serverless for Knative? It only means that you have some containers (if you want you could call them Functions, but of course if you want to deploy Funtions you will need light containers in which make them live) which are deployed in a serverless fashion, that is giving you nice properties such as scale-to-zero. Of course the "no servers" part is still there as in the case of AWS Lambda, because you can use automatic tools for Build and Serve your app, nonetheless gives you Eventing channels as AWS. Another difference is that, given that Knative can be deployed on any K8s Cluster, the pricing could change based on which resources you use for running the cluster. Hence in this vision, I wanted to ask you all, what do you think about implementing Infinicache over Knative?
How would you calculate the costs for such Infinicache deployment?

Thanks for any answer you could give me, your opinion is really important for me.

For your own cloud, it would be much easier IMO to handle small-object-intensive workloads, as you could allocate more concurrency or parallelism for handling bursts of small object requests within few cache instances. Yes, this limitation would not apply, to some extent.
I haven't put much thought on Q3. However, to calculate the cost for tenants' resource usage in an open deployment, it really depends. I believe you could use similar pricing model as AWS Lambda. Or, at least, you could map the same problem to priority-based multi-tenant resource sharing. This would be an interesting research question to explore.

Hi, for the Q3, since all the cache nodes are under AWS. The costs of the InfiniCache deployment are from AWS Lambdas. We collect log information from AWS.

Thanks to all! I will come back with observations about this topic when I will start deploying in Knative,

About lifetime
Lifetime is used to control the T_backup. while T_warm is controlled by the proxy.
infinicache-master/lambda/lifetime/lifetime.go - Backup interval controller.
infinicache-master/lambda/lifetime/session.go - Invocation and bill duration controller.
infinicache-master/lambda/lifetime/timeout.go - This file contains basic timer based operations that support billing duration control.

 How should I reconstruct it

The full function signature of EcGet is:
EcGet(key, size) (reqId, io.ReadCloser, success)
To read data, you can simply use the second returned value.

Hi everybody, if you don't mind I would have some questions for you:

Why you don't specify port on Lambda srv = redeo.NewServer(nil). I mean why you don't do something like this? :

lis, err := net.Listen("tcp", ":9736")
if err != nil {
	panic(err)
}
defer lis.Close()
srv.Serve(lis)

I would like to substitute the Lambda Handler with another Redeo command, which I will call "Invoke", hence in order to trigger the lambda I will have to call the lambda, but in order to do that I need to know the port. How You don't specify such port?

That is the difference between the Lambda and Knative. For the lambda, we invoke it by calling AWS api. There is no in-bound connection allowed within the Lambda runtime. And for Knative, you can't invoke a pod, only to scale out to have more pods, and then the function logic within the pod may be invoked. In another word, a more comparable subject of the Lambda function will be the pod itself.
As for the code, we use redeo as a reverse server, which means the server on the Lambda will reuse the connection the Lambda established as a client to the proxy, and serve the request issued by the proxy. You're welcome to make connections in the way suits Knative.

Guys thanks a lot for your suggestions. They were useful and gave me many insights about how to solve my issues.
I am happy to inform you that I have just finalized, what I called InfiniCache Lazy Migration over Knative and K8s.
Briefly I created a container for the proxy and one for the node. Then I made some modifications to the proxy and on the node in order to allow them to trigger the connection initialization using a little http server. Basically I have a GKE cluster where I installed Knative and where I am able to deploy the proxy and expose it as a K8s service and also deploy the lambda nodes as Knative serverless services/workloads. In the end I was also able to run the client example with a successful get/set.
My fork lacks of documentation now but you can have a look at the repo here if you want.
I would like to take kindly ask you if would be possible to open some kind of glitter channel where we could discuss further details about your implementation. Or if I can use this issue thread to interact with you when I do not understand something. However I would appreciate any kind of possibility and channel to remain in contact with you while my work evolves.
Thanks and bests

You are welcome. I will definitely take a look at code once I finished the project in hand. I guess it may be late in May. I'm thrilled we finally have a k8s based version.

I dedicated a branch to the Lazy Migration over K8s. You can see it here. Moreover I have added some docs in proxy/docs and also a README about how it should be installed.

Hi Guys! I am back with some new questions for my research.
Let suppose I would like to optimize infinicache to work with small objects (min 100 B, max 500 KB, just choose one). Assume a scenario where another service X is using infinicache to GET and SET always the same amount of data (min 10 KB, max 5 MB, just choose one) for each request it processes.
From the paper is clear that when the number of requests/sec exceeds 86, the pay-per-request model increases the costs too much compared to Elasticache cost which is constant.

Given that the objects are so small, could we reduce the number of Erasure Code shards in order to reduce the number of lambdas invocations and therefore increase the access rate without loosing the economical benefit or much performance?

Basically I am supposing that if with a RS code of 10+1 the maximum access rate before overshooting the economical benefit is 86 req/s. Reducing that number to 3+1 or max 5+1 will not deteriorate much the performance for so small objects but will decrease the economical cost for accessing the lambdas and therefore increase the admissible access rate.

Do you have any other idea about how we could optimize increase the admissible access rate?

I am supposing that the number of contacted lambdas from 10 to 3, the economical benefit would be of 7x. But I do not know how much the performance could be damaged.
Thanks for any possible idea

Depending on the characteristic of the workload, I think we may use different EC configurations for different sizes of objects. For small objects, it will practically fallback to P+1 replicas by using EC configuration like 1+P, which may be more cost effective.

Hi guys, can I ask why when the client detects failed chunks and tries to recover using func (c *Client) recover() does a SET() of the failed chunks? I really do not understand. Some comments say that we use a fixed lambdaID = 0. But I can't find where Infinicache discriminates on this value.

Hello! Sorry if i am bombing you with questions, I hope to not bother you.
I also want to be able to GET more than only 1 chunk from 1 cache node (through the same connection), or maybe none. However the number of chunks I may want is dynamic.
I want to do this because I am implementing an alternative mechanism to EC which is based on replication (I want to see if it works better for small objects), but for every GET operation I need the client to select which lambda to contact therefore how many connections to open and how many chunks to download from every connection. Therefore it should be able to know where the chunks are and then ask for data. Therefore I need the client to dynamically select which lambda to talk with.
I would like to kindly ask what you think about moving the map that records which chunk on which lambda is stored, from the proxy into the client. This would allow the client to automatically select the perfect number of lambdas and stream all needed chunks.
Do you think this is possible?
Do you have any suggestion regarding that or how to do it?
Thanks, and bests

Hi guys, can I ask why when the client detects failed chunks and tries to recover using func (c *Client) recover() does a SET() of the failed chunks? I really do not understand. Some comments say that we use a fixed lambdaID = 0. But I can't find where Infinicache discriminates on this value.

Sorry for the late response.
Because this type of error can be fixed ubiquitously by the erasure coding algorithm at the client side. Because the proxy as a meta server has the information of the lambda the missing chunk is located, there will be no need to pass lambdaID anymore.

The proxy will transform lambdaID passed by the client to real lambdaID based on multiple factors such as:

Load balancing
Data eviction

However, the client has solo freedom to choose EC configuration, such as 1+2 for replication (you may need to implement your own EC Encoder), or N+0 for no fault tolerance (Implemented as DummyEncoder). But you do need to create another client instance in the current implementation. I believe it is possible to implement a new client compatible with all these scenarios.
Does this answer your question?

Hi guys, can I ask why when the client detects failed chunks and tries to recover using func (c *Client) recover() does a SET() of the failed chunks? I really do not understand. Some comments say that we use a fixed lambdaID = 0. But I can't find where Infinicache discriminates on this value.

Sorry for the late response.
Because this type of error can be fixed ubiquitously by the erasure coding algorithm at the client side. Because the proxy as a meta server has the information of the lambda the missing chunk is located, there will be no need to pass lambdaID anymore.

Thanks for the answer. But why the client is sending a SET command to the proxy during a recover?

The proxy will transform lambdaID passed by the client to real lambdaID based on multiple factors such as:
1. Load balancing

2. Data eviction

However, the client has solo freedom to choose EC configuration, such as 1+2 for replication (you may need to implement your own EC Encoder), or N+0 for no fault tolerance (Implemented as DummyEncoder). But you do need to create another client instance in the current implementation. I believe it is possible to implement a new client compatible with all these scenarios.
Does this answer your question?

Thanks again. This makes sense. But  how could the client dynamically decide how many connections to open if the number of chunks I may want is dynamic as well?
I will try to explain better the use case:
Suppose that each CHUNK Infinicache stores in the nodes are Key-Value pairs their self. because the data is too small (each value is at most 10 KB, sometimes just 60 B), therefore (assumption) it doesn't make sense to EC encode them.
So instead of EC encoding we replicate the chunks (each chunk stores a Key-Value Pair) across a number of nodes for fault tolerance and for load balance.
Assume you have the following Key-Value Pairs Groups
Group 1:

Replication Factor = 5
Key Value Pairs = { <K1, V1>}

Group 2:

Replication Factor = 4
Key Value Pairs = { <K2, V2>, <K3, V3> }

Group 3:

Replication Factor = 3
Key Value Pairs = { <K4, V4>, <K5, V5>, <K6, V6>, <K7, V7> }

Each Group has a different replication factor, because they range from Frequently Accessed Data (high Replication Factor to allow load balancing) to Not So Frequently Accessed Data (so we can have a lower Replication Factor).
Moreover when the Replication Factor decreases (and therefore also the Access Frequency to that group) the number of Key-Value Pairs increases.
Assume also that these keys have some kind of relationship, which imply the following:

Every GET/SET operation that I send to INFINICACHE involves at least 1 Key-Value Pair for Group

Which means that whenever I try to GET a Key-Value pair from Group 1, I always need 1 Key-Value Pair from Group 2 and 1 Key-Value Pair from Group 3.
For this reason we may abstract the concept and say that Group 1, 2 and 3 contain Key-Value Pairs which belong to the same Entity. This Entity is represented by a High-Level-Key.
The High-Level-Key would be the one which creates the relationship between the Groups and allows us to select to right proxy.
So now I would like to implement the following operations:
1.
SET (High-Level-Key,
          {
          Group1: { <K1, V1> },
          Group2: { <K2, V2>, <K3, V3> },
          Group3: { <K4, V4>, <K5, V5>, <K6, V6>, <K7, V7> }
          })

Assuming that the Cache Pool is composed by the following lambdas {λa, λb, λc, ...., λz}
And given the above defined Replication Factors per Group.
The previous SET Operation should distribute the data in the following random 5 Cache Nodes, more or less in the following way:
λx: { <K1, V1>, <K2, V2>, <K3, V3>, <K4, V4>, <K5, V5>, <K6, V6>, <K7, V7> }
λy: { <K1, V1> }
λz: { <K1, V1>, <K2, V2>, <K3, V3>, <K4, V4>, <K5, V5>, <K6, V6>, <K7, V7> }
λr:  { <K1, V1>, <K2, V2>, <K3, V3> }
λp: { <K1, V1>, <K2, V2>, <K3, V3>, <K4, V4>, <K5, V5>, <K6, V6>, <K7, V7> }
As you can see the distribution respects the Replication Factors.

The second operation is obviously the GET which should be able to select some or all Key-Value Pairs of a High-Level-Key. However our assumption says that we should be able to get at least 1 Key-Value Pair per High-Level-Key. Therefore our generic GET example may be:

GET (High-Level-Key,
          {
          Group1: { <K1, V1> },
          Group2: { <K2, V2>},
          Group3: { <K4, V4> }
          })

The GET operation should have a interface similar to SET. But what should be interesting about the GET operation is that based on the Access Frequency to the cache nodes, the client should be able to decide:

from which cache nodes to download the data, and therefore how many connections to open (maybe also trying to keep the number of open connections very low, that way we can lower the economical costs of Infinicache; NB that the number of connections can range from 1 to 3, but not more)
and which CHUNKS to download from each node

This would allow to download all Key-Value Pairs specified by the GET operation either
(A) from the same node (e.g. when the Access Frequency to the entire system is low)
(B) or distribute the chunks requests when the nodes which contain all the keys I am interested in are over loaded
So my questions are

Under this scenario, how can the client decide from which cache nodes to download the data, and therefore how many connections to open? (That's why in my previous question I was asking if I could move the map that records which Chunks each Lambda is responsible fro, from the proxy in the client. This would allow the client to specify each time to check the Access Rate of the interested Lambdas and decide how many connections to open and which chunks to download from each Lambda. I am asknig this also because given the current design choices done by Infinicache, is the client which decides how many connections to open and which chunks to download.
Do you think this scenario makes sense?

Forgive me if I was too long, but I would really want to know what do you think about this mechanism.
Thanks and have a nice day

If you allow me I would also want to ask you, how do you deal with concurrent SET operations. In a sense that, how do you ensure that concurrent SET operations on the same object do not corrupt the data.
I imagine that, given that the objects are EC encoded, 2 concurrent SET operations on the same object, will produce different chunks if the data is different in the 2 operations (e.g. data has changed). That's why I think that 2 concurrent operations could corrupt the data if they arrive in different orders to lambdas.

I can first answer the short one. Theoretically, the value in the infinicache is unmodifiable, we should use version mechanism to deal with potential conflict (current version didn't implement this, but the experiments are based on this assumption).

Hi Guys, I do not understand something from the implementation.
Basically when you send a SET Request from Proxy to Lambda you add a Request to the Commands channel of an instance in this way:
	p.group.Instance(lambdaDest).C() <- &types.Request{
		Id:           types.Id{connId, reqId, chunkId},
		Cmd:          strings.ToLower(c.Name),
		Key:          chunkKey,
		ChanResponse: client.Responses(),
		EnableCollector: true,
	}

But then when you Prepare the Request for SET, you Write req.Body, which has not been set previously nowhere. Basically you do this:
func (req *Request) PrepareForSet(w *resp.RequestWriter) {
	w.WriteMultiBulkSize(6)
	w.WriteBulkString(req.Cmd)
	w.WriteBulkString(strconv.Itoa(req.Id.ConnId))
	w.WriteBulkString(req.Id.ReqId)
	w.WriteBulkString(req.Id.ChunkId)
	w.WriteBulkString(req.Key)
	if req.Body != nil {
		w.WriteBulk(req.Body)
	}
	req.w = w
}

Then my answer is: How and where is beeing `req.Body` set? PLEASE this question is very IMPORTANT.

And how do you think is the best way to send multiple Byte objects with the same request writer?

I was thinking to simply send a parameter which says how many Byte objects there are in the request and then simply read them in sequence.

Thanks in advance

Hi, the answer is simple. We used req.BodyStream to send body instead of req.Body. The req.BodyStream was been streamed directly to the destination in req.Flush(). Notice the CopyBulk() call, this was where streaming happened.
Sending multiple byte objects as stream is a little complicated, you may have to write special reader and writer to stream streams one by one. However, write multiple byte objects is nothing different from write multiple strings.

Hi guys! I was able to implement the idea that I mentioned in this previous post (https://github.com/mason-leap-lab/infinicache/issues/3#issuecomment-645915093) . So, I am transferring more than 1 chunk at the time using the same connection.
I am able to do that in both directions: SET and GET (proved by debug) but obviously I have a problem. It is related to the GET op:
Given that the chunks that I am transferring are so small (few Bytes), instead of streaming them, I am passing them as Byte Arguments. Therefore, for example on SET I do something like this in the client:
func (c *Client) multiKeySet(){
        ......
        w.WriteBulkString(pairs)
	for i := 0; i < pairs; i++ {
		var pair = keyValuePairs[i]
		w.WriteBulkString(pair.Key)
		w.WriteBulk(pair.Value)
	}

And I do similarly in order to transfer them from the proxy to the Lambda.
My problem arrives when the Client tries to receive data back from the Lambda during a GET op.
In the proxy I am able to receive and print the data received from the Lambda.
For example here I have created a handler for my MultiKeyGet operation, on receiving the MultiKeyGet Ack from Lambda:
func (conn *Connection) mkGetHandler(start time.Time)  {
	conn.log.Debug("mkGET from lambda %d", conn.instance.id)

	// Exhaust all values to keep protocol aligned.
	connId, _ := conn.r.ReadBulkString()
	reqId, _ := conn.r.ReadBulkString()
	chunkId, _ := conn.r.ReadBulkString()
	lowLevelKeysN, _ := conn.r.ReadInt()

	lowLevelKeyPairs := make(map[string][]byte)
	for i:=0;i<int(lowLevelKeysN);i++{
		lowLevelKey, _ := conn.r.ReadBulkString()
		var lowLevelValue []byte
		lowLevelValue, _ = conn.r.ReadBulk(lowLevelValue)
		lowLevelKeyPairs[lowLevelKey] = lowLevelValue
	}
	conn.log.Debug("received: %v", lowLevelKeyPairs)

      .....

      rsp := &types.Response{Cmd: "mkget"}
	rsp.Id.ConnId, _ = strconv.Atoi(connId)
	rsp.Id.ReqId = reqId
	rsp.Id.ChunkId = chunkId
	rsp.LowLevelKeyValuePairs = lowLevelKeyPairs

	conn.log.Debug("mkGOT %v, confirmed.", rsp.Id)
	if req, ok := conn.SetResponse(rsp); !ok {
            .......

And I am actually able to print the data received. So my GET op behaves correctly.
As you can see above I have modified the types.Response struct to add a field that I want to use instead of Body or BodyStreams. But actually I am not able to send them back to the client.
What I have done then  was to add this in the `Response.Flush()"  :
func (rsp *Response) Flush() error {
	if rsp.w == nil {
		return errors.New("Writer for response not set.")
	}
	w := rsp.w
	rsp.w = nil

	if rsp.BodyStream != nil {
		if err := w.CopyBulk(rsp.BodyStream, rsp.BodyStream.Len()); err != nil {
			return err
		}
	}

	l := len(rsp.LowLevelKeyValuePairs)
	if l > 0 {
		w.AppendInt(int64(l))
		for k, v := range rsp.LowLevelKeyValuePairs {
			w.AppendBulkString(k)
			w.AppendBulk(v)
		}
	}

But this does not seem to work. On the client if I simply do lowLevelKeyValuePairsN _ = c.Conns[addr][i].R.ReadInt() it always reads 0 (zero).
Please do you have any insight about that? I want to precise again that I am sure that the data gets stuck in the proxy, once received from the lambda. So the short question would be:
How do I transfer them to the client in an By Argument Style?
Thanks and bests

Did you call w.Flush() at the end of the Response.Flush?

Hey, thanks for the answer. Yes sure, is just omited from the code

Yo in the end I managed. Was a mistake of mine. Thanks

I can first answer the short one. Theoretically, the value in the infinicache is unmodifiable, we should use version mechanism to deal with potential conflict (current version didn't implement this, but the experiments are based on this assumption).

Hi Guys! I'd like to kindly ask some clarification about this answer if possible.
I am wondering how is testing possible without being able to modify the values inside it?
The only way that can come to my mind is to test the 2 operations separately.
For the SET operation we can simply implement many concurrent clients which only SET values and therefore analyze the performance.
For the GET operation I am thinking to populate first the store with preset kv pairs and then launch many concurrent clients which perform GET operations against Infinicache.
I would like to ask if this is what you had in mind?
And also I would like to know how do you think is the better way to launch concurrent clients?
Is ok to connect them against the same proxy? What about make them running on the same machine?
Thanks in advance for any possible answer and bests

Do do you mean performance testing? We did that by following ways:

Set different keys with one/multiple connection(s)
Get different keys with one/multiple connection(s)
Run real workload and analyse logged requests.

Since our paper targeted large objects and each client involved multiple connections, it can easily saturated the bandwidth of the machine. So our experiments deployed one client on each machine, and proxies were co-located with clients to form a cluster.

Thank you for the answer. Very clear! Do you think is ok to collocate more than 1 client with the same proxy?

Using version technique, key updates can be regarded as a key with different versions, which are equal to values with different keys. This transformation can be completed in the proxy, so that we can ensure objects are immutable.
Yes, colocating multiple clients is supported, if you care more about throughput instead of latency.

Hi! On your suggestion, I have created 2-proxy environment with 20 nodes each and I collocated 3 clients on each proxy in the following way:
func main() {
	var wg sync.WaitGroup
	for i:=0; i<3; i++{
		wg.Add(1)
		go test(i, &wg)
	}
	wg.Wait()
}

func test(i int, wg *sync.WaitGroup){
	defer wg.Done()
	var addrList = "<IP_ADDRESS>:6378,<IP_ADDRESS>:6378"
	// initial object with random value

	// parse server address
	addrArr := strings.Split(addrList, ",")

	// initial new ecRedis client
	cli := client.NewClient(10, 2, 32, 3)

	// start dial and PUT/GET
	cli.Dial(addrArr)
	var data [][3]client.KVSetGroup

	// ToDo: Replace with a channel
	var setStats []float64
	//var getStats []float64

	for k:=0; k<10000; k++{
		d := cli.GenerateSetData(5)
		data = append(data, d)
		key := fmt.Sprintf("HighLevelKey-%d", k)
		if _, stats, ok := cli.MkSet(key, d); !ok {
			log.Println("Failed to mkSET", i, key)
		}else{
			setStats = append(setStats, stats)
			log.Println("Successfull mkSET", i, key)
		}
	}

But at some point, after cca 3000 mkSET ops, a panic is thrown:
panic: runtime error: racy use of timers

goroutine 197 [running]:
time.startTimer(0xc000108968)
        /usr/local/go/src/runtime/time.go:114 +0x2b
time.(*Timer).Reset(0xc000108960, 0xdf8475800, 0xc0004efd00)
        /usr/local/go/src/time/sleep.go:127 +0x81
github.com/neboduus/infinicache/proxy/proxy/lambdastore.(*Instance).warmUp(0xc0002b91d0)
        /project/src/proxy/proxy/lambdastore/instance.go:499 +0x6a
github.com/neboduus/infinicache/proxy/proxy/lambdastore.(*Instance).flagValidated(0xc0002b91d0, 0xc000504300, 0x0)
        /project/src/proxy/proxy/lambdastore/instance.go:340 +0x7e
github.com/neboduus/infinicache/proxy/proxy/lambdastore.(*Connection).pongHandler(0xc000504300)
        /project/src/proxy/proxy/lambdastore/connection.go:242 +0x1cc
github.com/neboduus/infinicache/proxy/proxy/lambdastore.(*Connection).ServeLambda(0xc000504300)
        /project/src/proxy/proxy/lambdastore/connection.go:146 +0x76b
created by github.com/neboduus/infinicache/proxy/proxy/server.(*Proxy).Serve
        /project/src/proxy/proxy/server/proxy.go:81 +0x45

Do you have any isights about why this is happening? Thanks!

It said timer resetting is being called in multiple go-routings. So we may need to put the reset of the timer in a mutex block.

Actually adding a mutex on line 499 seems to solve the issue. I d like to ask also:

where and how is triggered the lambda that is used to backup another lambda?

I know that the proxy acts as a bridge between the 2 functions. And that the backup function is a replica of the node. But I cannot find the portion of code where this backup function is triggered. As happens for example for the "main" node in this function func (ins *Instance) triggerLambdaLocked(warmUp bool) {...}.

The function to invoke the backup is the in the lambda side. Specifically, it is in the /lambda/migrator/client.go, line 82.


[TC-DM-2.1]--While verifying few optional non global attribute to write chip-tool throws an error 0x86((UNSUPPORTED_ATTRIBUTE)) instead 0x88(UNSUPPORTED_WRITE(0x88)

#### Problem
[[TC-DM-2.1]--While verifying few optional non global attribute to write chip-tool throws an error 0x86((UNSUPPORTED_ATTRIBUTE)) instead 0x88(UNSUPPORTED_WRITE(0x88) those attributes are mentioned below.

CapabilityMinima attribute
Reachable attribute
UniqueID  attribute
SerialNumber_test attribute
ProductLabel  attribute
SoftwareVersionString attribute

expected behavior:-UNSUPPORTED_WRITE(0x88)
actual behavior:- CONSTRAINT_ERROR(0x87)
*** steps to reproduce**
sudo ./chip-tool basic write-by-id 0x14 partlabel 1 0
sudo ./chip-tool basic write-by-id 0x15 newserialnumber 1 0
sudo ./chip-tool basic write-by-id 0x17 0 1 0
sudo ./chip-tool basic write-by-id 0x18 newid 1 0
sudo ./chip-tool basic write-by-id 0x19 4 1 0
system configuration
Chip-Tool: RPI
DUT: RPI
Platform : IP and Wifi
Commit-ID:4d50b1084284018c4a628f919b0d2e01ea2a39f8
Dut: All-Clusters-app
PFA logs:-
DM-2.1.txt
Test-Plan reference;-
https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/devicemanagement.adoc#tc-dm-2-1-basic-information-cluster-server-attributes

Is the actual behavior CONSTRAINT_ERROR or UNSUPPORTED_ATTRIBUTE?  The above is not self-consistent....
UNSUPPORTED_ATTRIBUTE is somewhat expected if the attributes are actually not supported, no?

@Rajashreekalmane

@bzbarsky-apple As per the test-plan we should get 0x88(UNSUPPORTED_WRITE(0x88) but currently I am getting  0x86((UNSUPPORTED_ATTRIBUTE)).
expected behavior:-UNSUPPORTED_WRITE(0x88)
actual behavior:- 0x86((UNSUPPORTED_ATTRIBUTE))

sudo ./chip-tool basic write-by-id 0x14 partlabel 1 0
sudo ./chip-tool basic write-by-id 0x15 newserialnumber 1 0
sudo ./chip-tool basic write-by-id 0x17 0 1 0
sudo ./chip-tool basic write-by-id 0x18 newid 1 0
sudo ./chip-tool basic write-by-id 0x19 4 1 0

Every single one of those attribute ids does not exist in the spec.  So UNSUPPORTED_ATTRIBUTE is correct.
I assume you meant 14 instead of 0x14 and so forth (if the intent is to test ProductLabel and the like).  Those attribute ids work fine and return


Review use of Format Specifiers for potential information leakage

The Go format specifier %v (and variations) can dump an entire struct into a log message, potentially resulting in information leakage (e.g. if the struct contains an API key or other secret).
We should identify which format specifiers might cause this kind of leakage and ensure we're only using them where it's safe and there's no alternative.

(Preserving a point @matthchr made in chat.) In the case of the generator it's only operating on public information (API and schema definitions) so it doesn't have any private information to leak. This is more a consideration for the controller implementation which is passing around instances of the types we generate.
Although one possible thing to do for safety would be to generate a String() method on types that have secrets in them - is this something we can detect? From some testing it seems like that would protect against someone using %v or %+v, although not %#v.

From the discussion here there are also some subtleties around unexported fields but that's probably not an issue here since fields need to be exported for json/yaml serialisation anyway.


Vacuum Smart 700 - Can't start cleaning

Describe the bug
When I use HA to start cleaning the integration won't change the current command, so the vacuum doesn't start.
To Reproduce
After having the the vacuum added, here is the status when it's charging:

And here it is after I press 'Start Cleaning':

'Activate' changed to true, while the 'Command' remained the same.
Expected behavior
I expect the command (DP3) to change to 'Smart'.
Additional context
This used to work about 3 months ago, but in the meantime I sent my vacuum for repairs and now that it came back with the updated integration this behavior has appeared.
This is also an issue in the default HA integration, but there I'm able to manually change the command while it is not available with this integration.
Diagnostic: tuya_local-01J7GMRHP2K3S3K9E74S4BD34M-Robotnik-47ca15c70833a3db47d47ad5bab6c958.json
Logs:
home-assistant_tuya_local_2024-09-13T11-20-00.176Z.log
Thanks in advance!

I suspect this is related but I'm not sure on how to fix this:
https://github.com/make-all/tuya-local/commit/a7820d8d922aec3694dca4ccc1729801db9c5ce2

There isn't really any standard way for vacuums to work, so I don't know exactly what needs to be done here. It is not normal to automatically send the smart command when toggling on, and I don't think that can be done easily.

Thanks for the reply!
Would it be acceptable to expose an select entity for this vacuum model with the command?

If you're okay with that, I can open the PR.
Then I should be able to circumvent this issue in the same manner as the default cloud integration.

A better option would be to add a button to your UI that performs the sequence of actions needed to actually start a clean. The vacuum.send_command action is still available as a service, even if the command is not exposed in the default UI.

Nothing changed in the config for more than 1 year, so I suspect that when your vacuum was away for repairs, it was given a firmware upgrade that changed its behaviour.

A better option would be to add a button to your UI that performs the sequence of actions needed to actually start a clean. The vacuum.send_command action is still available as a service, even if the command is not exposed in the default UI.

Thanks for mentioning the send_command service, with this there's no need for new entities. That is enough to fix my issue, thanks a lot!
And yeah, this model works nice but it's a lemon in terms of software.


wip: experimental support for version ranges in unsat errors

I won't have time to finish up this PR properly, but I'm leaving it here because at least part of it might be interesting. The second commit could probably be cherry-picked and merged without too much additional work.
Experimental support for version ranges
@wolfv was enthusiastic about the idea, but we didn't really reach a consensus about how the feature would exactly work. The first commit contains a working implementation that you could tweak further. Check out the insta snapshots of the commit to see how it affects error messages.
Proper unicode characters in error messages
This means we stop using |-- and start using ├─, and similar. There is some complexity associated to this change, because now we need to do some tracking to know which character we need (├─, └─ or │). The implementation is in the second commit, but there are a few TODOs to clean up the code. Check out the insta snapshots of the commit to see how it affects error messages.

Closing this PR for now. We don't see an avenue forward for this as of yet.


MoveIt Robot motion

While working with the real robot I am not getting a sequential motion like
home > position1 > position2 > position3....
Instead I am getting motions like
home > poistion1 > home > position2 > home > poisition3....
When checked with gazebo simulation it works correctly whereas when I am working with the real robot I am getting the error mentioned above.

Could you tell me the AS system software version which you use.
You can check the version by Auxiliary function 0804  or Monitor command "ID".

To use our driver in F series, the robot controller's AS system should be higher than ASF_01000000W.
(docs/ConnectingRealRobot.md)
Could you ask your KHI partner to upgrade AS system?

To use our driver in F series, the robot controller's AS system should be higher than ASF_01000000W.
(docs/ConnectingRealRobot.md)
Could you ask your KHI partner to upgrade AS system?

Still I could not get the required target motion.
Instead I am getting motions like
home > poistion1 > home > position2 > home > poisition3.... This is the motion that is happening still.
These are my code snippets. I have connected the robot to the ubuntu through the vmware workstation. Does it have anything to deal with this ?

Still I could not get the required target motion.
Instead I am getting motions like
home > poistion1 > home > position2 > home > poisition3.... This is the motion that is happening still.
These are my code snippets. I have connected the robot to the ubuntu through the vmware workstation. Does it have anything to deal with this ?

About,

home > position1 > home > position2 > home > poisition3....

If,

you set set_max_velocity_scaling_factor(0.1)
you set a home position which is not equal to [JT1:0 JT2:0 JT3:0 JT4:0 JT5:0 JT6:0]
the arm moves home(0) > position1 > home(1) > position2 > home(2) > poisition3...,

How is the motion position1 > home(1) like?

Is the motion speed same as the speed : velocity_scaling(0.1)?
Is home(1) same as home(0), or  [JT1~6: 0]?

About,

home > position1 > home > position2 > home > poisition3....

If,

you set set_max_velocity_scaling_factor(0.1)
you set a home position which is not equal to [JT1:0 JT2:0 JT3:0 JT4:0 JT5:0 JT6:0]
the arm moves home(0) > position1 > home(1) > position2 > home(2) > poisition3...,

How is the motion position1 > home(1) like?

Is the motion speed same as the speed : velocity_scaling(0.1)?
Is home(1) same as home(0), or  [JT1~6: 0]?

Yes! The motion speed is same as velocity scaling but it is maintained only during the motion of home > poisition 1 and not during poisition 1 > home.
The home 1 is not same as home 0 but it is going near home 0

Yes! The motion speed is same as velocity scaling but it is maintained only during the motion of home > poisition 1 and not during poisition 1 > home.
The home 1 is not same as home 0 but it is going near home 0

It seems to be a problem on khi_robot side.
To reproduce your problem, let me know your system environment in detail.

Ubuntu/ROS version
Current AS system
khi_robot version or revision
Motion script: Is it same as https://github.com/Kawasaki-Robotics/khi_robot/issues/47#issuecomment-757865376 ?

It seems to be a problem on khi_robot side.
To reproduce your problem, let me know your system environment in detail.

Ubuntu/ROS version
Current AS system
khi_robot version or revision
Motion script: Is it same as https://github.com/Kawasaki-Robotics/khi_robot/issues/47#issuecomment-757865376 ?


EF Core 1.0 !!!

Now that EF7 has become EF Core 1.0, what will happen with package versions/names? And how will this affect NuGet versioning and providers? http://www.hanselman.com/blog/ASPNET5IsDeadIntroducingASPNETCore10AndNETCore10.aspx

@ErikEJ we are still discussing some options for the packing naming scheme of EF Core 1.0 components. Certainly, how it wpuld work for provider writers is one of the important aspects we are looking at. We should have a proposal very soon.

@ErikEJ I agree with you, that it could be real problem for developers who started development with ASP.NET 5. I posted the issue which described the issue with many renaming of packages. The issue was closed, because Microsoft don't take seriously the consequence of renaming the packages and the difficulties of finding the new names.
Renaming of "official names" follows renaming of assemblies, which are the names of packages. As the result the projects created before will not resolve to the new packages. One find a lot of examples on the stackoiverflow or in the documentation, which uses old package names. Thus it's really difficult to locale the correct name of some internal assembly after its renaming.

@OlegKi Sure, but keep in mind, that ASP.NET Core 1.0 has not been released yet, but is "beta" software

@ErikEJ What I mean is: it's no problem with changing the names of assemblies
Microsoft.AspNet.Mvc.Xml to Microsoft.AspNet.Mvc.Formatters.Xml
EntityFramework.SqlServer to EntityFramework.MicrosoftSqlServer
...
and so on, but the names will be used as the NuGet package name or id of the package. Thus the projects, which used "EntityFramework.SqlServer": "7.0.0-beta8" will get no advice that the Update to renamed package "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final" exist. You will still find a lot of code examples in Google where old name EntityFramework.SqlServer are used and a lot of developers will not update his project.json to new versions because or renaming of ids used for referencing. Many parts of official ASP.NET documentation will be corrected to new names weeks later.
It's clear for me that the ASP.NET is not final, but the most release candidates was more friendly to projects created using beta versions. One didn't renamed DLLs, registry keys and other internal information, which could be used by developers.
I expect a lot of renaming of packages after renaming of the product name. It will punish the developers, who already started migration his large projects to ASP.NET 5. It's really easy to extend the requests to NuGet server for the newest version so that the renamed packages would be used too.

@OlegKi totally hear what you are saying. For us it's a matter of "do we cause some pain now for folks who have adopted pre-release software or do we forevermore stick with a naming/versioning story that causes confusion". I'm not diminishing the impact that it has now... but in the long term I think adopting a strategy that correctly reflects what we are building is the clear winner.

Opened https://github.com/aspnet/EntityFramework/issues/4376 to track everything that needs to be done around this (and discuss naming etc. with provider writers).


How to avoid JSON fields from being indexed in GraphQL

Hello,
I have a category collection with a JSON field in where I stored an array with the order of products for display.
The problem is that I have to declare which products I want to fetch in the GraphQL query, but the products are not datafields, they are records in the database, so they are dynamic.
product_priorities is the JSON field, and tete is just a product in the database (record in the database)

So the product is dynamic, I just want to fetch product_priorities like if it was a text field, not like a 'GraphQL aware' field where I have to declare exactly what I want.
A product_priorities {*} in the GraphQL query would do the job, but that does not exist. Because in the code I don't know what will be in there.
Is a bit hard to understand my problem, feel free to ask for more clarification if needed. Any help would be welcome, thanks!

I'm using a text field to make it work. But there is JSON.parse and JSON.stringify all over, is becoming un-manageble


Add position top, right, bottom, left _utilities.scss

Add position top, right, bottom, left utilities

This is a no from me. No need for all of these in Bootstrap; the API is there for this to be customized by you.


Failing test for avoidRouterUpdate

Hm, as far as I can see avoidRouterUpdate is actually broken:

Error: Expected [ '/', '/foo' ] to equal [ '/' ]

This api is causing us a lot of complexity — should we just remove it? Can't you basically "fake" the same thing using replacePath?
(The docs mention forceRefresh, but I've never looked at it, so don't know if that's something that will break? Does the value gained outweigh the complexity?)
I'll keep playing around with the code and tests to try to find a solution.

The main problem is redux devtools. It makes things complicated because we can't rely on simple heuristics that worked before.
We probably are making it more difficult for ourselves by storing info like that in the state. Doing things like that confuses devtools: devtools uses the actions to track changes across time, but we probably shouldn't be doing stuff like storing avoidRouterUpdate and changeId if we want devtools to work easily.
At the same time, it gets difficult to implement other solutions. I think we're onto something in #109 though. I think we could use the fact that the route state object changes whenever we want to call into history. Even if the path and all the properties are the same, the instance of the state will be different. It seems like we could use that, combined with some local variables to track other state for stopping cycles, for a more robust solution.

Closing this in favour of #129


thunderbird: 31.7.0 -> 38.1.0

I did not test build this.

Doesn't build locally. See travis error. I'm closing this as it's too trivial.


count_total_set_bits

@all-contirubutors please add @10mohan for his code

@all-contributors please add @10mohan for his code


Add, improve and revise translations for Russian

Translated some previously untranslated lines. Improved a couple of questionable translations. Changed punctuation at a couple of places according to Russian rules (they differ from both English and French).

Thanks a lot!


If ASK_VERIFY_REQUESTS  is set then ASK_APPLICATION_ID is redundant

During the testing my Alexa skill, I set ASK_VERIFY_REQUESTS=False & ASK_APPLICATION_ID to the application id of the skill. When I send a request (which has a different applicationId), the request_verify works as expected (i.e. it's not getting verified) but I'd expect the request to fail due to the mismatching applicationId.
Looks like if ASK_VERIFY_REQUESTS is set then ASK_APPLICATION_ID is redundant. Because when I use my 'production' config it works fine and there I've only set APPLICATION_ID and not VERIFY_REQUESTS.
I do see in the code that the verify block is executed only if the ask_verify_requests is True:
ask_payload = self._alexa_request(verify=self.ask_verify_requests)
...
def _alexa_request(self, verify=True):
        raw_body = flask_request.data
        alexa_request_payload = json.loads(raw_body)

        if verify:
            ...

I think it 'd be a good idea to separate the checking of the signature, application_id and timestamp, so that the user can individually control what he wishes to verify.

That sounds reasonable to me


Added PopulateDefaultsMachineDeployment to sync

With missing this, there are a couple of nil pointer panics, because the
content to wich d (or deployment) is pointing to, is changed before
storing it again. Subsequent code assumes Strategy to be there, no
matter wether or not it is already in the DB.
The result also is, that Spec.Strategy is stored with defaults in the API
Feel free to just copy the line out of the code and close this PR. Just
thought this is easier than trying to describe what to do ;)

That's awesome thanks @chrigl - I'm hoping if I merge this PR it should add your commit to my PR and should all get attributed correctly upstream!


App Crash at ImageSource.camera

Hi i want to take picture with camera which i set a button to click on it and take picture here is my code
        XFile? _pickedFile = await _imagePicker.pickImage(source: ImageSource.camera);
            XFile? _imageFilePicked =XFile(_pickedFile!.path);
            print('image path is $_imageFilePicked');

also added
            android:requestLegacyExternalStorage="true"

but when i click button to get camera on then it on the camera and destroy the app on background i dont know what issue it is

Hi @itsdani121, thanks for filing the issue.
Please provide a complete minimal reproducible code sample as well as the output of flutter doctor -v and the crash message as well as the version of image_picker you're using.
Thank you

Image picker crashing on Samsung device (Galaxy A42 5G, Android 11), on other devices im not able to reproduce the same behavior.
Android SDK compiles & target SDK: 31
Flutter: 2.8
image_picker: ^0.8.4+4
flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 2.8.0, on macOS 12.0.1 21A559 darwin-arm, locale en-GB)
[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 13.2)
[✓] Chrome - develop for the web
[!] Android Studio
✗ Unable to find bundled Java version.
[✓] Android Studio (version 2020.3)
[☠] IntelliJ IDEA Community Edition (the doctor check crashed)
✗ Due to an error, the doctor check did not complete. If the error message below is not helpful, please let us know about this issue at
https://github.com/flutter/flutter/issues.
✗ FormatException: Unexpected extension byte (at offset 5)
[✓] VS Code (version 1.63.1)
[✓] Connected device (4 available)
! Doctor found issues in 2 categories.
Minimal setup
all premissions are enabled
Future<File?> pickImage() async {
   try {
      final result = await ImagePicker().pickImage(
          source: ImageSource.camera,
          maxHeight: 2048,
          maxWidth: 2048,
          imageQuality: 90);
      return result != null ? File(result.path) : null;
    } on PlatformException catch (e, s) {
      log('openCamera: $e', error: e, stackTrace: s);
    } catch (e, s) {
     log('openCamera: $e', error: e, stackTrace: s);
    }
return null;
}

crashes are not logged to Crashlytics..

I'm having exactly the same problem after setting targetSdkVersion to 31.

Hi @zoocityboy & @pafciu17, could either of you provide a list of devices that are affected by this?
As well as the output of flutter run -v with some logs that show what you're experiencing. A screen recording may also help in this case.
The output of flutter run -v is likely to be huge so kindly put it into a .txt file and attach it here.
Thank you


Docker Installation - /opt/tyger permissions

Hi folks,
When following the current Docker installation documentation, there's a bit of a hole that can be stepped into during the api install step. The documentation mentions that /opt/tyger must exist, and advises to create via sudo: https://github.com/microsoft/tyger/blob/353c5e9849dd8bb55d94f9e346dc8684cd61c9a5/docs/introduction/installation/docker-installation.md?plain=1#L101 This unblocks the install (or at least, appears to for the user), until the data plane container is stood up - at which point it fails its health check. Logs indicate the data-plane fails to bind the tyger.data.sock with a permissions error (since the tyger dir created with sudo is owned by root), which can be fixed with chmod on the tyger dir.
Since the instructions mentioned explicitly to create with sudo, I assumed the root permissions or privileged access might be handled by docker compose or uid in the containers. It might be worth updating the documentation to mention the purpose of the /opt/tyger dir, and necessary permissions - but I also wanted to ask what permissions should be used on the /opt/tyger directory if docker were used in production. I quick-fixed my install with 777 (based on .devcontainer/prepare-host.sh), which lets the installation finish but seems less than ideal? Would prepare-host.sh be worth mentioning in the documentation or using for local, manual docker installations?

andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ tyger api install -f test.yml
2024-10-25T08:53:40.335-04:00 INF Starting Tyger API install
2024-10-25T08:53:40.335-04:00 FTL error="error creating directory /opt/tyger: mkdir /opt/tyger: permission denied"

andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ sudo mkdir /opt/tyger
andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ tyger api install -f test.yml
2024-10-25T08:54:02.072-04:00 INF Starting Tyger API install
2024-10-25T08:54:02.175-04:00 FTL error="error creating database container: error creating directory /opt/tyger/database: mkdir /opt/tyger/database: permission denied"

andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ sudo mkdir /opt/tyger/database
andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ tyger api install -f test.yml
2024-10-25T08:54:16.489-04:00 INF Starting Tyger API install
2024-10-25T08:54:16.823-04:00 INF Pulling image mcr.microsoft.com/cbl-mariner/base/core:2.0
2024-10-25T08:54:17.002-04:00 INF Pulling image postgres:16.2
2024-10-25T08:54:21.420-04:00 INF Done pulling image mcr.microsoft.com/cbl-mariner/base/core:2.0
2024-10-25T08:54:28.697-04:00 INF Done pulling image postgres:16.2
2024-10-25T08:54:40.740-04:00 FTL error="error creating data plane container: error creating directory /opt/tyger/data-plane: mkdir /opt/tyger/data-plane: permission denied"

andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ sudo mkdir /opt/tyger/data-plane
andrew@andrew-Precision-Tower-7810:~/Documents/tyger$ tyger api install -f test.yml
2024-10-25T08:54:51.898-04:00 INF Starting Tyger API install
2024-10-25T08:54:52.425-04:00 INF Pulling image mcr.microsoft.com/tyger/tyger-server:v0.6.9
2024-10-25T08:54:52.433-04:00 INF Pulling image mcr.microsoft.com/tyger/tyger-data-plane-server:v0.6.9
2024-10-25T08:54:57.098-04:00 INF Done pulling image mcr.microsoft.com/tyger/tyger-data-plane-server:v0.6.9
2024-10-25T08:54:57.728-04:00 INF Done pulling image mcr.microsoft.com/tyger/tyger-server:v0.6.9
2024-10-25T08:55:58.001-04:00 FTL error="error creating data plane container: timed out waiting for container to become healthy. Current status: unhealthy"

Hi @Andrew-Dupuis, sorry for taking a while to respond. Your points are very valid and indeed there is more we need to do to lock down the directory and its contents, improving the installation experience, and documenting the necessary permissions.
We don't have time to do this all of this at the moment, but at a high level, we would make the following changes:

tyger api install would create the directories and/or update the ownership and permissions. It would likely need to run as root and therefore would require the userId and allowdGroupId fields to be set in the config.
The postgres database socket needs to be locked down to only the allowed user ID.

I will update this issue as soon as we have something to share.


app provider: allow insecure connection to wopiserver / wopibridge

@wkloucek we could fold this in https://github.com/cs3org/reva/pull/1785 as the whole appprovider is being refactored and extended

@wkloucek we could fold this in #1785 as the whole appprovider is being refactored and extended

That would be fine for me. I needed it for as a intermediate step for owncloud/ocis#2204 and owncloud/ocis-wopiserver#16 and if I don't open a PR in time I will forget about it later... So feel free to close this one at any time in favor of yours ;-)

Implemented by #1785


koa-router 7 is not working with 3 async middlewares

Get code from example and try to add another middleware with setTimeout inside

Thanks for pointing this out, @Alex0007.
The example should be more clear. next must be called and added to the promise chain or awaited for the middleware stack to continue executing. Please see if this commit and example in the commit message explains it sufficiently. If not, feed free to re-open this issue!


time.clock - > time.perf_counter

python 3.8 compatibility

LGTM. Thanks!

To future me: Here is where python says this is deprecated.


Let's Encrypt

Should probably consider integrating Let's Encrypt support into the web/update server, this way you can send updates over HTTPS so they cannot be intercepted.

You should use a reverse proxy instead. Please take a look at traefik. If you need help, I can send you my setup.

Hey, yeah I'd also go with a reverse proxy in front of docker-ddns to not increase it's complexity for stuff that can be offloaded. Traefik is a great choice alongside Caddy or nginx.
I'll close this for now but if you need any help, just let me know.


[fuzzer] Add function to retrieve Referral Entries from table.

Keyword Check:
/pins_upstream/sonic-buildimage/src/sonic-p4rt/sonic-pins$ ~/tools/keyword_checks.sh p4_fuzzer/
Keyword check Passed.

Build result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel build $BAZEL_BUILD_OPTS  ...
INFO: Analyzed 514 targets (0 packages loaded, 0 targets configured).
INFO: Found 514 targets...
INFO: Elapsed time: 0.782s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

Test result:
/sonic/src/sonic-p4rt/sonic-pins$ bazel test $BAZEL_BUILD_OPTS --cache_test_results=no ...
INFO: Analyzed 514 targets (0 packages loaded, 0 targets configured).
INFO: Found 344 targets and 170 test targets...
INFO: Elapsed time: 106.993s, Critical Path: 25.98s
INFO: 179 processes: 211 linux-sandbox, 18 local.
INFO: Build completed successfully, 179 total actions
//gutil:collections_test                                                 PASSED in 0.5s
//gutil:io_test                                                          PASSED in 0.8s
//gutil:proto_matchers_test                                              PASSED in 0.9s
//gutil:proto_ordering_test                                              PASSED in 0.9s
//gutil:proto_test                                                       PASSED in 1.0s
//gutil:status_matchers_test                                             PASSED in 0.8s
//gutil:test_artifact_writer_test                                        PASSED in 0.6s
//gutil:testing_test                                                     PASSED in 0.5s
//gutil:timer_test                                                       PASSED in 5.6s
//gutil:version_test                                                     PASSED in 1.3s
//lib:basic_switch_test                                                  PASSED in 0.9s
//lib:ixia_helper_test                                                   PASSED in 1.2s
//lib/basic_traffic:basic_p4rt_util_test                                 PASSED in 1.5s
//lib/basic_traffic:basic_traffic_test                                   PASSED in 15.1s
//thinkit:mock_generic_testbed_test                                      PASSED in 0.9s
//thinkit:mock_mirror_testbed_test                                       PASSED in 0.7s
//thinkit:mock_ssh_client_test                                           PASSED in 0.0s
//thinkit:mock_switch_test                                               PASSED in 0.7s
//thinkit:mock_test_environment_test                                     PASSED in 0.1s
//thinkit:switch_test                                                    PASSED in 0.7s
//sai_p4/instantiations/google/test_tools:table_entry_generator_test     PASSED in 13.0s
  Stats over 5 runs: max = 13.0s, min = 1.1s, avg = 7.1s, dev = 3.8s
//sai_p4/instantiations/google/tests:p4_constraints_integration_test     PASSED in 1.3s
  Stats over 5 runs: max = 1.3s, min = 0.7s, avg = 1.0s, dev = 0.2s

Executed 170 out of 170 tests: 170 tests pass.
INFO: Build completed successfully, 179 total actions

@VSuryaprasad-HCL Can you pls fix the conflicts

@VSuryaprasad-HCL Can you pls fix the conflicts

Hi @kishanps Could you please review the PR.
Thanks.


Pshowalter updates to README

Minor edits after Refactoring.
Formatting - using bold for buttons, or form Input and Output
New screen capture.
@jmccausland

@pshowalter we just changed the UI so wait for my PR to get merged for new screen captures


Transform raw VDS XML files into structured data

30 second samples seem to land as individual XML files which have flow, count, and occupancy values for each detector. We'll want to develop a pipeline to roll those samples into structured timeseries parquet files.

@pingpingxiu-DOT-ca-gov can you update here with what you have been learning about the internal caltrans ETL pipelines?

Similar to #77, we are having to re-think the data access story for the VDS data. Closing this issue as not planned.


When filter on log for transcriptomics with individual gene, diagram does not reflect data

I filtered on 3, but the diagram that shows the log ratio graph is empty.  There should be 16 comparisons that have this

I can't reproduce the problem. Please add a link to the page, if you still see the problem.

This gene:
fig|511145.12.peg.2835 | b2741 | VBIEscCol129921_2835 | rpoS | RNA polymerase sigma factor RpoS  expand or shrink bread crumb
https://www.beta.patricbrc.org/portal/portal/patric/TranscriptomicsGeneExp?cType=feature&cId=PATRIC.511145.12.NC_000913.CDS.2864581.2865573.rev&sampleId=&colId=&log_ratio=&zscore=
----- Original Message -----
From: "mshukla1"<EMAIL_ADDRESS>To: "PATRIC3"<EMAIL_ADDRESS>Cc: "Rebecca Wattam"<EMAIL_ADDRESS>Sent: Wednesday, June 10, 2015 11:50:22 AM
Subject: Re: [patric3_website] When filter on log for transcriptomics with individual gene, diagram does not reflect data (#265)
I can't reproduce the problem. Please add a link to the page, if you still see the problem.

Reply to this email directly or view it on GitHub:
https://github.com/PATRIC3/patric3_website/issues/265#issuecomment-110807996

fixed and available on beta.

Checked and verified.


add Thai language resource

Hacktoberfest notes:

due to volume of submissions, we may not be able to review PRs that do not pass tests and do not have informative titles.
please read our contributing guidelines
be sure to check the output of Travis-CI for linter errors
if this is your first open source contribution, make sure it's not your last!

What does this PR do?
Add Resource(s) | Remove Resource(s) | Add info | Improve Repo
Add Resource.
For resources
Description
This book is a java programming concept in Thai language.
Why is this valuable (or not)?
It's for studying basic java programming concept.
How do we know it's really free?
It's from e-tech for studying purpose.
For book lists, is it a book? For course lists, is it a course? etc.
It's a book.
Checklist:

[✓] Not a duplicate
[✓] Included author(s) if appropriate
[✓] Lists are in alphabetical order
[✓] Needed indications added (PDF, access notes, under construction)

Apparently the linter does check thai either #5031.
Thanks, and welcome to free-programming-books!


Releases: swap out current release token for carbon-bot token

What will this achieve?
This will remove the use of my personal npm token used to publish our packages to npm and in place use a token generated by carbon-bot, which is how most of the other packages from carbon are released. Taylor has already created a new token from carbon-bot for our team and it is already in our repo secrets, we just need to update our release workflows.

How will success be measured?
When new token is used to publish our packages to the npm registry.

Additional information

Designs
Existing code
etc

This is now complete, tomorrow's (Nov. 8) release and all future releases will be made by the carbon-bot npm account


Add txns to blocks

Skimmed the changes, asked some questions (not meaning to nit-pick at this stage, just curious :)). Exciting!


(preview) should be removed from Restricted app groups (preview).

We see still "preview" in Endpoint DLP page for "Restricted app groups (preview)". This feature has been GA since mid of 2022. So, "(preview)" should be removed from Restricted app groups (preview).
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 34f25ec8-d5a8-4a9d-ca72-bd61eacd9dbc
Version Independent ID: 34f25ec8-d5a8-4a9d-ca72-bd61eacd9dbc
Content: Configure endpoint DLP settings - Microsoft Purview (compliance)
Content Source: microsoft-365/compliance/dlp-configure-endpoint-settings.md
Service: o365-seccomp
GitHub Login: @chrfox
Microsoft Alias: chrfox

@YoshihiroIchinose Thanks for your feedback! New issues and pull requests for this content are closed while we update this content to support upcoming feature improvements.


Fails to compile when using both state machine and dependencies in action.

I attempted following the instructions in issue number #94 submitted by @redboltz. This doesn't compile for me when I use it in an on_entry event. See the below code and error. This is a sub state machine but I get the same results regardless of whether or not it's in the parent or the sub state machine.
namespace TcpClientStateMachine
{
    using namespace std;
    using AsyncTcpClientPrivate = detail::AsyncTcpClientPrivate;

    // states
    struct Resolving {};
    struct ConnectingSocket {};

    // actions
    struct Resolve {
        auto operator()(const auto event, auto& sm, AsyncTcpClientPrivate& client) {
            cout << "Resolving . . . " << endl;
            client.mConnection->asyncResolve([&sm](const std::error_code& error){
                if(!error) {
                    sm.process_event(Event::ResolveComplete{});
                } else {
                    sm.process_event(Event::ErrorConnecting{error});
                }
            });
        }
    };

    struct Connect {
        auto operator()() {
            cout << "Connecting . . . " << endl;
        }
    };

    struct ConnectingStateMachine {
        Resolve resolve;
        Connect connect;

        auto operator()() const {
            /**
             * Initial state: *initial_state
             * Transition DSL: src_state + event [ guard ] / action = dst_state
             */
            return make_transition_table(
                *state<Resolving>          + event<Event::ResolveComplete>    / connect    = state<ConnectingSocket>,
                state<Resolving>           + on_entry<_>                      / resolve
            );
        }
    };

}

compiler output:
include/client/AsyncTcpClient/ConnectingStateMachine.h:35:14: note: candidate: template<class auto:3, class auto:4> auto TcpClientStateMachine::Resolve::operator()(auto:3, auto:4&, TcpClientStateMachine::AsyncTcpClientPrivate&)
         auto operator()(const auto event, auto& sm, AsyncTcpClientPrivate& client) {
              ^~~~~~~~
include/client/AsyncTcpClient/ConnectingStateMachine.h:35:14: note:   template argument deduction/substitution failed:
In file included from include/client/AsyncTcpClient.h:21:0,
                 from src/client/AsyncTcpClient.cpp:8:
include/boost/sml.hpp:1752:18: note:   candidate expects 3 arguments, 4 provided
     return object(event, sm, deps, subs);
            ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~

This only occurs when I'm using the state machine. When only use the signature: auto operator()(const auto& event, AsyncTcpClientPrivate& client), it compiles fine.

@krzysztof-jusiak I've been going through the code to try to find the problem and am having trouble finding where to even start. If you can give me a hint, I can try to provide a pull request.

@krzysztof-jusiak I traced through gdb and it appears that it's being added to the queue. Maybe the event loop for the queue is not being processed?
Also, to clarify what's happening, this makes a call to asio::ip::tcp::async_resolve and, when it's complete, executes the callback which executes processEvent. This happens after the call to resolve is complete.

The problem it would seem still persists. My example:
#include <boost/sml.hpp>
#include <cassert>
#include <functional>
#include <iostream>
#include <queue>
#include <utility>

struct DoStuff {};
struct SomeEvent {};

template <typename func>
struct DelayedCall {
  void set_callback(func f) { f_ = std::move(f); }
  void invoke() { f_(); }
  func f_;
};

struct SomeTransitions {
  auto operator()() const noexcept {
    auto delayDoingStuff = [](
        boost::sml::back::process<DoStuff> processEvent,
        DelayedCall<std::function<void()>>& delayer) -> void {
      delayer.set_callback([processEvent]() mutable {
        processEvent(DoStuff{});
        std::cout << "Called it!" << std::endl;
      });
    };

    using namespace boost::sml;
    return make_transition_table(
        *"initial"_s + event<SomeEvent> / delayDoingStuff,
        "initial"_s + event<DoStuff> = X);
  }
};

int main(int argc, char* argv[]) {
  using namespace boost::sml;
  using namespace std;

  DelayedCall<std::function<void()>> delayer;
  sm<SomeTransitions, process_queue<queue>> machine{delayer};
  assert(machine.is("initial"_s));

  machine.process_event(SomeEvent{});
  delayer.invoke();

  assert(machine.is(X));
  return 0;
}

Output is:
Called it!
timed: ../src/main.cpp:47: int main(int, char **): Assertion `machine.is(X)' failed.
Aborted (core dumped)

I also have this problem.
A working example on how to do async calls that sends events would be really helpful to people I think.


Add KLayout

Needed for things like DRC checks and GDS generation.

Ref: hdl.github.io/awesome/items/klayout

Hi, I would like to work on this one. However, It looks like we could use just the KLayout python module. This way we don't need to include to the workspace a big dependency which is Qt.

Cool. If the python module works we can use that. Thanks for working on it.


Authorisation not working

Not much we can do about it - it stopped working in chrome too...

Login is no longer available via GCP (may be rutracker itself blocked it), not much we can do about it other then migrating to, for example, TOR.

Fixed with TOR.


Fix: Box plot does not plot columns against index

Fixes #226
Hi @risenW, how does this look?
Let me know if this needs any changes.
Thanks

Fixes #226
Hi @risenW, how does this look?
Let me know if this needs any changes.
Thanks

Looks good. Thanks!

Hey, unsure if this fix caused the docs to go out of date but i'm not getting what i would expect to see


Clean initial char

I using phone mask, when the activity start the edit text start with the first char '('
How I prevent this?
eg.:
(XXX)XXX-XX-XX
When activity start:
(

Any pull-request is very welcome. Please, don't hesitate to contact me in case if you're interested in it - I'll provide you all the support.


Players can not create strenght potion when old alchemy stand is enabled

Players can not create strenght potion when old alchemy stand is enabled.
It is because they can not put blaze powder to brewing slot (not fuel).

@gvlfm78, this is your module, just add a slot check and this'll be fixed :P

@piter909 Can you try with this test version and see if the bug persists?

Plugin updated. After restart (tomorrow), I will post if it works or not :) Thanks!

Do you not have plugman?

Reloaded with plugman. I was not doing it just for safety (crash risk etc.).
Plugin still does not work. I can now click on blaze powder and put it correctly. It is starting brew, but.. When I close stand and reopen it, brewing is stoping.

Yeah I realised too late. I believed I have fixed that in this other test version.

@piter909 Does it work with the test version?

I will test it today.

@piter909 So does it work? I will be releasing a new OCM version very soon


VBScriptEngine, security issues

I have an issue with all computers in AD:
I'm getting this error after creating VBScriptEngine with debugging flag:
either a required impersonation level was not provided, or the provided impersonation level is invalid. (Exception from HRESULT: 0x80070542)
The problem is around this line:
if (processDebugManager.TryAddApplication(debugApplication, out debugApplicationCookie))
Any ideas which restriction could it be?

@andreyzapt Are you aware of any policy settings that affect users' ability to debug applications?

@ClearScriptLib "Run As Administrator" -> same result. What policies are you talking about?

There's SeDebugPrivilege, but that shouldn't be required for debugging one's own processes.
The error code appears to be HRESULT_FROM_WIN32(ERROR_BAD_IMPERSONATION_LEVEL), but that isn't listed in the AddApplication documentation.
The odd thing is that TryAddApplication simply returns false for any error code, so it isn't clear how an exception is being raised. Can you provide a stack trace?

Please reopen this issue if you can provide further information.


Segmenting images with a nuclear stain and membrane stain

Could cellpose be trained to identify cell boundaries using a cell membrane stain and a nuclear stain such as the below example?  If so, which model should I start with to iteratively train?  Or should I start from scratch?  Thanks!

The cyto2 model works with this data.
Results below using just the green channel (mid) and with adding the optional nuclear blue channel (right).
(Cell diameter set at 80)

Thank you!  I tried cyto2 with cell diameter at 80 and the green and optional nuclear channels set as you recommended.  There is a caveat, this image is actually once slice in a 3D stack, and it works significantly less well when I run it on the whole stack.  A couple of questions:

Why are some of the borders of my 3d segmented imaged (below) so jagged and oversegmented?  Do I need to smooth my images before running cellpose?

If I do start iteratively building a model for 3D stacks, how to I inform the model that called ROIs on different z-planes are part of the same object?  Or should I stick to 2-d and assemble them into 3d objects in post?

Thanks!

Update:  I know now that cellpose trains only in 2D, but that the 2D training enhances the 3D accuracy.
I also realized that to segment in 3D i needed to run 'python -m cellpose --do_3D'.  This gave better results.

When I split the image below, how should chan be set?Thanks.

@jzinski @CQ981001 please look for usage-related help (ie things that are not bugs) on the image.sc forum


Allow data imports into Docsite Examples

Feature Request
Allow data imports into Docsite Examples, such as items for a List, in order not to import them every time in each example file.

Are you mean something like there?
import React from 'react'
import { Dropdown } from 'semantic-ui-react'

import { friendOptions } from '../common' // <-- shared data

const DropdownExampleSelection = () => (
  <Dropdown placeholder='Select Friend' fluid selection options={friendOptions} />
)

https://react.semantic-ui.com/modules/dropdown/#types-selection

Proposal:
Create a mocks folder under docs/ that takes a dependency on faker npm module. faker has a nice API for providing all sorts of mock data types. See:

repo: https://github.com/marak/Faker.js/
usage: docs/src/prototypes/chatPane/services/dataMock.ts) in our repo.

API:
Create a simple API for returning mock data; API proposal:
import { OneOrArray } from 'types/utils'

type MockDataValueType = string | number | Date | boolean
type MockDataType =
  | 'word'
  | 'sentence'
  | 'name'
  | 'number'
  | 'date'
  | 'dateAsString'
  | 'id'
  | 'boolean'
  | 'imgSrc'

type MockDataOptions = {
  props: { [key: string]: OneOrArray<MockDataType | ((index: number) => any)> }
  count?: number
}

const getMockData = (
  dataType: MockDataType,
  count: number = 1,
): OneOrArray<MockDataValueType> => {
  // TODO
  return undefined
}

const getMockDataObj = (
  options: MockDataOptions
): OneOrArray<{ [key: string]: any }> => {
  // TODO
  return undefined
}

export default {
  getMockData,
  getMockDataObj,
}

Usage:
These are examples from current examples for Accordion, Chat and FormFIeld components example.
Example 1 - AccordionExample:
const panels = getMockDataObj({
  props: { key: 'id', title: 'sentence', content: 'sentence' },
  count: 2,
})

instead of
const panels = [
  {
    key: 'p',
    title: 'What is a point?',
    content: 'Use the haptic SDD circuit, then you can index the redundant pixel!',
  },
  {
    key: 'd',
    title: 'What is a dimension of a point?',
    content: 'We need to copy the primary USB firewall!',
  },
]

Example 2 - AccordionExample:
const items = getMockDataObj({
  props: {
    key: 'id',
    media: index => <Image key={`image-${index}`} avatar src={getMockData('imgSrc')} />,
    header: 'word',
    stringProp1: 'word',
    stringProp2: 'word',
  },
  count: 3,
})

instead of
const items = [
  {
    key: 'a',
    media: <Image avatar src="//placehold.it/100" />,
    header: 'cat',
    stringProp1: 'test-word',
    stringProp2: 'test-word',
  },
  {
    key: 'b',
    media: <Image avatar src="//placehold.it/100" />,
    header: 'dog',
    stringProp1: 'test-word',
    stringProp2: 'test-word',
  },
  {
    key: 'c',
    media: <Image avatar src="//placehold.it/100" />,
    header: 'mouse',
    stringProp1: 'test-word',
    stringProp2: 'test-word',
  },
]

Example 3 - ChatExample:
const janeAvatar = {
  image: 'public/images/avatar/small/ade.jpg',
  status: { color: 'green', icon: 'check' },
}

const items = getMockDataObj({
  props: {
    content: index =>
      index === 4 ? (
        <Divider content="Today" type="primary" important />
      ) : (
        <Chat.Message
          {...getMockDataObj({
            props: {
              content: 'sentence',
              author: 'name',
              timestamp: 'dateAsString',
              ...(index % 2 ? { mine: () => true } : { avatar: () => janeAvatar }),
            },
          })}
        />
      ),
  },
  count: 6,
})

instead of
const janeAvatar = {
  image: 'public/images/avatar/small/ade.jpg',
  status: { color: 'green', icon: 'check' },
}

const items = [
  {
    content: (
      <Chat.Message content="Hello" author="John Doe" timestamp="Yesterday, 10:15 PM" mine />
    ),
  },
  {
    content: (
      <Chat.Message
        content="Hi"
        author="Jane Doe"
        timestamp="Yesterday, 10:15 PM"
        avatar={janeAvatar}
      />
    ),
  },
  {
    content: (
      <Chat.Message
        content="Would you like to grab a lunch?"
        author="John Doe"
        timestamp="Yesterday, 10:16 PM"
        mine
      />
    ),
  },
  {
    content: (
      <Chat.Message
        content="Sure! Let's try the new place downtown"
        author="Jane Doe"
        timestamp="Yesterday, 10:15 PM"
        avatar={janeAvatar}
      />
    ),
  },
  {
    content: <Divider content="Today" type="primary" important />,
  },
  {
    content: (
      <Chat.Message
        content="Let's have a call"
        author="John Doe"
        timestamp="Today, 11:15 PM"
        mine
      />
    ),
  },
]

Example 4 - FormExample:
const fields = [
  ...(getMockDataObj({
    props: {
      label: 'word',
      name: 'word',
      id: 'name',
      key: 'name',
      required: 'boolean',
    },
    count: 3,
  }) as []),
  getMockDataObj({
    props: {
      control: () => ({ as: Button, content: getMockData('word') }),
      key: 'id',
    },
  }),
]

instead of:

const fields = [
  {
    label: 'First name',
    name: 'firstName',
    id: 'first-name-shorthand',
    key: 'first-name',
    required: true,
  },
  {
    label: 'Last name',
    name: 'lastName',
    id: 'last-name-shorthand',
    key: 'last-name',
    required: true,
  },
  {
    label: 'I agree to the Terms and Conditions',
    control: { as: 'input' },
    type: 'checkbox',
    id: 'conditions-shorthand',
    key: 'conditions',
  },
  { control: { as: Button, content: 'Submit' }, key: 'submit' },
]

fyi @silviuavram

If we will implement something line this, there should be a way to show generated data as json. However, I'm not sure that it's the best option at all.
For me as newbie user will be unbelievable hard to understand what is going in example 3 or example 4.

yea, it's just something that I was thinking about a while ago and wanted to express it here as well; it's definitely not readable and simple enough in current form, the main idea is to find a way to send to our mock function what we'd like the mock data to look like.
The tricky thing here is to provide complete flexibility in shaping the mocked data to the following type:
type Primitive = string | boolean | number | Date
type Collection = { [key: string]: any} | any[]
type MockDataType = Collection<Collection | Primitive> | Primitive

Let's not do this.
Example criteria:

Fit on one page
User can understand the whole example from the source
User can copy/paste to their project and it works
Can be executed locally or in the browser

Moving critical parts, like data, out of the example make it to where the user can't see what's happening when reading the doc examples.  They also can't copy it and have it run elsewhere.  This would also complicate the codesandbox links since we'd need to include other local files.


[enhancement] Add missing bower.json.

Hey, maintainer(s) of mitsuhiko/classy!
We at VersionEye are working hard to keep up the quality of the bower's registry.
We just finished our initial analysis of the quality of the Bower.io registry:
7530 - registered packages, 224 of them doesnt exists anymore;
We analysed 7306  existing packages and  1070 of them don't have bower.json on the master branch ( that's where a Bower client pulls a data ).
Sadly, your library mitsuhiko/classy is one of them.
Can you spare 15 minutes to help us to make Bower better?
Just add a new file bower.json and  change attributes.
{
  "name": "mitsuhiko/classy",
  "version": "1.0.0",
  "main": "path/to/main.css",
  "description": "please add it",
  "license": "Eclipse",
  "ignore": [
    ".jshintrc",
    "**/*.txt"
  ],
  "dependencies": {
    "<dependency_name>": "<semantic_version>",
    "<dependency_name>": "<Local_folder>",
    "<dependency_name>": "<package>"
  },
  "devDependencies": {
    "<test-framework-name>": "<version>"
  }
}

Read more about bower.json on the official spefication and nodejs semver library has great examples of proper versioning.
NB! Please validate your bower.json with jsonlint before commiting your updates.
Thank you!
Timo,
twitter: @versioneye
email<EMAIL_ADDRESS>VersionEye - no more legacy software!

:+1:


empty data

Data is empty https://github.com/ipitio/backage/blob/master/index/LizardByte/Sunshine/sunshine.json

Me too https://github.com/ipitio/backage/blob/master/index/timeplus-io/proton/proton.json
Please help. Appreciate.

Sorry about that, I accidentally disappeared the data. The database will be rebuilt soon!

Might also be a good idea to publish the "data" to a gh-pages branch instead of the default branch. Then your data is separate from your source code... and it will be published to a site at ipitio.github.io/backage/...

I think the issue has been resolved for everyone here; I'll close it if no one says otherwise.

Working for me

Might also be a good idea to publish the "data" to a gh-pages branch instead of the default branch. Then your data is separate from your source code... and it will be published to a site at ipitio.github.io/backage/...

Just did this! Should be good for a while, until the index grows too large for Pages.

I don't think you'll have an issue. I have some gh-pages with 500,000 files.


Scale description in Sections

Proposed Changes

Is now:

I have also adapted the level mixin to be more friendly with future customization such as this one and extracted the rules that were not required in the mixin.
Potential Breaking Changes
None
Acceptance Criteria

[x] The proposed changes are covered by unit tests
[x] The potential breaking changes are clearly identified
[x] README.md is adjusted to reflect the proposed changes (if relevant)

Codecov Report

Merging #1339 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1339      +/-   ##
==========================================
- Coverage   99.43%   99.43%   -0.01%
==========================================
  Files         363      363
  Lines        9099     9098       -1
  Branches     2218     2215       -3
==========================================
- Hits         9048     9047       -1
  Misses         51       51

Impacted Files
Coverage Δ

...nts/navigation/pagination/NavigationPagination.tsx
100% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28b9746...c8bc2bb. Read the comment docs.


Fix intermittent failures with freetext and stamp tests

They're potentially due to some concurrent access to the system clipboard. So this patch makes them sequential.

/botio integrationtest

/botio integrationtest

/botio integrationtest


Deprecate Python version 2

Python v2 dies at the end of 2019, so ATCv4 - which will probably be released toward the end of this year or sometime the following year in 2020 - shouldn't support it anymore.

Closed by #3639


Fix/creating dir with os not fs

Codecov Report

Merging #58 (2d378b5) into main (5b94107) will decrease coverage by 0.21%.
The diff coverage is 40.00%.

@@            Coverage Diff             @@
##             main      #58      +/-   ##
==========================================
- Coverage   38.76%   38.54%   -0.22%
==========================================
  Files          10       10
  Lines         810      812       +2
==========================================
- Hits          314      313       -1
- Misses        465      468       +3
  Partials       31       31

Impacted Files
Coverage Δ

pkg/database/db.go
66.31% <40.00%> (-2.51%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5b94107...2d378b5. Read the comment docs.