Bug: Issues index dtype=i8 with Inner Product Metrics

Describe the bug
Hi, I am using uSearch with Integer 8 index dtype.
I am having 928K embedding which I try to add to both i8 and float16 index  with Inner Product Metrics and then search  10 starting embedding  to get top 10 matches.
I see with float16 I get same vector getting recognised in top 10 but with I8 I didn't get them.
Also when I change metrics from MetricKind.Inner Product to MetricKind.L2sq those vectors starts coming but using this the distances are getting scaled differently(distances for IP are of order 0.61365813 and for l2sq 4517 maybe these are also because of i8) which I am not able to understand.
Also adding embedding to MetricKind.InnerProduct with I8 dtype is  too slow.
Steps to reproduce
I use following code to create and populate index with I8 dtype:-
searcher = Index(
 ndim=1024,
 metric=MetricKind.InnerProduct,
 dtype="i8",
 connectivity=32,
 expansion_add=128,
 expansion_search=64,
 multi=False,
)

searcher.add(range(len(embeddings)), embeddings, log=True)
matches = searcher.search(embeddings[:10], 10)
print(matches.keys)

I get following results:

when I only change metric=MetricKind.L2sq I start getting the expected embeddings (same index as first key).
Note:- With other dtype like f16,f32 MetricKind.InnerProduct works completely fine.
Expected behavior
Ideally I should get something like this with I8+ MetricKind.InnerProduct
The following is result I get with MetricKind.L2sq:

USearch version
2.11.7
Operating System
Ubuntu 22.04
Hardware architecture
x86
Which interface are you using?
Python bindings
Contact Details
<EMAIL_ADDRESS>Are you open to being tagged as a contributor?

[ ] I am open to being mentioned in the project .git history as a contributor

Is there an existing issue for this?

[X] I have searched the existing issues

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi @keshusharmamrt! Can you please try the Cosine distance and share the dtype and shape of embeddings?

hi @ashvardanian , Thanks for prompt reply. I tried cosine distance too but it also seems to not work perfectly as I was getting keys via MetricKind.L2sq. Also using this approach all the distances I am getting are near about zero.
Sharing screenshot for same.

The Details of embeddings are as follows:

I think the issue might be in the distribution that that produces those embeddings. USearch has no learned quantization, all it does is linear scaling from [-1.0, 1.0] to integers in [-100, 100].
If you know that your values won't properly scale into that range, you may want to somehow normalize them or just directly convert to int8 and later pass to USearch. That should work 🤗
PS: I'd appreciate a PR for the main README, if you think it makes sense to list it there 🤗

Hi, Thanks for your suggestion but It doesn't seems to be working too.
Sharing the Example Code snippet which demonstrates the issue with uniform embeddings.
import numpy as np
from usearch.index import Index, MetricKind

np.set_printoptions(formatter={"float": "{: 0.4f}".format})
np.random.seed(42)
num_centroids = 200
num_examples = 100
num_embeddings = num_examples * num_centroids
dim = 1024

centroids = np.random.uniform(low=-1, high=1, size=(num_centroids, dim))
offsets = np.random.uniform(low=-0.05, high=0.05, size=(num_embeddings, dim))
centroids = np.tile(centroids, (num_examples, 1))
embeddings = centroids + offsets
embeddings = embeddings / np.linalg.norm(embeddings, axis=1)[:, None]

for dtype in ["f32", "i8"]:
    print("Running test for dtype = ", dtype)
    for metric in [MetricKind.IP, MetricKind.Cosine, MetricKind.L2sq]:
        print("Running test ", dtype, metric)
        searcher = Index(
            ndim=dim,
            metric=metric,
            dtype=dtype,
            connectivity=32,
            expansion_add=128,
            expansion_search=64,
        )

        searcher.add(range(len(embeddings)), embeddings, log=True)
        matches = searcher.search(embeddings[:5], 4)

        print("Keys = \n", matches.keys)
        print("Distances = \n", matches.distances)

        exact_distances = 1 - (embeddings[:5] @ embeddings.T)
        exact_distances.sort(axis=1)
        if metric == MetricKind.L2sq:
            exact_distances = 2 * exact_distances

        print("Exact Distances = \n", exact_distances[:, :4])

I get following observations by running above code:-

for f32 we see that Distances are equal to Exact Distances
for i8 - we get completely different values for each metric and differ by large margin. Also for int8 L2sq and Cos seem to return similar order.
for i8  with Inner Product the keys are ordered incorrectly, distances are negative.
We use L2 normalized vectors so IP and Cos metrics should give the same results
Maybe this code snippet can help understand issues we are facing with Integer 8 dtype.

Is this expected when quantizing? If one is quantizing the embeddings, shouldn't one prefer the Hamming distance?:

Binarization is to compress a full-precision weight into a binary code that only occupy 1 bit. It is widely used in the embedding-based similarity search of recommender systems [28, 54], since the binary embeddings have two distinct advantages compared to the full-precision ones: (1) less memory or disk cost for storing embeddings; (2) higher inference efficiency as the similarity (i.e., inner product) between binary embeddings can be calculated more efficiently through the Hamming distance, which has been proved in [73]

Reference: Li, S., Guo, H., Tang, X., Tang, R., Hou, L., Li, R., & Zhang, R. (2024). Embedding Compression in Recommender Systems: A Survey. ACM Computing Surveys, 56(5), 1-21.
Expanding on @keshusharmamrt example code snippet:
from sentence_transformers.quantization import quantize_embeddings

embeddings_f32 = embeddings.astype(np.float32)
embeddings_i8 = quantize_embeddings(embeddings_f32, precision="binary")

searcher1 = Index(
    ndim=embeddings_f32.shape[1],
    metric="hamming",
    dtype="i8",
    connectivity=32,
    expansion_add=128,
    expansion_search=64,
)
print("dim: ", embeddings_f32.shape[1])
searcher1.add(range(len(embeddings_f32)), embeddings_f32)

searcher2 = Index(
    ndim=embeddings_i8.shape[1],
    metric="hamming",
    dtype="i8",
    connectivity=32,
    expansion_add=128,
    expansion_search=64,
)
print("dim: ", embeddings_i8.shape[1])
searcher2.add(range(len(embeddings_i8)), embeddings_i8)

query = embeddings_f32[:5]
query_i8 = quantize_embeddings(query, precision="binary")

matches1 = searcher1.search(query, 4)
print("int8 quantization:")
print("Keys = \n", matches1.keys)
print("Distances = \n", matches1.distances)

matches2 = searcher2.search(query_i8, 4)
print("binary quantization:")
print("Keys = \n", matches2.keys)
print("Distances = \n", matches2.distances)

Results:
dim:  1024
dim:  128
int8 quantization:
Keys =
 [[    0 16000 12400   400]
 [    1 11801 14801  4801]
 [    2  2402  7002 19002]
 [    3  1403  1003  8603]
 [    4 17604  6404 14004]]
Distances =
 [[ 0.0000  309.0000  329.0000  343.0000]
 [ 0.0000  346.0000  377.0000  382.0000]
 [ 0.0000  328.0000  332.0000  341.0000]
 [ 0.0000  362.0000  366.0000  371.0000]
 [ 0.0000  334.0000  340.0000  354.0000]]
binary quantization:
Keys =
 [[    0 18000  1800 16600]
 [    1 17601 10001 18201]
 [    2  3002  8002 12402]
 [    3  7603 13003 16403]
 [    4  8804 12604  2604]]
Distances =
 [[ 0.0000  9.0000  10.0000  12.0000]
 [ 0.0000  12.0000  12.0000  12.0000]
 [ 0.0000  13.0000  14.0000  14.0000]
 [ 0.0000  5.0000  7.0000  8.0000]
 [ 0.0000  7.0000  8.0000  8.0000]]

Note that each query element is included in the corresponding matches and that $d(x,x) = 0$.

@keshusharmamrt, your points are extremely close to each other to differentiate them with a simple scale-up. You'd need a custom quantization scheme. I'd recommend to stick to f16. I've cross-validated the output keys with a slightly extended script:
import numpy as np
from usearch.index import Index, MetricKind

np.set_printoptions(formatter={"float": "{: 0.4f}".format})
np.random.seed(42)
num_centroids = 200
num_examples = 100
num_embeddings = num_examples * num_centroids
dim = 1024

centroids = np.random.uniform(low=-1, high=1, size=(num_centroids, dim))
offsets = np.random.uniform(low=-0.05, high=0.05, size=(num_embeddings, dim))
centroids = np.tile(centroids, (num_examples, 1))
embeddings = centroids + offsets
embeddings = embeddings / np.linalg.norm(embeddings, axis=1)[:, None]

for dtype in ["f32", "f16", "i8"]:
    print("Running test for dtype = ", dtype)
    for metric in [MetricKind.IP, MetricKind.Cosine, MetricKind.L2sq]:
        searcher = Index(
            ndim=dim,
            metric=metric,
            dtype=dtype,
            connectivity=32,
            expansion_add=128,
            expansion_search=64,
        )
        print("Running test ", dtype, metric, searcher.hardware_acceleration)

        searcher.add(range(len(embeddings)), embeddings, log=True)
        matches = searcher.search(embeddings[:5], 4)

        print("Keys = \n", matches.keys)
        print("Distances = \n", matches.distances)

        exact_distances = 1 - (embeddings[:5] @ embeddings.T)
        exact_keys = exact_distances.argsort(axis=1)
        exact_distances.sort(axis=1)
        if metric == MetricKind.L2sq:
            exact_distances = 2 * exact_distances

        print("Exact Keys = \n", exact_keys[:, :4])
        print("Exact Distances = \n", exact_distances[:, :4])

@adolfogc, sadly the functionality of quantize_embeddings isn't enough to account for such small perturbations. It would work very poorly.

To quantize float32 embeddings to binary, we simply threshold normalized embeddings at 0: if the value is larger than 0, we make it 1, otherwise we convert it to 0.

I also believe, that you should may have wanted to use a different precision argument for binary indexes. It's not mentioned on the main page, but is listed in the docs - b1. Here is the updated snippet:
import numpy as np
from usearch.index import Index, MetricKind

np.set_printoptions(formatter={"float": "{: 0.4f}".format})
np.random.seed(42)
num_examples = 100
num_embeddings = num_examples * num_centroids
dim = 1024

embeddings = np.random.uniform(low=-1, high=1, size=(num_embeddings, dim))

from sentence_transformers.quantization import quantize_embeddings

embeddings_f32 = embeddings.astype(np.float32)
embeddings_b1 = quantize_embeddings(embeddings_f32, precision="binary").astype(np.uint8)

searcher_f32 = Index(
    ndim=dim,
    metric="cos",
    dtype="f32",
)
print("dim: ", embeddings_f32.shape[1])
searcher_f32.add(range(len(embeddings_f32)), embeddings_f32)

searcher_b1 = Index(
    ndim=dim,
    metric="hamming",
    dtype="b1",
)
print("dim: ", embeddings_b1.shape[1])
searcher_b1.add(range(len(embeddings_b1)), embeddings_b1)

queries_f32 = embeddings_f32[:5]
queries_b1 = embeddings_b1[:5]

matches_f32 = searcher_f32.search(queries_f32, 4)
print("Original:")
print("Keys = \n", matches_f32.keys)
print("Distances = \n", matches_f32.distances)

matches_b1 = searcher_b1.search(queries_b1, 4)
print("Binary:")
print("Keys = \n", matches_b1.keys)
print("Distances = \n", matches_b1.distances)

I think I should update the main README snippet. Will mark you as a co-author 🤗

I also believe, that you should may have wanted to use a different precision argument for binary indexes. It's not mentioned on the main page, but is listed in the docs - b1. Here is the updated snippet:

@ashvardanian thank you for pointing that out! In that case, I think the snippet could use the ubinary precision provided by sentence_transformers:
quantize_embeddings(embeddings_f32, precision="ubinary")

Defined like so:
https://github.com/UKPLab/sentence-transformers/blob/de3b29890344139d7b0e582cc7e00f18b98dc3d3/sentence_transformers/quantization.py#L380
I thought that since the int8 dimension was "lower" the saved index would be smaller in size, but it turned out that using b1 also compressed the serialized index.
Also, perhaps this (using b1 and ubinary) could be suggested as an improvement to the semantic_search_usearch function of sentence_transformers, here:
https://github.com/UKPLab/sentence-transformers/blob/de3b29890344139d7b0e582cc7e00f18b98dc3d3/sentence_transformers/quantization.py#L238
What do you think?
BTW, superb library, thank you for making it available!

Hey @ashvardanian , @keshusharmamrt  has prepared very simple example which shows that for the same data when using different metrics and int8 quantization we are getting unexpected results. It looks that there is a bug in the int8 quantization (maybe only for python client). You can see it that for dot metric the order of keys is random, but for l2sq is fine, but values of distances are too large. Everything works as expected for f16 and f32 quantization.
We also compared usearch with faiss and qdrant for the same HNSW parameters and dot metric of normalized vectors.

Connectivity
Expansion Add
Expansion Search

32
128
64

Our results for 1M of embeddings of size 1024

Method
Recal@1

usearch f16
0.98672

usearch int8
0.91227

qdrant int8
0.96034

qdrant int8 (with rescore)
0.98555

faiss int8
0.98278

For faiss we use the index setup from usearch benchmark repository
As you can see the results for usearch int8 are far from faiss and qdrant equivalents.

https://github.com/UKPLab/sentence-transformers/blob/de3b29890344139d7b0e582cc7e00f18b98dc3d3/sentence_transformers/quantization.py#L238
What do you think?

@adolfogc, sure, that makes a lot of sense! I've proposed the patch.

@kmkolasinski and @keshusharmamrt thank you for benchmarks and comparisons, those are very helpful! Can I also ask you to report the index.hardware_acceleration output, the USearch version, as well as the OS and the CPU model? Depending on those parameters, I'll know if one of the following kernels from SimSIMD is used:
https://github.com/ashvardanian/SimSIMD/blob/4b6a0411dc807eae3749b2702acaa1e3d4bad020/include/simsimd/spatial.h#L315-L361
https://github.com/ashvardanian/SimSIMD/blob/4b6a0411dc807eae3749b2702acaa1e3d4bad020/include/simsimd/spatial.h#L669-L734
https://github.com/ashvardanian/SimSIMD/blob/4b6a0411dc807eae3749b2702acaa1e3d4bad020/include/simsimd/spatial.h#L968-L1008

On older versions I've made this mistake.
    // Unfortunately we can't use the `_mm512_dpbusd_epi32` intrinsics here either,
    // as it's assymetric with respect to the sign of the input arguments:
    //      Signed(ZeroExtend16(a.byte[4*j]) * SignExtend16(b.byte[4*j]))
    // So we have to use the `_mm512_dpwssd_epi32` intrinsics instead, upcasting
    // to 16-bit beforehand.
    ab_i32s_vec = _mm512_dpwssd_epi32(ab_i32s_vec, a_vec, b_vec);
    a2_i32s_vec = _mm512_dpwssd_epi32(a2_i32s_vec, a_vec, a_vec);
    b2_i32s_vec = _mm512_dpwssd_epi32(b2_i32s_vec, b_vec, b_vec);

hi, @ashvardanian ,Thanks for your support.
Sharing the Details as you asked.
index.hardware_acceleration Output:-

Usearch Version = 2.12.0
OS =Ubuntu 22.04.4 LTS
CPU Model:- 13th Gen Intel® Core™ i5-13600HX × 20

That means, this kernel is used. I've just tried implementing a few tests for it on random data, but can't yet see a noticeable difference in distance calculations between that and serial code. Any chance you have an example of two arrays for which you get a big error, calling:
from simsimd import cos
cos(a, b)

On integer representations like i8 the conventional notion of normalization won't apply, as there are no values between 0 and 1. So anything that computes a dot-product of integer arrays would still need to renormalize them. USearch will swap ip with cos under the hood.

hi, I also Tried to use cos for bench marking using usearch with f16 and i8 (here too I see with i8 performance is bad)
HNSW config:

Connectivity
Expansion Add
Expansion Search

32
128
64

Sharing Entire Comparison we did for 1M of embeddings of size 1024

Method
Recall@1

usearch f16(ip)
0.98672

usearch i8(l2sq)
0.91227

usearch f16(cos)
0.98244

usearch i8(cos)
0.76728

qdrant int8
0.96034

qdrant int8 (with rescore)
0.98555

faiss int8
0.98278

Clearly with i8  the Usearch Metrics are poor.

Hi, @ashvardanian  if int8 requires some special treatment from the user side, could you provide us example on how do it or maybe there are some docs for it ?
All libraries we have tried so far do the quantization under the hood, so things like bucketing, scaling etc is handled internally by the lib.

using other words, I wonder what we can do on our side to get similar results as other frameworks for int8 ? because there are two options, either we are doing something wrong or there is indeed some problem with int8 quantization.

hi @ashvardanian, do you have any update regarding this?

A bit overwhelmed by other USearch-related projects, and haven’t had the chance to investigate deeper, @keshusharmamrt 😔

All libraries we have tried so far do the quantization under the hood, so things like bucketing, scaling etc is handled internally by the lib.

It’s true, most other libraries come with a lot of options for different quantization schemes. We don’t currently aim to enforce any of them or implement them in the C++ layer. But maybe we can add a couple of simple quantization schemes as utilities in the Python layer for now, Any chance you can contribute, @kmkolasinski?

Hi, I can't promise it. The results are weird and at this moment it looks like there is some bug.

We can check for bugs in the SimSIMD kernels, but they may already be sufficiently covered with tests.

ok, last question: let's say I would like to use int8 quantization and dot product metric. Are there some requirements for vectors to make it work properly ? do you have some example code which we could use and benchmark on our data for such scenario ?

Sure, the most obvious benchmark/example would be to generate random integers (instead of quantizing random floats), constructing the index, and measuring self-recall. Thats should work fine.

Hi @ashvardanian  @kimihailv we tried last thing. We took the benchmarking script from https://github.com/unum-cloud/usearch-benchmarks/tree/main and we tried to compare the metrics on a smaller dataset which is base.10M.fbin. Here is what we got:

We though we can use groundtruth.10K.ibin to verify metrics together with query.10K.fbin and base.10M.fbin, but it seems groundtruth.10K.ibin are not compatible with these query and train vectors.

The repository only points to 1B dataset which is too large for us to process. Unfortunately, we couldn't manage to find 100M data which are mentioned in the usearch benchmarking blog post.

The vectors in query.10K.fbin are different from the official ones query.public.10K.fbin from the https://research.yandex.com/blog/benchmarks-for-billion-scale-similarity-search. Which means that query vectors has been selected, which raise various questions in our minds. For example, how we can reproduce this blog post (https://www.unum.cloud/blog/2023-11-07-scaling-vector-search-with-intel) metrics ?

Also, there is a bug in the recall computation in the benchmarking code. In the run.py file groundtruth vector is of shape [10_000, 1] (https://github.com/unum-cloud/usearch-benchmarks/blob/main/run.py#L49)

groundtruth = load_matrix(groundtruth_path)[:, :1]

later you use these groundtruth indices to compute recall https://github.com/unum-cloud/usearch-benchmarks/blob/main/run.py#L82
recall_at_one = recall(nn_ids, groundtruth, ats=(1,))

but this should be replaced with
recall_at_one = recall(nn_ids, groundtruth[:, 0], ats=(1,))

or recall function should be fixed, as it adds new axis to the tensor causing a weird behaviour. Small example which demonstrates the issue

We tried to reproduce the results using base.10M.fbin vectors, but we are getting the same conclusions as when using our data. Integer 8 quantization is about 10% worse than f16. Here is the code you can use to verify it by yourself:

from tqdm import tqdm
import numpy as np
from usearch.index import Index
from usearch.io import load_matrix
from utils.metrics import recall

train_embeddings = load_matrix("data/base.10M.fbin")[:500_000]
query_embeddings = load_matrix("data/query.10K.fbin")

def best_match(query):
    return (train_embeddings @ query[:, None]).ravel().argmax()

groundtruths = []
for query in tqdm(query_embeddings, desc="Computing ground truths"):
    best_index = best_match(query)
    groundtruths.append(best_index)

groundtruths = np.array(groundtruths)

print("Benchmark f16 index")
i_half = Index(ndim=96, metric="cos", dtype="f16")
i_half.add(None, train_embeddings, log=True)
result = i_half.search(query_embeddings, 1, log=True)
recall_at_one = recall(result.keys, groundtruths, ats=(1,))
print(f" >> recall@1 = {recall_at_one[0]:.5f}")

print("Benchmark i8 index")
i_quarter = Index(ndim=96, metric="cos", dtype="i8")
i_quarter.add(None, train_embeddings, log=True)
result = i_quarter.search(query_embeddings, 1, log=True)
recall_at_one = recall(result.keys, groundtruths, ats=(1,))
print(f" >> recall@1 = {recall_at_one[0]:.5f}")

After running this code you will see
Benchmark f16 index
Add: 100%|██████████| 500000/500000 [00:17<00:00, 28269.71vector/s]
Search: 100%|██████████| 10000/10000 [00:00<00:00, 33128.90vector/s]
 >> recall@1 = 0.97220
Benchmark i8 index
Add: 100%|██████████| 500000/500000 [00:11<00:00, 42828.76vector/s]
Search: 100%|██████████| 10000/10000 [00:00<00:00, 51415.34vector/s]
 >> recall@1 = 0.86640

I think that this is a clear evidence that there is a bug in the int8 index, and we cannot simply solve it by adding some customer wrapper for index class. Hope this helps somehow.

@ashvardanian  final remarks and conclusions after spending more time on this issue:

Indeed scaling vectors values to range -1 and 1 improves recall for the test vectors of dim=96. Something which bothers me is that with this approach we are loosing some potential information as int8 range is (-128, 127) but usearch only uses (-100, 100), with this truncation we drop about 20% of potential information while bucketing. This 20% maybe important in some cases.

I think it would be nice if this -1, 1 scaling should be explicitly mentioned in the docs or readme, maybe usearch could do some validation of the input vectors to warn the user, that the vectors are not scaled optimally.

There is also one potential problem with int8 quantization, which can be suprising for unaware users i.e. if vectors values are larger than +/-1 quantization will create overflow issues. Here is an example:

you can imagine that the value flip can introduce some dramatic changes in the distance values.

It looks like there is indeed a bug in the "Inner Product" metric, as according to documentation and the code IP should compute 1 - dot(a, b), but when testing it locally it returns different values, here is an example

and different results for f16

From this you can see that IP is actually computing dot(a, b) / ||a|| ||b|| . This also explains why IP metric is producing terrible recall as the code expects the distance not similarity measure. However, I don't see where it is defined in the code, I found this in the code: https://github.com/unum-cloud/usearch/blob/5ea48c87c56a25ab57634a8f207f80ae675ed58e/include/usearch/index_plugins.hpp#L977 which seems to be correct. I use usearch version 2.12.0.

Regarding my previous comment, please check the benchmarking scripts and the recall function. You can easily check that for incorrect groudtruth shape array it will overestimate recall@1 by large marging. This is especially visible for int8.

@keshusharmamrt @adolfogc @kmkolasinski, I believe this issue is resolved now. I've extended the docs to highlight the improved behavior. Let me know if the issue persists 🤗

okey sure and thanks let's close it, in the case of problems we will create new ticket :+1:


feat: actions for no cni eks cluster creations

In this commit we add the new way to create an eks cluster without addons. We then use coredns and kube-proxy as the approach here is just to create the cluster without CNI so we can install cilium directly. We then added another action to create the nodegroups in a different step of the workflow.
Please ensure your pull request adheres to the following guidelines:

[ ] For first time contributors, read Submitting a pull request
[ ] All code is covered by unit and/or runtime tests where feasible.
[ ] All commits contain a well written commit description including a title,
description and a Fixes: #XXX line if the commit addresses a particular
GitHub issue.
[ ] If your commit description contains a Fixes: <commit-id> tag, then
please add the commit author[s] as reviewer[s] to this issue.
[ ] All commits are signed off. See the section Developer’s Certificate of Origin
[ ] Provide a title or release-note blurb suitable for the release notes.
[ ] Are you a user of Cilium? Please add yourself to the Users doc
[ ] Thanks for contributing!

Fixes: #1515
<!-- Enter the release note text here if needed or remove this section! -->

Actually nevermind, can't test yet because the build workflow didn't run because of the initial related checks fail. Cancelling EKS run.

/ci-eks


maybe this function can not be used by default now

Hi,med0x2e:
I used to test the tool and some other way like this principle,but I can't execute my code.So I think maybe this had be patch by default.
I test the code,and the function like the web:https://research.checkpoint.com/2022/can-you-trust-a-files-digital-signature-new-zloader-campaign-exploits-microsofts-signature-verification-putting-users-at-risk/

Oh,this is my problem.the function can be used as before


Process bamToH5 (WTnone) terminated with an error exit status (1)

A workshop attendee's run failed on the vignette with:
NOTE: Process `bamToH5 (WTnone)` terminated with an error exit status (1) -- Error is ignored

bamToH5 failed on both WTnone and WT3AT.
Suggest rerunning the workflow:
$ nextflow run -ansi-log false prep_riboviz.nf -params-file vignette/vignette_config.yaml
``
Then, use the unique run name, and `nextflow log` to get the `work/` directory. For example (remember to replace `friendly_hawking` with your own unique run name):
```console
$ nextflow log friendly_hawking -f hash,workdir -filter "name == 'bamToH5 (WTnone)'"
b9/d4f1c6	/home/ubuntu/riboviz/work/b9/d4f1c6374f7f61ecf6fd08a520c919

Then view the bash command, standard output and standard error files. For example:
$ cat work/b9/d4f1c6374f7f61ecf6fd08a520c919/.command.sh
#!/bin/bash -ue
Rscript --vanilla /home/ubuntu/riboviz/rscripts/bam_to_h5.R            --num-processes=1            --min-read-length=10            --max-read-length=50            --buffer=250            --primary-id=Name                        --dataset=vignette            --bam-file=WTnone.bam            --hd-file=WTnone.h5            --orf-gff-file=yeast_YAL_CDS_w_250utrs.gff3            --is-riboviz-gff=true            --feature=CDS            --stop-in-feature=false

$ cat work/b9/d4f1c6374f7f61ecf6fd08a520c919/.command.out
[1] "Created by: RiboViz"
[1] "Date: 2021-07-02 06:07:51"
[1] "File: /home/ubuntu/riboviz/rscripts/bam_to_h5.R"
[1] "Version: commit 46c168c0ce6d154879bb9060010ff1736bd50742 date 2021-06-24 13:57:28 GMT"
[1] "bam_to_h5.R running with parameters:"
$bam_file
[1] "WTnone.bam"

$orf_gff_file
[1] "yeast_YAL_CDS_w_250utrs.gff3"

$feature
[1] "CDS"

$min_read_length
[1] 10

$max_read_length
[1] 50

$buffer
[1] 250

$primary_id
[1] "Name"

$secondary_id
[1] NA

$dataset
[1] "vignette"

$stop_in_feature
[1] FALSE

$is_riboviz_gff
[1] TRUE

$hd_file
[1] "WTnone.h5"

$num_processes
[1] 1

$help
[1] FALSE

[1] "Mapping reads to feature CDS"
[1] "Saving mapped reads in H5 file WTnone.h5"
[1] "bam_to_h5.R done"

$ cat work/b9/d4f1c6374f7f61ecf6fd08a520c919/.command.err

Please add your outputs below.
Other suggestions are:

Install/reinstall R rtracklayer package.
Install/update R here package.

The user confirmed the problem was with rtracklayer.


Create Ubuntu/Debian package script so that deb won't require libdb

Task description
Currently there is a requirement of using libDb (Berkeley DB) package of specific version, which is overlapped in many of modern Linux distributions (It is required for wallet-related functionality, but making several package versions is not convenient).
The goal is to get rid of this package dependency (not library itself), which will ease installation on them for most cases (binaries would be immediately runnable).
The scripts for building Ubuntu/Debian package can be created and placed in the 'tools' folder to be created in the repository.
There are various documents/information about creating a package, like:
https://debian-handbook.info/browse/stable/debian-packaging.html
and for bundling shared libraries in the package as well:
https://www.debian.org/doc/manuals/maint-guide/advanced.en.html
Acceptance criteria:

On a VM with clean image of Ubuntu 18.04 (e.g. in popular cloud service like GCloud) installing the created .deb package from a file will result in commands like BGLd, BGL-cli;
The commands are runnable (without installing libdb as prerequisite as mentioned here: https://github.com/wu-emma/bitgesell/releases/tag/0.1.2)

I've finished the task interrupting my work on this issue. I will continue working soon.

I need your fullname to write down it as the package maintainer name (I don't want to be a maintainer after fulfilling the bounty, unless you pay me additional money, so the maintainer is you).

Please confirm or reject explicitly if the built binary Debian package should contain GUI features.

Successfully compiled Debian packages on my Ubuntu 20.04.1. Commands do work (at least with --help) option.
It remains:

test on Docker (I will add the full test to the debian/ folder.)
change my name to your name (see above)
decide whether we compile the GUI things (your decision!)
decide if we keep debian/ in the package root (I prefer this variant) or move it to the tools, then add there a script that copies debian/ to package root and then builds it (silly).

Please answer the appeared questions.

I will also distribute libboost together with BGL, as it has the same troubles as libdb.

I will also distribute libboost together with BGL, as it has the same troubles as libdb.

Please increase funding because I need to do this additional work to produce a working .deb package.

Ugh, it appears impossible to make a package that works in both Ubuntu 20.04.1 and Ubuntu 18.04 :-(
When running the binary package built on 20.04.1 in Ubuntu 18.04:
BGLd: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by BGLd)

I set distributing three sources libdb4.8, libboost, and libevent (see https://github.com/vporton/bitgesell), but even after this it does not work. Static libc causes crash of running programs.
Looks like, there is no solution. It may be best to build a separate .deb for each Ubuntu/Debian version instead :-(

Ugh, it appears impossible to make a package that works in both Ubuntu 20.04.1 and Ubuntu 18.04 :-(

The solutions are either:

Use an old Docker image to compile (not just to run).

Distribute also libc and libstdc++ together with our package.

Which of these two solutions do you choose?
Note that I already changed your package to contain libboost (because it has a mess of different version on different versions of Ubuntu), db-4.8, libevent (It seems impossible to link to an old system package, even if it is installed, because there is only a single version of libevent-dev on Ubuntu 20.04.1.)
Please choose your solution (and increase funding, as the task appeared much more difficult than anticipated).

OK, I decided I will do compilation inside Docker. (There is no other choice.)

Victor thanks for your efforts! please ping me in the comment when I can check it, will also assess the effort and hours spent of course

Victor thanks for your efforts! please ping me in the comment when I can check it, will also assess the effort and hours spent of course

I didn't measure the hours exactly, but I think I already spent about 20-25 hours.

Thanks for the contribution guys! closing the issue.


glTF importer: "root scale" parameter appears to be ignored

Godot version: 3.1
OS/device including version: Linux, Ubuntu 18.04.2 LTS
Issue description: I have 3rd party assets which need to be scaled up.  Rather than manually adjusting them all post-import, the glTF "root scale" parameter seemed like a good fix (since they all need to be scaled uniformly by the same amount).  Unfortunately this parameter appears to have no affect, even after clicking "reimport", or exiting Godot, deleting the .import cache, restarting, etc.
Steps to reproduce:

Add a gltf asset
Select a scale factor (2 perhaps)
Reimport
Drag it into the scene
Note that it is not scaled.

Minimal reproduction project:  Here's a gltf of a simple 2x2x2 cube from blender.  I see the same issue with this as with more complex gltf assets.
blender-2x2x2-cube.zip

Looks like things do work if I drag the asset into the Scene's tree view pane instead of directly into the scene itself.  Not sure if that's a known limitation?

I had to click reimport and then open the scene again.
Godot Engine doesn't seem to refresh the existing scene. Dragging the scene from the filesystem doesn't work either. However, that's another bug.
Scaling the scene via import settings does work.

See https://github.com/godotengine/godot/issues/27409 for the other bug.


Feature request: Remember Strict context in functions and program

The script context is correctly interpreted however once the AST is built, there is no way to track if the program or functions have been parsed in strict mode. Analyzing the Directive in the body is not sufficient as it's inherited recursively during the parsing process.
I have made the changes on esprima.net already.
https://github.com/sebastienros/esprima-dotnet/commit/df52de89cd3c018ed20432e8feee9b8b643d2c7c

Arguably, this is something that should be done as a separate stage after the parsing (for instance, as part of the scope analysis). The reason is that a syntax tree can be constructed without a parser (think of a code generator or compiler) and hence, the strict context needs to be determined in that stage.

Agreed with @ariya. Please close.


License

Hi Joe,
Would you consider including a LICENSE like you have in begriffs/postgrest?

Sure. I never thought of this repo as being very fancy, just a basic sqitch example, but I can add an explicit license for you.

Thank you


Revisit SchemeBuilder/AddScheme

We probably need some more sugar around adding schemes today, namely:
authOptions.AddScheme<handlerType>(schemeName, displayName)
which today looks like:
authOptions.AddScheme(schemeName, o => {
   o.DisplayName = displayName;
   o.HandlerType = typeof<HandlerType>;
});

In fact, now that we don't have settings in the scheme anymore, we can probably just remove the scheme builder and only have this api.
Thoughts @Tratcher ?

The current one is more future proof, but adding the sugar on top wouldn't hurt.

Yeah I guess knowing how auth usually goes, the ability to add stuff later will probably be nice :)

df5c6730f772775bfc9c14fbff050a1a56d9fef6


Specify version number in url

Will it be possible to to specify a version number in a hyper url with the new version of hypercore?

I'll also talk to the holepunch folks to see if versions in the URL is something they're still into.

Thank you! I look forward to hearing what y'all decide.

Another thing we could think about is putting the version in the header or under a special folder. Like hyper://key_here/$/version/12/example.txt.
This might also be useful for preserving the "origin" across versions so that sites could retain their localStorage and cookies and the such.

Another thing we could think about is putting the version in the header or under a special folder. Like hyper://key_here/$/version/12/example.txt.

I like the special folder idea (it could provide a good parallel option for diffs (see https://github.com/RangerMauve/hypercore-fetch/issues/51#issuecomment-1441278979).
What did you have in mind for putting the version number in the header?

On a call with Joseph right now, we've decided that it would be more consistent with existing HTTP-using applications if the version number were given as a query parameter in the URL.  For example:
hyper://PUBLIC-KEY/example.txt?version=12

This would make it easy to parse the URL into component parts using existing libraries and avoid special-casing in our URL-handling code.

I would advise against putting the version into the querystring since it would break relative URLs.
E.g. if you use <script src="./script.js"></script> it will point to different files depending on how it's served.
If we use the querystrings like hyper://PUBLIC-KEY/example.txt?version=12 then the file would resolve to hyper://PUBLIC-KEY/script.js and lose the querystring.
With another approach where we put the version into the origin it could work like this: hyper://PUBLIC-KEY+12/example.txt which would more correctly resolve the script to hyper://PUBLIC-KEY+12/script.js. However this would create another issue where the "origin" is treated as a separate one and any cookies/localstorage/etc in web apps wouldn't carry over between versions. This means that checking out an older version of an app would mean losing all your saved data.
I think the "special folder" option is the best here since it preserves relative URLs and keeps the origin the same between versions. E.g. hyper://PUBLIC-KEY/$/version/12/example.txt maps to hyper://PUBLIC-KEY/$/version/12/script.js.
Sadly this means that absolute URLs are busted, but I think it's still more useful if app developers take into consideration that their code might be loaded from randoms static folders.

Talking with Joseph now, and reading your comment, I understand your point.  It seems like the hypercore /$/EXTENSION/... format is already a thing, so we should stick with that format.  We'll just need to make sure that we have solid URL-parsing code in our app that handles whatever special folders, like $/version/NN, that we use.

/$/version/NN is out in the latest version


Buttons not working

After redirect to AWS it's not possible to proceed because the S3 template URLs are not correct.
https://s3.amazonaws.com/awslabs-refarch/magento/latest/templates/magento-master.template
https://s3.amazonaws.com/awslabs-refarch/magento/latest/templates/magento.template
Maybe you want to change them to
https://s3.amazonaws.com/quickstart-reference/magento/latest/templates/magento-master.template (new VPC)
https://s3.amazonaws.com/quickstart-reference/magento/latest/templates/magento.template (existing VPC)

Archiving this repo in favour of https://github.com/aws-quickstart/quickstart-magento


Dark Pact Totem DPS calculation

Dark pact totem includes the "cast by  player" damage bonus, which is not correct as far as I know.

If a totem spends its own life on Dark Pact, it gets all the same benefits a player would for spending their own life on Dark Pact. You're just confused about how totems work.

Dark Pact totems receive the damage bonus.  Please see citation on the DP wiki page.

As mentioned, Totem-case Dark Pact does indeed gain the damage bonus.


Navigation bug

Steps to reproduce

Start the app fresh (onboarding)
add user information
on the success screen select create invoice
Create invoice
The List invoice page will be pushed in the stack instead of being the root of the stack. The user will be able to navigate back to the invoice screen. The arrow will also appear in the top bar.

Expected behavior
The invoice list is always the root of the stack.
Actual behavior
After the onboarding + create first invoice the List page is at the top of the stack.
Crash logs
No response
App version
v0.5.1
Device
All
Other details
@alexrodrigues I'm pretty sure that this bug was introduced by your last changes.
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have updated the app to version version.
[X] I will fill out all of the requested information in this form.

BTW. I think the success screen should also not be on the top of the stack. Why would you navigate back from a success screen?

Agreed!
Thanks for proving all this context I can see where I've made a mistake. I am kind connecting the onboarding and the normal flow. I am going to work on these fixes now 🚀


See existing commits, and start e.g. with feat|model|fix|build|docs|clean|format|refactor and (comm|core|k8s|tools|docs).

Additionally https://github.com/Ezard/semantic-prs may be used as a GitHub action to validate PRs in case contributors did not configure git hooks.


⚠ Bofrost.at has degraded performance

In 3071c8e, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1936 ms

Resolved: Bofrost.at performance has improved in 30c7e54.


Review request to my code partner

Hi, @MesakDuduCoder I request your help with the code review.
Thank you
Requirements

[ ] Create a new empty repo.
[ ] Copy your code from the "To Do list" project and commit it as one big chunk of code in a feature branch.
[ ] Open a pull request.
[ ] Ask your partner for a review on GitHub
[ ] Share the pull request link with your partner.

Very nice job @MarcoDiaz2000  keep up the good work your code is easy to read and follows DRY, KISS and YAGNI practise. You can merge your this branch with main now.


Private rooms

While currently not a big concern, multiple groups of players might want to play on the same server, while not seeing everyone else.

Unnecessary


Wrong validation errors for TD registration

There are some shacl validation errors upon registration. They can be opened in the TD repo if the shape is wrong in the TD in the first place.

Using enum in data schema.
Minimum maximum in data schema. There is also different behavior between number and integer
Having op anywhere
Using rel in links

I have tried some shacl validators but get different results from each. What is the reference validator and which one is used in this project?

Hi @egekorkan, thanks for your feedback. I will review your comments as soon as I'm back from vacation.

When you say the reference validator, do you mean what is used by wothive or what the community refers to as the reference implementation? It would be nice to have a validator that I can use without installing a software stack, so something that is hosted or dockerized. I have found this: https://github.com/ISAITB/shacl-validator

In this case wothive uses the jena validator, I'm not sure if this is a candidate implementation of the standard SHACL which is a W3C standard. Currently, the shapes in the WoT directory where not updated to work with the new TD spec and thus they return always error (for instance, the shapes are expecting to have dct:title but the new context translates the title as td:title).
I agree with you about this It would be nice to have a validator that I can use without installing a software stack. I also have this one Astrea. Its main functionality is not validating RDF, but it could be used as well for that purpose.


ArgumentError (wrong number of arguments (given 1, expected 0))

When I do:
render json: { data: UserSerializer.new(current_user) }

I get ArgumentError (wrong number of arguments (given 1, expected 0))
there isn't much of stacktrace, but it points to https://github.com/wmakley/simple_serializer/blob/master/lib/simple_serializer.rb#L99
However, if I do:
render json: { data: UserSerializer.new(current_user).as_json }

it works just fine.
I tried adding:
  def as_json(data)
    binding.pry
  end

to serializer, data seems to be {} 🤔

This is my recommended usage now:
render json: { data: UserSerializer.new(current_user).serialize }

I just added some parameters to fix your use case though. I just forgot that ActiveSupport expected #as_json and #to_json to take an options parameter.

Btw, just tried adding:
  def as_json(data = {})
    super()
  end

and it seems to be working.

Yeah the issue was my as_json implementation didn't take any arguments.


PR - Fix: Stable dependencies versions

Resolves #123
Merge message:
Set the version of the standard library dependency to the latest (2.34.2), updated the library to make it compatible, and removed the caret operator.

Will be done in https://github.com/Hypothesize/somatic.js/pull/121


ERROR:root:TraCI has disconnected with: 'NoneType' object has no attribute 'setOrder'

High Level Description
After running the RL training for 19 hours. It started to get error connect to TraCI server at localhost:54319 [Errno 111] Connection refused. I'm not sure whether this is more related to SUMO? This happened while running multiple jobs on parallel using the same python environment.
Desired SMARTS version
0.6.1
Operating System
ubuntu 20.04
Problems
Here is the traceback massege.
ERROR:root:Failed to initialize SUMO
                Your scenario might not be configured correctly or
                you were trying to initialize many SUMO instances at
                once and we were not able to assign unique port
                numbers to all SUMO processes.
                Check /ubc/cs/home/y/yuant426/.smarts/_sumo_run_logs/sumo-61a988b8 for hints
ERROR:root:TraCI has disconnected with: 'NoneType' object has no attribute 'setOrder'
WARNING:SumoTrafficSimulation:attempting to transfer SUMO vehicles to other providers...
ERROR:SMARTS:Failed to successfully reset after 2 times.
Traceback (most recent call last):
  File "competition/track1/train/run.py", line 281, in <module>
    main(args)
  File "competition/track1/train/run.py", line 95, in main
    run(envs_train=envs_train, envs_eval=envs_eval, config=config, wandb_run = wandb_run)
  File "competition/track1/train/run.py", line 142, in run
    model.learn(
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/ppo/ppo.py", line 299, in learn
    return super(PPO, self).learn(
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 250, in learn
    continue_training = self.collect_rollouts(self.env, callback, self.rollout_buffer, n_rollout_steps=self.n_steps)
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 184, in collect_rollouts
    if callback.on_step() is False:
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/common/callbacks.py", line 88, in on_step
    return self._on_step()
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/common/callbacks.py", line 192, in _on_step
    continue_training = callback.on_step() and continue_training
  File "/ubc/cs/research/plai-scratch/smarts/miniconda3/envs/smartsEnv/lib/python3.8/site-packages/stable_baselines3/common/callbacks.py", line 88, in on_step
    return self._on_step()
  File "/ubc/cs/research/plai-scratch/smarts/SMARTS_VCR/competition/track1/train/wandbCallback.py", line 133, in _on_step
    episode_rewards, episode_lengths, episode_infos = evaluate_policy(

Sorry for the late reply on this, the error comes from the cause that SUMO can have a race condition for ports when acquiring a port if many SUMO instances are generated simultaneously. We generally saw this as solved with a particularly tiny possibility because it has to fail a port acquisition 15 times.
The failure cases are as follows:

Connection to the SUMO process took longer than 5.05s

The SUMO process died
The SUMO process did not handshake in time (CPU lockup could do this)

traci attempted to connect to a port that is in use

traci tried to connect to another SUMO instance
traci tried to use a port that some other program is using

We struggle with this a bit since it is hard to guarantee SUMO since it can crash for legitimate reasons such as a malformed resource file.

I think there is still a bit more room to improve in handling these cases since we can perhaps check the exit-code of the SUMO process to see if it is still viable aside from the viability of the instance from port connection response.
@saulfield

I am actaully running 3 RL agent training on parallel, and this error kills all of them at the same time after 19 hours. I'm guessing this is indicating that running SUMO server on parallel may be a issue?

@yuant95 Would there happen to be stranded processes left over? It is also possible that SMARTS or the framework is not cleaning up the processes at the end.
If your model is very intensive it is just possible that SMARTS cannot afford to keep a timeout. I will look into adding in configuration to attempt retries until it succeeds.
I am also going to try using a stress test to see what might be up with it.

The newly merged #1776 should prevent this issue from occurring.


make make command work

Some imports are missing from anom.cpp and nodehash.cpp

@bhatiasiddharth I have made demo.sh file executable and uploaded it. I have tested this branch on my system and it is working. We don't have to change the README.md file. Please consider merging.


Improve multi record caching

This patch fixes a caching performance problem
where the cache looked up records from fulfil server
by reading records one at a time. Network latency
is more than the actual time to read a bulk of records.
This optimizes for network

LGTM


Ability to specify existing dynamodb source when adding Graphql api.

When adding graphql api , the amplify cli is creating a new dynamodb table and using it as a Data Source for Graphql. It would be really useful if there could be a feature where we can add an existing Dynamodb Table as the data source.

@tarunguddeti Thanks for the question. This have been discussed multiple times and is a feature that we would like to support. We can use this ticket to kick off a discussion around the design of this feature.
As I see it, the most obvious hook point would be to allow @model to specify an existing table by name & region. It is technically possible to call tables in different accounts but since this will require extra config in the other account, I suggest we do not support cross account access at first. I propose an updated @model directive.
directive @model(
    queries: ModelQueryMap,
    mutations: ModelMutationMap,
    subscriptions: ModelSubscriptionMap,
    table: TableConfig,
) on OBJECT
# You must provide a name. The region will default to the same region as the stack.
input TableConfig { name: String!, region: String }

# These are the same as before
input ModelMutationMap { create: String, update: String, delete: String }
input ModelQueryMap { get: String, list: String }
input ModelSubscriptionMap {
    onCreate: [String]
    onUpdate: [String]
    onDelete: [String]
}

If you already had a table named "Tasks" and you wanted to front it by an AppSync API, then you could use the following markup:
type Task @model(table: { name: "Tasks" }) {
  id: ID!
  title: String!
  content: String
}

This would result in the creation of the data source, data source iam role, and the CRUDL+sub resolvers but would target the existing table in your account by name instead of creating a new one. This solves part of the problem. Since the transform will no longer be administering the table, we need a way to inform the transform of the key schema and indexes and may also want to consider adding new directives for different access patterns.
We have plans to introduce an @index directive (see the design #314) that will allow for arbitrary key & index structures on all @model types. I view this as a pre-req for this feature. In terms of building support for new access patterns that allows querying indexes at the top level as well as using them to connect related data.
Currently the @connection directive is the only way to configure DynamoDB query operations (without using custom resolvers). The @connection directive does a few things:

It creates a GSI (in one-to-many situations)
It configures a DynamoDB query operation against that index
It passes along the parent object's $ctx.source.id as the hash key in the DynamoDB query which allows for the efficient look up of related info

This is a design to allow users to bring their own (pre-configured) tables, so it seems like it would be useful to have a directive that is capable of doing steps 2 & 3 without doing step 1. For example, I propose a new @query directive.
directive @query(index: String, fieldMappings: [FieldMapping]) on FIELD_DEFINITION
input FieldMapping { from: String!, to: String! }

Compounding on our example above, let's say the Task table used a primary key schema of (projectId [HASH], taskId [SORT]). Let's say it also provided a global secondary index on the (ownerId [HASH], taskId[SORT]) fields to allow users to easily lookup their owned items. You should be able to model this in its entirety using the new design. For example,
type Task @model(table: { name: "Tasks" }) {
  projectId: ID! @index(type: HASH)
  taskId: ID! @index(type: SORT) @index(name: "OwnerIndex", type: SORT)
  ownerId: ID! @index(name: "OwnerIndex", type: HASH)
  title: String!
  content: String
}

Let's say you had another table, Comments, that had a column that related it to a task by taskId.
type Task @model(table: { name: "Tasks" }) {
  # ... fields from above

  # Add a query operation. Omitting the "index" defaults to using the table's primary key.
  # The field mappings allow us to map a source attribute to an attribute in the other index.
  # I.E. the resultant query would allow you supply a sort key expression but the hash key would
  # be auto-set to $ctx.source.taskId.
  comments: [Comment] @query(fieldMappings: [{ from: "taskId", to: "taskId" }])
}
type Comment @model(table: { name: "Comments" }) {
  taskId: ID! @index(type: HASH)
  createdAt: ID! @index(type: SORT)
}

As an alternative approach, we could change the @connection directive to support similar behavior as above.
# Currently we have
directive @connection(name: String, keyField: String, sortField: String) on FIELD_DEFINITION

# We could change to
directive @connection(
  # You could use name, keyField, & sortField to create GSIs as usual.
  name: String, keyField: String, sortField: String,

  # OR you can use index/fieldMappings to use already existing indexes. You cannot mix and match.
  # For backwards compat, you may need to pass "primary" to hit a table's primary key
  # because @connection without args already has behavior.
  index: String, fieldMappings: FieldMapping
) on FIELD_DEFINITION

Let me know what you think. I am particularly interested in what you think of augmenting @connection vs adding new directives @query/@getitem etc. The FieldMapping concept is very fresh and I imagine there are many ways to accomplish a similar goal so I would be interested in hearing more options there.

Would be nice if we can add tables generated by amplify add storage on some similar way that we do with the current @function directive.
type User @model(table: "UsersTable-${env}") {
  id
}

@tarunguddeti Thanks for the question. This have been discussed multiple times and is a feature that we would like to support. We can use this ticket to kick off a discussion around the design of this feature.
Updates to @model
As I see it, the most obvious hook point would be to allow @model to specify an existing table by name & region. It is technically possible to call tables in different accounts but since this will require extra config in the other account, I suggest we do not support cross account access at first. I propose an updated @model directive.
directive @model(
    queries: ModelQueryMap,
    mutations: ModelMutationMap,
    subscriptions: ModelSubscriptionMap,
    table: TableConfig,
) on OBJECT
# You must provide a name. The region will default to the same region as the stack.
input TableConfig { name: String!, region: String }

# These are the same as before
input ModelMutationMap { create: String, update: String, delete: String }
input ModelQueryMap { get: String, list: String }
input ModelSubscriptionMap {
    onCreate: [String]
    onUpdate: [String]
    onDelete: [String]
}

If you already had a table named "Tasks" and you wanted to front it by an AppSync API, then you could use the following markup:
type Task @model(table: { name: "Tasks" }) {
  id: ID!
  title: String!
  content: String
}

This would result in the creation of the data source, data source iam role, and the CRUDL+sub resolvers but would target the existing table in your account by name instead of creating a new one. This solves part of the problem. Since the transform will no longer be administering the table, we need a way to inform the transform of the key schema and indexes and may also want to consider adding new directives for different access patterns.
The @index directive
We have plans to introduce an @index directive (see the design #314) that will allow for arbitrary key & index structures on all @model types. I view this as a pre-req for this feature. In terms of building support for new access patterns that allows querying indexes at the top level as well as using them to connect related data.
Updated @connection and/or new directives for plain DynamoDB operations.
Currently the @connection directive is the only way to configure DynamoDB query operations (without using custom resolvers). The @connection directive does a few things:

It creates a GSI (in one-to-many situations)
It configures a DynamoDB query operation against that index
It passes along the parent object's $ctx.source.id as the hash key in the DynamoDB query which allows for the efficient look up of related info

A @query directive
This is a design to allow users to bring their own (pre-configured) tables, so it seems like it would be useful to have a directive that is capable of doing steps 2 & 3 without doing step 1. For example, I propose a new @query directive.
directive @query(index: String, mapToKey: KeyMapping) on FIELD_DEFINITION
input KeyMapping { hash: String!, sort: String }

Compounding on our example above, let's say the Task table used a primary key schema of (projectId [HASH], taskId [SORT]). Let's say it also provided a global secondary index on the (ownerId [HASH], taskId[SORT]) fields to allow users to easily lookup their owned items. You should be able to model this in its entirety using the new design. For example,
# If using Amplify CLI storage tables with multienv, you could pass  "Tasks-${env}".
type Task @model(table: { name: "Tasks" }) {
  projectId: ID! @index(type: HASH)
  taskId: ID! @index(type: SORT) @index(name: "OwnerIndex", type: SORT)
  ownerId: ID! @index(name: "OwnerIndex", type: HASH)
  title: String!
  content: String
}

Let's say you had another table, Comments, that had a column that related it to a task by taskId.
type Task @model(table: { name: "Tasks" }) {
  # ... fields from above

  # Add a query operation. Omitting the "index" defaults to using the table's primary key.
  # The field mappings allow us to map a source attribute to an attribute in the other index.
  # I.E. the resultant query would allow you supply a sort key expression but the hash key would
  # be auto-set to $ctx.source.taskId.
  comments: [Comment] @query(mapToKey: { hash: "$ctx.source.taskId" }])
}
type Comment @model(table: { name: "Comments" }) {
  taskId: ID! @index(type: HASH)
  createdAt: ID! @index(type: SORT)
}

An updated @connection directive
As an alternative approach, we could change the @connection directive to support similar behavior as above.
# Currently we have
directive @connection(name: String, keyField: String, sortField: String) on FIELD_DEFINITION

# We could change to
directive @connection(
  # You could use name, keyField, & sortField to create GSIs as usual.
  name: String, keyField: String, sortField: String,

  # OR you can use index/mapToKey to use already existing indexes. You cannot mix and match.
  # For backwards compat, you need to pass "primary" (or similar) to hit a table's primary key
  # because @connection without args already has behavior.
  index: String, mapToKey: KeyMapping
) on FIELD_DEFINITION

The same example as above would look like this with @connection:
type Task @model(table: { name: "Tasks" }) {
  # ... fields from above

  # Add a query operation. Omitting the "index" defaults to using the table's primary key.
  # The field mappings allow us to map a source attribute to an attribute in the other index.
  # I.E. the resultant query would allow you supply a sort key expression but the hash key would
  # be auto-set to $ctx.source.taskId.
  comments: [Comment] @connection(index: "primary", mapToKey: { hash: "$ctx.source.taskId" }])
  # mapToKey is optional and if not provided, the user would have to provide the taskId
  # explicitly in the query. e.g. `query { task { comments(taskId: "123") { ... } }` which
  # is not ideal in this scenario.
}
type Comment @model(table: { name: "Comments" }) {
  taskId: ID! @index(type: HASH)
  createdAt: ID! @index(type: SORT)
}

Requesting feedback
Let me know what you think. I am particularly interested in what you think of augmenting @connection vs adding new directives @query/@getitem etc. The KeyMapping concept is very fresh, and I imagine there are many ways to accomplish a similar goal so would be interested in hearing more options.

@mikeparisstuff  is there any movement on this with  @model(table: { name: "Tasks" }) option???

One variation of this feature that would be handy to me: specify dynamodb table name to connect to in one env (prod), but have default tables that would be created by Amplify used in other envs.

+1

@clinicalinkgithub
i'm facing issue while connecting to exiting tables in dynamoDB is there any other way to connect to the existing tables? "amplify push" creates new tables
i'm also tested with your given code i'm facing some issue
Here is my code
`directive @model(
queries: QueryInput,
mutations: MutationInput,
subscriptions: SuscriptionInput,
table: TableConfig,
) on OBJECT
input TableConfig { name: String!, region: String }
input MutationInput { guid: String, name: String }
input QueryInput { guid: String, name: String }
input SuscriptionInput {
onCreate: [String]
onUpdate: [String]
onDelete: [String]
}
type Planet @model(table: { name: "Planet" }) @key(fields: ["guid"]) {
guid: String!
name: String!
}`
And when i run "amplify push " it gives the following  error
Error: Unknown argument "table" on directive "@model".
There can be only one directive named "model".

@clinicalinkgithub @hisham @khaschuluu @yuth @mikeparisstuff @kevinwolfdev
We are looking forward for your help this is urgent for me please participate with your experience with an mentioned issue thanks in advance.

I'm going to close this, as I believe everything mentioned here has either been implemented, can be accomplished via extensibility, or overlaps with the request in #140.


⚠ Bofrost.de has degraded performance

In d1bd2f9, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1338 ms

Resolved: Bofrost.de performance has improved in ffadb43.


[fix][doc] Fixed binary protocol naming and descriptions

Modifications

Fixed documentation of basic command structure to match the code

Verifying this change

[ ] Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.
Documentation
Check the box below or label this PR directly.
Need to update docs?

[ ] doc-required
(Your PR needs to update docs and you will update later)

[ ] doc-not-needed
(Please explain why)

[x] doc
(Your PR contains doc changes)

[ ] doc-complete
(Docs have been already added)

@codelipenghui Changed


Use object-spaced bounding boxes for putting stuff in crates

About the PR
This is my PR that requires an engine change.
This adds the ability to get object-spaced bounding boxes to the engine, and then uses it so that when crates attempt to get bounding boxes to determine object size, they use those object-spaced bounding boxes instead of world space bounding boxes. This change makes it so that the ability of an object to fit into crates is determined solely by the size of the object.
Requires https://github.com/space-wizards/RobustToolbox/pull/4545
Why / Balance
#19301
#19012
#19205
Spears and soaps will routinely not fit into crates due to, of all things, the rotation of the station in world space. They're meant to fit in, but can't. Also, while I've seen no reports about it, this same bug could also in theory allow objects that aren't intended to fit into crates to fit in at certain angles, since the ability to grow certain bounding boxes by rotation also entails the ability to shrink others.
Technical details
Primary code changes are in RobustToolbox, so technical review should go there. Change here is simply use of a different function once it exists.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
None, purely additive.


Fix resource issues

I tested recompiling with some random simple apps, e.g.
https://apkpure.com/typescript-tutorial/com.cosmiclearn.typescripttutorial
https://apkpure.com/gradle-dependencies-generator/com.androidgradle.app
and just fixed the first resource or gradle issue in each iteration. The first app now fails with a java issue. I used gradle 7.4.2 to take another bug (also reproducible with gradle 7.6).
For review: Each commit fixes one bug. I used apktool as a reference for the exported resources.
The following issues can now be closed: https://github.com/skylot/jadx/issues/1073 and https://github.com/skylot/jadx/issues/1682
For the next issues a data structure to aggregate infos for gradle build file generation would help, e.g. for a flag to track usage of "android:pathData-Attributes" + minSdkVersion < 21 to set "vectorDrawables.useSupportLibrary = true" or a legacy Apache http client detection....

@nitram84 thanks for fixes!
These sample apps are really very simple, so I think it is possible to fix java code compilation issues :+1:


Set obvious default values when Parent adding a child

country, city/town and address should default to be the same as the parent adding the child - makes the form less tedious to fill out (as it is tedious, e.g. why is 'alias' a required field?)

@DanielBrierton covered by you, am i right?

Yeah, this is covered by my new add child form. Will be sure to close when deployed.

Closing, it's been covered by my children && the new booking flow


IANA considerations for txn_token type

I believe we should add an IANA consideration and propose to register urn:ietf:params:oauth:token-type:txn_token.

Thanks for this email and adding the issues to GitHub. I will update the
spec shortly.
On Fri, Sep 15, 2023 at 9:48 AM Arndt Schwenkschuster <
@.***> wrote:

I believe we should add an IANA consideration and propose to register
urn:ietf:params:oauth:token-type:txn_token.
—
Reply to this email directly, view it on GitHub
https://github.com/SGNL-ai/transaction-tokens/issues/32, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB55UG2NFDHWDE7USMWVPDLX2SBGPANCNFSM6AAAAAA42CJ2YU
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Spike publishing for C++ shared libraries to a Maven repository

@adammurdoch commented on Mon Jun 05 2017
Split out of #21. Produce a spike to discover the issue in publishing and resolving C++ libraries via a Maven repository.

This is basically a duplicate of #129.


mac-39 lost external connection from phone device.

Type of Request
bug
Infrastructure Environment
LUCI, Github, Cocoon scheduler, Autosubmit, etc...
What is happening?
mac-39 lost external connection from phone device.

Steps to reproduce
Step 1:
Step 2:
..
Step n:
Expected results
I expect to see X when Y is finished.

Rebooting machine to restore connection.


Load resources lazily

Today, the various properties on References are declared as get-only properties with an initialiser. Example:
public static PortableExecutableReference Accessibility { get; } = AssemblyMetadata.CreateFromImage(Resources.Accessibility).GetReference(filePath: "Accessibility.dll", display: "Accessibility (net472)");

There's also a static All property which stores all of the other PortableExecutableReferences in an array.
All of these initialisers always run (when the References class is first accessed), so all of the embedded resources are always loaded, even if you only need to use one of them. It looks like the resources total around 50MB (which is retained until the process terminates, since they are static fields), but also on the Desktop Framework there's additional garbage generated by the stream.CopyTo call in GetResourceBlob.
Would you be open to loading the resources lazily so that we only pay for what we use?
private static PortableExecutableReference? _accessibility;
public static PortableExecutableReference Accessibility
{
    get
    {
        if (_accessibility == null)
        {
            _accessibility = AssemblyMetadata.CreateFromImage(Resources.Accessibility).GetReference(filePath: "Accessibility.dll", display: "Accessibility (net472)");
        }
        return _accessibility;
    }
}

This could be changed to use Lazy if you want to be certain to load the resource once only (thread safety). Would you accept a pull request containing a change like this?

I should add that the properties in the Resources class do already load the resources lazily, but they're all evaluated upon usage of References. So it's work-around-able (use the Resources directly and call CreateFromImage manually), but would be good to fix in the library if possible.


Crash while scrolling

Hi there!
I'm seeing the following crash when scrolling up a chat screen. This is happening randomly based on the actual chat content: fortunately I am able to always replicate it in a specific "chat room". The only way to avoid the crash is setting the showLoadEarlierMessagesHeader flag to NO. Any thoughts?
Thanks in advance!
: *** Assertion failure in -[UICollectionViewData validateLayoutInRect:], /BuildRoot/Library/Caches/com.apple.xbs/Sources/UIKit/UIKit-3512.30.14/UICollectionViewData.m:408
: *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'layout attributes for supplementary item at index path (<NSIndexPath: 0xc000000000000016>
{length = 2, path = 0 - 0}) changed from <JSQMessagesCollectionViewLayoutAttributes: 0x15fc8cbb0> index path: (<NSIndexPath: 0xc000000000000016> {length = 2, path = 0 - 0}
); element kind: (UICollectionElementKindSectionHeader); frame = (0 0; 375 32); zIndex = 10; to <JSQMessagesCollectionViewLayoutAttributes: 0x15fc749b0> index path: (<NSIndexPath: 0xc000000000000016>
{length = 2, path = 0 - 0}
); element kind: (UICollectionElementKindSectionHeader); frame = (0 0; 375 32); zIndex = -1; without invalidating the layout'

As far as I have seen this crash is happening only if you call reloadData too often.

+1

+1

+1

+1

+1

So I fixed this by just commenting out
else {     attributesItem.zIndex = -1; }
in JSQMessagesCollectionViewFlowLayout.m.

+1

Could anyone try this PR? https://github.com/jessesquires/JSQMessagesViewController/pull/1868

+1


Stage 3: source/exports.ts

I'm holding this open for the "hash-all-files" check, which I haven't added yet.


Use of shared cluster chart in CAPVCD

Motivation
In order to match the target state of our defined release framework we need to use the shared cluster chart in CAPVCD and extract all provider independent apps from cluster-cloud-director as well as introduce the use of the cluster chart into the cluster-cloud-director chart.
### Tasks
- [ ] Introduce the use of the cluster chart in cluster-cloud-director.
- [ ] Remove all apps in cluster-cloud-director that are now managed in the cluster chart.
- [ ] Clearify with Team Rocket how to roll out the change on all CAPVCD Installations
- [ ] https://github.com/giantswarm/giantswarm/issues/31193
- [ ] https://github.com/giantswarm/giantswarm/issues/31697
- [ ] https://github.com/giantswarm/giantswarm/issues/31698
- [ ] https://github.com/giantswarm/giantswarm/issues/31699
- [ ] https://github.com/giantswarm/giantswarm/issues/31700
- [ ] https://github.com/giantswarm/giantswarm/issues/31701

Outcome

CAPV now has the expected structure that will be aligned with the release framework across all providers.

done 🥳


⚠ ClamAV has degraded performance

In c6c2ed2, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 91 ms

Resolved: ClamAV performance has improved in fb88365.


Update MIDC variable mapping

~[ ] Closes #xxxx~
[x] I am familiar with the contributing guidelines
[ ] Tests added
~[ ] Updates entries in docs/sphinx/source/reference for API changes.~
[x] Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
[x] New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
[x] Pull request is nearly complete and ready for detailed review.
[x] Maintainer: Appropriate GitHub Labels (including remote-data) and Milestone are assigned to the Pull Request and linked Issue.

I got the following comments from the personnel responsible for operating MIDC. This PR implements the requested changes, which I think makes good sense.
Note, the change in how instruments are mapped is in a way a breaking change,

Just a heads up on the following variable mapping:

NWTC does not have PSP anymore (and they also have Direct and Diffuse now).
BMS still has the NIP, but the CHP1 is better (we had the CHP1 for a very long time).
UOSMRL does not have Diffuse Schenk anymore (they have a Diffuse CMP22).  Also they now have an instrument better than NIP that can be used instead (CHP1)

I’m not sure if this variable mapping is used for just current data or historical data, so maybe that is why the NIP was chosen so that it works over a larger time period, however for NWTC and UOSMRL you won’t have the PSP and Schenk anymore.
Also I highly recommend the VTIF be removed since its shutdown (had leveling problems) and since BMS is so close to it there is no reason to use it.

Test failures are related to upload to Codecov

To be clear, MIDC no longer offers data from those old instruments through their API? Or, are the old data still available somehow, and there are also data from new instruments?

To be clear, MIDC no longer offers data from those old instruments through their API? Or, are the old data still available somehow, and there are also data from new instruments?

The latter. At some periods that may be two of the same type of measurement,  e.g. DNI so we need to decide which gets mapped

If someone requests old data (prior to these MIDC changes), they will get old data. If they also ask for variable mapping, pvlib will rename any of the current fields, and leave the old ones alone. Is that right?

If someone requests old data (prior to these MIDC changes), they will get old data. If they also ask for variable mapping, pvlib will rename any of the current fields, and leave the old ones alone. Is that right?

Correct. No data is changed.
But if you upgrade your pvlib version, the dni data may seem to have changed because the data is renamed from a different column in some instances.

It's not obvious to me that pvlib.iotools should be in the business of choosing which of several DNI options is the "blessed" channel that gets mapped to be mapped to the special name dni.  What do we do with other datasets that provide multiple measurements of the same quantity?  With get_srml, don't we return names like dni_1, dni_2?

It's not obvious to me that pvlib.iotools should be in the business of choosing which of several DNI options is the "blessed" channel that gets mapped to be mapped to the special name dni.  What do we do with other datasets that provide multiple measurements of the same quantity?  With get_srml, don't we return names like dni_1, dni_2?

I believe so. In this case we could maybe do dni_chp1 and the likes?

In this case we could maybe do dni_chp1 and the likes?

I could support this.  The alternative I see is to not map at all in cases like this where there are several options.

It's not obvious to me that pvlib.iotools should be in the business of choosing which of several DNI options is the "blessed" channel that gets mapped to be mapped to the special name dni.

I want to emphasize that this is the data provider doing it in this case. So I don't see a problem with it. Also in this case there's a significant improvements in accuracy (thus a clear choice) and one instrument was decommissioned

@kandersolar I've implemented a bit of a compromise. For the stations that have multiple pyrheliometers, I have implemented subscripts for the instrument type, e.g., dni_chp1 and dni_nip.
For the case when the new column does not denote the instrument type, e.g., 'Diffuse [W/m^2], I have chosen to just map this to the corresponding variable name (for the example it would be dhi).

Maybe for cases like the MIDC where there are potentially multiple instruments, the rule should be to always append the instrument name when it is known, even if there is only one measurement of that type? So for example, BMS (which has only one GHI measurement listed) would have 'Global CMP22 (vent/cor) [W/m^2]': 'ghi_cmp22'.
This seems somewhat more consistent to me, and has the added benefit of being future-proof if/when the MIDC adds more instruments again in the future.

I don't think it is future proof given that SRRL for example has multiple of the same type, so it would have to be ghi_cmp22_1.

Side note: doesn't the BMS have dozens of instruments? Why do we only map 1-2 of each type?

Yes, I expect they have 20+ GHI sensors, so I think that's not useful to map. However, they specify one instrument as the main instrument, which is what we map in this case. I acknowledge that it is not ideal, but I still think that merging this PR is the current best option

merging this PR is the current best option

I agree


104 seed hs building map

[x] Adds rake task  export:export_to_seeds for

[x]  retrieving Point, PointOfInterest, Polyline, Wall, Room and Building from db

[x]  logging corresponding create statements

[x] seed building data (add logged statements to seeds.rb)

This is outdated, too many conflicts with dev


Can not edit/upgrade the app if the original chart is hidden by the rancher-version annotation

Setup

Rancher version: 2.6.0
Browser type & version: Chrome

Describe the bug

We are working on adding the following annotation to all rancher charts that should be available on rancher v2.5.x only
catalog.cattle.io/rancher-version: "< 2.5.99-0"

But it turns out the UI will go wrong after applying the annotation to the chart.
To Reproduce

It requires some extra steps to reproduce it:

clone the rancher/charts repo https://github.com/rancher/charts/tree/dev-v2.6
checkout the branch dev-v2.6
run rancher v2.6.0
on the cluster explorer UI, add a new repo pointing to your GH repo and branch
install the app rancher-alerting-drivers 1.0.100 from the repo
now update the branch to include the fix from this PR: https://github.com/rancher/charts/pull/1480
on the cluster explorer UI, refresh the repo
now the chart rancher-alerting-drivers 1.0.100 should NOT show in the UI anymore
go the the app -> edit -> change a value -> click on upgrade

Result

Expected Result

We should be able to

editthe existing app
upgrade it to the newer version

Possible that this is just the backend not returning chart at all, which would not be UI problem. Should be able to tell via the v1 api calls.

I'm going on the assumption that chart versions with an invalid catalog.cattle.io/rancher-version should be filtered out by the backend. If so...

v1/catalog.cattle.io.clusterrepos/<repo id>?link=index response contains invalid versions
From the screenshot it looks like v1/catalog.cattle.io.clusterrepos/<repod id>?link=info&chartName=rancher-alerting-drivers&version=1.0.100 then is correctly returning a 500

For app edit/upgrades the user should be offered the chance to select the version to use, the current version will always be offered as well (unless we can't find the chart, then we offer them the option to find it). From the screenshot it looks like this selection was left as-is... so the frontend is trying to request a bad version (from & to versions are displayed under the chart name in the header).
From my understanding then

Backend need to ensure it only provides valid chart versions for the rancher-version
Frontend chart install needs to do additional filtering on versions to ensure the current one exist (see pages/c/_cluster/apps/charts/install.vue and mixins/chart.js mappedVersions

If catalog.cattle.io/rancher-version applies to the target cluster and not local, so backend shouldn't filter out, then frontend needs to do additional filtering (just like we currently do or OS - CATALOG_ANNOTATIONS.SUPPORTED_OS)

My testings on the rancher 2.6.0 show that the annotation catalog.cattle.io/rancher-version: "< 2.5.99-0" takes effect on only the local cluster.
So we have 2 issues that need to be addressed:

on the local cluster, the installed app should be editable/upgradable even the original version of the chart is hidden after upgrading rancher
This is more like a design question: Should the label works on the downstream cluster? If yes, should we support it in the backend end or the front end?

cc @richard-cox @cbron

Seems like the server-side version filtering should be only on the listing of the whole index, so that you don't offer a new install of that chart.  But if we ask for a specific chart+version by name and you have that version: just return it, so that the normal edit flow can work.

Backend will fix primary problem in issue linked right there, this issue can exist for the non-existing case, which can be done later.

Based on Caleb's feedback, pushing the UI side out of 2.6.1.

The fix is validated on 2.6.1-rc5
see this comment for the detail: https://github.com/rancher/rancher/issues/34919#issuecomment-929733068

The only remaining bug seems to be:
The flow from apps & market place > charts > specific chart (for example, rancher-istio) > upgrade does not show the old existing installed version in the drop down

The flow from apps & market place > installed apps > specific chart (for example, rancher-istio) > edit/upgrade does show the old existing installed version in the drop down

Is it possible to upgrade so that both flows work?

Reopen the issue to bring attention.
Be free to move it back to to-test if it is a false alert.

For the reopened element, we'll push to 2.6.2. With everything else in 2.6.1, perhaps this can be split.

@gaktive since a partial fix was added making edit is no longer completely blocked, 2.6.2 should be okay for the remainder of the work.

Created https://github.com/rancher/dashboard/issues/4311 to track the outstanding issue based on a comment. Moving this one back To Test for additional QA.


Add data attributes to select2 select and select the new option

I use ajax to populate a select2 dropdown. The initial value is set in a regular . It all works until I seledct on of the options in the dropdown. Then the new item is not selected once I open the dropdown anymore and it doesn't seem to find $(this).val() on select2:selecting anymore. I use this code to add some data attributes to the newly created option on select2:select. What else can I do to tell select2 that it is the new selected option? The new option shows up in the select2 selection, the values are just not reachable for some reason.
Is there a better way to set an initial value, without making an ajax request for it? Do I need a trigger("change") somewhere to clarify? Ideas?
.on('select2:select', function (e) {
var $self = $(this);
var data = e.params.data;
$self.children('[value="'+data.id+'"]').attr(
    {
        'data-title':data.title,
        'data-text':data.text,
        'data-subvalue':data.subvalue
    }
); // [...]

Closed due to user error...

hello,

i have had the same problem since a long time, and problem is reported also on Stack overflow (https://stackoverflow.com/questions/25187926/how-to-set-selected-value-of-jquery-select2/35131804).
How did you solve it ?

Thank you.


T-56 : Timer Driver Interface and Implementation(LPC17xx)

This commit implements one shot driver functionality. Each timer is mapped to a HW Timer.

Collaborator:
Review for this pull request was created. Link to review

Corresponding review in collaborator has been closed. Link to review


Unauthorized fatal: Could not read from remote repository.

Unauthorized
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Failure
<EMAIL_ADDRESS>(exit code: 1)

I am having the same problem

Hey, it seems not possible at least for Bitbucket because it allows using "read-only" ssh access keys, so, Bitrise doesn't have write permission to a Bitbucket repository.

Hi @sergpetrov I have same problem for github. What is the problem?

Same issue with Gitlab, I've checked the repo has write permission


Setup organisation table

Basic organization model and migration is already there: https://github.com/CorrelAid/engage-2/blob/development/backend/src/models/organization.py


Standalone Invoke AI app

Summary
InvokeAI is currently installed through a Python script that can be confusing to install for users who are not technically savvy and are looking for an easy way to get started with InvokeAI.
Packaging InvokeAI into an Electron app that is able to be downloaded and ran without needing to worry about environment setup will allow the project to reach a much wider user base, encouraging adoption of the platform by users of all backgrounds.
Intended Outcome
InvokeAI is able to be downloaded as a desktop application and does not require environment setup through the terminal.

Why removal? :(


COMP 151 & 139

COMP-151: Change 'Certificate Holder' label to 'Regulated Party'
COMP-139: Issue with "Create New Case File" Button when Linking Inspection

Codecov Report
Attention: Patch coverage is 0% with 9 lines in your changes missing coverage. Please review.

Project coverage is 42.63%. Comparing base (17656d0) to head (93d9c97).

Files with missing lines
Patch %
Lines

...src/components/App/CaseFiles/LinkCaseFileModal.tsx
0.00%
5 Missing :warning:

...ance-web/src/components/App/ProjectDetailsForm.tsx
0.00%
2 Missing :warning:

...rc/components/App/Complaints/ComplaintFormUtils.ts
0.00%
1 Missing :warning:

.../components/App/Inspections/InspectionFormUtils.ts
0.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff             @@
##           develop      #65      +/-   ##
===========================================
- Coverage    42.72%   42.63%   -0.09%
===========================================
  Files           81       81
  Lines         1463     1466       +3
  Branches       342      389      +47
===========================================
  Hits           625      625
- Misses         836      839       +3
  Partials         2        2

Flag
Coverage Δ

compliance-web
42.63% <0.00%> (-0.09%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


App mob status

Description
Adding Application mobility status field and also checking services dependent on application-mobility if its succeeded in deploying
https://jira.cec.lab.emc.com/browse/KRV-15319
GitHub Issues
List the GitHub issues impacted by this PR:

GitHub Issue #

Checklist:

[ ] I have performed a self-review of my own code to ensure there are no formatting, vetting, linting, or security issues
[ ] I have verified that new and existing unit tests pass locally with my changes
[ ] I have not allowed coverage numbers to degenerate
[ ] I have maintained at least 90% code coverage
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have maintained backward compatibility

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Please also list any relevant details for your test configuration

Can we verify with a pflex installation as well? Just to double check backwards compatibility

Can we verify with a pflex installation as well? Just to double check backwards compatibility


Is the tokenzier's output exactly the same as BertTokenizer?

I have verified the tokenize result on 180000 Chinese sentences. It is exactly the same as BertTokenizer.


[master] Bump Mesos to nightly master 40210db

High-level description
This is a routine bump to the latest Mesos and mesos-modules.
Related JIRA Issues
Checklist for all PRs

[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[x] Changelog:
https://github.com/apache/mesos/compare/26e68578f4390e16e21caca40783a1748bef5990...40210dbcfb218d6ef3f647b047afbdda24133f4d
https://github.com/dcos/dcos-mesos-modules/compare/0d80a848041a2d82a4e444a17e0ff72e28140e82...0143dac6622e89c8ef98683a1c9d8d1113877e79

[ ] Test Results: [link to CI job test results for component]

[ ] Code Coverage (if available): [link to code coverage report]

@mesosphere-mergebot bump-ee

Closed in lieu of https://github.com/dcos/dcos/pull/4056


Distinguish not-yet-evaluated alert from empty alert

Like we do for targets (UNKNOWN) I'd like to see alerts that have mnot yet been evaluated (after a restart or  new alerts) marked as unknown.

Hmm, what happens if there's two alerting rules with the same name?

This should be easy to fix as there is already a Health() that will return HealthUknown if the rule is not yet evaluated.

@brian-brazil valid concern; no matter how many alerting rules share the name there are always a sum of n active alerts plus m alert defintions with unknown state.
(n active and m alerting rules unknown) sounds a bit kludgy, but it would fit the intention exactly.

Mentioning (n active and m alerting rules unknown) sounds like a good idea.
If there is an approval, I can open PR for this

No need for approval, go ahead :)
Sent by mobile; please excuse my brevity.
On Fri, Aug 24, 2018, 16:19 Harsh Agarwal<EMAIL_ADDRESS>wrote:

Mentioning (n active and m alerting rules unknown) sounds like a good
idea.
If there is an approval, I can open PR for this
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prometheus/issues/4510#issuecomment-415772992,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAuEI-C9eor5r-vqTHledls-K5dSvr1Uks5uUAtWgaJpZM4V_eqL
.

@RichiH Looks like PR by @sipian is stale now. I am looking for new open source project to contribute to. This can be a good start for me.
Should I try this, if this is still available?

@sipian do you plan to continue your PR? :)

@sipian said (offline) he won't be able to continue that PR. @pradeepgangwar it's all yours now.

@pradeepgangwar
You can go ahead with the PR.

Please do it on the new UI.

Hello from the bug scrub! Keeping open for the open PR.


Bayesian tutorial

added tutorial and tutorial material

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@3e42a5b). Click here to learn what that means.
The diff coverage is 0%.

@@            Coverage Diff            @@
##             master     #226   +/-   ##
=========================================
  Coverage          ?   73.18%
=========================================
  Files             ?      134
  Lines             ?    14180
  Branches          ?        0
=========================================
  Hits              ?    10378
  Misses            ?     3802
  Partials          ?        0


3il.txt

@Bastiengrsd As far as I see, the official title of the insitution is "ÉCOLE D'INGÉNIEURS EN INFORMATIQUE". Could you please change the "3il Group" in the file in your request to "ÉCOLE D'INGÉNIEURS EN INFORMATIQUE" in order to match the official name? Thank you!

@Bastiengrsd Pull request merged. Thank you!


will it support slimerjs in docker container(ubuntu)

On windows, run test with slimerjs(0.10.3)+firefox(50.1.0) +mocha-phantom-js, it will output the expected result;
On docker container(Ubuntu): I try same steps, but there is no output and no errors;
I try to use xvfb-run slimerjs test/test.js, but also no output.
I wonder: will mocha-phantomjs-core support slimerjs in a  docker container?
Thanks!

It doesn't support it, firefox + xvbf does that support. Remember mocha-phantomjs-core runs within whatever phantomjs-like environment and that environment does the headless graphics rendering. (Also this issue should have been opened against mocha-phantomjs-core)
This is the same environment as travis - see this build. Note slimerjs is failing tests (see nathanboktae/mocha-phantomjs-core#25) but I'll have to try that combination you're using.

Yes，solved by using xvbf.  and mocha-phantomjs-core support the latest slimerjs well.
Thank you!
By the way, have you tried any docker IE test? I‘v tried winetricks on my linux machine ,it can run ie8, but I'm disappointed that ie8 can't be installed by winetricks(cause X server & $DISPLAY problems).
TrifleJS seems to be a promising solution, but I it failed to run  on Windows machine.

docker ie
Not possible. Docker virtualizes the Linux kernel and IE is Windows only often relying on the latest or even private APIs.

If you need to do cross browser testing you should use Karma or hosted testing like SauceLabs. I highly recommend the later. I've had IE bugs only repro on specific windows versions with specific windows features turned on and off.

Thank you, I have a brief look at SauceLab, I'm not sure will it support our gitlab CI test.
And I found a docker image suchja/x11server ; if I run x11server before running wine image, wine can open IE from docker image  successfully.
but I'm still not sure if it possible to run test through docker IE.

Even if you get that running, you will constantly be maintaining that Frankenstein setup. Don't do it.


adult DUM neuron can perhaps be deprecated in favor of adult VUM neuron

I see that FBbt:00049554 is adult dorsal unpaired median neuron and my paper is cited. A few years ago we had some internal discussion about whether DUM neurons and VUM neurons both exist or are different names for the same thing – it seems like the existence of two separate terms VUM and DUM is due in part to cross-species differences in the soma location of a single type of cells. Some people said they thought DUM and VUM neurons were separate things that both existed in some species, but we didn't really resolve the conversation definitively. Anyway, what is certain is that the cells in Drosophila reported in my paper that are referred to by FBbt:00049554 actually do have their cell body on the ventral side of the VNC, and are certainly the same cells that Erika refers to in FBbt:00052750. (I think we may have had the term "DUM neuron" in our preprint but then in the published paper we changed it to "UM" neuron. I don't believe the term "DUM neuron" in Drosophila exists really, or at least I couldn't tell you what cells it would refer to, other than being a synonym for these VUM/UM neurons.)
I would be in favor of removing/deprecating/marking as obsolete/whatever you call it the term FBbt:00049554

Thanks for the clarification - I have to run any term obsoletions/merges past the FlyBase curators, but if nobody objects I will do the merge before the next release (keeping the VUM term).


Add a new "Resiliency" section that references our Azure AD resiliency guidance at aka.ms/resilientaad

Would be helpful to include a new section where we reference our clear guidance on creating a resiliency plan, before a disruption. This is an important step and should be included as part of our cloud-based MFA deployment guidance.
Perhaps a few points and link to our recommendations found here https://docs.microsoft.com/en-us/azure/active-directory/authentication/concept-resilient-controls#microsoft-recommendations

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f132318f-8670-fd0d-2524-01305ccff2a1
Version Independent ID: 0849b4a9-ff14-1d61-47f2-a2daecf3cde9
Content: Plan and execute an Azure Multi-Factor Authentication deployment - Azure Active Directory
Content Source: articles/active-directory/authentication/howto-mfa-getstarted.md
Service: active-directory
Sub-service: authentication
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@christinamorillo Thanks for the feedback. We are currently investigating into the issue and will update you shortly.

@MicrosoftGuyJFlo  coudl you please review this documentation update request .

Its in one of the linked docs on emergency access accounts already in the doc.

#please-close


Mutliple 'user' models for authentication

Is it possible to have two models, both of which are not called User to login (sort of like two roles but split into models)?
If that is not possible, is it possible to have a single model not named User to login? Say my user model is named Student or something else? I can't seem to find any documentation on this.

I don't think it's possible to have two separate "User" models to log in (but I'm not an authority on this).
What you're looking for on the second bit is the userEndpoint option when you configure authentication (check it out in the docs).
You'll want to configure that endpoint in your services, of course.
If that doesn't make enough sense, I can give you examples (I just had to figure this all out myself).

@ruddfawcett a fix is coming in 0.8 for this so that you can authenticate with any/multiple entities

Great, thanks @ekryski!

@ekryski Was this actually implemented? Documentation seems to be scarce on this point!