Added Quint specification

Overview
This PR introduces a formal specification of the NMT proof/verification logic in Quint.
The benefits of having a Quint specification are threefold:

It is a precise description of the expected behavior, and yet it resides on a higher level of abstraction than the code. Furthermore, it is executable, which makes it easier to spot and eliminate mistakes in the specification. Module nmt captures the logic of namespace proof generation and verification, and the invariant verificationAlwaysCorrect is an example of a property against which a specification can be checked.
It allows for test generation. Module nmtTest iteratively generates proofs and non-deterministically corrupts them. These generated test runs are exported in a json format (e.g., file ITF_traces/runTest.itf.json). To be executed as a part of the standard suite, an adapter simulation_test.go is necessary. (The adapter iterates through the json-represented execution state and translates them to function calls.) The generation of the tests happens through simulation. In that sense, it is similar to the existing tests fuzz_test.go, except that it also adds corruption of the data.
Having specifications written in Quint makes it possible to change tests/specs quickly: either by taking advantage of updates to Quint (e.g., going from simulation to exhaustive checks by changing a command) or by virtue of making changes on the level higher than code (and thus less details need to be changed).

Current limitations:

the specification does not model absence proofs
there is an assumption that every tree is full and complete
the specification does not model special handling of parity namespace (ignoreMaxNamespace option). Modelling it correctly depends on the decision of what the desired behaviour is (issue #148 )

Checklist

[ ] New and updated code has appropriate documentation
[ ] New and updated code has new and/or updated testing
[ ] Required CI checks are passing
[ ] Visual proof for any user facing features like CLI or documentation updates
[ ] Linked issues closed with keywords

Here is how to start experimenting with Quint and the model.
Installation

install quint tool by running npm i @informalsystems/quint -g  (more details here)
install the VSCode plugin from here. While this is not necessary, it is of great help when writing models.

Basic resources

a sequence of very gentle and fun tutorials here
language cheatsheet
documentation for built-in operators here

REPL
After installing the quint tool, run the REPL by typing quint to terminal.
Then you can play with the REPL by trying some basic expressions, e.g.

2+3
val x = Set(1,3,4)
x.powerset()
x.map(i => i*2)

Inspecting the NMT model
As a first step, examine the model operators from within REPL.
Example commands to try out after running quint within the formal_spec folder:
.load nmt.qnt

import basics.*

import nmt_helpers.*

import nmt.*

val c_leaves = [{ value: ("data", 0), namespaceId: 0 }, { value: ("data", 1), namespaceId: 2 }, { value: ("data", 2), namespaceId: 2 }, { value: ("data", 3), namespaceId: 4 }]

val c_tree = BuildTree(c_leaves)

val proofZero = CreateProofNamespace(0, c_tree)

CreateProofNamespace(2, c_tree)

val c_root = c_tree.hashes.get(1)

verifyInclusionProof(proofZero, c_root, 0, [{ value: ("data", 0), namespaceId: 0 }])

MerkleRangeHash(proofZero, [{ value: ("data", 0), namespaceId: 0 }])

verifyInclusionProof(proofZero, c_root, 0, [{ value: ("data", 0), namespaceId: 2 }])

import nmtProofVerification.*

// runs the initialization action
init

// runs one step of the model
step

// runs another step of the model
step

After getting acquainted with all the operators, you can simulate the model behavior by running

quint run --main=nmtProofVerification --max-samples=1 --max-steps=100 nmt.qnt --invariant=verificationAlwaysCorrect
This command will run the module nmtProofVerification, which iteratively generates data, creates a proof for a namespace and then verifies it. It will run 100 steps of the model and will check that the property verificationAlwaysCorrect always holds. It will output the steps to the terminal. (For options, run quint run --help)

quint run --main=nmtTest --max-samples=1 --max-steps=100 nmt.qnt --out-itf=ITF_files/out.itf.json
This command will simulate the module nmtTest, which non-deterministically corrupts the generated proof and outputs the simulation steps to the file ITF_files/out.itf.json, to be used in tests.

Running tests
Once a test file is generated, it is read by simulation_test.go, a regular go test. (A path to the json file that should be tested needs to be given - at the moment hardcoded in the code.)

Thank you for providing instructions on using Quint and running tests. I am currently reviewing the PR and will provide my comments as I go. However, I have an immediate question:
In the ITF_files directory, there are two JSON files - panicissue.itf.json and runTest.itf.json. Can you please explain how they were generated and how they differ from the output generated by the following command:
quint run --main=nmtTest --max-samples=1 --max-steps=100 nmt.qnt --out-itf=ITF_files/out.itf.json

Thank you for providing instructions on using Quint and running tests. I am currently reviewing the PR and will provide my comments as I go. However, I have an immediate question: In the ITF_files directory, there are two JSON files - panicissue.itf.json and runTest.itf.json. Can you please explain how they were generated and how they differ from the output generated by the following command:
quint run --main=nmtTest --max-samples=1 --max-steps=100 nmt.qnt --out-itf=ITF_files/out.itf.json

Another question: What is the expected output when running TestFromITF using the panicIssue.itf.json file.

The runTest.itf.json was created exactly by running the command that you cite (perhaps with more execution steps, not fully sure about that). Then, when I ran the test, I noticed it caused the panic. So I inspected which steps were the one causing the panic and then I removed all those into a separate file, panicIssue.itf.json. Now, for panicIssue.itf.json I expect the test to panic (until the problem is fixed).

Thanks @ivan-gavran for addressing my previous comments.
After examining the current Quint model in greater detail and reviewing some of the examples available in the Quint repository on GitHub, there are certain aspects that still require further clarification. I have listed these aspects below and would appreciate your insights on them.

It is a precise description of the expected behavior, and yet it resides on a higher level of abstraction than the code.

Can you explain the level of abstraction provided by the Quint model? For instance, when looking at the CreateProofNamespace function (https://github.com/ivan-gavran/nmt/blob/c3cc6b7acba34c97a1a4d5e8fa4be1d355535c1e/formal_spec/nmt.qnt#L277
), the code appears to be just as detailed and complex as the original Go version. It would very helpful if you please clarify how the Quint model abstracts away the implementation details?

It allows for test generation.

How does the process of generating test cases in the Quint model compare to generating them programmatically in Go? Specifically, the Quint model requires us 1) to write the code to generate the test vectors e.g., creating a tree 2) to specify the corruption scenarios, and develop necessary code to produce corrupt inputs. Can you please discuss the advantages of using the Quint model for test vector generation, and how it compares to generating test vectors programmatically in Go?

In the nmtTest module of the nmt.qnt file, the sequences of steps are generated randomly. This means that depending on the value passed in the --max-samples option and the randomness used to pick those step sequences, we may or may not cover all the corruption cases in the output test cases. I am interested to know your perspective on the confidence we can have in the test vectors generated by the Quint model and the coverage it provides. Additionally, do you anticipate that Quint testing will uncover more bugs than traditional Go testing?

Having specifications written in Quint makes it possible to change tests/specs quickly

Could you provide an example to clarify this aspect? It seems that for every new feature, we must write code in both the Quint language and the original Go code. Would this dual coding process result in a quicker change process?  Specifically, how does the use of the Quint language reduce the overall time and effort required to implement new features or changes to existing features in comparison to only using the Go code?

Hi @staheri14 , thanks for the questions and for digging deeper into this PR.
I am answering your question one-by-one below.

Can you explain the level of abstraction provided by the Quint model? For instance, when looking at the CreateProofNamespace function (https://github.com/ivan-gavran/nmt/blob/c3cc6b7acba34c97a1a4d5e8fa4be1d355535c1e/formal_spec/nmt.qnt#L277
), the code appears to be just as detailed and complex as the original Go version. It would very helpful if you please clarify how the Quint model abstracts away the implementation details?

You are right that the model version and Go version of the computation here are at a similar level of abstraction and complexity.
This is because NMT is more about computation than communication or interleaving of messages.
My quoted comment was referring to general benefits of using models.
Where we do gain on readability of NMT are the test cases: using the model, we treat test-cases explicitly as data (whereas in the original Go code they are intertwined with test execution).

How does the process of generating test cases in the Quint model compare to generating them programmatically in Go? Specifically, the Quint model requires us 1) to write the code to generate the test vectors e.g., creating a tree 2) to specify the corruption scenarios, and develop necessary code to produce corrupt inputs. Can you please discuss the advantages of using the Quint model for test vector generation, and how it compares to generating test vectors programmatically in Go?

The current test generation (as present in this PR) is the most similar to the existing fuzz_test.go.
With Quint, we could additionally generate particular high-level scenarios. (This is not present in the PR, sending a commit tomorrow to illustrate.) For instance: generate tests that fail, generate tests that fail for a particular reason, or generate tests that succeed.  (In the currently existing tests, these cannot be generated programmatically, but are instead hand-written).
The same Quint model can thus be used for tests such as those in nmt_test.go (except that those are explicitly hand-written) and tests such as those in fuzz_test.go.

In the nmtTest module of the nmt.qnt file, the sequences of steps are generated randomly. This means that depending on the value passed in the --max-samples option and the randomness used to pick those step sequences, we may or may not cover all the corruption cases in the output test cases. I am interested to know your perspective on the confidence we can have in the test vectors generated by the Quint model and the coverage it provides. Additionally, do you anticipate that Quint testing will uncover more bugs than traditional Go testing? How do you compare the tests generated by Quint with the existing tests in the nmt repository?

This touches on the three modes I was describing: a random simulation, symbolic simulation, and full verification.
Random simulation would probably work equally well as the current fuzz_test.go.
The other two modes (still to be added to Quint, by connecting it to the Apalache model checker) provide better coverage, by not depending on sheer luck to stumble upon an interesting specified(!) scenarios, but guaranteeing to find them if they exist.
I'd like to emphasise that Quint tests are really Go tests. The difference comes from separating the test-case generation and execution. In the "traditional testing approach", the test cases (such as this tabular sequence) are written by hand; in the "Quint testing", the test cases are generated by the tool.
This makes it easier to generate a much larger number of test cases (depending on the state-space size, even exhausting all possible test cases up to a certain number of steps) and in that way making it more likely that a rare bug would be found.

Having specifications written in Quint makes it possible to change tests/specs quickly

Could you provide an example to clarify this aspect? It seems that for every new feature, we must write code in both the Quint language and the original Go code. Would this dual coding process result in a quicker change process?  Specifically, how does the use of the Quint language reduce the overall time and effort required to implement new features or changes to existing features in comparison to only using the Go code?

The point I wanted to make was that as the codebase evolves (e.g.,  parameters of a certain function are changed, or the logic changes making some test-cases behave differently etc.), we do not have to re-write a large number of manually written test cases. Instead, we could change the model and the go function which reads the generated test cases and executes them, and have again a huge number of test cases re-generated.

Finally, it is fair to say that Quint is a new technology so best testing practices are still being established and evaluated.
What I believe to be the most useful feature of Quint is that it takes care of generating tests through different algorithms, at the same price of having to write a model + Go adapter.  The overall effect should be that we can write tests much faster (since one never has to write by hand tabular tests and make sure not to forget a case).

Thanks @ivan-gavran for your thorough response!
Please see my followup questions below:

You are right that the model version and Go version of the computation here are at a similar level of abstraction and complexity.
This is because NMT is more about computation than communication or interleaving of messages.
My quoted comment was referring to general benefits of using models.

Where we do gain on readability of NMT are the test cases: using the model, we treat test-cases explicitly as data (whereas in the original Go code they are intertwined with test execution).

Thanks for clarifying your point. Based on your explanation, the Quint model for nmt simplifies and automates the process of generating test data, which has two primary benefits (echoing what you said essentially): 1) it allows for the identification of hidden corner cases that would otherwise be difficult to discover manually; and 2) it reduces the workload for reproducing test data and refactoring code when new features are added.
However, as the current state of the PR, the test data generation follows a random simulation, hence, coverage-wise (referring to the first point above) the generated test data is similar to ones generated in the fuzz_test.go, right?
I would also  appreciate it if you could clarify whether Quint can be used as a specification for nmt, I would be curious to learn more about how that would work. For example, could the quint model (compared to the current go implementation) be leveraged to facilitate the implementation of nmt in a different language?  My understanding is that it cannot replace a formal english specification, and we should not rely on Quint as a replacement of specifications. Would like to know your point of view on this.

How does the process of generating test cases in the Quint model compare to generating them programmatically in Go? Specifically, the Quint model requires us 1) to write the code to generate the test vectors e.g., creating a tree 2) to specify the corruption scenarios, and develop necessary code to produce corrupt inputs. Can you please discuss the advantages of using the Quint model for test vector generation, and how it compares to generating test vectors programmatically in Go?

The current test generation (as present in this PR) is the most similar to the existing fuzz_test.go.
With Quint, we could additionally generate particular high-level scenarios. (This is not present in the PR, sending a commit tomorrow to illustrate.) For instance: generate tests that fail, generate tests that fail for a particular reason, or generate tests that succeed. (In the currently existing tests, these cannot be generated programmatically, but are instead hand-written).
The same Quint model can thus be used for tests such as those in nmt_test.go (except that those are explicitly hand-written) and tests such as those in fuzz_test.go.

I'm interested  to know how to generate test data that intentionally fail for specific reasons using quint and how to incorporate that data into the tests of the nmt_test.go file. I assume that with your upcoming commit, this process will become clearer. My understanding is that, as each test in the nmt_test.go file requires data to be corrupted in a particular way, it seems that we need to add all those corruption scenarios to the quint model under the nmt_test module. Then, using the go adapter, we can parse the JSON file containing the test data, examine the type of corruption in each case, and call the corresponding test function from nmt_test.go (or alternatively make each test in the nmt_test.go read the JSON file and finds data that is relevant to its test). Is this the intended workflow for using the resulting test data to replace the hardcoded test vectors in nmt_test.go?
Also, I think quint-generated test data are not meant for unit testing of the functions? For example, tests such as https://github.com/celestiaorg/nmt/blob/05017fc627d0cdb0aaba84293141f67010485648/hasher_test.go#L67 (although, I do not see any immediate reason that this would not be possible). Can you please elaborate on this?

In the nmtTest module of the nmt.qnt file, the sequences of steps are generated randomly. This means that depending on the value passed in the --max-samples option and the randomness used to pick those step sequences, we may or may not cover all the corruption cases in the output test cases. I am interested to know your perspective on the confidence we can have in the test vectors generated by the Quint model and the coverage it provides. Additionally, do you anticipate that Quint testing will uncover more bugs than traditional Go testing? How do you compare the tests generated by Quint with the existing tests in the nmt repository?
This touches on the three modes I was describing: a random simulation, symbolic simulation, and full verification.
Random simulation would probably work equally well as the current fuzz_test.go.
The other two modes (still to be added to Quint, by connecting it to the Apalache model checker) provide better coverage, by not depending on sheer luck to stumble upon an interesting specified(!) scenarios, but guaranteeing to find them if they exist.

The point I wanted to make was that as the codebase evolves (e.g., parameters of a certain function are changed, or the logic changes making some test-cases behave differently etc.), we do not have to re-write a large number of manually written test cases. Instead, we could change the model and the go function which reads the generated test cases and executes them, and have again a huge number of test cases re-generated.

I see your point, thanks for clarification!
Currently, the test data generated in this PR is using the random simulation, is that correct? How can we use the other two modes?

I like this discussion :)

[...] Based on your explanation, the Quint model for nmt simplifies and automates the process of generating test data, which has two primary benefits (echoing what you said essentially): 1) it allows for the identification of hidden corner cases that would otherwise be difficult to discover manually; and 2) it reduces the workload for reproducing test data and refactoring code when new features are added.
However, as the current state of the PR, the test data generation follows a random simulation, hence, coverage-wise (referring to the first point above) the generated test data is similar to ones generated in the fuzz_test.go, right?

That is correct!

I would also appreciate it if you could clarify whether Quint can be used as a specification for nmt, I would be curious to learn more about how that would work. For example, could the quint model (compared to the current go implementation) be leveraged to facilitate the implementation of nmt in a different language? My understanding is that it cannot replace a formal english specification, and we should not rely on Quint as a replacement of specifications. Would like to know your point of view on this.

I believe Quint could be used as a specification for nmt. The most promising approach would be to interleave English descriptions (for intuition) with Quint expressions (for precision).
For the NMT spec, the description of treating the ignoreMaxNamespace flag (here) caused some confusion for me as a reader. If that part was instead replaced by a Quint function, that would make it clear and would allow me to inspect the spec for different inputs.
One example of such approach is an (unfinished) intro to the bank module - linking in particular descriptions of ViewKeeper's functions.
More generally, I see Quint code as a replacement for pseudocode - and now you have specs with pseudocode you can simulate, automatically reason about, or connect to test cases.

I'm interested to know how to generate test data that intentionally fail for specific reasons using quint and how to incorporate that data into the tests of the nmt_test.go file. I assume that with your upcoming commit, this process will become clearer. My understanding is that, as each test in the nmt_test.go file requires data to be corrupted in a particular way, it seems that we need to add all those corruption scenarios to the quint model under the nmt_test module. Then, using the go adapter, we can parse the JSON file containing the test data, examine the type of corruption in each case, and call the corresponding test function from nmt_test.go (or alternatively make each test in the nmt_test.go read the JSON file and finds data that is relevant to its test). Is this the intended workflow for using the resulting test data to replace the hardcoded test vectors in nmt_test.go?
Also, I think quint-generated test data are not meant for unit testing of the functions? For example, tests such as
https://github.com/celestiaorg/nmt/blob/05017fc627d0cdb0aaba84293141f67010485648/hasher_test.go#L67
(although, I do not see any immediate reason that this would not be possible). Can you please elaborate on this?

In the latest commit, I wrote the spec and the Go adapter that mimic the tests from TestNamespacedMerkleTree_ProveNamespace_Ranges_And_Verify.
You will notice that the way in which I generate different scenarios (following the scenarios from the original test) is by defining an invariant. For instance, for the test case described by
{
			"three leaves and not found but with range", 2,
			append(repeat(generateLeafData(2, 0, 1, []byte("_data")), 2), newNamespaceDataPair([]byte{1, 1}, []byte("_data"))),
			[]byte{0, 1},
			2, 3,
			false,
		},

the invariant is defined by
     val fiveLeavesAndNotFoundButWithinRange =
        val leavesSet = listToSet(leaves_namespace_idx_v)
        and{
            length(leaves_namespace_idx_v) == 5,
            not(leavesSet.contains(namespace_v)),
            min(leavesSet) < namespace_v,
            max(leavesSet) > namespace_v
        }

The difference between the original test description and Quint description is that the original description is simultaneously describing a scenario and technically achieving it. In the commit, the Quint part only describes what a test scenario should be about, while the technical details (e.g., byte representation) are left to the adapter.
After negating the invariant, a test case is found as a violation of that (negated) invariant.
This example illustrates that one does not have to explicitly write desired corruption cases. What we do instead is describe the data by an invariant.
You are right that Quint is not meant primarily for units tests, but can be used for it, too (as the example illustrates).

The point I wanted to make was that as the codebase evolves (e.g., parameters of a certain function are changed, or the logic changes making some test-cases behave differently etc.), we do not have to re-write a large number of manually written test cases. Instead, we could change the model and the go function which reads the generated test cases and executes them, and have again a huge number of test cases re-generated.

I see your point, thanks for clarification! Currently, the test data generated in this PR is using the random simulation, is that correct? How can we use the other two modes?

Yes, the test data is generated using random simulation. The other two modes are in the coming soon stage: they depend on connecting Quint to the Apalache model checker (here is the tracking issue)

In the previous comment, I mentioned the new commit, which mimics tests from TestNamespacedMerkleTree_ProveNamespace_Ranges_And_Verify, but written using Quint. I'd like to add some more details about how that is done:

there are three variables of interest: namespaces of leaves, the namespaces that is queried, and whether the queried namespace is among leaves
unlike the spec from nmt.qnt, here I do not generate all leaves at once, but generate one leaf per step. (An approach in which I would generate all at once is also possible. This touches on the @staheri14 comment on using Quint for unit tests)
whenever we wanted to create a test scenario, we write a property describing it.
For instance, for the property that was in the .go file given by a description and exact values with:

{
			"two leaves and found", 1,
			append(generateLeafData(1, 0, 1, []byte("_data")), generateLeafData(1, 1, 2, []byte("_data"))...),
			[]byte{1},
			1, 2,
			true,
		}

, Quint defines it by
val twoLeavesAndFound =
        and{
            length(leaves_namespace_idx_v) == 2,
            val leavesSet = listToSet(leaves_namespace_idx_v)
            leavesSet.contains(namespace_v)
        }

Finally, we negate the property and use the negated property as an invariant, whose violation is then a test case.
Concreately, to obtain the test case data, we can run  quint run --max-samples=1000 --max-steps=10 tests.qnt --invariant=twoLeavesAndFoundTest

DISCLAIMER: the final json file, consisting of all test cases, scenario.json, is manually stitched together. (Generation of multiple test cases for multiple properties is in the pipeline.)

Yes, the test data is generated using random simulation. The other two modes are in the coming soon stage: they depend on connecting Quint to the Apalache model checker (https://github.com/informalsystems/quint/issues/12 is the tracking issue)

Would the current quint model for nmt be applicable for the other two simulation modes or would a different quint model be required for those modes?

Would the current quint model for nmt be applicable for the other two simulation modes or would a different quint model be required for those modes?

The same model can be used for all three modes. The difference is only in how they are looking for an invariant violation (and, therefore, what guarantees they can provide):

random simulation: at every step selects all the values (action to execute, nondetermined values etc. randomly). No guarantees, but it's very fast. Thus, if we run many simulations, we might stumble upon an execution that violates the invariant.
symbolic simulation: at every step it selects an action to execute at random, but, for the selected action, exhaustively inspects all possible values. A bit slower, but gives a guarantee that for all inspected action sequences, if no violation was found, then there is no violation (for those sequences).
verification: for a given number of actions n, it will make sure that all possible combinations (of different actions and different values) are explored and that if no violation is found, then there is not violation (of length up to n).

In this commit I corrected a mistake that was introduced while prettifying the spec, which resulted in generation of empty test-data.
I also added comments around Quint functions and did some go linting. Marking now this PR as ready.

@ivan-gavran Thanks a lot for addressing the comments, they look good to me! The issue with gobencher is currently being looked into. As soon as it's resolved, I'll let you know. At that point, I'll kindly request you merge the master branch into your branch one last time. After that, we should be ready to merge your PR :)

@ivan-gavran Would you mind updating your branch with the master branch for the last time, then we are good to merge this PR (given that CIs are fine excluding the gobencher). Thanks!


Strengthen WebAssemblyArrayBuffer.GrowMemory

Make reporting external memory allocation cleaner with a wrapper and ensure we don't end up in a bad state after ReAlloc

This change is

@akroshg can you take a look on my review fixes ?

I have reworked WebAssemblyArrayBuffer::GrowMemory a lot, so I decided to just squash my changes + rebase on top of #3479.
@tcare Could you take a look at how I use AutoDisableInterrupt as well

Interrupt use LGTM

:shipit:


Image is deprecated

In the visio-drawing here: https://github.com/microsoft/CloudAdoptionFramework/blob/master/ready/enterprise-scale-architecture.vsdx Management and Monitoring is D, not E. Image should be updated accordingly.
Likewise, https://github.com/MicrosoftDocs/cloud-adoption-framework/blob/master/docs/ready/enterprise-scale/platform-automation-and-devops.md needs to have the picture updated. I is the current reference in the image on the page it is H.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 13a01671-cf1f-63e1-b526-7346953d8042
Version Independent ID: 9ae96c16-1771-f098-f581-6a5abc630ed6
Content: Management and monitoring - Cloud Adoption Framework
Content Source: docs/ready/enterprise-scale/management-and-monitoring.md
Service: cloud-adoption-framework
Sub-service: ready
GitHub Login: @BrianBlanchard
Microsoft Alias: brblanch

Thanks for reporting - this issue is under review.  This is a Microsoft Internal DevOps Bug ID AB#270099

This issue has been resolved.


Question - Any way to import data files?

Hello - I just found out about this project yesterday and got it all set up and running, and it is awesome! Thanks.
I have a self-developed project that has been collecting /data/ endpoint JSON for about 9 months now. Is it possible to import these files into the TeslaMate application? I am happy to do some manual work to make it work. Just looking for some pointers. Thanks!

This is only possible via CSV files formatted as TeslaFi exports. If you can transform the collected data into this format, you can import it (here is an example:: https://github.com/adriankumpf/teslamate/tree/master/test/fixtures/import/01_complete). Note, however, that data with sufficiently high resolution is required: https://github.com/adriankumpf/teslamate/issues/452


Support non local requests if an authorization header is set

I don't currently have a great way to keep sqsd local to workers without scaling sqsd cron. I don't have a great answer for de-duplicating cron jobs other than keeping cron sqsd services running on a single instance. I still need to scale the workers that support cron. So this PR allows jobs to come in from anywhere so long as a provided authorization header matches what is in Amplify's environment. I think a secret is sufficient to prevent jobs from being spoofed however the risk could be compromise if I'm wrong

Is this why all jobs were failing in proton? This library always assumed local requests?
No this change is for my re-work of the services that splits off the service into many services. I was actually writing up a description that digs into the why in the PR when you commented this


Update sbt to 1.5.2

Updates org.scala-sbt:sbt from 1.5.1 to 1.5.2.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "sbt" } ]

labels: library-update, semver-patch

Superseded by #323.


Add checkboxes for each plotted line to toggle visibility

Add checkboxes for each plotted line in a plot to toggle their visibility.

Moved to https://github.com/learningsimulator/learningsimulator/issues/112


[spike] Discovery of possible optimization of a Trie structure for a prefix iterator

As implemented currently the Trie enum has two major variants: Leaf, which contains K, and V, and a Node which contains 255 pointers in a trie. Each pointer in the Node variant is either Leaf pointer or Node pointer. In the prefix iterator implementation when we descend into the requested prefix we have to also navigate all the trie pointer’s to get the actual Key value based on the matching prefix. The idea of this discovery ticket is to determine if we can optimize this by moving K from Trie::Leaf variant into the LeafPointer variant inside pointer block, so we can short circuit the last step, and just return the K from the LeafPointer without reading another Trie object at a pointer to get that K. We know for a fact that random IO on the slowest supported machine is super slow which led us to implement the batching migration mechanism, so we think this is worth  measuring at least before making decision.

@mpapierski this is something we want to do, but it is not urgent, or the most important thing for you to do at the moment.   Please finish what is in the sprint before taking this on

To do: Have a conversation with Michal and Bart before L1 Sprint planning. @devendran-m

Follow-up action as per the discussion on L1 Daily on 17-Sep-24;

Michał to review the code to confirm if the work is already completed. If it is done, close the ticket as completed. If not, discuss with Bartłomiej to pick up the remaining work.


Implement new PagesMenu API

Per discussion on #2089, implement the following:

[x] A Timber\PagesMenu class.
[x] A timber/pages_menu/classmap filter.
[x] A Timber::get_pages_menu() function.
[x] A Timber\PagesMenuFactory class - used under the hood in ::get_pages_menu(), calls the timber/pages_menu/classmap filter.
[x] Make Menu and MenuItem constructors protected.
[x] Draft the relevant documentation as well as the notes in the Upgrade Guide for this.

There is already a test-timber-pages-menu.php test file. We may want to flesh this out with a few more tests, or redesign some of the existing tests.

I think the documentation is already done for this with #2254.

Implemented in #2562.


Updated the feature flag wiki page

Updates to the Launching-new-features wiki page.

Hey @seanlip, PTAL.
Thanks

@lkbhitesh07 Would it be possible to get this PR in soonish? Contributors are still using the old way of writing tests and it would be helpful to have somewhere to point them to for the right approach. Thanks!

@lkbhitesh07 Would it be possible to get this PR in soonish? Contributors are still using the old way of writing tests and it would be helpful to have somewhere to point them to for the right approach. Thanks!

Sorry for the delay @seanlip, I have updated it now.
Thanks


Images don't load on Discord

The images no longer load because the Cert seems to have expired on the server.

oh right, just noticed that, will work on the issue asap!


{AppConfig} az appconfig: Update azure appconfiguration sdk to version 1.7.0

Related command

az appconfig
Description

This change update the azure-appconfiguration package to the latest stable version.
Testing Guide

History Notes

{AppConfig} az appconfig: Update azure appconfiguration sdk to version 1.7.0

This checklist is used to make sure that common guidelines for a pull request are followed.

[ x ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ x ] I adhere to the Command Guidelines.

[ x ] I adhere to the Error Handling Guidelines.

AppConfig

@albertofori Please resolve these conflicts~

Please note that we are launching the release for this sprint this week. Please resolve all comments and CI issues by tomorrow, otherwise the release of this PR will have to be postponed to the next sprint (on 11-05)


Enable use for ReCaptcha enterprise

The changes needed to be made are the following:

Make the script URL configurable. The enterprise script is located here: https://www.google.com/recaptcha/enterprise.js?render=reCAPTCHA_site_key
Make the calls to the enterprise grecaptcha API instead:

grecaptcha.enterprise.ready(function() {
      grecaptcha.enterprise.execute('reCAPTCHA_site_key', {action: 'homepage'}).then(function(token) {
       ...
    });
});

More documentation about this Google Cloud feature.

up

Upvote

Please, this is important

So?

what is the solution ?
I need help please

Hey! Sorry for late response. I need an Enterprise key to test.

Perhaps setup a branch with these changes so we can test it?

Hey! Sorry for late response. I need an Enterprise key to test.

I could provide an enterprise key for testing.

Hey! Sorry for late response. I need an Enterprise key to test.

I could provide an enterprise key for testing.

DM me please<EMAIL_ADDRESS>

any updates?

any updates?

I'm waiting for the key and get to work right away

@mvrlin i could prob test for you. I should be getting keys shortly but I can’t share them as I’m not the account holder. Could you post a branch?

@serge-wq released the 1.1.0 https://github.com/nuxt-community/recaptcha-module/releases/tag/v1.1.0 🎉

Mmmm… I believe you need you to change to this._grecaptcha.enterprise.excute or any other call as enterprise is from a new object grecaptcha.enterprise. Also you use mode but then I see you assign it from version. I apologize I only quickly looked at the plug-in folder. @mvrlin

Mmmm… I believe you need you to change to this._grecaptcha.enterprise.excute or any other call as enterprise is from a new object grecaptcha.enterprise. Also you use mode but then I see you assign it from version. I apologize I only quickly looked at the plug-in folder. @mvrlin

https://github.com/nuxt-community/recaptcha-module/blob/v1.1.0/lib/plugin.js#L151 the assignment is happening here.


Unable to insert new entry that has navigation link to singleton

I'm new to OData and this great library but I'm trying to do something quite simple here. Insert an item that has a link to another item that happens to be a singleton. The code looks like so:
var env = client.For("Environment").FindEntryAsync().Result;
await client.For("WebCapabilities").Set(new { id = "wc1", Environment = env } ).InsertEntryAsync();
Where:
"Environment" is a navigation property on the "WebCapability" entity type.
The error produced is a NullReferenceException in RequestWriter.cs at :
var linkSet = _model.SchemaElements
.Where(x => x.SchemaElementKind == EdmSchemaElementKind.EntityContainer)
.SelectMany(x => (x as IEdmEntityContainer).EntitySets())
.BestMatch(x => x.EntityType().Name, linkTypeWithKey.Name, _session.Pluralizer);
var formattedKey = _session.Adapter.GetCommandFormatter().ConvertKeyValuesToUriLiteral(
linkKey.ToDictionary(x => x.Name, x => linkEntry[x.Name]), true);
linkUri = linkSet.Name + formattedKey;
The problem seems to be that the code only iterates over entity sets so ignores all the Singletons contained within the Entity Container.
Is this desired behavior or a bug? I would love to know if their is a workaround to this or will it require a fix to the code ?
Thanks,
paul.
n.b:
Excluding setting Environment would produce the following odata response as you would expect:
HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Content-Language: en
OData-Version: 4.0
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Thu, 02 Jul 2015 14:31:36 GMT
Connection: close
6a
{"error":{"code":"2000","message":"The request does not specify the navigation links for 'Environment'."}}
0

Thank you for a comprehensive bug report. Yes, this is a bug. Should be easy to fix but I am away now so it may take some time until the new version is available. I will let you know. Unfortunately I don't think there is a workaround for this.

Thanks, looking forward to the new version :)

I believe I've made a fix. You can test the pre-release of the forthcoming version 4.7.3 by downloading a package from Myget:
https://www.myget.org/gallery/simple-odata-client
Please give it a try.

Hi, many thanks for that fix. I've tested the changes. It didn't quite work but perhaps this is not a fault of your changes? Have a look and see what you think...
When using my sample code with your fixes the request body now looks like this:
{  "@odata.type":"#Tridion.WebDelivery.Platform.WebCapability","id":"wc1","Environment@odata.bind":"http://cdwstest.ams.dev:8082/test/discovery.svc/Environment('DefaultEnvironment')"
}
At first glance it looks good but sadly the service returns a bad request here because the singleton Environment can't handle the fact a key predicate was passed. If I manually browse to http://cdwstest.ams.dev:8082/test/discovery.svc/Environment('DefaultEnvironment') it fails. If you loose the key predicate it works so changing the bit above to this:
"Environment@odata.bind":"http://cdwstest.ams.dev:8082/test/discovery.svc/Environment"
Do you think this is a problem with the library or perhaps shudder its an issue with my odata 4 service?
Thanks again
paul

Hi Paul,
Looks you hit another singleton-related issue. Will work on it, thanks for the notice.

Hi,
I've just pushed a new pre-release version (at myget.org) that distinguishes between singleton and entity set. You can try that, and in case you have no other collection than singleton with Environment entity type, it should work. But if in addition to Environment there are entity sets that have the same type, for the time being the only way to hint that singleton should be taken is to use typed syntax and define two types: for entity set and for singleton, e.g.:
class Environment {...}
class OtherEnvironment : Environment {}
Then Simple.OData.Client will be able to choose the right collection when creating a link. But if you only have a single (singleton) collection of Environment type, this shouldn't be an issue.
Let me know how it goes.

Did you have a chance to test the new version?

Ahh sadly no, I'll hopefully try it out on monday for you and let you know.

Just tested the changes and it works fine now :)
Many thanks
Paul


Considerably slower than chrome is.

If i use chrome to download the files from zippyshare, 500 megs generally downloads in less than a minute.
This program takes 4-5 minutes to do the same.
How do I fix this?

Hello,
Please provide more details:

What OS do you use?
What is the maximum download speed of your Internet connection?
Could you please send me zippyshare link(s) that you are using for the test?
Please send me also applog_* file(s) from zdownloader.
If you like you can send me links and logs to e-mail: pawniz at onet.pl

i use windows 10.
Internet speed test
015102050100+
164.2
Megabits per second
Testing upload...

169.5
Mbps download
164.2
Mbps upload
Latency: 18 ms
Server: Chicago
Your Internet connection is very fast.
Your Internet connection should be able to handle multiple devices
streaming HD videos, video conferencing, and gaming at the same time.

any zippyshare link with a large file is suitable for testing. I'm
having difficulty finding one i can actually share, but you should have no
trouble locating a 500MB rar file or two to download for a test.

a snippet, showing typical results with the downloader.
[20210301.08:41:56][D] SPEED: 12.74 AVG SPEED: 15.66 Mb/s
[20210301.08:41:58][D] Progress: 3.52% [17.61/500.00MB] ETA: 4m50s ActSeg:
1/1 - xxxx.rar
.
(snip)
.
[20210301.08:45:56][D] SPEED: 15.89 AVG SPEED: 15.85 Mb/s
[20210301.08:45:58][D] Progress: 99.52% [497.61/500.00MB] ETA: 1s ActSeg:
1/1 - xxxx.rar
[20210301.08:45:58][D] HTTP: xxxx.rar from
https://www119.zippyshare.com/x/xxxxxx/xxxxx/xxxx.rar

real filename and URL omitted.
I generally get 8 megaBYTES per second, which is 64 megabits.  and this is
with no download manager at all. Single segment, bog standard browser
download.
Even with 8 segments, i still can't get quite that fast, though it's
close.  Chrome gets that in ONE.

@zaphod77
Please send me content of zdownloader.ini file.
I have additional questions:

Are you using same download folder for zdownloader and Chrome? Are zdownloader and Chrome downloading the file to the same HDD/SDD?
What CPU do you have?
What is the average CPU usage(in percents) of the zdownloader during the download test?

yes and yes.
cpu is Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz   3.00 GHz. i have 8 gig
of ram.
2-3% cpu. 7 megs of ram.  i suspect that chrome is allocating much more
memory than zdownloader is.

my ONLY change from default is max_segments_per_file=1 (i did this because
i wanted to be a good little netizen because my chrome downloads were so
fast, i shouldn't NEED to steal more bandwidth than zippyshare was giving
me.) The legit form of segmented download is downloading same file from
multiple servers.
On Thu, Mar 4, 2021 at 3:41 AM sin2000<EMAIL_ADDRESS>wrote:

@zaphod77 https://github.com/zaphod77
Please send me content of zdownloader.ini file.
I have additional questions:

Are you using same download folder for zdownloader and Chrome? Are
zdownloader and Chrome downloading the file to the same HDD/SDD?
What CPU do you have?
What is the average CPU usage(in percents) of the zdownloader
during the download test?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sin2000/zdownloader/issues/20#issuecomment-790477105,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQL7ZNLFYE56NXU4SJLOLTTB5IPBANCNFSM4YMU65MA
.

okay i just bumped up the network read and file write buffers to 8 megs, and now i can get 64Mbps downloads with one segment.
It seems zdownloaders buffer defaults are very conservative.  :)

okay i just bumped up the network read and file write buffers to 8 megs, and now i can get 64Mbps downloads with one segment.
It seems zdownloaders buffer defaults are very conservative. :)

I doubt if this is a buffer problem (unless you have very slow HDD/SDD writes like less than 12 MB/s).
I checked myself and it seems that zippyshare limiting dl speed to ~15Mbps when some optional HTTP headers missing.
I didn't notice this before because I have not using application for quite some time.
I have attached(unreleased yet) zdownloader(win7_8_10-x64_beta23.zip) for Windows with some fixes and additional HTTP headers for zippyshare.
Could you please reset zdownloader.ini to defaults(especially buffers, also set max_parallel_file_downloads and max_segments_per_file to 1) and check this attached zdownloader?
Please let me know if you have better download speed with this modified app.
BTW. This probably proves that the zippyshare admin directs his attention to my application. LOL (You aren't zippyshare admin, aren't you? ;) )
win7_8_10-x64_beta23.zip

no, i'm not zippyshare admin. :) But naturally they don't want you bypassing the popups. So they are not wrong to pull that throttle.
But right now with the large buffer i'm getting Zippy speeds with 1 segment.
when i had tested today before my buffers change, i was getting 15Mbps with one segment.  But i will test this version after my current downloads finish.

Okay, tested again with your change. with default buffer settings i get 15Mbps.  While my drives aren't SSDs, they aren't THAT slow.
if i bump the buffers up to 8 megs, i get full speed.  Maybe there's some overhead from the stopping and starting when the network buffer is filled?
Doesn't seem to matter what drive i write to. perhaps it's windows itself causing the issue.  But 2M network and 1M disk buffer is slow, and bumping both to 8 meg is amazingly fast.

I have checked this issue again and you are right that buffers values(segment_network_read_buffer_kb and segment_file_write_buffer_kb) are too low.

Not sure what the best defaults are for buffers. but i will stick with 8 megabytes (8192), as i am happy with it's speed.


Mapper documentation confusing text

Under the Mapper section of Parsing Language Reference Guide (https://help.sumologic.com/docs/cse/schema/parsing-language-reference-guide/#mapper).  This paragraph is confusing and the second sentence is not clear.
Specify the product, vendor, and event_id for the message. If you identify the mapper using , all three are required: you’ll define three MAPPER attributes. Templating is allowed for each value. However, the most common and best practice is to define vendor and product using static strings. event_id often varies based on the log type so it’s more common to use templating when defining that.
https://user-images.githubusercontent.com/106690344/198131927-4573df76-568e-4e36-ba4e-9580a04e00e1.png

@MariePMcGarry I believe this one is yours.


feat(minifier): minimize computed property access

Ported ConvertToDottedProperties partially. The rest seems to be mostly handled by #8147.

This falls under substitute alternative syntax. I'll move the code around when I get the time.

=> #8175, #8176


Question: dpms should be off?

I've got DPMS enabled and it suspends the laptop after 10 minutes of inactivity as expected.
Now that I've plugged in this amazing tool, it's not clear if I should xset -dpms at startup? Or leave it be? Because the two seems to be doing the same thing.
Thanks!

I suppose that is up to you whether you want to use xidlehook for suspending or leave that to DPMS. xidlehook will let you detect a running fullscreen application or one that plays audio, and might be favorable.
I'm not sure if xset -dpms is enough, however: I stumbled upon this ArchWiki section on XScreenSaver and DPMS when Googling. I must admit I don't actually know much about DPMS!


Unable to rekey some private folders

Hi,
All my existing devices cannot open some private folders. Those devices are probably too new.
I've tried keybase rekey paper with an old paper key but it doesn't seem to work.
keybase rekey status returns No TLFs need rekeying..
Is resetting the folders the only way to move forward? Or is there anything I could do to confirm my existing keys are definitely unable to access those folders?
Thanks.

@domon if you run keybase log send and report the resulting log ID here, I can check each folder with problems individually and let you know if they can be recovered.  If keybase rekey status doesn't show anything, that could mean that none of them are recoverable by you.  But if they are folders with other users, they might be able to recover them.
If your private personal folder is one of them though, and you've revoked or lost all the devices that once had access, resetting is the only option.  (Note that recent folders can no longer get into this state -- either you have access to the folder or you have to reset your whole Keybase account, nothing in between.)

@Domon if you run keybase log send and report the resulting log ID here, I can check each folder with problems individually and let you know if they can be recovered.

Hi @strib,
I have sent the log. The log ID is fccdd83da90f58d9a74a851c.
My private personal folder is one of the folders that I can't access.

Ok, I see two problems with issues, and they both need to be reset.  I will send you a private message on Keybase with instructions on how to authorize them to be reset, so I don't expose the other private folder name here on github.

We reset the folders and all is good now. Thanks for the report!


Modal not working with Angular

Describe the bug
Hi, i'm using Flowbite with Angular. I have an error with the modals: Flowbite: Instance with ID offer does not exist. Modal with id offer has not been initialized. Please initialize it using the data-modal-target attribute. But I have already entered the data-modal-target attribute on my button.
this is the button code:
<button data-modal-target="card_modal" data-modal-toggle="card_modal"                     class="text-white bg-[#7E2BB5] font-medium rounded-lg text-sm px-5 py-2.5 text-center dark:bg-blue-600 dark:hover:bg-blue-700 dark:focus:ring-blue-800">               Change             </button>
and this is the modal  code:
<div id="card_modal" data-modal-backdrop="static" tabindex="-1" aria-hidden="true"      class="hidden fixed top-0 left-0 right-0 z-50 w-full p-8 overflow-x-hidden overflow-y-auto md:inset-0 h-[calc(100%-1rem)] max-h-full">
And the errors in the console is this:

Flowbite: Instance with ID card_modal does not exist.
Modal with id card_modal has not been initialized. Please initialize it using the data-modal-target attribute.

My issue is the same, but I am using Flask.

My solution is to ensure that the entire page is loaded before getting the modal instance:
window.addEventListener('load', () => {
  const modal = FlowbiteInstances.getInstance('Modal', 'modal-id')

  modal.show()
})

It works fine.

In my case, FlowbiteInstances is a global object imported to my code via CDN, that means it comes inside the window object after the DOM has rendered.
window.FlowbiteInstances

the docs.


New command: teams chat member add

Usage
m365 teams chat member add
Description
Adds a member to a Microsoft Teams chat conversation.
Options

Option
Description

-i, --chatId <chatId>
The ID of the chat conversation.

--userId [userId]
The ID of the user. Specify either userId or userName but not both.

--userName [userName]
User's UPN (user principal name, e.g. johndoe@example.com). Specify either userId or userName but not both.

--role [role]
The role for the user to add. Valid values are: owner, guest. Defaults to owner.

--visibleHistoryStartDateTime [visibleHistoryStartDateTime]
Include chat history until a certain point in time. Specify either visibleHistoryStartDateTime, includeAllHistory, or neither.

--includeAllHistory
Include all chat history. Specify either visibleHistoryStartDateTime, includeAllHistory, or neither.

Examples
Add a member by ID to a Teams chat without sharing any chat history.
m365 teams chat member add --chatId<EMAIL_ADDRESS>--userId bd94e214-7852-48b0-a326-5a34b2a02183

Add a member by UPN to a Teams chat and share chat history until a specific point in time.
m365 teams chat member add --chatId<EMAIL_ADDRESS>--userName<EMAIL_ADDRESS>--visibleHistoryStartDateTime 2023-05-03T12:00:00Z

Add a guest member by UPN to a Teams chat and share all chat history.
m365 teams chat member add --chatId<EMAIL_ADDRESS>--userName<EMAIL_ADDRESS>--role guest --includeAllHistory

Remarks
We should give an extra explanation for the option role. Value owner is only used for internal users, guest is used for guest users.
Additional Info
https://learn.microsoft.com/en-us/graph/api/chat-post-members

Thank you for this suggestion @Aadit-Ambadkar, seems like this is indeed a missing command.
Specs are already a good start, however I notice that we should change a few things.

Let's use -i as short for chatId to be consistent with teams chat member list
Let's make chatId required instead of making it a flag, so -i, --chatId <chatId>.
Same with userId
Let's also add userName as option
We should also expose a property named role
Let's add a property to include all chat history
Let's add a property to include chat history to a certain point in time

I'll update your specs with the feedback listed above.

Updated the specs, now let's wait for another maintainer to agree with it before we start working on it.
@pnp/cli-for-microsoft-365-maintainers 👀

Can I work on this one?

Definitely!

@Aadit-Ambadkar the command has been created and is now available in our beta release.
You can run npm i -g @pnp/cli-microsoft365@next to try it out.

That's cool! (first time actually using GitHub the way it was intended). Unfortunately, school's out right now so my Teams subscription doesn't work, but I'll be sure to check it in the fall.

School's out right now so my Teams subscription doesn't work, but I'll be sure to check it in the fall.

By that time it will be in our stable release 😊
If you find more missing commands, don't hesitate to create a new issue. We greatly appreciate it!


Rebuilding a docker file

Hi,
I am using a command on a daily basis in update my software on a container.
docker-compose build -d my-app
Is it possible to rebuild an image with portainer?
Thanks

You can use the Build an image feature via Images > Build a new image to build an image from a tarball that is available on your machine.
It is not possible to use the docker-compose build equivalent directly into Portainer tho. You can ask any other questions in our community support channel: https://portainer.slack.com


keeps adding /_karma_webpack_ to file paths?

My build was working fine, and then when I sent it to travis I noticed that it blew up for some reason, anyway cleaned my node_modules and re installed and now for some reason I keep getting errors where it is saying the file does not exist, and is prefixing with /_karma_webpack_, so here is an example:
07 07 2016 10:00:58.013:ERROR [karma]: { [Error: no such file or directory]
  code: 'ENOENT',
  errno: 34,
  message: 'no such file or directory',
  path: '/_karma_webpack_/tests/specs/rules/max-value-rule-tests.ts' }
Error: no such file or directory
    at MemoryFileSystem.readFileSync (C:\Code\Web\treacherous\treacherous\node_modules\memory-fs\lib\MemoryFileSystem.js:114:10)
    at MemoryFileSystem.readFile (C:\Code\Web\treacherous\treacherous\node_modules\memory-fs\lib\MemoryFileSystem.js:297:21)
    at doRead (C:\Code\Web\treacherous\treacherous\node_modules\karma-webpack\index.js:156:26)
    at Plugin.readFile (C:\Code\Web\treacherous\treacherous\node_modules\karma-webpack\index.js:160:3)
    at nextTickCallbackWith0Args (node.js:419:9)
    at process._tickCallback (node.js:348:13)

The file lives in tests/specs/rules/max-value-rule-tests.ts but no idea why it is looking there, my config looks like:
module.exports = function(config) {
    config.set({
        files: [
            'node_modules/bluebird/js/browser/bluebird.js',
            'tests/specs/**/*.ts'
        ],

        preprocessors: {
            'tests/specs/**/*.ts': ['webpack']
        },

        webpack: {
            module: {
                loaders: [
                    { test: /\.tsx?$/, loader: 'ts-loader', exclude: /node_modules/}
                ]
            },
            resolve: {
                extensions: ['.ts', '.js', '.tsx', '.jsx', '']
            }
        },

        frameworks: ['mocha'],

        // test result reporter
        reporters: ['progress'],

        // web server port
        port: 9876,

        // enable / disable colors in the output (reporters and logs)
        colors: true,

        // level of logging
        logLevel: config.LOG_INFO,

        // enable / disable watching file and executing tests whenever any file changes
        autoWatch: false,

        // start these browsers
        browsers: ['PhantomJS'],

        // Continuous Integration mode
        singleRun: true
    });
};

@grofit - Can you paste the os & node sections of your travis.yml?
i.e.
os:
  - linux
  - osx
node_js:
  - node
  - "6"
  - "5"
  - "4"

@grofit I'm also wondering if adding an output.path to you your config would help. I think that __karma_webpack__ is the default output.path and output.publicPath

I am pretty sure I fixed the issue whatever it was, given that I raised this a few weeks back I have completely forgotten the context as to what was actually happening. I didnt need to add output path but it all seems fine now, so just close it if needed.

@grofit - If you happen to remember, please append it to the issue

Had the same trouble:
preprocessors: {    ''tests/index.js': ['webpack'] // reported _karma_webpack_/tests/index,js could not be resolved }
After setting up basePath in karma.conf it worked.
Out of plugin sources I do not see how output.publicPath can be helpful, but I put it there also just in case.
Hope that helps.


WIP - Feature 88 - Fetch and display engagement data on rota screen

Stuff to do:

currently all engagements are fetched, limit the engagements to currently viewed week
only the manager part of the view works, we need to decide how to best handle the user view (e.g. use different component or check permissions within the rota component)
decide what to do with the colours, currently hard-coded as a colour array and cycle when the number of users is greater than the number of colours - perhaps there is a better solution?
The text in rendered boxes is cut off - decide how we want the text to be displayed if there are multiple user engagements at the same time. Perhaps a different calendar view would be more appropriate (but timeline-view https://fullcalendar.io/docs/timeline-view requires premium version of fullcalendar.io)

Looks good


Build error in build with GO_TAGS=stablediffusion

LocalAI version:

commit f505d7ab3f4dabf927413d42691adb37bd46f131 (HEAD -> master, origin/master, origin/HEAD)
Environment, CPU architecture, OS, and Version:

Darwin Samarths-MacBook-Pro-495.local 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64
Describe the bug

I tried building locally using the command
make GO_TAGS=stablediffusion build

on MacBook M1  Pro
I followed the setup guide on the website and was able to build and use without errors using:
make build

To Reproduce

On Mac M1 Pro using the series of commands:
brew install abseil cmake go grpc protobuf wget protoc-gen-go protoc-gen-go-grpc

git clone https://github.com/go-skynet/LocalAI.git

cd LocalAI

make build

# worked fine till now and able to use the build

make GO_TAGS=stablediffusion build
# gives error

Expected behavior

Logs

Error logs
In file included from /opt/homebrew/include/absl/strings/string_view.h:39:
In file included from /opt/homebrew/include/absl/base/attributes.h:37:
In file included from /opt/homebrew/include/absl/base/config.h:92:
/opt/homebrew/include/absl/base/policy_checks.h:79:2: error: "C++ versions less than C++14 are not supported."
#error "C++ versions less than C++14 are not supported."
 ^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:108:
In file included from /opt/homebrew/include/google/protobuf/stubs/common.h:21:
In file included from /opt/homebrew/include/absl/strings/string_view.h:40:
In file included from /opt/homebrew/include/absl/base/nullability.h:153:
/opt/homebrew/include/absl/base/internal/nullability_impl.h:40:16: warning: variable templates are a C++14 extension [-Wc++14-extensions]
constexpr bool IsSupportedType = IsNullabilityCompatible<T>::value;
               ^
/opt/homebrew/include/absl/base/internal/nullability_impl.h:56:55: error: no template named 'remove_cv_t' in namespace 'std'; did you mean simply 'remove_cv_t'?
  static_assert(nullability_internal::IsSupportedType<std::remove_cv_t<T>>,
                                                      ^~~~~~~~~~~~~~~~
                                                      remove_cv_t
/opt/homebrew/include/absl/meta/type_traits.h:238:1: note: 'remove_cv_t' declared here
using remove_cv_t = typename std::remove_cv<T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:108:
In file included from /opt/homebrew/include/google/protobuf/stubs/common.h:21:
In file included from /opt/homebrew/include/absl/strings/string_view.h:40:
In file included from /opt/homebrew/include/absl/base/nullability.h:153:
/opt/homebrew/include/absl/base/internal/nullability_impl.h:64:55: error: no template named 'remove_cv_t' in namespace 'std'; did you mean simply 'remove_cv_t'?
  static_assert(nullability_internal::IsSupportedType<std::remove_cv_t<T>>,
                                                      ^~~~~~~~~~~~~~~~
                                                      remove_cv_t
/opt/homebrew/include/absl/meta/type_traits.h:238:1: note: 'remove_cv_t' declared here
using remove_cv_t = typename std::remove_cv<T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:108:
In file included from /opt/homebrew/include/google/protobuf/stubs/common.h:21:
In file included from /opt/homebrew/include/absl/strings/string_view.h:40:
In file included from /opt/homebrew/include/absl/base/nullability.h:153:
/opt/homebrew/include/absl/base/internal/nullability_impl.h:72:55: error: no template named 'remove_cv_t' in namespace 'std'; did you mean simply 'remove_cv_t'?
  static_assert(nullability_internal::IsSupportedType<std::remove_cv_t<T>>,
                                                      ^~~~~~~~~~~~~~~~
                                                      remove_cv_t
/opt/homebrew/include/absl/meta/type_traits.h:238:1: note: 'remove_cv_t' declared here
using remove_cv_t = typename std::remove_cv<T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:108:
In file included from /opt/homebrew/include/google/protobuf/stubs/common.h:23:
In file included from /opt/homebrew/include/google/protobuf/stubs/port.h:22:
/opt/homebrew/include/google/protobuf/port_def.inc:139:1: error: static assertion failed due to requirement '201103L >= 201402L': Protobuf only supports C++14 and newer.
static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer.");
^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:108:
In file included from /opt/homebrew/include/google/protobuf/stubs/common.h:35:
/opt/homebrew/include/google/protobuf/port_def.inc:139:1: error: static assertion failed due to requirement '201103L >= 201402L': Protobuf only supports C++14 and newer.
static_assert(PROTOBUF_CPLUSPLUS_MIN(201402L), "Protobuf only supports C++14 and newer.");
^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:36:
In file included from /opt/homebrew/include/absl/log/internal/nullstream.h:36:
/opt/homebrew/include/absl/base/log_severity.h:102:3: warning: use of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions]
  switch (s) {
  ^
/opt/homebrew/include/absl/base/log_severity.h:108:3: warning: multiple return statements in constexpr function is a C++14 extension [-Wc++14-extensions]
  return "UNKNOWN";
  ^
/opt/homebrew/include/absl/base/log_severity.h:103:36: note: previous return statement is here
    case absl::LogSeverity::kInfo: return "INFO";
                                   ^
/opt/homebrew/include/absl/base/log_severity.h:104:39: note: previous return statement is here
    case absl::LogSeverity::kWarning: return "WARNING";
                                      ^
/opt/homebrew/include/absl/base/log_severity.h:105:37: note: previous return statement is here
    case absl::LogSeverity::kError: return "ERROR";
                                    ^
/opt/homebrew/include/absl/base/log_severity.h:106:37: note: previous return statement is here
    case absl::LogSeverity::kFatal: return "FATAL";
                                    ^
/opt/homebrew/include/absl/base/log_severity.h:116:21: warning: variable declaration in a constexpr function is a C++14 extension [-Wc++14-extensions]
  absl::LogSeverity n = s;
                    ^
/opt/homebrew/include/absl/base/log_severity.h:117:3: warning: use of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions]
  if (n < absl::LogSeverity::kInfo) n = absl::LogSeverity::kInfo;
  ^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:37:
In file included from /opt/homebrew/include/absl/log/internal/strip.h:24:
In file included from /opt/homebrew/include/absl/log/internal/log_message.h:41:
In file included from /opt/homebrew/include/absl/log/log_entry.h:35:
/opt/homebrew/include/absl/time/time.h:278:22: warning: 'constexpr' non-static member function will not be implicitly 'const' in C++14; add 'const' to avoid a change in behavior [-Wconstexpr-not-const]
    constexpr HiRep& operator=(const int64_t value) {
                     ^
                                                    const
/opt/homebrew/include/absl/time/time.h:255:7: warning: use of this statement in a constexpr constructor is a C++14 extension [-Wc++14-extensions]
      *this = value;
      ^
/opt/homebrew/include/absl/time/time.h:259:22: warning: variable declaration in a constexpr function is a C++14 extension [-Wc++14-extensions]
      const uint64_t unsigned_value =
                     ^
/opt/homebrew/include/absl/time/time.h:284:11: error: cannot assign to non-static data member within const member function 'operator='
      hi_ = static_cast<uint32_t>(unsigned_value >> 32);
      ~~~ ^
/opt/homebrew/include/absl/time/time.h:278:22: note: member function 'absl::Duration::HiRep::operator=' is declared const here
    constexpr HiRep& operator=(const int64_t value) {
    ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/include/absl/time/time.h:285:11: error: cannot assign to non-static data member within const member function 'operator='
      lo_ = static_cast<uint32_t>(unsigned_value);
      ~~~ ^
/opt/homebrew/include/absl/time/time.h:278:22: note: member function 'absl::Duration::HiRep::operator=' is declared const here
    constexpr HiRep& operator=(const int64_t value) {
    ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/include/absl/time/time.h:286:14: error: binding reference of type 'HiRep' to value of type 'const absl::Duration::HiRep' drops 'const' qualifier
      return *this;
             ^~~~~
/opt/homebrew/include/absl/time/time.h:283:18: warning: variable declaration in a constexpr function is a C++14 extension [-Wc++14-extensions]
      const auto unsigned_value = static_cast<uint64_t>(value);
                 ^
/opt/homebrew/include/absl/time/time.h:284:7: warning: use of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions]
      hi_ = static_cast<uint32_t>(unsigned_value >> 32);
      ^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:37:
In file included from /opt/homebrew/include/absl/log/internal/strip.h:24:
In file included from /opt/homebrew/include/absl/log/internal/log_message.h:41:
In file included from /opt/homebrew/include/absl/log/log_entry.h:36:
In file included from /opt/homebrew/include/absl/types/span.h:70:
/opt/homebrew/include/absl/types/internal/span.h:119:21: error: no template named 'remove_const_t' in namespace 'std'; did you mean simply 'remove_const_t'?
  using Container = std::remove_const_t<T>;
                    ^~~~~~~~~~~~~~~~~~~
                    remove_const_t
/opt/homebrew/include/absl/meta/type_traits.h:241:1: note: 'remove_const_t' declared here
using remove_const_t = typename std::remove_const<T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:37:
In file included from /opt/homebrew/include/absl/log/internal/strip.h:24:
In file included from /opt/homebrew/include/absl/log/internal/log_message.h:41:
In file included from /opt/homebrew/include/absl/log/log_entry.h:36:
In file included from /opt/homebrew/include/absl/types/span.h:70:
/opt/homebrew/include/absl/types/internal/span.h:130:24: error: no template named 'enable_if_t' in namespace 'std'; did you mean simply 'enable_if_t'?
using EnableIfIsView = std::enable_if_t<IsView<T>::value, int>;
                       ^~~~~~~~~~~~~~~~
                       enable_if_t
/opt/homebrew/include/absl/meta/type_traits.h:307:1: note: 'enable_if_t' declared here
using enable_if_t = typename std::enable_if<B, T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:37:
In file included from /opt/homebrew/include/absl/log/internal/strip.h:24:
In file included from /opt/homebrew/include/absl/log/internal/log_message.h:41:
In file included from /opt/homebrew/include/absl/log/log_entry.h:36:
In file included from /opt/homebrew/include/absl/types/span.h:70:
/opt/homebrew/include/absl/types/internal/span.h:133:27: error: no template named 'enable_if_t' in namespace 'std'; did you mean simply 'enable_if_t'?
using EnableIfNotIsView = std::enable_if_t<!IsView<T>::value, int>;
                          ^~~~~~~~~~~~~~~~
                          enable_if_t
/opt/homebrew/include/absl/meta/type_traits.h:307:1: note: 'enable_if_t' declared here
using enable_if_t = typename std::enable_if<B, T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:110:
In file included from /opt/homebrew/include/absl/log/absl_check.h:38:
In file included from /opt/homebrew/include/absl/log/internal/check_impl.h:19:
In file included from /opt/homebrew/include/absl/log/internal/check_op.h:37:
In file included from /opt/homebrew/include/absl/log/internal/strip.h:24:
In file included from /opt/homebrew/include/absl/log/internal/log_message.h:43:
/opt/homebrew/include/absl/strings/has_absl_stringify.h:56:8: error: no template named 'enable_if_t' in namespace 'std'; did you mean simply 'enable_if_t'?
    T, std::enable_if_t<std::is_void<decltype(AbslStringify(
       ^~~~~~~~~~~~~~~~
       enable_if_t
/opt/homebrew/include/absl/meta/type_traits.h:307:1: note: 'enable_if_t' declared here
using enable_if_t = typename std::enable_if<B, T>::type;
^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:112:
In file included from /opt/homebrew/include/absl/strings/cord.h:79:
In file included from /opt/homebrew/include/absl/container/inlined_vector.h:53:
In file included from /opt/homebrew/include/absl/container/internal/inlined_vector.h:31:
In file included from /opt/homebrew/include/absl/container/internal/compressed_tuple.h:40:
/opt/homebrew/include/absl/utility/utility.h:54:12: error: no member named 'index_sequence' in namespace 'std'
using std::index_sequence;
      ~~~~~^
/opt/homebrew/include/absl/utility/utility.h:55:12: error: no member named 'index_sequence_for' in namespace 'std'
using std::index_sequence_for;
      ~~~~~^
/opt/homebrew/include/absl/utility/utility.h:56:12: error: no member named 'integer_sequence' in namespace 'std'; did you mean '__integer_sequence'?
using std::integer_sequence;
      ~~~~~^~~~~~~~~~~~~~~~
           __integer_sequence
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk/usr/include/c++/v1/__utility/integer_sequence.h:25:8: note: '__integer_sequence' declared here
struct __integer_sequence {
       ^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:112:
In file included from /opt/homebrew/include/absl/strings/cord.h:79:
In file included from /opt/homebrew/include/absl/container/inlined_vector.h:53:
In file included from /opt/homebrew/include/absl/container/internal/inlined_vector.h:31:
In file included from /opt/homebrew/include/absl/container/internal/compressed_tuple.h:40:
/opt/homebrew/include/absl/utility/utility.h:57:12: error: no member named 'make_index_sequence' in namespace 'std'
using std::make_index_sequence;
      ~~~~~^
/opt/homebrew/include/absl/utility/utility.h:58:12: error: no member named 'make_integer_sequence' in namespace 'std'; did you mean '__integer_sequence'?
using std::make_integer_sequence;
      ~~~~~^~~~~~~~~~~~~~~~~~~~~
           __integer_sequence
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.5.sdk/usr/include/c++/v1/__utility/integer_sequence.h:25:8: note: '__integer_sequence' declared here
struct __integer_sequence {
       ^
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/tools/caffe/caffe2ncnn.cpp:19:
In file included from /Users/samarthpusalkar/Desktop/LocalAI/sources/go-stable-diffusion/ncnn/build/tools/caffe/caffe.pb.h:20:
In file included from /opt/homebrew/include/google/protobuf/io/coded_stream.h:112:
In file included from /opt/homebrew/include/absl/strings/cord.h:79:
In file included from /opt/homebrew/include/absl/container/inlined_vector.h:53:
In file included from /opt/homebrew/include/absl/container/internal/inlined_vector.h:31:
In file included from /opt/homebrew/include/absl/container/internal/compressed_tuple.h:40:
/opt/homebrew/include/absl/utility/utility.h:82:12: error: no member named 'in_place_t' in namespace 'std'
using std::in_place_t;
      ~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
10 warnings and 20 errors generated.
make[4]: *** [tools/caffe/CMakeFiles/caffe2ncnn.dir/caffe2ncnn.cpp.o] Error 1
make[3]: *** [tools/caffe/CMakeFiles/caffe2ncnn.dir/all] Error 2
make[2]: *** [all] Error 2
make[1]: *** [ncnn/build/src/libncnn.a] Error 2
make: *** [sources/go-stable-diffusion/libstablediffusion.a] Error 2

`
Additional context

Yes, I confirm that with GO_TAGS=stablediffusion not works. Well, following build instructions from website. Also, is necessary opencv lib and ncnn. But some errors on building.


WIP: WRKLDS-1067: Add e2e test

SSIA

/retest

/retest

I'm not sure image all these image failures. Will close/reopen to trigger;
/close

/reopen

/retest

/lgtm


Empty dir not created

#70 added a test case for that (but the .download() call was wrong and corrected by #82).  We'll need the storage api to support mkdir: in addition to file: for this to work.

should work in 8


Show caption instead of filename (in my contributions and Explore)

Caption from Structured Commons, localized in the user's language.
Fall back to filename if no caption is available.

Hello @nicolas-raoul i am an outreachy applicant..I would like to work on this issue.

@BinduVerma123 It is yours :-)
Here is the API you will have to use: https://github.com/commons-app/apps-android-commons/issues/2337#issuecomment-470782124

@nicolas-raoul does this mean we should remove Title field from media details and add Captions instead of it?

Yes, if a caption is available in the user's language, it should be shown, and the title not shown.
(for those wondering: the title can still be obtained if needed via the "copy wikitext" button)

In that case some minor changes in #2662 will be enough to solve the issue.

@BinduVerma123 Any progress on this?
Please share anything you might have found out, things that worked and also things that did not work if any :-)

@vanshikaarora This needs to be implemented. It is the most important part of the GSoC to make the app more localized :-)
How to reproduce:

Set Android language to English or French or Japanese
Open the app
Tap "Explore"
Tap the search icon
Enter "tantanmen"
Scroll until you see this picture: https://commons.wikimedia.org/wiki/File:Tantanmen.jpeg

Expected: The app labels the picture using its caption in Android's language, for instance English or French or Japanese
Problem: The app labels the picture using its filename "Tantanmen" (which is not any of the captions)
Screenshots:

@nicolas-raoul I am currently working on this :)

I believe this is now implemented in all places where titles (filenames) were shown before.
(in Vanshika's upload_caption_depict branch)
If you find any other place where captions could be used, please let us know :-)


Change provider header logo size to align with product name

Context
It was identified in product review that the GOV.UK logo was too large to align with the product text. This PR reduces the size of the logo so that it aligns correctly.
Changes proposed in this pull request
Before:

After:

Guidance to review

Does this look okay to you?

Link to Trello card

https://trello.com/c/st1Mq1h6/2005-build-implement-the-new-header-for-manage-applications

Things to check

[x] This code doesn't rely on migrations in the same Pull Request
[x] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[x] API release notes have been updated if necessary
[x] New environment variables have been added to the Azure config

I think the second commit accidentally reverted the first!

I think the second commit accidentally reverted the first!

Sorry this was by intent - conversations progressed and it was agreed to go with the default GOV.UK styles which are larger than the initial change in this PR.


Feature/dx 945

Updates the image so default commmand runs serve and watch with port binding.
I'll look a bit further into how to get the console to print Server address: http://<IP_ADDRESS>:4000 instead of Server address: http://<IP_ADDRESS>:4000 to make it easier for those that does not have intimate knowledge.

Why can't this just be easy 😢 Maybe I'm chasing an impossible task.


Add a brightness magic variable [FEATURE]

Description of the requested feature
Add a brightness magic variable for actions related to controlling brightness
Proposed configuration syntax
No response
Additional context
No response

Hi Dear
I created the code i tested it on my system (Arch x86-64) it currently only works on Linux not macos(i don't have a mac )
Here is the branch Brightness Magic var
the use case is simple { <name_of_backlight>: { brightness, max_brightness, percent } } similar like EWW_DISK

Thanks!


Queries are very slow even in a basic table when the previous query was too slow

When a query was very slow in its execution, the following queries also became slow regardless of whether the consulted table is small, in this case, the solution is to close omniDB and open it again.
OmniDB 2.15.0
PostgreSQL 10.5

Hi @erdusa ,
Nice to hear from you again :)
Please provide more information so we can try to reproduce this. Suppose you are talking about:

Q1: a real slow query which was executed first and was also slow on OmniDB. So far so good.
Q2: a fast query which was executed after Q1 and was slow on OmniDB. But executing elsewhere, the query is fine.

Please answer this, providing as many details as you can:

Were Q1 and Q2 executed in the same Query Tab?
When you executed Q1, did you let it run to the end, or did you cancel it by clicking on the Cancel button?
When you executed Q1, was autocommit enabled or disabled?
Did you change autocommit after executing Q1 and before executing Q2?
Does Q2 perform fast when you try to execute it in a different, new Query Tab, or is it always slow after execution of Q1?
If Q1 is not executed at all, does Q2 perform fast as it should?
Are you using OmniDB server or app?
If you are using OmniDB server, when you say "solution is to close OmniDB and open it again", do you have to restart server as well or only the browser?
Does this issue happen in Console Tab?

And also other information that you may find relevant, please.
Thanks in advance,
William

Yes, that is the case, Q1 is fast, Q2 is slow, when I execute Q1 after Q2., now Q1 is slow, but at the finish OmniBD shows the duration is short though it waits for a long time for finish and show results.
This has happened to me several times, when I have this behavior again, I will answer your questions, I hope that is soon.

Hi @wind39, look this:

The Query only returns 7 rows in a simple table, and the total time is 30 seconds, :(
Now, I close OmniDB and open it again, and the result is different:

I could not identify what the problem was

Hi @erdusa ,
Thanks for the use case. Did you execute anything before the query started to take 30 seconds? Did you cancel the previous query?
Regards,
William

No, I didn't, I am thinking that the other possible slow query is not the problem

Hi @wind39
look at this new find

It is the same query executed in both applications regardless of whether the query is cached or not

Hi @erdusa,
So this specific query takes around 30 seconds every time you execute in OmniDB? When you click on Fetch All, does that return the rest of records quickly?
We need to try to reproduce this. Does this happen with other queries too?

hi @rafaelthca
Now I am running the query en omnidb and it takes 59 seconds every time, all time, and when I click on Fetch all takes 30 seconds, but in PGAdmin always takes 3.4 seconds
PgAdmin

OmniDB:

click on fetch all:

And with others query also it happens, but in sometimes, I don't know which is the problem

Ok thanks. Did that happen in 2.14.0 or earlier versions? If I create a custom build with additional debug messages, would you test for us? The idea is to install the custom build and reproduce the issue again. Then, send us the logs.
If you are willing to do that, please confirm the version. I can see you are using windows, app version, right?

Hi @rafaelthca, Yes of course, I will do it.
I'm using the 2.15.0 OmniDB version, and yes it is Windows and app version

Hi @erdusa , @rafaelthca
I stumped into this issue by chance, but I was executing the same very simple query against 2 OmniDB instances:

Local app connected to a remote database via SSH tunnel;
Remote server connected to a remote database, which was close to the server.

And the 30 seconds "penalty" happened only on the instance that was connected to the database via SSH tunnel, and not the remote one. This local app was open by several days already. Restarting the OmniDB app re-created the SSH tunnel and fixed the issue.
Perhaps you were connected via tunnel too?
Regards,
William


C++ Inference Failed

Error Message:
tvm/src/runtime/module.cc:58: Check failed: f != nullptr Loader of (module.loadfile_so) is not presented.
terminate called after throwing an instance of 'dmlc::Error'
  what():  [16:49:05] tvm/src/runtime/module.cc:58: Check failed: f != nullptr Loader of (module.loadfile_so) is not presented.

Reproducing step:

Get the pretrained resnet18 model and dump the necessary files: graph json, parameters and library file

# -*- coding: utf-8 -*-

import mxnet
import nnvm
import tvm
import numpy as np
import os

from mxnet.gluon.model_zoo.vision import get_model
from PIL import Image
import matplotlib.pylab as plt
project_root = r'/home/xxx/data/tvm_demo'
block = get_model('resnet18_v1', pretrained=True)
synset_name = os.path.join(project_root, 'imagenet1000_clsid_to_human.txt')
img_name = os.path.join(project_root, 'cat.png')
with open(synset_name) as f:
    synset = eval(f.read())
image = Image.open(img_name).resize((224, 224))
plt.imshow(image)
# plt.show()

def transform_image(image):
    image = np.array(image) - np.array([123., 117., 104.])
    image /= np.array([58.395, 57.12, 57.375])
    image = image.transpose((2, 0, 1))
    image = image[np.newaxis, :]
    return image

x = transform_image(image)
print('x', x.shape)

sym, params = nnvm.frontend.from_mxnet(block)

sym = nnvm.sym.softmax(sym)

import nnvm.compiler

target = 'llvm'
shape_dict = {'data': x.shape}
graph, lib, params = nnvm.compiler.build(sym, target, shape_dict, params=params)

from tvm.contrib import util
# dumping model files
print("Dumping model files...")
lib_path = os.path.join(project_root, 'resnet18_deploy.so')
lib.export_library(lib_path)

graph_json_path = os.path.join(project_root, 'resnet18.json')
with open(graph_json_path, 'w') as fo:
    fo.write(graph.json())

param_path = os.path.join(project_root, 'resnet18.params')
with open(param_path, 'wb') as fo:
    fo.write(nnvm.compiler.save_param_dict(params))

from tvm.contrib import graph_runtime
ctx = tvm.cpu(0)
dtype = 'float32'
m = graph_runtime.create(graph, lib, ctx)

# set inputs
data_x = tvm.nd.array(x.astype(dtype))
cat_file = os.path.join(project_root, 'cat.bin')
data_x.asnumpy().tofile(cat_file)
m.set_input('data', data_x)
m.set_input(**params)
# execute
m.run()
# get outputs
tvm_output = m.get_output(0, tvm.nd.empty((1000,), dtype))
top1 = np.argmax(tvm_output.asnumpy())
print('TVM prediction top-1:', top1, synset[top1])

The above script works well

Build tvm as a static library(using bazel like build system) and get libtvm.a

Test cpp inference using code from this page and get the error above.

Could you tell me how to fix this problem or am I doing anything wrong???

What is your argument for LoadFile ?
You could move this general discussion to discuss.tvm.ai

@srkreddy1238 resnet18_deploy.so, as output from the above python script, I have also post a question in here, feel free to close this issue if discuss.tvm.ai is better

Replied on discussion forum.
I can't close this here. You should be able to :)

What was the resolution to this problem? Moving to discuss.tvm.ai screws everyone who follows on similar issues.


Does not work with private Github repos

when using private repos, it requires a username and password, when using .replit, it does not ask for this, but simply runs and exits, does not ask for input.
GitHub also tells me this is a to be deprecated form of login
"Basic authentication using a password to Git is deprecated and will soon no longer work. Visit https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information around suggested workarounds and removal dates."

Hi, unfortunately this is the case because the app directly uses the URL to fetch the replit-deploy.json file from the GitHub repo.
This is technically possible to do, but it would require implementing authentication on the daemon side.
I'm not willing to implement this myself, but I'd be glad to accept a PR


Added calculateChangedBits()

This is a polyfill of the React API, it will not be adding any additional API or making any changes

This is part of the React API

https://github.com/facebook/react/blob/29e8924c70856bef9b11e0c74a450140bfcce773/packages/react/src/ReactContext.js#L16

Ready

Ready


Please document the '14.x' notation?

Description:
In the examples, there is a .x notation for the node-version. I could not find the spec for that notation. Could you link to that from near the example?
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
  with:
    node-version: '14.x'

Justification:
It is unclear which version-spec is used.
Are you willing to submit a PR?
If I could find the source, yes :)

Hello @d2a-raudenaerde. Thank you for your report. You do not need to provide a version as 14.x. You can use it as 14 and it will be used as a range for SemVer notation.

Or maybe better, from the Supported version syntax header?

I think we can use the similar approach as for the setup-go action and attach the link to the SemVer documentation.


Added functionality to generate invoice

Description

Checklist:

[x] I have performed a self-review of my own code.

@GauravGusain98 What we are trying to achieve in this PR, what were the requirements to create this PR, can you share it ?


Issue 11296 Fixes - AWS Rule

Required items, please complete
Change(s):

Issue #11296

Reason for Change(s):

Modified condition to handle array match

Version Updated:

Required only for Detections/Analytic Rule templates
See guidance below

Testing Completed:

See guidance below

Checked that the validations are passing and have addressed any issues that are present:

See guidance below

@microsoft-github-policy-service agree

Hi @ashwin-patil, is it correct that only comment has been added in the above query? is the only change which you have done in the query. Thanks...!!

Hi @ashwin-patil, is it correct that only comment has been added in the above query? is the only change which you have done in the query. Thanks...!!

Yes, i committed fix already by directly to master by mistake so raised this PR to link issue and updating solution package with actual fix.


Navbar

Ill create a navbar where the user can add links to external sites, e.g. solar panel data graph page, router webinterface, or temperature logger...

OK but can we discuss position? A typical navbar is on the top because its a curtail part of the website. But we dont need any kind of site navigation right now. So I would suggest to but it either on the left/right side as a "pop out" or at the button in a footer

yep, i will change it this weekend

There is a problem with mobile devices. The Sidebar will toggle automatically on small screens. Dont know how to fix this

New sidebar is working well 5dffdda56bc0f9afb401d5f37749b678c4a6a193
We need to talk about the creation of links. I have some ideas how to do this.


Relax assertions about diagnostics from missing protocol requirements

In some Swift 6 toolchains, the diagnostic about missing protocol requirements is emitted on the protocol in the inheritance clause instead of the conforming type. Relax a few assertions in the tests.

@swift-ci Please test


pathlib for tests

Checklist
GitHub

[ ] I've given this PR a concise, self-descriptive, and meaningful title
[ ] I've linked relevant issues in the PR body
[ ] I've applied the relevant labels to this PR
[ ] I've assigned a reviewer

PR contents

[ ] I've consulted ivadomed's internal developer documentation to ensure my contribution is in line with any relevant design decisions
[ ] I've added relevant tests for my contribution
[ ] I've updated the relevant documentation for my changes, including argparse descriptions, docstrings, and ReadTheDocs tutorial pages

Description

Linked issues

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+4.8%) to 74.898%

Totals

Change from base Build<PHONE_NUMBER>:
4.8%

Covered Lines:
4395

Relevant Lines:
5868

💛  - Coveralls


Data set production

I want to know about the LFW data set, what I found is in JPG format, how did the author convert it to .pkl format?

Hi @kuaileyuandi thank you for the question. What you get by running the download_data.sh script are the feature representations that we extracted for each image of the LFW158 dataset. If you want to know more about this step, you can find all the details in the associated paper.