Update GitLab_MaliciousIP.yaml

excluding local IP/PrivateIP from the TI_ipEntity.
Required items, please complete
Change(s):

added exclusion for private IP for GitLab_MaliciousIP.yaml

Reason for Change(s):

sometimes we are getting private IP in TI_ipEntity that needs to be excluded

Testing Completed:

yes

Checked that the validations are passing and have addressed any issues that are present:

yes

Guidance <- remove section before submitting

Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.

Details of the code changes in your submitted PR.  Providing descriptions for pull requests ensures there is context to changes being made and greatly enhances the code review process.  Providing associated Issues that this resolves also easily connects the reason.

Change(s):

Updated syntax for XYZ.yaml

Reason for Change(s):

New schema used for XYZ.yaml
Resolves ISSUE #1234

Version updated:

Yes
Detections/Analytic Rule templates are required to have the version updated

The code should have been tested in a Microsoft Sentinel environment that does not have any custom parsers, functions or tables, so that you validate no incorrect syntax and execution functions properly.  If your submission requires a custom parser or function, it must be submitted with the PR.

Testing Completed:

Yes/No/Need Help

Note: If updating a detection, you must update the version field.

Before the submission has been made, please look at running the KQL and Yaml Validation Checks locally.
https://github.com/Azure/Azure-Sentinel#run-kql-validation-locally

Checked that the validations are passing and have addressed any issues that are present:

Yes/No/Need Help

Note: Let us know if you have tried fixing the validation error and need help.

References:

Guidance for Detection checks
General contribution guidance
PR validation troubleshooting

Changes submitter for Exclude local addresses from the result.


Our NuGet needs to have a README file

https://devblogs.microsoft.com/nuget/add-a-readme-to-your-nuget-package/

This is now implemented


Thank you for your great program! It is excellent, but today I was unable to decode secrets generated using the 512 hash algorithm widely used nowadays. I had to decode such secrets, so I modified the program, and now it supports such algorithms. I hope these contributions will be beneficial for the other users as well.

Made configurable HMAC hash functions. The program was previously hard-coded to HMAC-SHA-256, now also 386 and 512-bit SHA hashes can be specified. Added explanation on how to use these hash functions to the READEME.md file. Replaced some unsafe typecasts, made consistent defaults and usage display.

Great news! Thank you very much!

Great news! Thank you very much!

Hello Brendan,
If you will be doing further updates to the program, please add version number somewhere, since not it has no version number.
--
Maxim Masiutin

Hello Brendan,
If you will be doing further updates to the program, please add version number somewhere, since not it has no version number.
--
Maxim Masiutin


Detachable console/debugger window.

If the terminal has a detachable options, that would really add greatly to the application use.
This could help resize console and editor side by side and sometimes be really useful.  Just like chromium console. xx

Looks same as #10121


A character outside of a comment is painted over

Godot version
4.0 (alpha 13)
System information
Ubuntu 18.04
Issue description

Steps to reproduce
var test = []# test
Minimal reproduction project
No response

It's a broader issue of unwanted ligatures screwing with the highlighting.
]# becomes considered as one character, similar to := or &&. Then the entire ligature steals the comment color.
I don't know of any other instances of ligatures messing up highlighting on valid syntax atm, however, if my #63326 gets merged, -- and ++ will join the party.

CC @Paulb23 @bruvzg

I do not think we can do anything with it, ligatures are either on or off (some fonts include style sets to switch some ligatures individually, but not JetBrainsMono).

Actually, CodeEdit/TextEdit can break ligatures by submitting parts of the string as a separate spans to the TextServer:
But I'm not sure what's the best way to do it, something like will work (obviously not a proper solution, just a POC):

diff --git a/scene/gui/text_edit.cpp b/scene/gui/text_edit.cpp
index 630a3316d6..46ffa30737 100644
--- a/scene/gui/text_edit.cpp
+++ b/scene/gui/text_edit.cpp
@@ -190,7 +190,16 @@ void TextEdit::Text::invalidate_cache(int p_line, int p_column, bool p_text_chan
 		}
 	} else {
 		if (p_text_changed) {
-			text.write[p_line].data_buf->add_string(text[p_line].data, font, font_size, language);
+			const String &txt = text[p_line].data;
+			int i = txt.find("]#");
+			int e = 0;
+			while (i >= 0) {
+				text.write[p_line].data_buf->add_string(txt.substr(e, i + 1 - e), font, font_size, language);
+				e = i + 1;
+				i = txt.find("]#", i + 1);
+			}
+			text.write[p_line].data_buf->add_string(txt.substr(e, -1), font, font_size, language);
+			//text.write[p_line].data_buf->add_string(text[p_line].data, font, font_size, language);
 		}
 		if (!text[p_line].bidi_override.is_empty()) {
 			TS->shaped_text_set_bidi_override(text.write[p_line].data_buf->get_rid(), text[p_line].bidi_override);

This will require code highlighting to be integrated at the text shaping step, instead of the drawing step. And as a free bonus, it allows using different fonts for highlighting (like highlighting with a bold or italic instead of color).

In the meantime, I recommend always adding a space before inline # comments (or ideally two, like in Python's code style guidelines).

Now that #63326 has been merged, I identified 25 ligatures in the default font that can possibly cause incorrect highlighting. I believe this list is complete.
Changed up the highlighting of numbers a little to make the issue more clear:

Without ligatures, all of these look properly:

Here's the code from above, just copy it into a script:
var A = []# '#' shouldn't highlight something before it
var B = 6 <#-- Same mistake as above
~@export var C   # '@' shouldn't highlight something before it
var D = 99 <$Mario.coins   # '$' shouldn't highlight something before it
var E = &=   # Unary '&' iss highlighted like StringName as it expects a string
var F = ^=   # Same mistake as above, but with '^' and NodePath
var G = true &&&"Lmofa" == &"Lmfao"
var H = 6 <+9
var I = 6 <-9
var J = 6 <--9
var K = 6 >-9
var L = 6 ++9
var M = 6 +++9
var N = 6 --9
var O = 6 ---9
var P = 6 ~~9   # The nonnexistent binary '~' uses symbol color
var Q = 6 ~-9   # Same mistake as above
var R = 6 -~9
var S = 6 <~9
var T = 6 <~~9
var U = 6 |-(-9)
var V = 6 <<-(-9)
var W = 6 >>-(-9)
var X = false ||-6 < 9
var Y = false <!--6 < 9

This will require code highlighting to be integrated at the text shaping step, instead of the drawing step.

Experimented with this a bit. Highlighter seems to be invalidation all lines from the previous edited one to the end of the document, so moving it to the shape step directly will significantly decrease performance, so probably this will require making highlighter asynchronous to be usable.


Unable to print dates and times

This example is using the chrono trait, but its' a broader issue I can't identify. This is
With rtt, you could do this:
rprintln!("DEBUG date: {:?}", &rtc.get_datetime().unwrap());

When I try with defmt:
    |     defmt::info!("DEBUG date: {:?}", &rtc.get_datetime().unwrap());
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `Format` is not implemented for `NaiveDateTime`

Any ideas? Thank you.

This is because defmt relies on a custom Format trait, which does not use the implementations provided by the rust core traits Debug and Display.
On custom types you automatically implement the Format trait via #[derive(Format)] .
As NaiveDateTime is a type provided by chrono you can not add this impl directly to it.
A way to resolve this could be to use the NewType Pattern to implement Format or get a meaningful primitive type out of NativeDateTime like u32 etc., which is supported by defmt.
There is probably also a better way I can't think of as of now.

The easier way of doing what @Sh3Rm4n proposed, is available in the defmt::Debug2Format & defmt::Display2Format here: https://github.com/knurling-rs/defmt/blob/main/src/lib.rs#L653-L687

Thank you - I'll check those out.

This is behaving as expected, but marking as documentation improvement just in case we can do better there

I think another option is make it work the way rprintln does.

I think another option is make it work the way rprintln does.

Is it unfeasable to make it handle this (perhaps as a fallback?)

currently, yes. at least until 'unsized values' are supported in defmt (see #292).

Or is not using Debug and Format core to how this works?

no, defmt does deferred formatting so it doesn't use core::Debug or core::fmt. (if you are interested in reading how defmt does deferred formatting you can check out the design / impl docs).

Is it unfeasable to make it handle this (perhaps as a fallback?)

currently, yes. at least until 'unsized values' are supported in defmt (see #292).

Or is not using Debug and Format core to how this works?

no, defmt does deferred formatting so it doesn't use core::Debug or core::fmt. (if you are interested in reading how defmt does deferred formatting you can check out the design / impl docs).


Use java-checks-testkit to detect rule violations

This PR fixes #75.
We have two groups of processors: sonar-based and spoon-based. The difference between them is that in sonar-based processors the detection of rule violations is performed with Sonar web API, and in spoon-based processors the detection is purely done with Spoon. However, per discussion (see #75), we would like to have the help of SonarJava in the detection of rule violations, without the usage of the web API. This PR addresses this, and make all processors compatible.
On SonarJava side: It's not possible to directly retrieve the issues found by SonarJava. To do so, then, I followed what had started here, and my changes are here. With those changes we are able to access the issues found by SonarJava. Because I changed the SonarJava source code, we use java-checks-testkit through a jar added in Sonarqube-repair. Additionally, there was a bug in SonarJava when multiple files are analyzed, which is in a module other than java-checks-testkit, and it was also fixed, this is why we have a second jar (i.e., java-frontend-5.14.0-SNAPSHOT.jar) in our project.
On Sonarqube-repair side: In summary, in the method isToBeProcessed of each processor now, let's say the processor for rule X, there is a check to see if, according to SonarJava, there is a violation of the rule X in the position (considering file and line number) of the element that entered in the isToBeProcessed. I already realized that only this check is not enough to just return true in every processor we have. Because of this, I kept the other checks done with Spoon for now (we can clean unnecessary checks later, if any).
Some random points:

The logic of CompareToReturnValueProcessor (see rule description), recently implemented by @henry-lp, was completely changed; the thing is that Henry was filtering CtMethod (because we wanted compareTo methods), but SonarJava actually creates an issue in the return line (which is in fact the noncompliant line), and then the line number of what Henry was filtering didn't match the line number of what SonarJava was returning. I fixed it.
Because some processors used the Sonar web API, we had an argument named projectKey, which now was completely removed from the project.
Not only all tests are passing, but I also compared the output files and they are exactly the same as before those changes.

I would like that at least 2 collaborators review this PR before merging, since it's a major change.

This PR addresses this, and make all processors compatible.

Congrats!
Overall, it's good.
We don't want to push custom jars file in the repo. Instead, we add in the .travis.yml the lines to build it from your fork, something like
git clone --branch modified-version https://github.com/fermadeiral/sonar-java/
mvn install

do you plan to make a PR upstream on sonar-java?

We don't want to push custom jars file in the repo. Instead, we add in the .travis.yml the lines to build it from your fork, something like

Done in the last commit, please check.

Plus do you plan to make a PR upstream on sonar-java?

No.
About the change regarding retrieving the issues found by SonarJava: I believe that their API is like that because the expected usage of the tool is not how we use it. I can open a PR about it, but I don't think it will be accepted.
About the change regarding fixing the bug around multiple files being analyzed: I don't know if this bug still exists in SonarJava. Let me explain. We are using the version 5.14.0, which is from August 2019. There are several versions after that one. We don't use a more recent one because all versions after 5.14.0 have a conflict with Spoon: apparently both of them use different versions of jdt. I don't know how to fix this issue, and to be honest I didn't try to search too much because my priority is to make all processors compatible and the version we are using is not that old (for now). However, I'm going to create an issue in our repo about this after this PR is merged, because we want to be able to upgrade SonarJava here one day. Finally, I don't know if the bug still exists in SonarJava because we are not using latest versions and maybe the bug was already fixed in them.

Thanks for the explanation, I agree with the prioritization (follow-up is #89).
LGTM.
@henry-lp would you have a look?

@monperrus yes :) .
@fermadeiral , we need to install your modded sonar-java as a local dependency to run ?
This might be a bit problematic later when we run this in the CI like Jenkins. We can either :

Make double checking with sonar-java as an optional step specified through JSAP
Deploy the modded version to maven repo and import that for now (Until we maybe can merge these to the main sonar-java repo sometime in the future)

WDYT ? :)

we need to install your modded sonar-java as a local dependency to run ?

Yes, see the changes in the README.md and .travis.yml files.

This might be a bit problematic later when we run this in the CI like Jenkins (Not every Jenkins server has Maven).

How are we going to run sonarqube-repair in the first place, then?

Make double checking with sonar-java as an optional step specified through JSAP

I don't follow you. What does that mean?

Deploy the modded version to maven repo and import that for now

I believe we can do that, but I'm not sure what that changes, maybe I don't follow the root cause of the problem you are predicting.

@fermadeiral , I see. If the deployed version of Sonarqube repair contains your modded dependency then there's no problem. :)

@henry-lp are you sure? Because we can find alternatives if how the things are now will complicate our life later.

I am sure about the integration part with Repairnator , since I import this as a dependency and as long as that contains the correct packages then I have no issues :) .

Ok, then. Thank you for your feedback, @henry-lp!

I performed two more commits, but only with minor updates according to the major changes this PR contains.


Homepage

Homepage
Thinking about what it could be, I will suggest multiple versions so we can discuss about them here.
Movie catalog

@Tommass @pierr ready to merge, just waiting for @Tommass to implement the Java side.
Functionnal on the mocked API

@Tommass see this implementation as a starting point for the Java side

Many components could be pure functions, no?
The only problem is for @TranslationBehaviour which add a method inside the this

But it's great!

I let @Tommass merge it

@pierr you were right, I fixed it, and use focus-core/translation/translate

@Bernardstanislas Cool!

@Bernardstanislas  I was wondering about translationBehaviour if we could pass the function as props instead of using this

T

But this is more a FocusComponents related issue

@pierr that could work but you would need to check whether the component provided to the behaviour is a function or a class, so that we keep the legacy compatibility by putting a this.i18n in classes and a translate prop in pure components

@Bernardstanislas Why wouldn't it work for both class and function, only focus-components uses this behaviour. Its just a rename this.i18n to this.props.i18n for classes

@pierr yes but it would imply that all project might change their this.i18n to this.props.i18n

ctrl+shift+r this.i18n replace this.props.i18n


Install fails on Python 3.6 with 'br' is invalid keyword argument

Attempting to install latest markdown on Python 3.6.0b1 fails with this error:
$ python -m pip install -t foo markdown
Collecting markdown
  Using cached Markdown-2.6.7.zip
Installing collected packages: markdown
  Running setup.py install for markdown ... error
    Complete output from command /Library/Frameworks/Python.framework/Versions/3.6/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-kwhawl52-record/install-record.txt --single-version-externally-managed --compile --home=/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpowua21z4:
    running install
    running build
    running build_py
    creating build
    creating build/lib
    creating build/lib/markdown
    copying markdown/__init__.py -> build/lib/markdown
    copying markdown/__main__.py -> build/lib/markdown
    copying markdown/__version__.py -> build/lib/markdown
    copying markdown/blockparser.py -> build/lib/markdown
    copying markdown/blockprocessors.py -> build/lib/markdown
    copying markdown/inlinepatterns.py -> build/lib/markdown
    copying markdown/odict.py -> build/lib/markdown
    copying markdown/postprocessors.py -> build/lib/markdown
    copying markdown/preprocessors.py -> build/lib/markdown
    copying markdown/serializers.py -> build/lib/markdown
    copying markdown/treeprocessors.py -> build/lib/markdown
    copying markdown/util.py -> build/lib/markdown
    creating build/lib/markdown/extensions
    copying markdown/extensions/__init__.py -> build/lib/markdown/extensions
    copying markdown/extensions/abbr.py -> build/lib/markdown/extensions
    copying markdown/extensions/admonition.py -> build/lib/markdown/extensions
    copying markdown/extensions/attr_list.py -> build/lib/markdown/extensions
    copying markdown/extensions/codehilite.py -> build/lib/markdown/extensions
    copying markdown/extensions/def_list.py -> build/lib/markdown/extensions
    copying markdown/extensions/extra.py -> build/lib/markdown/extensions
    copying markdown/extensions/fenced_code.py -> build/lib/markdown/extensions
    copying markdown/extensions/footnotes.py -> build/lib/markdown/extensions
    copying markdown/extensions/headerid.py -> build/lib/markdown/extensions
    copying markdown/extensions/meta.py -> build/lib/markdown/extensions
    copying markdown/extensions/nl2br.py -> build/lib/markdown/extensions
    copying markdown/extensions/sane_lists.py -> build/lib/markdown/extensions
    copying markdown/extensions/smart_strong.py -> build/lib/markdown/extensions
    copying markdown/extensions/smarty.py -> build/lib/markdown/extensions
    copying markdown/extensions/tables.py -> build/lib/markdown/extensions
    copying markdown/extensions/toc.py -> build/lib/markdown/extensions
    copying markdown/extensions/wikilinks.py -> build/lib/markdown/extensions
    running build_scripts
    creating build/scripts-3.6
    copying and adjusting bin/markdown_py -> build/scripts-3.6
    changing mode of build/scripts-3.6/markdown_py from 644 to 755
    running build_docs
    creating build/docs
    copying docs/authors.txt -> build/docs
    Converting docs/authors.txt -> build/docs/authors.html
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/setup.py", line 270, in <module>
        'Topic :: Text Processing :: Markup :: HTML'
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/install.py", line 545, in run
        self.run_command('build')
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/setup.py", line 184, in run
        out = template % self._get_context(src, outfile)
      File "/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/setup.py", line 116, in _get_context
        c['body'] = self.md.convert(src)
      File "build/lib/markdown/__init__.py", line 375, in convert
        newRoot = treeprocessor.run(root)
      File "build/lib/markdown/treeprocessors.py", line 361, in run
        brs = root.getiterator('br')
    TypeError: 'br' is an invalid keyword argument for this function

    ----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/3.6/bin/python3 -u -c "import setuptools, tokenize;__file__='/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-kwhawl52-record/install-record.txt --single-version-externally-managed --compile --home=/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpowua21z4" failed with error code 1 in /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-build-3fubfbqm/markdown/

This is a bug in Python 3.6 (already fixed upstream). A workaround is also in the latest Python-Markdown Git. See #499 and #500 for details.

Excellent. Thanks!


Cor transaction code 21 validation error

Closes #457

Codecov Report

Merging #459 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #459   +/-   ##
=======================================
  Coverage   86.62%   86.62%
=======================================
  Files         112      112
  Lines        6983     6983
=======================================
  Hits         6049     6049
  Misses        658      658
  Partials      276      276


getrandom build error with "target is not supported" under some circumstances

Crate Name: getrandom https://docs.rs/getrandom

NOTE: When using risc0-zkvm version 0.14 or later, guests may use getrandom and rand crates without issue. This issue is only relevant to you if you see the error described below. If not, getrandom is working.

Description of Error
getrandom is the standard crate for getting random bytes from the OS. RISC Zero provides a getrandom handler, but you may get a build error if the custom feature is not enabled.
error: target is not supported, for more information see: https://docs.rs/getrandom/#unsupported-targets getrandom
   --> ./src/lib.rs:286:9
    |
286 | /         compile_error!("target is not supported, for more information see: \
287 | |                         https://docs.rs/getrandom/#unsupported-targets");
    | |________________________________________________________________________^

Workarounds
With risc0 v0.14 or later, adding getrandom as a direct dependency in the guest Cargo.toml and enabling the custom feature should address this issue.
NOTE: If you depend risc0_zkvm, you likely do not directly depend on getrandom, it is likely you do not need to add this line to your config as risc0_zkvm enables the custom feature. Only add this line if you see the error mentioned above.
[dependencies]
getrandom = { version = "0.2", features = ["custom"] }

Long term solution(s)
Permanently handling this issue in all cases is best accomplished by getting support added directly to getrandom. Once the RISC Zero zkVM has a stable target triple for Rust, a PR should be submitted to getrandom.

@nategraf thanks for posting this issue and your workaround! I encountered the same error, but it was due to another dependency pulling in getrandom 0.1.16. Since v1 doesn't support the "custom" feature, I used "dummy" instead, which allowed me to compile successfully.
[dependencies]
getrandom = { version = "0.1", features = ["dummy"] }


Refactor docker/* targets in Makefile

Fixes #647

@maxfisher-g please review

Hi @gliptak, thanks for your continued work on this issue!
Looking at the linked issue and example Makefile, I think the 'top level' recipes should be verbs like build, sync, push etc. We probably don't need the docker prefix for everything.
Additionally, can we make it so that

make build will build the prod images and sandboxes
make push will push the prod images and sandboxes (after building, as currently)
make sync will sync all sandboxes (after building, as currently)

Thanks very much!

updated. please let me know

@gliptak nice, thank you!
One more thing, if you look at the original comment in #647, the format of the recipes is like build/image/XXX. This means that build/image will build all the non-sandbox images and build/sandbox will build all the sandboxes, etc.
Currently, the recipes in our Makefile look like e.g. build/XXX_image, sync/XXX_sandbox. Are you able to change the format from this to e.g. build/image/XXX, sync/sandbox/XXX?
This would be much appreciated :)

@maxfisher-g thank you. updated

Thanks for the update @gliptak!
Could you please add an entry for sync to the output of make help, and then I think we can merge this PR :)
The group targets e.g. make build/image , make sync/sandbox don't work currently, but this can be fixed in another PR.

@maxfisher-g updated sync

@gliptak thank you! I'll merge ;)


Safari support?

You already support Chrome and FF, why not Safari? Packaging Web Extensions for Safari looks pretty trivial to me.

While Chrome has an entry fee of $5 and Firefox has no entry fee to publish in their stores, in order to officially distribute an extension for Safari you must be part of the Apple Developer Program which costs $100 per year.
As I am not willing to pay that much to publish a single extension through official means offered by Apple, I will keep targeting only the 3 stores I originally had interest in publishing (Chrome, Firefox and Edge), unless there is enough demand to get the extension to another store where publishing doesn't cost me that much money.

You know that there are other ways to distribute software than the App Store?

unless there is enough demand to get the extension to another store where publishing doesn't cost me that much money.

My comment was clear on the "official means", if I put the effort into packing the extension for Safari it's to distribute it through these means. Apple's officially documented way of deploying web extensions is through the App Store.
Not to mention the requirement of Xcode to pack the extension which is also a blocker for me and makes a Safari build from me even less likely.

Would you accept a PR to add an Xcode project for the Safari extension? Then, interested people (like me) would be able to build it ourselves. No changes are required to the current extension code (beyond the addition of new files in the Xcode project directory).

I have mixed feelings about it. Having code that is bound to be unmaintained by me and which can go out of date at any given time without appropriated care doesn't align with my quality standards - especially with large scale changes bound to happen in 2022 (Manifest v3 migration). The lack of testing on the Safari build also throws me off a bit.
To me, anything in the repository is considered "officially supported" which simply wouldn't be the case here (and I myself have PronounDB related projects that are not part of the repo because not "official" despite being by the original author, me).
I'm not against having community builds (provided license compliance, which isn't too hard with a permissive license), and I think you can probably spin something with git submodules to review the impact of upstream changes on Safari build (which is probably one of the "best" solutions).


[regression] Crashlytics always succeeds while it shouldn't

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I have read the Contribution Guidelines

Issue Description
Crashlytics action succeed the upload, but cannot add testers due to inactive app id. In 2.36.0 version this was totally fine and crashlytics action were throwing an error, that could be handled by script. I was using this to launch the app in simulator via scan, so app could be activated on crashlytics:
    tries = 3
    begin
      crashlytics(
		ipa_path: './App.ipa',
		groups: "test-group",
		notifications: false
	  )
    rescue Exception => e
      tries -= 1
      if tries > 0
        scan(
		  workspace: "App.xcworkspace",
		  scheme: "Release",
		  clean: false
		)
        retry
      else
        raise e
      end
    end

This is what it looked like on 2.36.0:
[17:32:25]: -------------------------
[17:32:25]: --- Step: crashlytics ---
[17:32:25]: -------------------------
[17:32:26]: Uploading the build to Crashlytics Beta. Time for some ☕.
[17:33:19]: 2017-06-01 17:32:26.425 submit Crashlytics: Crashlytics.framework/submit 1.3.5 (18)
2017-06-01 17:33:19.144 submit Crashlytics: Unable to add testers: Project some-inactive-appid is inactive
2017-06-01 17:33:19.144 submit Crashlytics: Failed to add testers/groups to the release
[17:33:20]: Sending crash report...
[17:33:20]: The stacktrace is sanitized so no personal information is sent.
[17:33:20]: To see what we are sending, look here: /Users/travis/.fastlane/latest_crash.json
[17:33:20]: Learn more at https://github.com/fastlane/fastlane#crash-reporting
[17:33:20]: You can disable crash reporting by adding `opt_out_crash_reporting` at the top of your Fastfile
[17:33:20]: ------------------
[17:33:20]: --- Step: scan ---
[17:33:20]: ------------------

This is what it looks like now:
[09:48:59]: -------------------------
[09:48:59]: --- Step: crashlytics ---
[09:48:59]: -------------------------
[09:49:00]: Uploading the build to Crashlytics Beta. Time for some ☕.
[09:50:51]: Shell command exited with exit status 1 instead of 0.
[09:50:51]: 2017-06-07 09:49:00.800 submit Crashlytics: Crashlytics.framework/submit 1.3.5 (18)
2017-06-07 09:50:51.648 submit Crashlytics: Unable to add testers: Project some-inactive-appid is inactive
2017-06-07 09:50:51.648 submit Crashlytics: Failed to add testers/groups to the release
[09:50:51]: Build successfully uploaded to Crashlytics Beta 🌷
[09:50:51]: Visit https://fabric.io/_/beta to add release notes and notify testers.
[09:50:51]: Cruising back to lane 'ios beta' 🚘
[09:50:51]: Cruising back to lane 'ios auto' 🚘

This leads to a successful build, but no binary on fabric. How can I automate crashlytics activation now?
I believe, this is a regression from #9313
Complete output when running fastlane, including the stack trace and command used
Environment

Wow, interesting find!  @giginet, do you think this behavior could have been triggered by the changes in #9313?

Any ETA on fixing this? Really annoying, had to roll back to 2.36.0

@mpirri @KrauseFx at least, can I use raise_shell_error: true in crashlytics action?

Looking into this now, @Saga-Zero - sorry for the delay!

@mpirri when may I expect the fix to be released?


[BEAM-5663] Add issue links to failing Python 3.6 tests

This is is part of a series of PRs with goal to make Apache Beam PY3 compatible. The proposal with the outlined approach has been documented here: https://s.apache.org/beam-python-3.
This PR follows up on #7988 to add issue links to all skipped tests.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

R: @tvalentyn @aaltay


win 10 edits

There was an error while validating job c317db6f-81be-48ec-a20a-e91588297d41 Message:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
Details:Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase+ContextCommandException:
Result status: Failed
There was a problem accessing Github repository 'https://github.com/Microsoft/cordova-docs.git'.
---> System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Error clonning repository ---> LibGit2Sharp.NotFoundException: Branch 341 not found. Available branches: origin/master
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetBranch(Repository repo, String name)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<>c__DisplayClass34.b__29(KeyValuePair2 branch)    at System.Collections.Generic.List1.ForEach(Action1 action)    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.<>c__DisplayClass30.<CloneRepository>b__28()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()    at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func1 func)
at Microsoft.Practices.EnterpriseLibrary.TransientFaultHandling.RetryPolicy.ExecuteAction(Action action)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)    --- End of inner exception stack trace ---    at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository(String url, IDictionary2 branches)
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.CloneRepository()
at Microsoft.DPS.Services.LibGit2SharpSourceControlProvider.GetVersionAsync(String path)
at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)    --- End of inner exception stack trace ---    --- End of inner exception stack trace ---    at Microsoft.LightweightJobEngine.Core.Language.ContextCommandBase.ThrowIfErrorsLogged()    at Microsoft.DPS.Commands.IdentifyDocumentsToPublish.CheckExternalServicesCommand.Do(IDictionary2 context)
at Microsoft.LightweightJobEngine.Core.Language.JobWorkflow1.RunCommands(Guid jobId, IDictionary2 context, CancellationTokenSource cancelSignal)


PS5 Support

I downloaded the Unity source and was able to add support to the python/CMake system and successfully build a PS5 compatible phonon runtime library, however I this only appears to be the Unity part, the underlying library seems to be supplied as binary only.
Am I missing anything? If the source is available, I would like to use it to make a PS5 dynamic library. If the source isn't available have I reached a deadend? Or does Valve have a PS5 library that they can provide me if I prove my PS5 developer status?

Steam Audio supports Windows (32 bit and 64 bit), Linux (32 bit and 64 bit), macOS, and Android (armv7, arm64, x86, x64) platforms.

Source code is available for integration layers (C-API, FMOD, Unity, Unreal), not the underlying (audio implementation) library. It is likely that a PS5 version does not exist.

@Scraft - Sent you a note. We have not yet released the C-API source code. You will need to port that to PS5 as well in addition to integration layers for a full PS5 port.


Fix scoped storage issue

Description
Fixes #4585
What changes did you make and why?
Added permission android:requestLegacyExternalStorage = "true" and also updated compile SDK version to 29 because it is needed for this.
Tests performed
Tested betaDebug on Google Pixel 4 with API level 30

@neslihanturan Sorry for the late reply, it seems like the permission is still not granted for some version of OS, I tested on API 29

So merging this one and keeping the issue open :)


Different hydration options for different components

Hello! I experience an issue with the hydration options.
Let's say I have multiple components on my page that are client-side hydrated.
Then it seems like the settings for the first component is what affects the loading.
How to reproduce:
Clone the template and copy the Clock component twice, with identical copies called something like Alock and Block.
Import them at the top
import Clock from '../../components/Clock.svelte';
import Block from '../../components/Block.svelte';
import Alock from '../../components/Alock.svelte';

and use all three in the place where the one currently is used in Home.svelte
<Clock hydrate-client={{}} hydrate-options={{ loading: 'eager', preload: true }} />
<Block hydrate-client={{}} hydrate-options={{ loading: 'lazy' }} />
<Alock hydrate-client={{}} hydrate-options={{ loading: 'lazy' }} />

With this setting, all three components will be eagerly loaded and preloaded in the top of the HTML file.
I would expect only the first component to have this behaviour and the rest of them to be lazy-loaded.

@myrsmedenstorytel Thanks for the bug report. Did a major refactor of how we mount Svelte components to make it all much more test able. Ironically it was the absolutely last file that I did that had the bug. Push a fix shortly #81


added a plist file!!

@paetling this actually took me a while to make/debug.  Can it just live in the dusty repo here?

I think it is fine to stay if we create a folder like basic_setup or example_setup.  Then added in a readme in that file or what these things were.   Because this is a very gc specific thing (we want to run it like this) but it could be useful to people down the road

I'm unsure exactly how gc-specific it is... travis seems to think that making a service to run dustyd automatically is what everyone would want.  I'll leave the PR open for now

ok


Add resolve vanityurl and option to retrieve stats

Added a function to resolve vanity url if we don't have the steam id.
Added an option to User/Stats.php GetUserStatsForGame so we can retrieve not only achievements but stats too! See: CSGO or DOTA2.

I will merge this next time I am at a comp. Need to make sure it works for L4 and L5 and that it has docs with examples.

Thanks!

NP.  I'll tag releases once I get working examples and docs.


[bug] Missing support for private image repositories in Backup and Metrics

Documentation

[x] I acknowledge that I have read the relevant documentation.

Describe the bug
Installing mariadb-operator using private docker repositories requires the definition of registries and imagePullSecrets.
This generally works for the deployment of mariadb-operator via the values.yaml overrides:
---
image:
  <private registry>/mariadb-operator/mariadb-operator
imagePullSecrets:
  - name: <registry secret>
webhook:
  image:
    <private registry>/mariadb-operator/mariadb-operator
  cert:
    certManager:
      enabled: true
  imagePullSecrets:
    - name: <registry secret>
metrics:
  enabled: true

Unfortunately for the Backup Job as well as for the Metrics Exporter the image information is taken from a predefined CM:
apiVersion: v1
kind: ConfigMap
metadata:
  name: mariadb-operator-images
  namespace: mariadb-operator
data:
  MARIADB_OPERATOR_IMAGE: ghcr.io/mariadb-operator/mariadb-operator:v0.0.24
  RELATED_IMAGE_EXPORTER: prom/mysqld-exporter:v0.15.1
  RELATED_IMAGE_MARIADB: mariadb:11.2.2

Additionally for the JobTemplate used by the Backup CronJob an imagePullSecret cannot be added
Expected behaviour
The configuration options in the mariadb-operator to override images and imagePullSecrets should be taken into account in the jobTemplate for the Backup CronJob.
Additionally for "metrics" an "image" and "imagePullSecrets" config options could be added on operator level
Steps to reproduce the bug

deploy mariadb-operator including private registry and imagePullSecrets
create mariaDB and Backup resources
Check the create CronJob

Hey there @addreas ! Thanks for reporting this
You are right, neither the Backup nor the Restore specs support providing a custom image at the moment, we are using the same as the MariaDB resource they point to:

https://github.com/mariadb-operator/mariadb-operator/blob/main/docs/API_REFERENCE.md#backupspec
https://github.com/mariadb-operator/mariadb-operator/blob/main/docs/API_REFERENCE.md#restorespec

We should include this fields here and default to the MariaDB images when not available. Contributions are welcome!

Additionally for "metrics" an "image" and "imagePullSecrets" config options could be added on operator level

spec.metrics does have an exporter field which supports image and imagePullSecrets.  Please take a look at the API reference:
https://github.com/mariadb-operator/mariadb-operator/blob/main/docs/API_REFERENCE.md#exporter


adding in ability to provide per repo hints

After trying a few approaches, this simple one still seems potentially relevant (you can put specific instructions for goose in a repo).

@baxen yeah - this was nonsense test code, removed it for now so we can merge it in.


Please provide an AppImage for Linux

Providing an AppImage would have, among others, these advantages:

Applications packaged as an AppImage can run on many distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
One app = one file = super simple for users: just download one AppImage file, make it executable, and run
No unpacking or installation necessary
No root needed
No system libraries changed
Works out of the box, no installation of runtimes needed
Optional desktop integration with appimaged
Optional binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
Can optionally GPG2-sign your AppImages (inside the file)
Works on Live ISOs
Can use the same AppImages when dual-booting multiple distributions
Can be listed in the AppImageHub central directory of available AppImages
Can double as a self-extracting compressed archive with the --appimage-extract parameter

Here is an overview of projects that are already distributing upstream-provided, official AppImages.
The https://github.com/probonopd/linuxdeployqt tool makes it straightforward to bundle an application as an AppImage. See https://github.com/probonopd/linuxdeployqt#using-linuxdeployqt-with-travis-ci for an example.
If you have questions, AppImage developers are on #AppImage on irc.freenode.net.

Quite intresting, this will make Nitroshare easier to install.

To be precise, this will remove the need for installation altogether. Just download and run, like a "portable application" on Windows or most applications on the Mac.

I will attempt to get an AppImage open when I get home. Note that this might take a few days.

Work is underway to get NitroShare working with linuxdeployqt.
CI builds are here: https://ci.quickmediasolutions.com/job/nitroshare-appimage/

Looks like there is an issue with qmake. Can you use Qt from sudo add-apt-repository ppa:beineri/opt-qt593-trusty -y?

I think I just need to point linuxdeployqt to the correct location of qmake within the container - I'm building the application in a Docker container to avoid the need to install a bunch of packages on the Jenkins slave.

Possibly...

Well, the first AppImage built without errors but it doesn't work correctly, so more debugging is needed. Running the AppImage with --help displays usage information (as expected) but running with no arguments does nothing, leading me to believe that either the plugins aren't loading or something is preventing the tray icon from showing up.
I'll try extracting the squashfs filesystem to see where everything ended up.

It also appears that linuxdeployqt added itself to the AppImage 🤔

$ Downloads/NitroShare-x86_64.AppImage --appimage-extract
~$ squashfs-root/AppRun
(... just hangs)

Inspecting it with strace shows where it is hanging; it is trying to do something with Qt Accessibility (but the Qt Accessibility plugin is not bundled).
sendmsg(12, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\1\1\232\0\0\0\6\0\0\0\177\0\0\0\1\1o\0\25\0\0\0/org/fre"..., 144}, {"\225\0\0\0type='signal',sender='org.a1"..., 154}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 298
sendmsg(12, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\0010\0\0\0\7\0\0\0\202\0\0\0\1\1o\0\37\0\0\0/org/a11"..., 152}, {"\5\0\0\0:1.11\0\0\0\37\0\0\0/org/a11y/atspi/"..., 48}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 200
poll([{fd=12, events=POLLIN}], 1, 25000) = 1 ([{fd=12, revents=POLLIN}])
recvmsg(12, {msg_name(0)=NULL, msg_iov(1)=[{"l\1\0\0010\0\0\0gE\0\0\215\0\0\0\1\1o\0\37\0\0\0/org/a11"..., 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 328
recvmsg(12, 0x7ffc4bf29df0, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
write(5, "\1\0\0\0\0\0\0\0", 8)         = 8
read(6, "\1\22\0\1\0\0\0\0", 8)         = 8
recvmsg(7, 0x7ffc4bf2a740, 0)           = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}], 6, 0) = 1 ([{fd=5, revents=POLLIN}])
read(5, "\7\0\0\0\0\0\0\0", 16)         = 8
write(5, "\1\0\0\0\0\0\0\0", 8)         = 8
write(5, "\1\0\0\0\0\0\0\0", 8)         = 8
recvmsg(7, 0x7ffc4bf2a740, 0)           = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}], 6, 0) = 1 ([{fd=5, revents=POLLIN}])
read(5, "\2\0\0\0\0\0\0\0", 16)         = 8
stat("/home/me/squashfs-root/usr/plugins/accessible/.", 0x7ffc4bf29db0) = -1 ENOENT (No such file or directory)
stat("/home/me/squashfs-root/usr/bin/accessible/.", 0x7ffc4bf29db0) = -1 ENOENT (No such file or directory)
recvmsg(7, 0x7ffc4bf2a740, 0)           = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=5, events=POLLIN}, {fd=6, events=POLLIN}, {fd=7, events=POLLIN}, {fd=8, events=POLLIN}, {fd=11, events=POLLIN}, {fd=12, events=POLLIN}], 6, 0) = 0 (Timeout)
read(5, 0x7ffc4bf2a8f0, 16)             = -1 EAGAIN (Resource temporarily unavailable)
recvmsg(7, 0x7ffc4bf2a740, 0)           = -1 EAGAIN (Resource temporarily unavailable)

Are you aware of some accessibility functionality in this application?

The AppImage building is currently disabled - why? Is there anything wrong I could help with?

Hope it will be restored because no binaries are available for Debian Buster...

Any progress on this?


Benchmark uncertainty methods

already benched uncertainties which is used by point
also test quantities UncertaintyQuantity
new uncertainty wrapper using ALGOPY

Uncertainty benchmarks for the following:

Uncertainties
AlgoPy
Numdifftools
Statsmodels.tools.numdiff
for simple and complex, multi observation cases and for standard deviation only and covariance.

Outcome

statsmodels.tools.numdiff is fastest, but doesn't work for multiple observations.
UncertaintyWrapper uses same algorithm as statsmodels.tools.numdiff but adds extra dimension for extra observations which for small array size imposes small load, which relative to Jac * Cov * Jac^T is small and also small as array size grows large.
other methods either:

required rewriting code,
couldn't be used as wrapper for C/C++ extensions or
were too slow.


Deserialization of discord template into DiscordTemplate struct fails

Deserializing a discord template into the DiscordTemplate struct doesn't work as it's using the standard Guild struct for the serialized_source_guild field. See https://github.com/bwmarrin/discordgo/blob/a90485df0cd41873bc381e79f8fb6bb99c05f48b/structs.go#L1103
This fails because all IDs in the discord template data are integers instead of strings.

Since this is a bug with the Discord API (snowflakes must always be returned as strings in the HTTP API), I have created an issue on their side: https://github.com/discord/discord-api-docs/issues/6609
In the meantime, discordgo may want to consider using json.Number as the type for snowflake fields, since json.Unmarshal is capable of decoding both numbers and strings into fields of this type. Other fixes may resut in runtime errors if/when Discord fixes the return type in the API.

This has always been the case for templates, so I honestly don't think it's a bug. Changing it now would be a breaking change in the Discord API which won't happen until the next major version

This has always been the case for templates, so I honestly don't think it's a bug. The snowflakes in guild templates are just serial numbers starting at 0. Changing it now would be a breaking change in the Discord API which won't happen until the next major version. Let's wait for a response from Discord staff on the issue tho

I agree it would be a breaking change, but it's still a bug, since the field is snowflake:

Because Snowflake IDs are up to 64 bits in size (e.g. a uint64), they are always returned as strings in the HTTP API to prevent integer overflows in some languages

Returning an int breaks the invariant that snowflake is always a string.
I don't know what the policy on breaking changes is in this project, but I'll have a look into changing snowflake fields here to json.Number.

I don't think we need to update all snowflake fields to support passing in numbers. Because this behavior should only occur in guild templates.


Pangolin: Update URL

Name
Pangolin
Description
Recently added Gnostos Safe support on the official interface with this we want to update the IFPS URL for https://app.pangolin.exchange/
Thank you Gnosis Safe Team for the great job.
Type

Update

App URL
https://app.pangolin.exchange/
Issue
https://github.com/gnosis/safe-apps-list/issues/102

hey, the app has been updated :)


cloud_firestore IllegalStateException: Task is already complete

Steps to Reproduce

Create or open a new Flutter app
Add the current version of cloud_firestore (0.8.0) to dependencies.
Add a transaction call to the app (for example, the one specified in the official Cloud Firestore + Flutter codelab)
flutter run this in Android emulator or device
Initiate the transaction many times in quick succession (in the codelab example, tap on a name many times in a row)

This should fail with a PlatformException (or better, a TimeoutException, see https://github.com/flutter/flutter/issues/22440). Instead, it fails with java.lang.IllegalStateException: Task is already complete.
This bug doesn't manifest on iOS, as far as I can tell.
Logs
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): Failed to handle method call result
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): java.lang.IllegalStateException: Task is already complete
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.google.android.gms.common.internal.Preconditions.checkState(Unknown Source:8)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.google.android.gms.tasks.zzu.zzdr(Unknown Source:6)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.google.android.gms.tasks.zzu.setResult(Unknown Source:3)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.google.android.gms.tasks.TaskCompletionSource.setResult(Unknown Source:2)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at io.flutter.plugins.firebase.cloudfirestore.CloudFirestorePlugin$3$1.success(CloudFirestorePlugin.java:287)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at io.flutter.plugin.common.MethodChannel$IncomingResultHandler.reply(MethodChannel.java:169)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at io.flutter.view.FlutterNativeView.handlePlatformMessageResponse(FlutterNativeView.java:199)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at android.os.MessageQueue.nativePollOnce(Native Method)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at android.os.MessageQueue.next(MessageQueue.java:325)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at android.os.Looper.loop(Looper.java:142)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at android.app.ActivityThread.main(ActivityThread.java:6494)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at java.lang.reflect.Method.invoke(Native Method)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
E/MethodChannel#plugins.flutter.io/cloud_firestore( 9569): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

cc @kroikie @bparrishMines

had the exact same issue, not use traction to bypass it for now

I am also getting this issue right now... Hoping for the fix...
E/MethodChannel#plugins.flutter.io/cloud_firestore(22983): Failed to handle method call result
E/MethodChannel#plugins.flutter.io/cloud_firestore(22983): java.lang.IllegalStateException: Task is already complete

I have made a temporary fix.. Open the cloud_firestore package manually and go to CloudFirestorePlugin.java
Replace this code
result.success(transactionResult);
With
if (!transactionTCSTask.isComplete()) { // fix for error (java.lang.IllegalStateException: Task is already complete)
  result.success(transactionResult);
}

I hope this helps.

I were experiencing the same issue, but it's actually misleading error message
I had following code
dbRef.runTransaction((trans) async {
      final or = await trans.get(selfRef);
      final Map<String, dynamic> aa = or.data['aa'] ?? {};
      aa[id] = 1;
      await trans.update(selfRef, {'aa': aa});
});

It's not obvious, but or.data['aa'] is subtype of Map<dynamic,dynamic> so code inside failed
After I fixed that, following code worked without issue
dbRef.runTransaction((trans) async {
      final or = await trans.get(selfRef);
      final Map aa = or.data['aa'] ?? {};
      aa[id] = 1;
      await trans.update(selfRef, {'aa': aa});
});

@filiph
This issue has been moved to https://github.com/FirebaseExtended/flutterfire/issues/637. Any further collaboration will be done there.


Support Wikilink Syntax

I would like to use Nuxt Content to generate HTML form Obsidian files (see https://www.obsidian.md). Obsidian uses the Wikilink syntax to reference files.
e.g.
[[anotherfile]] references the file "anotherfile.md" in the current directory.
This syntax is (as far as I could see) not supported by nuxt content. I could do some preprocessing to convert that syntax to the standard markdown syntax. But that would not work if I use the content-directory directly as the root folder for the Obsidian vault.
Therefore it would be great if you could provide a remark plugin and a configuration example that handles that.

I love the idea, but in order for the links to work we need to know the route associated by it, for this template we know that one page = one document but that may not be the case for all websites using Nuxt Content.
Have you tried https://github.com/landakram/remark-wiki-link ?
See https://content.nuxtjs.org/api/configuration#markdown

See https://content.nuxtjs.org/api/configuration#markdown

@Atinux side-note: I noticed that https://content.nuxtjs.org/api/configuration#markdown does a 302 redirect to https://content.nuxtjs.org/api/v1/getting-started/configuration#markdown with a "Page not found" message.

Should be fixed for the redirection thanks @sig9

@henriette-einstein
For Obsidian wiki-links with Absolute paths, you should follow these steps

Use remark-wikilinks plugin as suggested by @Atinux, or create a custom remark plugin that visits text node in tree and using simple regex, grab the href and alt text of the link, and create a new html type node.
Then, change the tagName of such links to md-a or whatever, and create that component in components/content folder.
Inside that, accept props or use fall-back attributes, and using those, fetch the requisite article by transforming the string of href accordingly.

For example, if you are using Obsidian's Vault folder as source folder, it might have multiple sub-folders. Yet the absolute wiki-links inside the Obsidian Vault will be just name of the file, without any path info (unless a file with similar name exisit in some other sub-folder)
So, inside the nuxt app, when you use the obtained href to fetch the article, how will the Content Module know where to find it relative to the source folder?
So, for that, you can have simple logic that, if the href contains /, that article does not have a unique name, so Obsidian must have added entire path info as the href. And, if the href doesn't contain /, then, it's a unique article.
So, while fetching the article, inside the where query, you can use either path or slug respectively to match it with href and fetch the article.

@ManasMadrecha
Thanks a lot for that information. I tried to implement a custom remark plugin as you described and configured it in Nuxt-config. The plugin is super-simple:
import { visit } from 'unist-util-visit'

// The RegEx to match WikiLinks
const wikiRegex = /\[\[.*?\]\]/g

export default function remarkWikilink () {
  function transformer (ast) {
    visit(ast, 'text', node => {
      if (wikiRegex.test(node.value)) {
        const newValue = node.value.replace(wikiRegex, match => {
          return (
           '<w-link to="">' +
            match.substring(1, match.length - 1) +
            '</w-link>'
          )
        })

        Object.assign(node, { type: 'html', value: newValue })
      }
    })
  }

  return transformer
}

The returned value is just a test here. However, the plugin is never invoked. Instead
# Wikilink Test

[[Link to Page|Link Text]]

is rendered as
Link to Page|Link Text

If I use a different REGEX and a different input text, I can see that the plugin is involved. The plugin configuration therefore seems to work correctly.
It looks like double square brackets are somehow treated before they arrive in the plugin.
I will try to fiddle with the remark-wikilinks plugin, even though I have not seen a possibility to change the resulting tag-name in this plugin.

@henriette-einstein Unless some other plugin is also installed that treats the [[ as start of link, it will remain as text only. So, some other plugin must be treating it first, before your plugin. Try your custom plugin, by removing remark-wikilink plugin, or even remark-mdc plugin (enabled by default).
Also, did you register it correctly inside nuxt.config.js, i.e., inside remark and not rehype?
Then, the [[... will just be normal text node. And since you are visiting text node inside your plugin, it will work fine.

You can change the tagName of links even with remark-wikilink plugin, by having a custom rehype plugin. I think that plugin adds some classes like internal to the a links, so inside your own rehype plugin, you can visit such element with tagName as a and with node.properties.className containing internal, and then inside the visitor function, just change the node.TagName to md-a.

@ManasMadrecha
Thanks for your support! I have a minimal nuxt.config.js:
export default defineNuxtConfig({
  modules: ['@nuxt/content'],
  content: {
    markdown: {
      remarkPlugins: ['remark-wikilink']
    }
  }

})

I have not configured any other plugin, therefore the behavior must be included in Nuxt-Standard.
I have already found another solution using a Nitro plugin.
Nitro-Plugin
Create a file wikilinkPlugin.ts in the directory server/plugins. My initial version is
// The RegEx to match WikiLinks
const wikiREGEX = /\[\[.*?\]\]/g
// The Regex for the file part of a Wikilink
const fileREGEX = /(?<=\[\[).*?(?=(\]|\|))/;
// The Regex for the optional title part of a Wikilink
const titleREGEX = /(?<=\|).*(?=]])/;

function transformLinks (text: string): string {
    let theLinks = text.match(wikiREGEX);
    let linksFound = []

    if (theLinks) {
        for (var theLink of theLinks) {
            console.log(theLink)
            let theLinkRef = theLink.match(fileREGEX);
            if (theLinkRef) {
                let theTitle = theLink.match(titleREGEX);
                let newLink = `<m-a href="${theLinkRef[0]}">`+ theTitle?theTitle[0]:theLinkRef[0] + '</m-a>'
                linksFound.push(
                    {
                        'oldText': theLink,
                        'newText': newLink
                    }
                )
            }
        }
    }
    for(var i of linksFound) {
        text = text.replace(i.oldText,i.newText)

    }
    return text;
}

export default defineNitroPlugin((nitroApp) => {
    nitroApp.hooks.hook('content:file:beforeParse', (file) => {
      if (file._id.endsWith('.md')) {
        file.body = transformLinks(file.body)
      }
    })
})

and configure the plugin in 'nuxt.config.js`.
import { defineNuxtConfig } from 'nuxt'

export default defineNuxtConfig({
  modules: ['@nuxt/content'],
  nitro: {
    plugins: ['~/server/plugins/wikilinkPlugin.ts']
  }
})

The code is still incomplete. Local links and transfusions are not handled to name only the problems I can force by now. But it may be a start.
For some reason I don't understand, the final HTML will contain additional whitespace I did not produce. This may be a more complicated issue.
<m-a href="Text"> Text </m-a>

instead of
<m-a href="Text">Text</m-a>

@henriette-einstein OMG, you have complicated it so much 😂
Well, the issue lies with mdc.
With it enabled by default, this:
[[page]] some text

become this:
page some text

So, previously your remark plugin was not working, because you were visiting text node. While because of mdc, in the ast, the page no more remains inside a text node; each [ becomes a span. That's the way mdc works.
With it diabled, now, your remark plugin will work, and you don't need nitro-plugin.

Note: If you don't want to disable mdc, you will have to visit span and not text node.

Here is the my code (with mdc diabled):
Note: This code also considers the attachments (embeds) in md files.
The file is plugins/markdown/remark-obsidian-links/index.js
import { visit } from 'unist-util-visit'

function transformer(tree, { data }) {

	visit(tree, 'text', (node, i, parent) => {
		const txt = node.value;

		// const EMBED_REGEX = /(!\[\[)([^\[\[]+)(\]\])/g
		// const LINK_REGEX = /(\[\[)([^\[\[]+)(\]\])/g

		const EMBED_REGEX = /(!\[\[)(.*?)(\]\])/g
		const LINK_REGEX = /(\[\[)(.*?)(\]\])/g

		let newEmbed = txt.replace(EMBED_REGEX, (_, m1, m2, m3) => {

			let src, title;
			src = title = m2;
			if (src.includes('|')) {
				let parts = src.split('|')
				src = parts[0];
				title = parts[1];
			}

			return `<md-embed src="${src}" data-title="${title}"></md-embed>`

			/* Now, inside `md-embed` component, considering `src ends with what, check which type of embed is it (e.g., image, PDF, audio, or even an MD file, and accordingly use element using `is`.)

			EXAMPLES:

			If Image, then `alt` will be title. Also, consider if using Obsidian's width feature, then using `^(\d+)` regex, use that as `width` of the image, and not alt.

			If audio, then use `audio` native element with `control`, and inside it, add child `source` element, whose src will be above `src`.

			If it is an MD file, you can use `ContentDoc` component, and fetch the file accordingly to display it. Also, take care of `src` that has `#`, i.e., sections/headings.

			*/
		})

		let newLink = newEmbed.replace(LINK_REGEX, (_, m1, m2, m3) => {

			let href, title;
			href = title = m2;
			if (href.includes('|')) {
				let parts = href.split('|')
				href = parts[0];
				title = parts[1];
			}

			return `<md-a href="${href}" data-title="${title}">${title}</md-a>`
			/* Now, inside `md-a` component, fetch the article based on `href` prop.
			- If `href` contains `/`, then inside `where` query, use `path`, else use `slug`.
			- Also, take care of the `source`, if you using Content Module's multi-source feature.
			*/

		})
		Object.assign(node, { type: 'html', value: newLink })

	})

}

export default function () {
	return transformer
}

The next file is plugins/markdown/remark-obsidian-links/package.json
{
    "name": "remark-obsidian-links",
    "version": "1.0.0",
    "type": "module",
    "main": "index.js",
    "author": {"name": "ManasMadrecha", "url": "https://manas.madrecha.com"},
    "dependencies": {
        "unist-util-visit": "4.1.0"
    }
}

Install it in app's package.json
  "dependencies": {
    "remark-obsidian-links": "file:./plugins/markdown/remark-obsidian-links"
  },
  "devDependencies": {
    "@nuxt/content": "^2.0.0",
    "nuxt": "^3.0.0-rc.4"
  }

Inside, nuxt.config.js,
import { defineNuxtConfig } from 'nuxt'

// https://v3.nuxtjs.org/api/configuration/nuxt.config
export default defineNuxtConfig({
  modules: ['@nuxt/content'],
  content: {
    markdown: {
      remarkPlugins: ['remark-obsidian-links'],
      mdc: false
    },
  }
})

@ManasMadrecha Thanks a lot for your guidance and solutions! I'll now write and refine the components to see how that works.
After that, I'll try to implement a backlink component. Hope I don't have to bother you again then.


Is this valid use of model?

Hi,
I'm using heavy key-lookup in a joined relation. So I created an instance method like below:
export default class OptionGroup extends Model {
  static entity = "optionGroups";

  static fields() {
    return {
      id: this.string(""),
      ...
      options: this.hasMany(Option, "optionGroupId"),
    };
  }

  // Memoize on first use, then return memoized value
  get optionsById() {
    if (!this._optionsById) {
      this._optionsById = {};
      this.options.forEach(option => (this._optionsById[option.id] = option));
    }
    return this._optionsById;
  }
}

Basically, on first call I add an attribute to instance (_optionsById) and filled it with data. For the subsequent calls, I return that data.
const optionGroups = OptionGroup.query().with("options");
const option = optionGroups[0].options[0];
const otherOption = optionGroups[0].optionsById["someId"];

It seems working as expected, but I wonder is there any problem with that approach?
EDIT 2: I don't want _optionsById stored in vuex store. Does adding this attribute to model instance cause addition of this to vuex-store?

No, you wouldn't want to do this. The reason is because currently, Vuex ORM is storing the class instance as is to the Store. Which means, the behavior might violate Vuex immutability. Which is, you shouldn't mutate the store state outside of store mutation.

I don't want _optionsById stored in vuex store. Does adding this attribute to model instance cause addition of this to vuex-store?

Yes it will. Because, Vuex ORM stores class instance as is to the Store.

Thanks for warning. I will not use it.


Multiple errors / Not understanding how to use this with Traefik

Hi guys,
I'm using Traefik (last version) with multiple services on a Debian machine with Docker Swarm.
When setting up an error page for all my traefik routers, I would need a wildcard certificate for *.mydomain.com
I saw that one way of doing this, keeping self-hosted, was to deploy acme-dns.
What I did :

Deployed acme-dns on my traefik stack like this :

  acmedns:
    image: joohoi/acme-dns:latest
    ports:
      - "5443:443"
      - "5080:80"
      - "53:53/udp"
      - "53:53"
    networks:
      - traefik-public
    volumes:
      - acmedns_config:/etc/acme-dns:ro
      - acmedns_data:/var/lib/acme-dns
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.acme_dns.rule=Host(`auth.mydomain.com`)"
        - "traefik.http.routers.acme_dns.entrypoints=websecure"
        - "traefik.http.routers.acme_dns.tls=true"
        - "traefik.http.routers.acme_dns.tls.certresolver=myresolver"
        - "traefik.http.services.acme_dns.loadbalancer.server.port=80"

My acme-config is like this :
[general]
# DNS interface. Note that systemd-resolved may reserve port 53 on <IP_ADDRESS>
# In this case acme-dns will error out and you will need to define the listening interface
# for example: listen = "<IP_ADDRESS>:53"
listen = "<IP_ADDRESS>:53"
# protocol, "both", "both4", "both6", "udp", "udp4", "udp6" or "tcp", "tcp4", "tcp6"
protocol = "both"
# domain name to serve the requests off of
domain = "auth.mydomain.com"
# zone name server
nsname = "auth.mydomain.com"
# admin email address, where @ is substituted with .
nsadmin = "myemail.me.com"
# predefined records served in addition to the TXT
records = [
    # domain pointing to the public IP of your acme-dns server
    "auth.mydomain.com. A XX.XXX.XXX.22",
    # specify that auth.example.org will resolve any *.auth.example.org records
    "auth.mydomain.com. NS auth.mydomain.com.",
]
# debug messages from CORS etc
debug = true

[database]
# Database engine to use, sqlite3 or postgres
engine = "sqlite3"
# Connection string, filename for sqlite3 and postgres://$username:$password@$host/$db_name for postgres
# Please note that the default Docker image uses path /var/lib/acme-dns/acme-dns.db for sqlite3
connection = "/var/lib/acme-dns/acme-dns.db"
# connection = "postgres://user:password@localhost/acmedns_db"

[api]
# listen ip eg. <IP_ADDRESS>
ip = "<IP_ADDRESS>"
# disable registration endpoint
disable_registration = false
# listen port, eg. 443 for default HTTPS
port = "80"
#port = "80"

# possible values: "letsencrypt", "letsencryptstaging", "cert", "none"
#tls = "letsencryptstaging"
tls = "none"

# only used if tls = "cert"
tls_cert_privkey = "/etc/tls/example.org/privkey.pem"
tls_cert_fullchain = "/etc/tls/example.org/fullchain.pem"
# only used if tls = "letsencrypt"
acme_cache_dir = "api-certs"
# optional e-mail address to which Let's Encrypt will send expiration notices for the API's cert
notification_email =<EMAIL_ADDRESS># CORS AllowOrigins, wildcards can be used
corsorigins = [
    "*"
]
# use HTTP header to get the client ip
use_header = false
# header name to pull the ip address / list of ip addresses from
header_name = "X-Forwarded-For"

[logconfig]
# logging level: "error", "warning", "info" or "debug"
loglevel = "debug"
# possible values: stdout, TODO file & integrations
logtype = "stdout"
# file path for logfile TODO
# logfile = "./acme-dns.log"
# format, either "json" or "text"
logformat = "text"

In my Traefik container I have :
    environment:
      - "ACME_DNS_API_BASE=https://auth.mydomain.com"
      - "ACME_DNS_STORAGE_PATH=/acme-dns.json"

In my Traefik configuration I have :
  myresolver-dns:
    acme:
      email<EMAIL_ADDRESS>      storage: "acme-dns.json"
      dnsChallenge:
        provider: acme-dns
        delayBeforeCheck: 0

My DNS configuration (on my host, Infomaniak.com) is :
auth            3600 IN NS    auth.mydomain.com.
_acme-challenge 3600 IN CNAME 5183ea08-6f3d-43d5-b914-9d3374d1fc03.auth.mydomain.com.

I was able to use this command to create an user :
curl -s -X POST https://auth.mydomain.com/register |python3 -m json.tool
I used those values to setup my _acme-challenge CNAME on my host DNS.
My problem is that I think I do not completely understand how this should be used.
Watching the video on https://github.com/joohoi/acme-dns, I tried to register (successful) and then update.
When updating I got the error Forbidden.
Now when trying to even register a new user I got :
curl -s -X POST https://auth.mydomain.com/register |python3 -m json.tool
Expecting value: line 1 column 1 (char 0)

Would someone help me understand and point me to the right direction ?
My objective is to use a wildcard certificate on a service declared like this :
  errors_custom:
    image: tarampampam/error-pages:latest
    restart: unless-stopped
    networks:
      - traefik-public
    environment:
      - "TEMPLATE_NAME=shuffle"
      - "SHOW_DETAILS=true"
    deploy:
      labels:
        # Traefik - CatchAll router for error handling
        - "traefik.http.routers.traefik_catchall.rule=HostRegexp(`^.+.mydomain.com$$`)"
        - "traefik.http.routers.traefik_catchall.entrypoints=websecure"
        - "traefik.http.routers.traefik_catchall.tls=true"
        - "traefik.http.routers.traefik_catchall.tls.certresolver=myresolver-dns"
        - "traefik.http.routers.gitlab.tls.domains[0].main=mydomain.com"
        - "traefik.http.routers.gitlab.tls.domains[0].sans=*.mydomain.com"
        - "traefik.http.routers.traefik_catchall.priority=1"
        - "traefik.http.services.traefik_catchall.loadbalancer.server.port=8080"
        - "traefik.http.routers.traefik_catchall.service=traefik_catchall"

Thank you very very much (for reading this until this point at least) !

Something I will add :
I cannot create in my host DNS both :

NS entry : auth.mydomain.com -> auth.mydomain.com
CNAME entry : *.mydomain.com -> {server_public_ip}


Undefined Reference to rediscpp::make_stream

I compiled redis-cpp with following code
`

cd /opt &&\
git clone https://github.com/tdv/redis-cpp.git &&\
cd redis-cpp &&\
mkdir build  &&\
cd build  &&\
cmake -DREDISCPP_HEADER_ONLY=OFF -DREDISCPP_PURE_CORE=ON ..  &&\
make  &&\
make install`

On this line
auto redis_client = rediscpp::make_stream(this->redis_host, this->redis_port);
I getting below error
undefined reference to rediscpp::make_stream(std::basic_string_view<char, std::char_traits >, std::basic_string_view<char, std::char_traits >)'
collect2: error: ld returned 1 exit status`
My question is that, How to find redis-cpp in cmake and link with my code??
Any help would be appreciated.
@tdv

Hello.
Thank you for the request.
There is no problem related to undefined references.
I've notices that you use the flag REDISCPP_PURE_CORE=ON. That means only pure code without any buildin streams from the library. You can use that flag only in case when you'd like to extend the library by your own stream implementation or you need to use only a protocol supporting. If you want to have transport supportion out of the box, you need to use the library without flag REDISCPP_PURE_CORE or disable the one. If you saw the stream.h file, you couls notice a guard at the beginning. I have not exclude that file from the installation process when REDISCPP_PURE_CORE is enabled. I admit that's my omission and will fix it soon.
In addition, you can see how to use 'pure code', for instance, in the example 'resp' (about the protocol parsing) https://github.com/tdv/redis-cpp/tree/master/examples/resp
Regards.

@tdv Thanks for your response.
At first, compiled this library with flag REDISCPP_PURE_CORE=OFF and REDISCPP_HEADER_ONLY=ON, there was no undefined reference but I got error of multiple definitions. I wrote a class to do all redis related tasks and when include wrote class into another class, i got multiple definition errors. I also used HEADER_ONLY flag in cmake. Can you please help me with that.
Regards,

Unfortunately, I can't deep dive into your issue, because I'm lacked of your project's sources. Maybe, you've encountered a rare case. Nevertheless, I would propose an easy recipe for how to use the library

Copy all include files of the library (https://github.com/tdv/redis-cpp/tree/master/include) into your project
Include the library with the macro definition (#define REDISCPP_HEADER_ONLY) like in examples, for instance, described in README
Configure your project to use the right libraries' paths
Use the library like a header-only library without any need to build the one.

Hopefully, that will work and there will not be any problem to use the library.
Good luck.
Regards.


Issue #119 First POC typed akka actors integration

Hi @softinio @mtsokol
Here's the first POC using typed actors.
Some points:

! is the only method available for ActorRef[]
Ask pattern is only accessible inside actor behavior.
actorSelection method is no longer available in typedActorSystem

Thanks for your comments!

Hi @gastonlucero, thank you for working on it!
Let me check it out briefly, and also let's ask @softinio and @mijicd for a review.
General questions:

! from typed.ActorRef is great for fire-and-forget, but I think we can try to introduce ? also via ask pattern from outside an Actor and handle returned Future as ZIO.fromFuture. Please see this docs link
Great to see this as a separate module! 🙂 How about renaming it to zio-actors-akka-interop?

Hi @gastonlucero, thank you for working on it!
Let me check it out briefly, and also let's ask @softinio and @mijicd for a review.
General questions:

! from typed.ActorRef is great for fire-and-forget, but I think we can try to introduce ? also via ask pattern from outside an Actor and handle returned Future as ZIO.fromFuture. Please see this docs link
Great to see this as a separate module! 🙂 How about renaming it to zio-actors-akka-interop?

Hey @mtsokol , I was looking ask documentation, and internally akka creates an actor (like a replyTo actor), [docs link] (https://github.com/akka/akka/blob/v2.6.3/akka-actor-typed/src/main/scala/akka/actor/typed/scaladsl/AskPattern.scala#L107)
Thats why  in AkkaTypedActorRefLocal  -> ?, the argument is fa: typed.ActorRef[A] => F[A], that is the way I get the response  ZIO.fromFuture(_ => akkaActor.ask[A](hiddenAkkaRef => fa(hiddenAkkaRef)))`

@gastonlucero can you rebase with master please to resolve conflict. WIll review as soon as you do that. Thank you for this work sir. Excited!


Add instructions for ARM v8 instruction sets

Changes proposed

[x] Add more ARM v8 Targets to the installation instructions

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @jamesmunns (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.


Optional field for mapping Test_id to candidate should be there on admin panel

Optional field for mapping Test_id to candidate should be there on admin panel

Fixed in PR #19. Hence closing this issue.


Update perf tool from iperf to qperf

Update perf tool from iperf to qperf since the qperf
provides richer metrics.
Merge bandwidth_test.go into performance_test.go.

Hi @weiqiangt
qperf do not support parallel streams, right? As a workaround, we have to create multiple processes for clients if we want to scale. Can it support TCP_NODELAY option? I could not find that in its man page. There could be some more gaps. Wondering if we should have both tools images for testing purposes rather than sticking to one? Also curious, when you say richer metrics--do you mean latency metric provided by qperf or something else?

Thanks Srikar.
The main purpose of this PR is for adding latency metrics of current performance tests.  And for the last question, I actually specify the latency measurement of qperf.
Thanks for your information, I did not consider parallel streams and TCP_NODELAY before and there is no test designed for them. Thus I think maybe we should keep iperf installed since the size of this image is not that matter, and we can add some other tests with less changes.

The main purpose of this PR is for adding latency metrics of current performance tests. And for the last question, I actually specify the latency measurement of qperf.
Thanks for your information, I did not consider parallel streams and TCP_NODELAY before and there is no test designed for them. Thus I think maybe we should keep iperf installed since the size of this image is not that matter, and we can add some other tests with less changes.

Thanks. Latency sounds good, but qperf seems to provide only average per-hop latency and not e2e latency. It is probably worth mentioning in the comments.
To run the iperf based test in the future, we still need a separate iperf image with a different entry point right? Is there any issue with providing docker files for both images separately?

Thanks. Latency sounds good, but qperf seems to provide only average per-hop latency and not e2e latency. It is probably worth mentioning in the comments.
To run the iperf based test in the future, we still need a separate iperf image with a different entry point right? Is there any issue with providing docker files for both images separately?

The qperf is used to measure specific test cases, I would like to update code comments.
We can set run arguments in Pod spec to override entry point, thus we don't need different images.


Heroku new app deployment failing

System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Herkou via https://dashboard.heroku.com/new?template=https://github.com/chakki-works/doccano
Python version:
heroku/python buildpack (3.6.8)

Describe the problem
The Heroku create app flow is failing at the ls ./static/bundle step in heroku.sh during a brand new app deployment.
ls: cannot access './static/bundle': No such file or directory
Removing the ls ./static/bundle line allows the app to successfully deploy on heroku, and if that step isn't relevant anymore I can submit a pull request removing/fixing it.

-----> Node.js app detected

-----> Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error
       NODE_ENV=production
       NODE_MODULES_CACHE=true
       NODE_VERBOSE=false

-----> Installing binaries
       engines.node (package.json):  8.x
       engines.npm (package.json):   unspecified (use default)

       Resolving node version 8.x...
       Downloading and installing node 8.16.0...
       Using default npm version: 6.4.1

-----> Installing dependencies
       Installing node modules (package.json)
       up to date in 0.49s
       found 0 vulnerabilities

-----> Build
       Running heroku-postbuild

       > @ heroku-postbuild /tmp/build_d09457687117af539176b710a5db0835
       > sh tools/heroku.sh build

       ><EMAIL_ADDRESS>postinstall /tmp/build_d09457687117af539176b710a5db0835/app/server/static/node_modules/swiper
       > node -e "console.log('\u001b[35m\u001b[1mLove Swiper? Support Vladimir\'s work by donating or pledging on patreon:\u001b[22m\u001b[39m\n > \u001b[32mhttps://patreon.com/vladimirkharlampidi\u001b[0m\n')"

       Love Swiper? Support Vladimir's work by donating or pledging on patreon:
        > https://patreon.com/vladimirkharlampidi

       added 54 packages from 326 contributors and audited 8430 packages in 8.086s
       found 20 vulnerabilities (16 moderate, 4 high)
         run `npm audit fix` to fix them, or `npm audit` for details

       ><EMAIL_ADDRESS>install /tmp/build_d09457687117af539176b710a5db0835/app/server/static/node_modules/husky
       > node ./bin/install.js

       husky
       setting up hooks
       Can't find .git directory, skipping Git hooks installation
       added 910 packages from 495 contributors and audited 8430 packages in 26.165s
       found 20 vulnerabilities (16 moderate, 4 high)
         run `npm audit fix` to fix them, or `npm audit` for details
       Hash: c8b9653f73fa11a7b580
       Version: webpack 4.12.0
       Time: 32119ms
       Built at: 2019-06-07 15:11:50
                                 Asset      Size  Chunks                    Chunk Names
                          guideline.js  1020 KiB       9  [emitted]  [big]  guideline
         upload_text_classification.js  3.07 MiB       0  [emitted]  [big]  upload_text_classification
                     upload_seq2seq.js  3.07 MiB       2  [emitted]  [big]  upload_seq2seq
                              stats.js  3.41 MiB       3  [emitted]  [big]  stats
                  sequence_labeling.js  3.15 MiB       4  [emitted]  [big]  sequence_labeling
                            seq2seq.js   3.1 MiB       5  [emitted]  [big]  seq2seq
                           projects.js   923 KiB       6  [emitted]  [big]  projects
                              label.js   936 KiB       7  [emitted]  [big]  label
                              index.js   656 KiB       8  [emitted]  [big]  index
           upload_sequence_labeling.js  3.07 MiB       1  [emitted]  [big]  upload_sequence_labeling
       download_text_classification.js  3.07 MiB      10  [emitted]  [big]  download_text_classification
         download_sequence_labeling.js  3.07 MiB      11  [emitted]  [big]  download_sequence_labeling
                   download_seq2seq.js  3.07 MiB      12  [emitted]  [big]  download_seq2seq
            document_classification.js  3.13 MiB      13  [emitted]  [big]  document_classification
                   demo_translation.js   949 KiB      14  [emitted]  [big]  demo_translation
           demo_text_classification.js   978 KiB      15  [emitted]  [big]  demo_text_classification
                  demo_named_entity.js   990 KiB      16  [emitted]  [big]  demo_named_entity
                            dataset.js   127 KiB      17  [emitted]         dataset
       Entrypoint dataset = dataset.js
       Entrypoint demo_named_entity [big] = demo_named_entity.js
       Entrypoint demo_text_classification [big] = demo_text_classification.js
       Entrypoint demo_translation [big] = demo_translation.js
       Entrypoint document_classification [big] = document_classification.js
       Entrypoint download_seq2seq [big] = download_seq2seq.js
       Entrypoint download_sequence_labeling [big] = download_sequence_labeling.js
       Entrypoint download_text_classification [big] = download_text_classification.js
       Entrypoint guideline [big] = guideline.js
       Entrypoint index [big] = index.js
       Entrypoint label [big] = label.js
       Entrypoint projects [big] = projects.js
       Entrypoint seq2seq [big] = seq2seq.js
       Entrypoint sequence_labeling [big] = sequence_labeling.js
       Entrypoint stats [big] = stats.js
       Entrypoint upload_seq2seq [big] = upload_seq2seq.js
       Entrypoint upload_sequence_labeling [big] = upload_sequence_labeling.js
       Entrypoint upload_text_classification [big] = upload_text_classification.js
       [366] ./pages/index.js + 3 modules 249 KiB {8} [built]
             | ./pages/index.js 744 bytes [built]
             |     + 3 hidden modules
       [367] ./pages/seq2seq.js + 5 modules 11.8 KiB {5} [built]
             | ./pages/seq2seq.js 197 bytes [built]
             |     + 5 hidden modules
       [368] ./pages/document_classification.js + 5 modules 11.2 KiB {13} [built]
             | ./pages/document_classification.js 297 bytes [built]
             |     + 5 hidden modules
       [369] ./pages/label.js + 5 modules 14.5 KiB {7} [built]
             | ./pages/label.js 157 bytes [built]
             |     + 5 hidden modules
       [370] ./pages/download_seq2seq.js + 5 modules 3.7 KiB {12} [built]
             | ./pages/download_seq2seq.js 195 bytes [built]
             |     + 5 hidden modules
       [371] ./pages/download_sequence_labeling.js + 5 modules 3.25 KiB {11} [built]
             | ./pages/download_sequence_labeling.js 232 bytes [built]
             |     + 5 hidden modules
       [372] ./pages/demo_translation.js + 5 modules 11.2 KiB {14} [built]
             | ./pages/demo_translation.js 235 bytes [built]
             |     + 5 hidden modules
       [373] ./pages/upload_seq2seq.js + 5 modules 5.22 KiB {2} [built]
             | ./pages/upload_seq2seq.js 187 bytes [built]
             |     + 5 hidden modules
       [374] ./pages/projects.js + 5 modules 11.1 KiB {6} [built]
             | ./pages/projects.js 171 bytes [built]
             |     + 5 hidden modules
       [375] ./pages/demo_text_classification.js + 5 modules 10.8 KiB {15} [built]
             | ./pages/demo_text_classification.js 303 bytes [built]
             |     + 5 hidden modules
       [376] ./pages/download_text_classification.js + 5 modules 3.83 KiB {10} [built]
             | ./pages/download_text_classification.js 240 bytes [built]
             |     + 5 hidden modules
       [377] ./pages/upload_sequence_labeling.js + 5 modules 5.52 KiB {1} [built]
             | ./pages/upload_sequence_labeling.js 224 bytes [built]
             |     + 5 hidden modules
       [381] ./pages/demo_named_entity.js + 10 modules 15.4 KiB {16} [built]
             | ./pages/demo_named_entity.js 275 bytes [built]
             |     + 10 hidden modules
       [382] ./pages/sequence_labeling.js + 10 modules 16.2 KiB {4} [built]
             | ./pages/sequence_labeling.js 273 bytes [built]
             |     + 10 hidden modules
       [439] ./pages/dataset.js 333 bytes {17} [built]
           + 425 hidden modules

       WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB).
       This can impact web performance.
       Assets:
         upload_text_classification.js (3.07 MiB)
         upload_sequence_labeling.js (3.07 MiB)
         upload_seq2seq.js (3.07 MiB)
         stats.js (3.41 MiB)
         sequence_labeling.js (3.15 MiB)
         seq2seq.js (3.1 MiB)
         projects.js (923 KiB)
         label.js (936 KiB)
         index.js (656 KiB)
         guideline.js (1020 KiB)
         download_text_classification.js (3.07 MiB)
         download_sequence_labeling.js (3.07 MiB)
         download_seq2seq.js (3.07 MiB)
         document_classification.js (3.13 MiB)
         demo_translation.js (949 KiB)
         demo_text_classification.js (978 KiB)
         demo_named_entity.js (990 KiB)

       WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.

       For more info visit https://webpack.js.org/guides/code-splitting/
       Done webpack build.
ls: cannot access './static/bundle': No such file or directory
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! @ heroku-postbuild: `sh tools/heroku.sh build`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the @ heroku-postbuild script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /tmp/npmcache.Y9Shk/_logs/2019-06-07T15_11_50_856Z-debug.log
-----> Build failed

       We're sorry this build is failing! You can troubleshoot common issues here:
       https://devcenter.heroku.com/articles/troubleshooting-node-deploys

       If you're stuck, please submit a ticket so we can help:
       https://help.heroku.com/

       Love,
       Heroku

 !     Push rejected, failed to compile Node.js app.
 !     Push failed

Thank you for reporting the issue!. It seems to be caused by folder structure change.
ls ./bundle should be work (ls is used just for confirmation, but it is useful to check webpack behavior).


Peer-to-peer code review

Great job on your project! Here is the feedback!
👍 good use of classes
👍 JS modules don't overlap and work properly
Keep going!

Greet work on your project
good way of using modules
Keep coding

Great job on getting your project approved
✔ Great layout and markup
👌 Nice job on using ES6 syntax
Congratulations and happy coding!!!!

@NataliaPoletaeva  @kingqabil @tmampa
You are the best 💯
Thank you for the awesome review.


Better select with Postgres jsonb

Problem
With the current implementation of select, relationships can easily be queried. It would be nice to have the same feature with json fields.
Suggested solution
The following example refers to table where sections is a json field (an array with nested data) and country is a relationship. With sections, I would like to only retrieve the attributes name and results (and with the results array, I would limit the response to the attribute name). It is basically a select statement which allows to only retrieve fragments of a json tree (similar to what Postgres offers with json-functions).
Note: The field country is only there to show how it works with relationships. It is not directly related to the issue.
const data = await prisma.data.findMany({
    select: {
        sections: {
            select: {
                name: true,
                results: {
                    name: true
                }
            }
        },
        country: {
            select: {
                id: true,
                name: true
            }
        }
    }
})

Alternatives
Raw SQL query.
Additional context
Postgres made lots of improvements with json and it would be nice to reflect this with Prisma. It is probably necessary to define the structure of the json field with the Prisma schema. Because results could both be an object or an array and we cannot distinguish that by the select statement (at least I cannot think of a solution).

I'd also love to see better support for selecting specific fields from Json. For context, my requirement is related to providing per-locale translations. Here's a example model for a Product.
model Product {
  id      String  @id @default(cuid())
  notes   Json
  premium Boolean
}

The notes field, for example, would have a structure like this. That is, it'd contain arrays of strings per locale, which I'd pass as an arg to the query.
"notes": {
  "de": [
    "...",
    "..."
  ],
  "en": [
    "...",
    "...",
  ],
  "es": [
    "...",
    "..."
  ]
}

It'd be great if I could create a query like this and have the notes return a single array for only the provided locale.
t.list.field('productsByLocale', {
  type: 'Product',
  args: {
    locale: nonNull(stringArg()),
  },
  resolve: (_parent, { locale }, ctx) => {
    return ctx.prisma.product.findMany({
      select: {
        notes: {
          by: locale, // `locale` would be passed as an argument such as "en"
        },
      },
    })
  },
})

That way, I'd send only what the current user needs and it'd be flexible enough to add a bunch of languages in the future.

Just following up on this, has there been any improvement with JSONB handling by Prisma in 2022?