Auto correct 2020 05 23

Starts 1st daily PR per https://www.pivotaltracker.com/story/show/172890204
This is a Work In Progress, and therefore a Draft PR.
It should be changed to a regular PR after:

200 files are changed, see protocol at https://www.pivotaltracker.com/story/show/172890204;
Rubocop 0.83 #538 is merged; and
Any conflicts with #538 are resolved.

Coverage remained the same at 91.929% when pulling b28546632f8d6a847be6a881d6498c887654c77d on JoeCohen:auto-correct-2020-05-23 into fa7d47da7a8ff31d41172b8cfc0f21c7b8af9962 on MushroomObserver:master.

Questions about work flow when fixing easily auto-correctable offenses. (Pivotal #172890204).

Multiple Layers of Auto-Correctable Offenses: Auto-correcting an offense can sometimes create new auto-correctable offenses, and that process can iterate. I have followed that process iteratively thinking that the object is to fix auto-correctable offenses. Should I have done that?
Increased numbers of some non-auto-correctable offenses: Auto-correcting occasionally creates non-auto-correctable offenses. I (almost) never fixed those, thinking that the object is to fix the auto-correctable ones ASAP, leaving the rest till later. Should I have addressed new no-auto-correctable offenses?
Coverage decrease: Coveralls reports a drop in Coverage. I ignored it for two reasons: (1) all the "new missed lines" are old uncovered lines which have changed; (2) the decrease is trivial (-0.007%). Should I try to add some tests so that coverage does not drop at all.
Regenerating .rubocop_todo.yml: I did this as the last step in this PR. Is that how it should work?

PS: Scorecard
Total Offenses -- Before: 3134, After: 2909. About 7% decrease
Increased number of these non-auto-correctable offenses:

Lint/UselessAssignment 54 => 55


[GHSA-rrr8-f88r-h8q6] find-my-way has a ReDoS vulnerability in multiparametric routes

Updates

Affected products
CVSS v3

Comments
follow-my-way v4.1.0 is NOT exploitable by CVE-2024-45813. The following test does NOT fail on Node 12 NPM 6 follow-my-way v4.1.0:
test('prevent back-tracking', (t) => {
  t.plan(0)
  t.setTimeout(20)
  const findMyWay = FindMyWay({
    defaultRoute: () => {
      t.fail('route not matched')
    }
  })
  findMyWay.on('GET', '/:foo-:bar-', (req, res, params) => {})
  findMyWay.find('GET', '/' + '-'.repeat(200000000) + 'a', { host: 'fastify.io' })
})

The reason is that CVE-2024-45813 exploits mechanism that does not exist in v4.1.0. CVE-2024-45813 exploits a mechanism to check the validity of a given regex, and could cause a ReDoS. At some point after 4.1.0, the maintainer transformed the function router.prototype._on to use loops instead of recursion. This mechanism does NOT exist in version 4.1.0.

Hi there @delvedor! A community member has suggested an improvement to your security advisory. If approved, this change will affect the global advisory listed at github.com/advisories. It will not affect the version listed in your project repository.
This change will be reviewed by our Security Curation Team. If you have thoughts or feedback, please share them in a comment here! If this PR has already been closed, you can start a new community contribution for this advisory

Hey @sealonohana, thanks for the PR. We got out ranges from the maintainer's repo advisory
https://github.com/delvedor/find-my-way/security/advisories/GHSA-rrr8-f88r-h8q6
Would you mind reaching out to them to determine a lower bound for the advisory?

The vulnerability is the result of the following commit: https://github.com/delvedor/find-my-way/commit/66fa03923355b8da1db4ba572d66a4fee4a57cf5
Which means the vulnerability is from version 5.5.0

@darakian I've posted here the correct commit.

V4 of fmw is long unsupported and should not be used.

Which means the vulnerability is from version 5.5.0

I went ahead and updated the lower bound to be 5.5.0 based on the commit 👍


Add --mount syntax documentation to CLI reference

This is a first stab at adding some information about the --mount syntax and options. Should fix https://github.com/docker/docker/issues/24277 once merged
ping @sfsmithcha @cpuguy83 PTAL; I'm sure there's lots of improvements to be made (but I found a couple of issues / bugs, so it was useful :smile:)

thanks for this @thaJeztah . Loads of important information re: mounts.

Thanks for reviewing @sfsmithcha, I'll work on your suggestions, and ping after I have made change

@sfsmithcha PTAL, I pushed changes as a separate commit (for easier reviewing), will rebase/squash after review is done.
I did notice that we need to make some changes to the stylesheets, because bullet-lists inside a table get a different font-size than regular text inside a table:

It's easy to fix though, so I can open a PR for docs-base if needed;

We should probably make some recommendations here, and likely point out some of the short-comings that currently exist... namely:

Don't use named volumes with a not-distributed (local/node-level) driver, there's really no reason to.
Mounts are not volumes, even though they can take a volume.
Scaling services that require persistent storage (per-task) is going to be problematic and generally not supported right now.

What's the limitation on persistent storage atm? I'm almost 100% sure flocker can handle this and my kit can too...

@erikh Only way to have a scaled service + volume mount is for all tasks to get the same volume.

@thaJeztah I pushed a commit to my fork under branch pr/25776 to go on top of this PR but I'm not sure how to get it into the PR. PTAL I think it addresses some of @sfsmithcha comments and some formatting issues. In addition, I worked with @stevvooe on bind propagation. Still need to test that on Monday and still consider this a work in progress. He also asked me to put some docs in a few source files about bind propagation.

ping @cpuguy83 @stevvooe do you think this is ready to merge?  We can tweak / improve later, but I think it'd be nice to have this published as a start

LGTM

author LGTM

@thaJeztah Atm it seems from the reference doc like we have different syntax for docker run(https://github.com/docker/docker/blob/master/docs/reference/commandline/run.md#add-bin-mounts-or-volumes-using-the---mount-flag) and service create(https://github.com/docker/docker/blob/master/docs/reference/commandline/service_create.md#create-a-service-that-uses-a-bind-mounted-host-directory). I think we should use consistent syntax in the examples, not sure which one though.


Whenever I try to use create-llama through CLI, it throws the error: The data stream is hanging. Did you forget to close it with data.close()?

Here's a full copy of what I did on the CLI and the respective output:
Microsoft Windows [Version 10.0.22631.3593]
(c) Microsoft Corporation. All rights reserved.
C:\Users\lamak>npx create-llama@latest --ask-models
√ What is your project named? ... my-app
√ Which template would you like to use? » Chat
√ Which framework would you like to use? » NextJS
√ Would you like to set up observability? » No
√ Which model provider would you like to use » Gemini
√ Please provide your Google API key (or leave blank to use GOOGLE_API_KEY env variable): .
√ Which LLM model would you like to use? » gemini-1.5-pro-latest
√ Which embedding model would you like to use? » text-embedding-004
√ Which data source would you like to use? » Use an example PDF
√ Would you like to add another data source? » No
√ Would you like to use LlamaParse (improved parser for RAG - requires API key)? ... no / yes
√ Would you like to use a vector database? » No, just store the data in the file system
√ Would you like to build an agent using tools? If so, select the tools here, otherwise just press enter »
√ How would you like to proceed? » Generate code, install dependencies, and run the app (~2 min)
Creating a new LlamaIndex app in C:\Users\lamak\my-app.
Using npm.
Initializing project with template: streaming
Using vector DB: none
No tools selected - use optimized context chat engine
Installing dependencies:

@radix-ui/react-collapsible
@radix-ui/react-hover-card
@radix-ui/react-slot
ai
ajv
class-variance-authority
clsx
dotenv
llamaindex
lucide-react
next
pdf2json
react
react-dom
react-markdown
react-syntax-highlighter
remark
remark-code-import
remark-gfm
remark-math
rehype-katex
supports-color
tailwind-merge
vaul
@llamaindex/pdf-viewer
@e2b/code-interpreter
uuid

Installing devDependencies:

@types/node
@types/react
@types/react-dom
@types/react-syntax-highlighter
autoprefixer
cross-env
eslint
eslint-config-next
eslint-config-prettier
postcss
prettier
prettier-plugin-organize-imports
tailwindcss
tsx
typescript
@types/uuid

npm warn deprecated<EMAIL_ADDRESS>This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated<EMAIL_ADDRESS>Use @eslint/config-array instead
npm warn deprecated<EMAIL_ADDRESS>Rimraf versions prior to v4 are no longer supported
npm warn deprecated<EMAIL_ADDRESS>Glob versions prior to v9 are no longer supported
npm warn deprecated<EMAIL_ADDRESS>Use @eslint/object-schema instead
npm warn deprecated<EMAIL_ADDRESS>Use mz or fs-extra^3.0 with Promise Support
npm warn deprecated<EMAIL_ADDRESS>dommatrix is no longer maintained. Please use @thednp/dommatrix.
added 986 packages, and audited 989 packages in 2m
291 packages are looking for funding
run npm fund for details
3 high severity vulnerabilities
Some issues need review, and may require choosing
a different dependency.
Run npm audit for details.
Created '.env' file. Please check the settings.
Generating context data...
Copying data from path: C:\Users\lamak\AppData\Local\npm-cache_npx\7bfc2205dda2d438\node_modules\create-llama\dist\templates\components\data\101.pdf
Running npm run generate to generate the context data.

<EMAIL_ADDRESS>generate
tsx app\api\chat\engine\generate.ts

Using 'gemini' model provider
Generating storage context...
No valid data found at path: cache\doc_store.json starting new store.
No valid data found at path: cache\index_store.json starting new store.
No valid data found at path: cache\vector_store.json starting new store.
Storage context successfully generated in 10.826s.
Finished generating storage.
Initialized a git repository.
Success! Created my-app at C:\Users\lamak\my-app
Now have a look at the README.md (file://C:\Users\lamak\my-app/README.md) and learn how to get started.
Running app in C:\Users\lamak\my-app...

<EMAIL_ADDRESS>dev
next dev

▲ Next.js 14.2.4

Local:        http://localhost:3000
Environments: .env

✓ Starting...
✓ Ready in 2.6s
○ Compiling / ...
✓ Compiled / in 10s (2917 modules)
GET / 200 in 11380ms
○ Compiling /favicon.ico ...
✓ Compiled /api/chat/config in 5.7s (2812 modules)
✓ Compiled in 0ms (1489 modules)
✓ Compiled in 0ms (1489 modules)
✓ Compiled /api/chat in 1ms (1489 modules)
GET /favicon.ico 200 in 4956ms
✓ Compiled (2363 modules)
GET /api/chat/config 200 in 6989ms
Using 'gemini' model provider
GET /api/chat/config 200 in 932ms
GET /api/chat/config 200 in 11ms
GET /api/chat/config 200 in 10ms
The data stream is hanging. Did you forget to close it with data.close()?
POST /api/chat 200 in 7783ms

related to create-llama, tracked in https://github.com/run-llama/create-llama/issues/125


Update aws-core, http-client-spi, ... to 2.17.247

Updates

software.amazon.awssdk:aws-core
software.amazon.awssdk:http-client-spi
software.amazon.awssdk:netty-nio-client

from 2.17.239 to 2.17.247.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "software.amazon.awssdk" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "software.amazon.awssdk" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #923.


Better configuration for examples

We should pull in ini.c from aerc and add examples/config.c

fixed by #120


Would be nice that it initialize the test instance with the before / after from the extensions

Currently it only execute the test method from a fresh object instance, but it doesn't respect the before / after which can be annotated, thus some of the object fields are not set, and test may fail.

Thanks for your advice, I'll add this feature later.


extending 'tf.keras.Model.evaluate' API to permit using class_weight directly.

Issue : #35825

Oh yes. Sorry, i missed that earlier.
@qlzh727  I have made the changes.

@qlzh727  #37463 I am facing this issue while performing the unit test

@qlzh727  Can you help me with performing the unit test ? I am trying it for a long time, i am facing some problems.

I would expect some new unit test being added for this new param, and I didn't find that in this PR.
Do u have any trouble for writing new unit test? You can follow the existing example in the same package for the fit() method with class weights.

I tried to write the unit test but got this error : TypeError: evaluate() got an unexpected keyword argument 'class_weight'
I couldn't get how to solve this.

I think the current PR looks reasonable. What's the issue u are seeing in the test? Btw, I am kicking off a test on our side, and we can see the test result soon.

@qlzh727  I updated the docstring. Thank you.

@ayushmankumar7 Can you please check build failures. Thanks!

Is there any solution please to add class_weight to the evaluate() method?

@qlzh727  I tried updating the API .pbtxt ... #38350 issue.  I am working on it. Will update it asap. Thank you.

@qlzh727 Please check. I updated the golden files. Successfully regenerated the .pbtxt files. Thank you for the guidance.

@qlzh727 I made the requested changes

@qlzh727 I am sorry for that small mistake. I missed a comma. I fixed it. Thank you.

@ayushmankumar7 Can you please fix build failures ? Thanks!

@gbaned  I am encountering this error :

I am not sure how to solve it. As suggested, i updated the golden file, wrote unit tests. If i get some help, i will be able to make the changes succesfully. Thank you.

Why we have the last commit for revert back? https://github.com/tensorflow/tensorflow/pull/37425/commits/0baa946a97098e51f1bb662b551ec9f264a5c0b6

@qlzh727  I went though the master branch, i saw that i added 'class_weight' to evaluate_generator which wasn't the objective of the PR. So, i reverted it back to the what it was initially and concentrating only on evaluate.

I think the eval and eval_generator should have the same API interface for consistency.

Okay, then i am making the changes. Thanks

@qlzh727 I made the requested changes.

@qlzh727 I solved it. Please have a look. Thank you.

What's the status of this PR ? @gbaned

@qlzh727  Thanks for the approval :)

Sorry for the long wait. I checked with Francois about the PR, and seems that it is expected to not having class_weights for model.evaluate.
Quote from Francois:
"The reason we don't support class weights in evaluate is that the class_weight argument represents sample weights that are computed from the labels, but the labels should not be an input to the model during evaluation. During training this is fine, but during evaluation this represents a data leak from the labels to your metrics. If you used class weighting in evaluate, your score would not be reproducible on real test data (when you don't have the labels).
So this is conceptually wrong."
I am closing this PR since it is working as intended.


support loading standard library zig files?

** (File.Error) could not read file "/home/ityonemo/code/zocket/lib/std.zig": no such file or directory
probably if we can't find a zig file in the main path, don't call an error during the early phase.

fixed in c88e8d2


[Question] How to interpret the gt_results in relation to the --ref-fna?

Running version: maast=1.0.8=py310hc2b1e32_0
Running the following command:
maast end_to_end --in-dir assemblies/DENV2/ --rep-fna References/DENV2.fa  --out-dir maast_output/DENV2/ --skip-centroid --keep-redundancy

Trying to understand the output better:
Local Pos: up to seven digits which specifies the local position of a SNP on a contig
Global Pos: up to seven digits which specifies the global position of a SNP in a species, served as sort of ID
I’m assuming that Column 2 and 3 are one and the same and identifies the NT position of the SNP.
Allele 1: single character, A, C, G or T, which specifies allele 1 of a SNP I
I am not sure if this base is referring to the refeq’s (wildtype) base? That is the DENV.fa file provided for --rep-fna
Allele 2: similiar as Ref allele but specifies allele 2 of a SNP
Does this refer to the SNP of the sample that is different from Column 4 and hence identifies the SNP?

Hi jolespin, thanks for trying maast.

I’m assuming that Column 2 and 3 are one and the same and identifies the NT position of the SNP.

They are not the same. Column 2 is the position of the SNP on chromosome (Column 1), Column 3 is the unique identifier that is derived from the global genomic position. For example, three SNPs, A, B and C, can have the following values in their first three columns: Chrom 1 Pos 1 ID 1, Chrom 1 Pos 2 ID 2, Chrom 2 Pos 1 ID 3. As you can see, column 3 always increases thus will be unique, and column 2 always indicates the position on chomosome.

I am not sure if this base is referring to the refeq’s (wildtype) base? That is the DENV.fa file provided for --rep-fna
Does this refer to the SNP of the sample that is different from Column 4 and hence identifies the SNP?

Good questions! These two questions are related and hence are answered here together. Allele 1 is actually the major allele identified in the input genomes, and Allele 2 is the minor one. Allele 1 serves as reference allele but it does not mean it is always the one on the reference genome (perhaps calling it representative genome is less confusing). Practically, it allows to solve the conflicts when the actual allele on the reference genome is extremely rare.
Let me know please if you have further questions.

I'm still not entirely clear to be honest.  Would you be able to do a detail explanation on how to interpret this using reads data and separately using contigs as input on your main repo? Maybe a FAQ section? I'd love how easy it was to use this tool but I'm not sure how to interpret the results so I can't really use it in the paper I'm working on at the moment.

We actually had a demonstration study which did a bit more detailed walk through of results interpretation. Check it out at https://www.sciencedirect.com/science/article/pii/S2666166722008449. Please feel free to leave your suggestions on FAQ, too. Thanks.


Fix typo in docs to correctly show asymmetric

I believe there is a typo in the docs that should say asymmetric instead of symmetric. This potentially could be confusing to readers. I have attached a picture with a red box around what I believe should be changed.
Thanks for the awesome library!

Isn't symmetric the correct word?  It looks to be HMAC which is symmetric not asymmetric

This is symmetric indeed.


[TRA-4197] Integration Tests for PCAP files

similar idea of testing PCAP files already merged in https://github.com/up9inc/mizu/pull/767
so I will close this


Hotfix reference link

Ref #816

Coverage increased (+0.04%) to 91.346% when pulling 8b11d80d30274136b589c38587c6a1e7bc543882 on takahi-i:hotfix-reference-link into e25d95ba71d28555259d68e090791b82c2d9970e on redpen-cc:master.

Coverage increased (+0.04%) to 91.346% when pulling 8b11d80d30274136b589c38587c6a1e7bc543882 on takahi-i:hotfix-reference-link into e25d95ba71d28555259d68e090791b82c2d9970e on redpen-cc:master.

I'm sorry that I now don't have an environment to build Java application. Could you try this file and confirm no error? If no error, it must work for me.

Thank you very much for giving us the sample text @rhysd. I succeeded to run fixed redpen command for the sample.
➜  redpen-distribution-1.10.1-SNAPSHOT git:(hotfix-reference-link) bin/redpen  ~/tmp/README.md
[2017-12-15 00:52:23.117][INFO ] cc.redpen.Main - Configuration file: /Users/takahi-i/IdeaProjects/redpen/redpen-distribution/target/redpen-distribution-1.10.1-SNAPSHOT/conf/redpen-conf-en.xml
[2017-12-15 00:52:23.125][INFO ] cc.redpen.config.ConfigurationLoader - Loading config from specified config file: "/Users/takahi-i/IdeaProjects/redpen/redpen-distribution/target/redpen-distribution-1.10.1-SNAPSHOT/conf/redpen-conf-en.xml"
[2017-12-15 00:52:23.139][INFO ] cc.redpen.config.ConfigurationLoader - Succeeded to load configuration file
[2017-12-15 00:52:23.139][INFO ] cc.redpen.config.ConfigurationLoader - Language is set to "en"
[2017-12-15 00:52:23.139][WARN ] cc.redpen.config.ConfigurationLoader - No variant configuration...
[2017-12-15 00:52:23.142][INFO ] cc.redpen.config.ConfigurationLoader - No "symbols" block found in the configuration
[2017-12-15 00:52:23.145][INFO ] cc.redpen.config.SymbolTable - Default symbol settings are loaded
[2017-12-15 00:52:23.153][INFO ] cc.redpen.parser.SentenceExtractor - "[., ?, !]" are added as a end of sentence characters
[2017-12-15 00:52:23.153][INFO ] cc.redpen.parser.SentenceExtractor - "[', "]" are added as a right quotation characters
[2017-12-15 00:52:23.563][WARN ] cc.redpen.parser.markdown.ToFileContentSerializer - Failed to add parent for a Section
[2017-12-15 00:52:23.662][INFO ] org.reflections.Reflections - Reflections took 50 ms to scan 1 urls, producing 6 keys and 58 values
[2017-12-15 00:52:23.751][WARN ] cc.redpen.validator.ValidatorFactory - cc.redpen.validator.sentence.SpaceBeginningOfSentenceValidator is deprecated
[2017-12-15 00:52:23.755][WARN ] cc.redpen.validator.ValidatorFactory - cc.redpen.validator.section.VoidSectionValidator is deprecated
[2017-12-15 00:52:23.809][INFO ] org.reflections.Reflections - Reflections took 3 ms to scan 1 urls, producing 175 keys and 180 values
[2017-12-15 00:52:24.011][INFO ] cc.redpen.util.DictionaryLoader - Succeeded to load UnexpandedAcronymValidator default dictionary.
[2017-12-15 00:52:24.016][INFO ] cc.redpen.util.DictionaryLoader - Succeeded to load weak expressions.
[2017-12-15 00:52:24.017][INFO ] cc.redpen.validator.JavaScriptValidator - JavaScript validators directory: js
README.md:81: ValidationError[UnexpandedAcronym], The expanded form of the acronym API is not present in the document. at line: MIT license
README.md:59: ValidationError[EmptySection], The section "How to get GitHub API token" is empty. at line: How to get GitHub API token
README.md:12: ValidationError[SentenceLength], The length of the sentence (121) exceeds the maximum of 120. at line: github-clone-all is a small command to clone all repositories matching to given query and language via GitHub Search API.
README.md:25: ValidationError[EndOfSentence], Found invalid end of sentence "'.". at line:  But quoting the query is recommended to avoid conflicting with shell special characters as github-clone-all 'foo bar'.
README.md:51: ValidationError[Hyphenation], This phrase should be hyphenated (ie: "set-to"). at line: If the token is set to $GITHUB_TOKEN environment variable, omitting -token should also work fine.
README.md:59: ValidationError[Hyphenation], This phrase should be hyphenated (ie: "How-to"). at line: How to get GitHub API token

[2017-12-15 00:52:24.051][ERROR] cc.redpen.Main - The number of errors "6" is larger than specified (limit is "1").

Merged the hotfix.


no detailed page documenting about an Overlaps function

Re : no detailed page documenting about an Overlaps function
Hello,
There is no specific page to detail how to compute an Overlaps map.
Yours sincerely,

A keyword search provides only Changelogs information, no documentation ...
https://qgis.org/en/search.html?q=overlaps&check_keywords=yes&area=default

@pblottiere This is the repository of QGIS website which is not intended to provide documentation on features. Please refer to https://docs.qgis.org for anything related to features documentation and if not documented check whether an issue is not already available for the feature at https://github.com/qgis/QGIS-Documentation/issues. Thanks.
Closing.


Trying to fix test coverage for the two lines

In relation to #64 .
I tried making a test that creates Agents that shutdown after a certain timeout, which should give NameServerProcess.shutdown() enough time to enter the while loop and cover those lines in the test. However, this is failing. My personal guess is that the GIL is somehow related, but I don't know how to get around it.

Already merged #69.


taskserver: integrate pki scripts

This PR integrates the pki convenience scripts into the taskserver package.
In the patchPhase all scripts related to generating a CA for taskwarrior are written into share/taskd and patched to actually work within nixos environment.
Things done:

[x] Tested using sandboxing
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

More
I saw there is an open PR #14476 for integrating the whole taskserver into nixpks - still i think providing the original scripts as described in the official documentation.
cc @matthiasbeyer

Looks good to me and builds, so ACK.

@makefu you might want to have a look at #14476 as well.

@matthiasbeyer as i wrote in the in comments above - i saw the PR being open but i think providing the original scripts from upstream to be usable has some advantages - especially when running on a non-nixos machine.


llms: Add support for using the whisper model to transcribe audio

PR Checklist

[x] Read the Contributing documentation.
[x] Read the Code of conduct documentation.
[x] Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
[x] Describes the source of new concepts.
[ ] References existing implementations as appropriate.
[x] Contains test coverage for new functions.
[x] Passes all golangci-lint checks.

I agree with @eliben's intuition here, I'm not sure if audio transcription as a concept fits right into our llm namespace. I'm open to exposing this and generalizing over providers but I think it belongs in a different namespace.

@tmc some update ?


ISSUE-793: Make ManagerStatus and ManagerStatus.Leader nullable

Fix NodeInfo.ManagerStatus and ManagerStatus.Leader. Both fields can be null.

Codecov Report

Merging #802 into master will increase coverage by <.01%.
The diff coverage is 66.66%.

@@             Coverage Diff              @@
##             master     #802      +/-   ##
============================================
+ Coverage     66.04%   66.04%   +<.01%
  Complexity      713      713
============================================
  Files           162      162
  Lines          3060     3063       +3
  Branches        350      351       +1
============================================
+ Hits           2021     2023       +2
- Misses          884      885       +1
  Partials        155      155


Timings always show in terminal

Not sure when this started, but any command that is slightly longer than instantaneous gets the time printed. This clutters the terminal and can make messages hard to read. Any way to turn this off?
Simple example when running devtools::check() on a package:

Weirdly enough, the timings often go away if I resize the terminal window. Would be great if the timings could be cleared away automatically,

this might actually be specific to itbble or pillar-formatted objects.

Closing, seems to be tibble/pillar specific. See #942.


what is the default cloud-config option

As raised by @detiber here, the default we use for the path to the cloud-config file is /etc/cloud-sa/cloud-sa.json, yet this is a holdover from GCP which has no meaning in the EQXM context.
We should change this.

An update to this.
The instancesv2 PR (#185) introduced a change in the name of the variable that users supply to define the cloud-config file.
Previously --provider-config was used.  This argument was required and had no default.
That argument is no longer honored. It has been replaced with --cloud-config which is standard among cloud-provider binaries. The argument is inherited from the upstream packages, as seen in --help:
      --cloud-config string                      The path to the cloud provider configuration file. Empty string for no configuration file.

This argument also has no default value and is required.
The default deployment templates now include '--cloud-config=/etc/cloud-sa/cloud-sa.json'.

It's also confusing that the default secret used by CPEM is metal-cloud-config, this is suggestive that this is used by MetalLB.

/remove-lifecycle rotten

/reopen

/triage accepted


how to change search value in router

from the link  > www.abc.com/search?rlz=1C1CHBF_env  to  > www.abc.com/search
i want to change the link when i click to another page
now, i  get the data of router b using  mapStateToProps

This is a bug tracker, not a support system. For usage questions, please use Stack Overflow or Reactiflux where there are a lot more people ready to help you out. Thanks!


Add PendingApprovalRetriever

For https://github.com/ProfessionalWiki/PageApprovals/issues/24
DatabasePendingApprovalRetriever and DatabasePendingApprovalRetrieverTest are, in large part, AI-generated, and I have not fully reviewed them from first principles yet.
Query turned out to be more complex than I had imagined, but I don't see a way to simplify

Fixed CS issues with https://github.com/ProfessionalWiki/PageApprovals/commit/dcb59d82f4b86b960b94d17de68b63e397395593


stylus global import not work!

i use stylus ,
min.styl
$main-color = #000000

App.vue
  @import "../assets/min.styl"

Holle.vue
  .button
      $main-color

holle.vue can't find $main-color
what can i do

Please ask questions on the forum or StackOverflow.

I think you have to import the stylus file in the style section

@posva I don't want to do in every vue file,Is any other way to do

Not that I know. You should search about how to do it in plain stylus
files, but it's not related to Vue
On Fri, 21 Apr 2017, 05:48 BusyHe<EMAIL_ADDRESS>wrote:

@posva https://github.com/posva I don't want to do in every vue file,Is
any other way to do
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/vuejs/vue-cli/issues/441#issuecomment-296046156, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAoicZyb8cvuiHMFd13j3vlHFb6-WGC1ks5ryCcZgaJpZM4NCluU
.

but it is related to the vue-cli. Importing fonts doesn't work either.

I just wanted to leave a note to say @sky790312 's comment above helped me figure this out.
My vue.config.js which works
// vue.config.js
var path = require('path');
module.exports = {
  css: {
    loaderOptions: {
      stylus: {
        import: [path.resolve(__dirname, "src/stylus/variables")]
      }
    }
  }
};


Create release assets

Create the workflow for git tag.
As par of https://github.com/zhangyuan/google-authenticator-extractor/issues/3

https://github.com/zhangyuan/google-authenticator-extractor/pull/24


React-color 2.17 is breaking

I'm using storybook addon-knobs, it uses react-color as a dependency, and it has an error in it's story files, I guess it tries to find the .storybook directory which is missing, I tried uninstalling knobs and only installing react-color, it still breaks the build with the same error.
ERROR in ./node_modules/react-color/lib/components/swatches/story.js
Module not found: Error: Can't resolve '../../../.storybook/report' in '/Users/lna11126/Desktop/my-app/node_modules/react-color/lib/components/swatches'
 @ ./node_modules/react-color/lib/components/swatches/story.js 9:14-51
 @ . sync .story\.jsx?$
 @ ./.storybook/config.js
 @ multi<EMAIL_ADDRESS><EMAIL_ADDRESS>./.storybook/config.js (webpack)-hot-middleware/client.js?reload=true

This is the file: react-color/lib/components/swatches/story.js

It looks like you are using a broad glob pattern when searching for stories in your project and it is pulling in stories from this package. Are you doing this intentionally? If you omit it from searching the node_modules folder it will fix this problem.


powershell/collection/screenshot

The script that appears in the comments (https://github.com/PowerShellMafia/PowerSploit/blob/master/Exfiltration/Get-TimedScreenshot.ps1) seem to have an interval parameter. However the module in Empire only takes Ratio and Agent parameters. Is it possible to use it for repeated screenshots with a time interval?

@poopaapoopaa You can modify the module to try repeated screenshots. I'm not sure how empire would handle the output.


ci: update githash for libhg

Updates the git hash after merging in libhg repo.

I assume that is the correct hash


Let use explicit plugins/presets with a ! and the end of the name

Related to #42

Not sure why the checks failed :(


Delete default permissions methods for project (for WB, DS, Flow etc) not working for TSC

Describe the bug
I am using the TSC methods to delete the default permissions method for WB, DS etc and it is throwing an error message. Only the delete project permissions method is working but the delete permissions method for WB, DS and Flow for a project is failing.
Versions
I tried it for both 0.24 and 0.25 (latest) and it fails.
To Reproduce
Use the following code. You will have to uncomment or comment certain lines based on what methods you want to call.
import hvac
import sys
import argparse
import getpass
import os
import re
import smtplib
import tableauserverclient as TSC
import csv
import docopt
from configparser import ConfigParser
import logging.config
def main():
print('Hello World')
tabuname = 'abcd'
tabuserpwd = 'xxxx'
tabsite = 'Test'
tabserver = 'https://server.com/'
tableau_auth = TSC.TableauAuth(tabuname, tabuserpwd, tabsite)
server = TSC.Server(tabserver, use_server_version=False)
server.add_http_options({'verify': False})
# change the REST API version to match the server
server.use_server_version()
print(f'Server: {tabserver} : username: {tabuname} site: {tabsite} (blank means default) tabuserpwd:{tabuserpwd} ')
with server.auth.sign_in(tableau_auth):
# Loop through and find out the schedules
print(f'TabServer Logged in...')
#exit()
#tabserver = 'https://10ax.online.tableau.com/'
#tableau_auth = TSC.PersonalAccessTokenAuth('Nirav2023-05', 'PATTOKEN')
#server = TSC.Server(tabserver, use_server_version=False)
#server.add_http_options({'verify': True})
## change the REST API version to match the server
#server.use_server_version()
with server.auth.sign_in(tableau_auth):
print(f'Start:Projects metadata collection')
projs = list(TSC.Pager(server.projects))
WBs = list(TSC.Pager(server.workbooks))
DSs = list(TSC.Pager(server.datasources))
#users = list(TSC.Pager(server.users))
for wb in WBs:
if wb.name == 'dsNirav2_WB':
wbFound = wb
print(f'TabCloud wbFound.name={wbFound.name}, project={wbFound.project_name}')
for proj in projs:
if proj.name == 'xNiravTest1':
project1 = proj
if proj.name == 'xNiravTest2':
project2 = proj
print(f'Project1={project1.name}, Project2={project2.name}')
# Delete existing permissions of Project2 first
#
#server.projects.populate_workbook_default_permissions(project1)
#tgtpermissions = project1.default_workbook_permissions
server.projects.populate_permissions(project1)
tgtpermissions = project1.permissions
#server.projects.populate_datasource_default_permissions(project1)
#tgtpermissions = project1.default_datasource_permissions
print(f'tgtpermissions:{tgtpermissions}', {len(tgtpermissions)})
for permission in tgtpermissions:
grantee = permission.grantee
capabilities = permission.capabilities
tgt_del_rules = [TSC.PermissionsRule(grantee, capabilities)]
print(f'grantee={grantee}  ,  capabilities={capabilities}')
server.projects.delete_permission(project1, tgt_del_rules)
#server.projects.delete_workbook_default_permissions(project1, tgt_del_rules)
#server.projects.delete_datasource_default_permissions(project1, tgt_del_rules)
Results
What are the results or error messages received?
When I call the delete_workbook_default_permissions or delete_datasource_default_permissions etc.
Then I get the following error message : 'list' object has no attribute 'capabilities'
NOTE: Be careful not to post user names, passwords, auth tokens or any other private or sensitive information.

Forgot to add. These methods do not work for both the on-prem as well as TabCloud. The server names, username and the pwd or the personal access token are all imaginary. Please replace them with the right credentials for your environment to try it out. Thanks.

It looks like your problem is that projects.delete_permission takes a list of rules, and delete_x_default_permissions takes only one rule. We should add typings for these methods; currently the only hint there is the names of the parameters:
server.projects.delete_permission(self, item, rules)
server.projects.delete_datasource_default_permissions(self, item, rule)

Either the documentation or the code is wrong. (and based on the implementation "update_permission", I guess it's the code)
The documentation clearly states:

Parameters

Name
Description

item
A project object.

rules
A list of PermissionsRule objects or a single PermissionsRule object.

While in the code, the method header does not use a "Sequence" when compared to the update method.
def update_default_permissions(
        self, resource: BaseItem, permissions: Sequence[PermissionsRule], content_type: Resource
    ) -> List[PermissionsRule]:
# vs
def delete_default_permission(self, resource: BaseItem, rule: PermissionsRule, content_type: Resource) -> None:

The docs were wrong. I have opened a pull request to correct the error.


restore should do both restore and restore --staged.

Since oaf papers over the distinction between staged and unstaged changes, restore should undo both.

Due to being implemented via checkout, this is already done.


How to create and retrieve the query generated by the query builder

Just checked the demo of query builder at https://zebzhao.github.io/Angular-QueryBuilder/demo/
I have a query as how the query gets generated in the backgroud with the various filters selected and how can we retrieve this generated query.
I need to save the generated filtered query in my DB which will further be used as where clause.

See the plunker demo .. although it not working but you can see the code to retrieve this generated query.


river format the repo

using ./util/agentfmt.sh

looks like some more content got merged to main so I'll close this and create a new PR formatting the latest on main.


xarray, chunking and rolling operation adds chunking along new dimension (previously worked)

I was testing the latest version of xarray (0.12.3) from the conda-forge channel and this broke some code I had.  Under the defaults installation not using conda-forge (xarray=0.12.1),  the following code works correctly with desired output:
Test code
import pandas as pd
import xarray as xr
import numpy as np

s_date = '1990-01-01'
e_date = '2019-05-01'
days = pd.date_range(start=s_date, end=e_date, freq='B', name='day')
items = pd.Index([str(i) for i in range(300)], name = 'item')
dat = xr.DataArray(np.random.rand(len(days), len(items)), coords=[days, items])
dat_chunk = dat.chunk({'item': 20})
dat_mean = dat_chunk.rolling(day=10).mean()

print(dat_chunk)
print(' ')
print(dat_mean)

dat_std_avg = dat_mean.rolling(day=250).std()

print(' ')
print(dat_std_avg)

Output (correct) with xarray=0.12.1 - note the chunksizes
<xarray.DataArray (day: 7653, item: 300)>
dask.array<shape=(7653, 300), dtype=float64, chunksize=(7653, 20)>
Coordinates:
  * day      (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01
  * item     (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299'

<xarray.DataArray '_trim-8c9287bf114d61cb3ad74780465cd19f' (day: 7653, item: 300)>
dask.array<shape=(7653, 300), dtype=float64, chunksize=(7653, 20)>
Coordinates:
  * day      (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01
  * item     (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299'

<xarray.DataArray '_trim-2ee90b6c2f29f71a7798a204a4ad3305' (day: 7653, item: 300)>
dask.array<shape=(7653, 300), dtype=float64, chunksize=(7653, 20)>
Coordinates:
  * day      (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01
  * item     (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299'

Output (now failing) with xarray=0.12.3 (note the chunksizes)
<xarray.DataArray (day: 7653, item: 300)>
dask.array<shape=(7653, 300), dtype=float64, chunksize=(7653, 20)>
Coordinates:
  * day      (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01
  * item     (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299'

<xarray.DataArray (day: 7653, item: 300)>
dask.array<shape=(7653, 300), dtype=float64, chunksize=(5, 20)>
Coordinates:
  * day      (day) datetime64[ns] 1990-01-01 1990-01-02 ... 2019-05-01
  * item     (item) object '0' '1' '2' '3' '4' ... '295' '296' '297' '298' '299'

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
...

ValueError: For window size 250, every chunk should be larger than 125, but the smallest chunk size is 5. Rechunk your array
with a larger chunk size or a chunk size that
more evenly divides the shape of your array.

Problem Description
Using dask + rolling + xarray=0.12.3 appears to add undesirable chunking in a new dimension which was not the case previously using xarray=0.12.1  This additional chunking made the the queuing of a further rolling operation fail with a ValueError.  This (at the very least) makes queuing dask based delayed operations difficult when multiple rolling operations are used.
Output of xr.show_versions() for the not working version

INSTALLED VERSIONS
------------------
commit: None
python: 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 22:01:29) [MSC v.1900 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 23 Model 113 Stepping 0, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
libhdf5: 1.10.4
libnetcdf: 4.6.1
xarray: 0.12.3
pandas: 0.25.1
numpy: 1.16.4
scipy: 1.3.1
netCDF4: 1.4.2
pydap: None
h5netcdf: None
h5py: 2.9.0
Nio: None
zarr: None
cftime: <IP_ADDRESS>
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.2.1
dask: 2.3.0
distributed: 2.3.2
matplotlib: 3.1.1
cartopy: None
seaborn: 0.9.0
numbagg: None
setuptools: 41.0.1
pip: 19.2.2
conda: 4.7.11
pytest: None
IPython: 7.8.0
sphinx: None

Apologies if this issue is reported, I was unable to find a case that appeared equivalent.

Yikes. Thanks @p-d-moore .
Looks like the rolling mean changes the chunks
>>> print(dat_chunk.chunks)
((7653,), (20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20))

>>> print(dat_mean.chunks)
((5, 7648), (20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20))

A workaround is to do dat_mean = dat_mean.chunk({'day': None}) and then calculate the rolling std

Some additional notes:
The bug appears for xarray=0.12.2 as well.  Other rolling operations are affected as well - replace .mean() in the above code with .count(), .sum(), .std(), .max() etc results in the same erroneous chunking behaviour.
Another workaround is to downgrade xarray to 0.12.1

Confirm bug is still present in 0.13.0.

'''
Note the chunksize is now (20, 7648) instead of (20, 7653).
I believe it might be related to https://github.com/pydata/xarray/pull/2942 -> I think disabling bottleneck for dask arrays for the rolling operation caused the bug above to appear (so the bug may have been there for a while, but doesn't appear because bottleneck was being used).
Doing a quick trace, in rolling.py , I think it's the line windows = self.construct(rolling_dim) in the reduce function which creates windows with incorrect chunking, possibly as a consequence of some transpose operations and dimension mix up?
It seems strange that other applications aren't having problems with this, unless I am doing something different in my code?

Error still present in 0.14.0
I believe the bug occurs in dask_array_ops.py: rolling_window
My best guess at understanding the code:  I believe there is an attempt to "pad" rolling windows to ensure the rolling windows doesn't miss data across chunk boundaries.  I think the "padding" is supposed to be truncated later, but something is miscalculated and the final array ends up with the wrong chunking.
In the case I presented, the "chunking" happens along a different dimension to the "rolling" and padding is not necessary.  Perhaps something goes haywire because the code was written to guard against rolling along a chunked dimension (and missing data across chunk boundaries)?  Additionally, as the padding is not necessary in this case, there is a performance penalty that could be avoided?
A simple fix for my case is to not do any "padding" whenever the chunksize along the rolling dimension is equal to the arraysize along the rolling dimension.
e.g.  in the function dask_array_ops.py: rolling_window
pad_size = max(start, end) + offset - depth[axis]

becomes
if a.shape[axis] == a.chunksize[axis]:
    pad_size = 0
else:
    pad_size = max(start, end) + offset - depth[axis]

This fixes the code for my usage case, perhaps someone could advise if I have understood the issue correctly?


feat: make the archive resource change when its content change

I don't think that this is so much releavant since #521 was merged.


Support for multiple values in defaults

This is a really great change, thanks!
My only real question is whether \n is a typical separator for environment variables in this situation. It seems like that could cause issues on Windows. Maybe default it to os.PathListSeparator?

If you can fix up a couple of the comments (use ellipsis in help, and add docs to the Envar method, I'd be happy to merge!

Regarding the separator for environment variables, I am afraid os.PathListSeparator might be breaking some existing apps that would require to escape it to keep their single-value default (which is still 99% of the use cases IMHO and that we shouldn't complicate). I thought newline was a more exotic value for flag values in a CLI and therefore settled on that.
I agree windows might be an issue, I'll try to follow https://golang.org/pkg/bufio/#ScanLines's approach instead of my naive \n splitting. Would that be OK for you?

commits amended, inline comments

Looks good, thanks!

I was about to push again but you were quick :) it seems that flags.formatPlaceHolder() is dead code, that's why I didn't add ellipsis support there after your review


Revise Questions and Content on Parent Tab  (FT only)

@all This requires UX Designer updates how can we handle this?
Current Behavior

Student application identifies student as dependant, and prompts the 'parent' tab on application form. The Following questions appear:
"Is your parent(s) a resident of BC"? (if 'no' is selected, no exception is triggered)
Does your parent(s) have valid SIN? (if 'no' is selected, student self-reports their parental income)
How many parents do you live with? (the number selected determines how many slots of parents info are required to progress app)

Problems:

If student is a dependant and parents are not residents, student should not be eligible for funding. Therefore selecting 'no' should trigger an exception path.
Question suggests BOTH parents have to be resident, but that's not the case. Only one parent needs to be resident to satisfy policy requirements.
Ordering of questions and wording of questions is confusing

Expected Behavior:

Student application identifies student as dependant, and prompts the 'parent' tab on application form. Questions should appear as follows, and be re-worded to reflect the following:
"How many parents/sponsors/legal guardians/step-parents support you financially?" (This is what will dictate how many parent info slots are required to be filled out by parents OR through the self-report feature.
"Is at least one of your parents/sponsors/legal guardians/step-parents a resident of B.C.?" (If 'no' is selected for this question, the exception path MUST be triggered)
"Are each of your parents/sponsors/legal guardians/step-parents eligible for a B.C. Services Card?". Help text should read "All applicable parents/sponsors/legal guardians/step-parents must provide their information to StudentAid BC by logging in with a B.C. Services Card. If they are ineligible for a B.C. Services Card (for e.g. if they do not have a valid SIN), select 'no' to this question and provide information on their behalf." If 'no' is selected to this question, add a banner message which reads "If parents cannot access a B.C. Services Card, the student will need to report parental information on their behalf. Please be aware this information will be subject to review by StudentAid BC and may delay a student's application"

Private Zenhub Image
Acceptance Criteria

[ ] Re-configure questions consistent with the above screenshot

[Uploading... Parent Tab.JPG]

@JerPearson Feel free to close if you think your FT app review has covered the concerns on the ticket.

Closing ticket as work captured already on other tickets.


style: try running black

Just to get the ball rolling.
Tried to run black 20.8b1 21.5b1 with the configuration
[tool.black]
    target-version = ["py35"]
    skip-string-normalization = true
    line-length = 88

The skip-string-normalization avoids the conversion ' to " if we want to keep the current flake8 rules
TODO:

[x] re-run with line length 88
[x] cleanup string concatenation in the same line (like 'foo bar ' 'baz foobar')
[x] review where we need to add / remove tailing commas to improve the styling
[ ] update the documentation with the changes done to the example folder

Closes #1453

I've updated the change with the latest black.

The default of 88 is fine, and avoids excessive line wrapping esp. for deeply nested control structures. If we increased the limit much further I would argue for a separate standard for keeping comments shorter (even if we have to write our own checker for it).
Regardless, I'll take a look through the diff soon and see how things look now.

I'll re-reun with the 88 line.
If you are not familiar with black, note that you can tell it to split things in multiple lines by setting a comma at the end.
Like
def foo(x,):
  pass
# becomes
def foo(
  x,
):
  pass

So we may need to add / remove some to get the desired formatting.
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html?highlight=comma#the-magic-trailing-comma

I'm a bit orthodox when it comes to line length, 100+ sounds a bit too long.
But I'm fine with black's 88, it does sound like a good compromise, and... if we're going to use black, we don't need to care about manually keeping lines at 79 or 88 anyway, that's what black is for.

Ok, I initially went with 100 both because we do have some lines that are on the long side in the source and it's what I usually use.
I'll re-run with the default 88 and update this.

Updated with 88. From the change lines it seems that our average line was a bit longer, since this version adds a 1000 new lines.
For reference the previous with length 100 was more or less a wash regarding line count

I'll wait for https://github.com/falconry/falcon/pull/1709 before rebasing and polishing this

@kgriffs @vytas7 I've updated the docs to use the new formatting, bug I'm not sure I updated all of them.
I've also made a quick pass to the files to check that no funny formatting was added. For this I think we can correct eventual leftover in the future

And a more general question: should we try updating Cython code for the delimited stream reader etc? When I wrote it, it was loosely based on the Python version, with changes specific to Cython.

black does not support cython, so the formatting would be manual.
Personally I think we could leave it as is or update as needed as a follow on change (maybe in the same issue as the one setting flake line length to 88)

I've opened https://github.com/falconry/falcon/issues/1926 to do the follow up on this change.
Please add there if there were other deferred points from this change. I'll try to get to it this we


[EG] devices required/optional

all required except moduleID

API change check
APIView has identified API level changes in this PR and created following API reviews.
Microsoft.ApiCenter


Extra Utilities 2 Support: Quantum Quarry crashes game

I thought I'd add this here. I'm not sure if other mods try to generate chunks for a world that doesn't exist, but it might be worth nothing:
https://pastebin.com/RJ3Jtz7N
Extra Utilities 2, version 1.12-1.7.2 from CurseForge.
MalisisCore 6.3.0.
CubicChunks 809 (compiled from git on Wed 17 Jan 2018).
Again, I'm not sure how relevant it is, but I'm not sure how many other mods implement a "miner" that grabs ores from another dim (if that is the problem).
I can confirm that the same versions work if CubicChunks is disabled.
Edit Thu 18 Jan 2018: I used the upload log function in MultiMC to get another log: https://paste.ee/p/sLLZn
This one happens at y 69, not below y = 0.
Also, note that an RF source must be used to start the quarry; I used the infinite one from EU2

Is this still the case?

Closing for now. If it's still an issue, report to compatibility reports page.


IPv6 compatibility

To be compatible with IPv6-only networks (such as Thread), maybe consider changing the hardcoded NTP pool address to the NIST '2' pool (2.pool.ntp.org)? NIST currently only serves AAAA records for pool 2.
http://www.internetsociety.org/deploy360/resources/ipv6-time-servers-ntp/

Closed via #2


Switching to log4j2.

Description of changes:

Switching the logging backend from log4j to log4j2
Had to include a new log4j2.properties file as the upgrade isn't 100% backwards compatible

I also had a question as to if we should keep the log4j runtime configuration stuff via DP config, as it seems we're already passing in a log4j.properties file as part of the startup command - figured we can discuss that here.
I'll go back and clean up some of the docs around the old log4j stuff once this is merged.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Approving, on the condition that we manually tested that the log level configuration still works. For now, the only way we have of verifying the log level configuration is by setting the log level via config, and looking at the log file or console to see that the logging is correct. We can chat offline about how to do that if you like

Discussed offline, tested the log4j2.properties file to adjust log thresholds. Will likely remove the "log configuration via data-prepper.yml feature" as users can already just pass in their modified .properties file. WIll be a separate PR with documentation cleanup.


Database Deadlock

03-22 17:11:14.797  11546-25818/at.... W/SQLiteConnectionPool﹕ The connection pool for database '.../databases/MyDatabase.db' has been unable to grant a connection to thread 26636 (Thread-26636) with flags 0x2 for 30.001001 seconds.
    Connections: 0 active, 1 idle, 0 available.

I sometimes get a deadlock when working on the database on the same time with different threads.
When writing bulk data I do something like this often:
    SQLiteDatabase db = Ollie.getDatabase();
    db.beginTransaction();
    try {
        ...
        // saving operations
        db.setTransactionSuccessful();
    } finally {
        db.endTransaction();
    }

Is there a way to prevent this deadlock?

It seems that the deadlock appears when I execute a Model.save() operation while another thread executes a transaction.
Thread A: execute transaction (like the one I wrote in the first comment)
Thread B: someModel.save()
Other transactions and queries are queued/blocked after the transaction in Thread A and don't produce a deadlock (problem lies only with Model.save()).
It seems that I got rid of it by wrapping the model and execute every save() operation in a transaction. (I'm not 100% sure, maybe I just got another timing by this approach so that my testcases won't trigger the deadlock anymore, I'm still new with sqlite)

So here is a testcase where I get a deadlock every time. Am I doing something wrong here?
    // Just getting some test data (10000 entries with name test)
    final List<Measure> measures = Select.from(Measure.class).where(Measure.NAME + "=?", "test").fetch();

    // Thread1, write in transaction
    new Thread(new Runnable() {
        @Override
        public void run() {

            SQLiteDatabase db = Ollie.getDatabase();
            db.beginTransaction();
            try {
                Timber.d("Start writing in thread1");
                for (int i = 0; i < 10000; i++) {
                    Timber.e("writing: " + i);
                    Measure measure = new Measure();
                    measure.mName = "kek";
                    measure.save();
                }
                db.setTransactionSuccessful();
            } finally {
                db.endTransaction();
            }
            Timber.e("End writing thread");
        }
    }).start();

    // Thread2, write without transaction
    new Thread(new Runnable() {
        @Override
        public void run() {
            Timber.d("Start, writing thread without transaction --> Thread2");
            Timber.d("Thread2, measure size: " + measures.size());
            for (int i = 0; i < measures.size(); i++) {
                if (i % 100 == 0) { // do only some writes here, no reads
                    Timber.d("database access" + i);
                    measures.get(i).mName = "kek";
                    measures.get(i).save();
                    Timber.d("finished database access" + i);
                }
            }
            Timber.d("Writing Thread2 finished");
        }
    }).start();

Deadlock appears when Thread2 starts a saving operation and Thread1 makes a few saving operations in between. Then Thread2 can't finish its saving operation (finished database access isn't shown) and both threads are stuck with the message, that the database is unable to grand access to thread2.
Is there anything unexpected in my code?

Ok strange, nobody working with transactions here?
I fixed the issue simply by locking the transaction with Ollie.class. The code looks like this (inspired by how sugar orm makes transactions)
public class TransactionHelper {
	public static void doInTransaction(Callback callback) {
		synchronized (Ollie.class) {
			SQLiteDatabase db = Ollie.getDatabase();
			db.beginTransaction();
			try {
				callback.manipulateInTransaction();
				db.setTransactionSuccessful();
			} finally {
				db.endTransaction();
			}
		}
	}

	public interface Callback {
		void manipulateInTransaction();
	}
}

Maybe I will create a PR.


move the minutia of build into the Makefile

Makefile updates

Leaned heavily into make target dependencies to build up to producing the ZIP file for the layer.

Added a 'make test` target for convienence in dev. The CircleCI config uses a go orb's default for test execution.

Includes comments for targets that produce help text when used with remake:

» remake --tasks
artifacts            where release artifacts will be built
artifacts/extension.zip the extension's zipfile for publishing to Amazon
artifacts/extensions where the contents of the extention will be gathered
artifacts/extensions/honeycomb-lambda-extension the extension's executable
build                compile the extension executable
clean                clean up from builds and tests
publish_aws          publish the extension as a lambda layer to a specified AWS_REGION
test                 run tests with coverage report
zip                  package up the extension in a ZIP file

WIP: fan out lambda-layer publishing in CI?
Kind of neat to make Circle do all the publishing in parallel. Not 100% sure it's better.
It takes about 1 minute for the aws-cli orb to spin up and configure itself. Then it takes about 2 minutes total to publish the extension to all listed regions in a loop.
Possible advantages to publishing fan out:

slightly faster because parallelism?
look up a region's ARN by looking at the output in the job with a name
for that region

Possible disadvantages to fan out:

not really that much faster
all the resultant ARNs in one job's output makes it easy to scan and
check that the regions have a consistent layer version at the end

@robbkidd is this still a draft?

Moving to draft until it's ready for review 👍🏻

Closing per @robbkidd 's suggestion


interval() accumulates error

Bug Report
Current Behavior
Let's say I use the interval() function to generate an event every S seconds, and then take N of these events. The last event will be delivered much later than S*N, because the error (inherent how async events are delivered) is accumulated across all intervals.
The interval() function doesn't take into account that the async scheduler calls the callback with a slight delay, and doesn't adjust the next period accordingly.
Reproduction
The following observable should deliver the last event after ~1000s, but depending on how busy the system is, it may happen at much later time (in the order of dozens of seconds later).
interval(1000).pipe(take(1000))

Expected behavior
The interval() function should take into account the delay between the desired delay and actual delay and adjust the next interval accordingly.
Environment

Runtime: Node v12
RxJS version: 6.5.3

I believe there was similar q somewhere but failed to search. @benlesh can double confirm - afaik this is expected, we try best to put an action in the queue with properly specified time but do not guarantee each action's margin bubbles up into queue execution time.

That makes interval() more like a recursive / recurring setTimeout() instead of a setInterval()… I think that is unfortunate since this makes it impossible to implement an accurate timer.

It should not be the case that the next period needs any adjustment. All of the async functionality is implemented using setInterval and when the period/duration is unchanged, clearInterval is not called and new interval is not created:
https://github.com/ReactiveX/rxjs/blob/5bbf0aae880052fc676868c3be716f7888c036a4/src/internal/scheduler/AsyncAction.ts#L77-L79
So the behaviour should be identical to using setInterval alone.

Alright. I'll investigate my code closer. I've previously had code in the form of interval(1000).pipe(take(1000)) and I've changed it to something like this and it solved any unwanted delays:
merge(...Array.from({ length: N }).map((_, t) =>
  interval(((t + 1) * 1000).pipe(take(1), mapTo({ type: "VCLOCK" }))
)

Essentially generate N events, each emitted at a different interval.

Ok, it appears to be an issue with setInterval as well. I created this script which tests the drift of a single setTimeout, a setInterval which counts to the end, and an observable which uses interval().
const { interval } = require("rxjs");
const { endWith, mapTo, take } = require("rxjs/operators");

const HZ = 10;
const timeLeft = 60;

const obs = interval(1000 / HZ).pipe(
  take(timeLeft * HZ),
  mapTo({ type: "VCLOCK" }),
  endWith({ type: "END" })
);

const t0 = Date.now();

function log(s) {
  const t = Date.now();
  console.log(`[${((t - t0) / 1000).toFixed(3).toString().padStart(8)}]`, s);
}

log("start");
setTimeout(() => {
  log("setTimeout end");
}, timeLeft * 1000);

let counter = 0;
setInterval(() => {
  if (counter++ === timeLeft * HZ) {
    log("setInterval end");
  }
}, 1000 / HZ);

obs.subscribe(ev => {
  if (ev.type == "END") {
    log('observable interval end');
  }
});

The output indicates that setInterval and observable interval are delayed by almost two seconds:
[   0.000] start
[  60.016] setTimeout end
[  61.831] observable interval end
[  61.932] setInterval end

Js timer is not high resolution timer from beginning. Not sure about diff though.

Closing as this occurs in bare js.


About Apiparams a little suggest of swagger

Sometime,When I define an interface parameter.I found that some fields of the entity class are not required.like this ↓
@RequestMapping(value = "/update", produces = {"application/json;charset=UTF-8"})    public void update(@RequestBody Book param) {}
But I only need a few fields in the entity,How do I hide the other fields.Do I have to define a new entity? That's a hassle.

@poppycokey yes atleast until we get #2056 merged.


add Drives Endpoints

Description
This holds the spec for https://github.com/owncloud/ocis/blob/de36293a72dd552ccb37a035926fb2a84b425ad5/docs/adr/0005-api-for-spaces.md

@refs @butonic @kulmann
according to the spec we should add a response 401 when the user doesn't provide a bearer token.

@butonic @dragotin IMO this could be merged now. Please review again.
@refs I removed the title atribute completely. I found no other public resource using it. For me it works perfectly without it.


Add support for 'org.apache.commons:commons-dbcp2:2.9.0

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

No.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Add support for 'org.apache.commons:commons-dbcp2:2.9.0.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Null.

Additional context
Add any other context or screenshots about the feature request here.

Metadata generated at https://github.com/linghengqian/graalvm-trace-metadata-smoketest/issues/1 .

Linkes to https://github.com/apache/shardingsphere/issues/21347 .

I'm working on this issue.

A strange behavior was reported at https://github.com/oracle/graal/issues/5913.


Limit Concurrent Builds

Is there a way to limit concurrent builds?  We are currently on a resource constrained server and have run into some issues when a dev pushes several commits at once.
Also is there a better place for community support?  Not sure this is ticket worthy, but I didn't see any forums, etc.

I think this is the default behavior now: https://github.com/Block8/PHPCI/blob/master/PHPCI/Command/RunCommand.php#L104

@SamMousa It seems, but sometimes it fires multiple times at a time.
Yesterday I was doing some tests using a local phpci and I tried to add multiple builds and tried to execute the phpci:run-builds multiple times at a time. Every time I've tried to run the phpci:run-builds it showed me a notice like "There's a currently running build".
Today I've updated phpci in my office server and after some commits there has been one push that triggered four builds at a time. A part from the fact that the server slowed down too much, some of that builds started showing this error:
fatal: destination path '/var/www/phpci/PHPCI/build/50' already exists and is not an empty directory.

Failed to clone remote git repository.
Exception: Could not create a working copy.

Probably is another issue, but started showing with this huge build.
In my case I'm using the normal cron for phpci:run-builds instead of the daemon.

As said in my previous comment:

I think that I can confirm that when it starts making such a lot of builds, builds coming later crash with the following error(s):
fatal: destination path '/var/www/phpci/PHPCI/build/70' already exists and is not an empty directory.

Failed to clone remote git repository.
Exception: Could not create a working copy.

This is common too:
Cloning into '/var/www/phpci/PHPCI/build/78'...
error: cannot run /var/www/phpci/PHPCI/build/78.sh: El fitxer o directori no existeix
fatal: unable to fork

Exception: Warning: unlink(/var/www/phpci/PHPCI/build/78.key): No such file or directory in /var/www/phpci/PHPCI/Model/Build/RemoteGitBuild.php line 108

The sentence El fitxer o directori no existeix means "The file or folder does not exist".
After crashing this way, if I trigger a manual build everything seems to work as expected.

Do not start several run-builds at all. In the current state of the code, run-builds fetches a list of builds to run at the very start of its execution and try to build them all without refreshing its data between each build. Several run-builds in parallel will stomp on each other toes, trying to build the same jobs at nearly the same time.

And that's why it crashes?
What's the solution then? Create a custom script like described by @robertscherer?

That's it.

@elboletaire: also, if the flock command is available, you can try this (much simpler):
flock --exclusive --nonblock /var/run/phpci.lock /var/www/phpci/console phpci:run-builds >> /var/log/phpci.log

This will exclusively lock the file /var/run/phpci.lock during the execution of phpci:run-builds, silently failing if the file is already locked (i.e. another instance of phpci:run-builds is running).

That's a little better.

Why not use a true task scheduler instead of trying to reinvent the wheel?
https://github.com/illuminate/queue
For example would allow multiple runners relatively easy.
Also, there could be some pruning in each build, for example:

Build task starts.
Build task checks if the commit it must be run against fits the requirements (like "there must not be a commit on this branch, after the current commit, that has a build finished or in queue")
Build task runs

Since step 1 and 2 are fast it can skip task 3 if 2 is not ok; this will keep it simple and makes it so very little actual queue management is required.


removed 'tips' prefix

Docs Build status updates of commit de0d0e0:
:clock10: Full build: the average full build time is 8 min(s) 41 sec(s), this is based on the last 879 build(s) for this repository.

Docs Build status updates of commit de0d0e0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/use-cli-effectively.md
:bulb:Suggestion
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)
Details

docs-ref-conceptual/TOC.yml
:white_check_mark:Succeeded
View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid) View (azure-cli-latest)

docs-ref-conceptual/use-cli-effectively.md

Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 121: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/overview?toc=./ref/toc.json&view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/resources/application?view=graph-rest-1.0 NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.
Line 123: [Suggestion-preserve-view-not-set] Did you mean to link to a specific version? If so, add &preserve-view=true to the URL: /graph/api/application-update?view=graph-rest-1.0&tabs=http NOTE: This Suggestion will become a Warning in early December, 2020.

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


WIP: 500 error in VTN when VEN creates a single report

Hi Stan,
I discovered a bug in Openleadr's XML-to-object behaviour. The error path is approximately this:

My VEN client had only a single report (created with my_OpenADRClient.register_reports(report=[...]))
This gets correctly turned into XML and sent to the VTN.
On the VTN, xmltodict.parse turns that single report entry (in the <oadr:oadrPendingReports> tag) into a dict with a bare string in it -- if there were more entries, it would produce a dist with a list of strings. (Arguably, this is the source of the error ...)
openleadr.util.normalize_dict fails to normalize that structure, because it only expects a dict with a list of strings. (... but this is the easiest place to fix it.)
Further down the line, the VTN's openleadr.service.report_service.ReportService.on_created_report blows up and we get a 500 error.

For now, our workaround is to register a second, dummy, report on our VEN.
About this PR:

First commit in this PR adds the failing test.
Second commit in this PR contains one possible fix -- using glom, a library that specializes in extracting data from nested data structures.

Motivation:

The error is essentially a schema error: xml2dict.parse's result has a different schema depending on if there are 0, 1, or more pending reports, and then openleadr.util.normalize_dict only handles one of those schemas.
Python is really bad at checking a nested data structures: it forces you to write error-prone code like if isinstance(d[key], dict) and 'report_request_id' in d[key] and isinstance(d[key]['report_request_id'], list).  (Footnote: Python 3.10's structural pattern matching is perfect for extracting data from nested structures, and handling multiple cases. But I figure you want to keep supporting Python<=3.9 for a while longer.)
So a library that makes it easy to check and use schemas will make openleadr.util.normalize_dict easier to maintain. For this WIP proposal, I chose glom. (Another option might be schema might also be an option?)

Choices from here, depending on how you feel about Glom:

"Not feeling it": Rewrite the pending_reports schema checks using plain python x in y and isinstance(y[x], mytype) and ...
"Looks nice enough": Accept this PR, including Glom, as is; leave the rest of normalize_dict, we can use glom if more bugs crop up
"Let's try a few more cases": First (someone, probably me) rewrites the other ~16 transformations in normalize_dict to also use Glom where it makes sense. That's not much work.

"Let's get radical": Fix this by solving #17 .

Any thoughts?
Cheers,
Sietse

Hi Sietse,
Thanks so much for the detailed analysis, useful suggestions, and a working test and fix.
As you noted by referencing #17, I was already not very fond of the xml-to-object functionality and the normalization. It started out as a good idea to make the unwieldy OpenADR / EiInterop schemas more user friendly (which I think it does), but it is a little error-prone and inefficient.
For the moment, I added another manual check that report_request_id is in fact a list, and then turn it into a list if it is not, like I did for some of the other proprties that undergo the normalization. Mostly because I don't (yet) know Glom or Schema, but it looks like an elegant and more expressive way to get the data in the correct shape.
(In the process of thinking about this problem, I found another possible failure in this code (which your fix does not yet address, I think): if the report_request_id is strictly numeric (like "123"), it will get turned into an int instead of a str. That should now be fixed as well.)
I will be looking into some of the alternatives that you mentioned, and I'd love to hear your suggestions on #17 if you have any (and on other areas of improvemen for OpenLEADR).
Thanks again, hopefully we'll speak again.


Should make full use of ASM9

The project already depends on ASM9, but instructs ASM to only use its capabilities from version 8: https://github.com/autonomousapps/dependency-analysis-android-gradle-plugin/blob/e8879cb762c6c795b628b2ac41c2e8f2e63f00cf/src/main/kotlin/com/autonomousapps/internal/asm.kt#L20.

what is the advantage to making such a change?

Support for new Java features like sealed classes.


Migrate to baseline profile plugin

ℹ Running the default gradlew :app:generateProdReleaseBaselineProfile task leads run a Gradle error where it says the target app is not installed on the device 🤯
Unfortunately I did not saved the stacktrace.
But running gradlew :app:installProdRelease :app:generateProdReleaseBaselineProfile seems to have fixed it.

@keyboardsurfer can you share with us what is the expected run configuration for this project?
BuildType/Flavor:

ProdRelease: unable to reproduce since production API keys are not provided here (although we can extract them from the PlayStore apk) and therefore the listings would be empty
DemoRelease: not necessarily representative since it does not go through the whole network stack (data is fetched from a local JSON file).

Device:

pixel6api31aosp: anyone can "reproduce" since it's a Gradle Managed Device, but it comes with its own downsides that can be ignored with testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "EMULATOR":
Click to show warning message
/**
 * ERROR: Running on Emulator
 *     Benchmark is running on an emulator, which is not representative of
 *     real user devices. Use a physical device to benchmark. Emulator
 *     benchmark improvements might not carry over to a real user's
 *     experience (or even regress real device performance).
 *
 * While you can suppress these errors (turning them into warnings)
 * PLEASE NOTE THAT EACH SUPPRESSED ERROR COMPROMISES ACCURACY
 */

Custom rooted device: not able to reproduce (for obvious reasons)

It would be really great if it could be reproduced on CI builds before releases.

Although the BaselineProfileGenerator test succeeded, I got the following error:
FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':benchmarks:collectDemoNonMinifiedReleaseBaselineProfile'.
> kotlin.io.NoSuchFileException: nowinandroid\benchmarks\build\intermediates\managed_device_android_test_additional_output\demoNonMinifiedRelease\pixel6api31aospDemoNonMinifiedReleaseAndroidTest\BaselineProfileGenerator_generate-baseline-prof-2023-07-28-19-46-26.txt: The source file doesn't exist.

Could this be related to running baseline profile tests on emulator?

Re #issuecomment-1656245386: This means the baseline profile output file wasn't found after the test was created. Can you try again with saveInSource disabled? If that works we might have to file a bug against the plugin.
You can create a Baseline Profile on an emulator and should not see any errors along the way.
Measuring its effectiveness requires a physical device. The "running on emulator" error above should only pop up when you want to measure performance.
Paging @dturner for advice on the variant to build.

I guess you made huge progress on #880 making this PR a bit outdated 🙈

Thank you for submitting this pull request. We have decided not to accept it at this time.
Feel free to comment or open an issue for further discussion if necessary.


Unable to request whole CompleteState on the ControlInterface

Currently, it is not possible to fetch the whole CompleteState over the ControlInterface. On the workload site the empty CompleteState is received (workloads: None, workloadStates: None).
Current Behavior

Empty CompleteState received over the ControlInterface when requesting the whole CompleteState.
Expected Behavior

A workload shall be able to receive the whole CompleteState shall be received over the ControlInterface.
Steps to Reproduce

Navigate to the control interface example folder inside the repository: cd examples
Modify the create_request_for_complete_state function body inside the python_control_interface example (adjust the filterMask to include the desiredState):

    return control_api.ToAnkaios(
        request=ank_base.Request(
            completeStateRequest=ank_base.CompleteStateRequest(
                fieldMask=["desiredState", "workloadStates"]
            ),
            requestId=REQUEST_ID,
        )
    )

Change the permissions of the startConfig.yaml file to allow the workload to request the whole CompleteState:

...
        - type: StateRule
          operation: Read
          filterMask:
            - "*" # change to wildcard to allow full read access
...

Run the example: ANK_BIN_DIR=/workspaces/ankaios/target/x86_64-unknown-linux-musl/debug ./run_example.sh python_control_interface
Check the log output of the workload:

podman logs -f $(podman ps -a | grep control_interface | awk '{print $1}')

You should see an empty CompleteState:

Context (Environment)

Release v0.4.0
Linux
All Control Interface examples and all workloads requesting the whole CompleteState over the Control Interface are affected
Logs

After checking the server logs there is an issue with the tagging when using flatten and enums:

Like in the picture the error string is a serde_yaml conversion error untagged and internally tagged enums do not support enum input
Additional Information

Final result

To be filled by the one closing the issue.

I have analyzed the issue and I have found an issue opened on the serde_yaml part having problems since serde_yaml = "0.9" where the tagging was introduced: https://github.com/dtolnay/serde-yaml/issues/344
I know that @krucod3 was discussing that with @christoph-hamm and I know that @krucod3 left some comment in #22 about an fix to use the old still supported singleton_map_recursive serializer.
I have tried to search the code base if the old singleton_map_recursive serializer fix is already included inside the Authorization changes of #301, but it is not.
I tried to annotate each struct using enums in containers (part of the desiredState field) with the singleton_map_recursive annotation (like shown in the comment of @krucod3), but no combination has fixed the issue.
Next, I downgraded serde_yaml to version 0.8 and the problem is resolved. So, it is indeed the tagging introduced with serde_yaml >= 0.9.
But downgrading is not a real fix, so I decided to check if the annotations with singleton_map_recursive would fix the bug with version 0.9 when trying out the example code of the issue https://github.com/dtolnay/serde-yaml/issues/344.
And indeed when putting the singleton_map_recursive not on a container but on the field inside a struct using the enum which causes problems in combination with the serde flatten option, then the issue is resolved.
use serde::{Deserialize, Serialize};

fn main() {
    // this works
    let gender = Gender::Other("F22".into());
    assert_eq!(
        &gender,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&gender).unwrap())).unwrap()
    );

    // this still works
    let info = Info { gender };
    assert_eq!(
        &info,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&info).unwrap())).unwrap()
    );

    // this errors but works now because of "serde_yaml::with::singleton_map_recursive" below inside Gender struct
    let user = User { info };
    assert_eq!(
        &user,
        &serde_yaml::from_str(&dbg!(serde_yaml::to_string(&user).unwrap())).unwrap()
    );
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
enum Gender {
    Female,
    Male,
    Other(String),
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
struct Info {
    #[serde(with = "serde_yaml::with::singleton_map_recursive")]
    gender: Gender,
}

#[derive(Clone, Debug, Eq, PartialEq, Deserialize, Serialize)]
struct User {
    #[serde(flatten)]
    info: Info,
}

Now, I tried to put the annotation not on containers like suggested in #22 but instead in other places on field members to get it fixed like in the example code of the other issue above. But no success, still the same issue.
Then, I decided to migrate from serde_yaml to serde_json because in the issue it was mentioned that serde_json does not have this problem and indeed, it works. There are a few modifications on the production and test code necessary, e.g. we must migrate the code of common/src/state_manipulation/object.rs to use serde_json instead of serde_yaml and we must change two places in the CLI code expecting now an serde_json::Error. But at the end the issue is resolved with that. The workload now gets the whole CompleteState and the tagging issue is not inside serde_json.
@krucod3: I have prepared a branch (sub-branched from release v0.4.0) with the production code changes to migrate internally to serde_json for the filtering instead of serde_yaml, since for me it looks like the serde_json is more stable and does not have the tagging problem that affects us. Please have a look if you want to migrate to the serde_json or if you want to search for an alternative solution.

@krucod3: After some additional investigation I found out that the allow access rule rights are deserialized differently:
The problems are th !stateRule internal tags inside the sequence. The first line in the screenshot is a part of the whole current CompleteState inside ServerState::get_complete_state_by_field_mask(...) (line 166). It's the proto object from the current complete state after the ".into()" call. The second line is the filtered complete state inside the variable return_state output just before the return_state.try_into() call that fails when converting back the filtered Object to proto. The object.rs uses serde_yaml and when doing the filtering and inserting mappings and values it uses the tag format with (!stateRule) which confuses the serde_yaml because of the mixed tag variants used when deserializing back to proto.

@krucod3: I have now also implemented a bug fix to keep the serde_yaml for parsing inside the object.rs. Like said in the previous comment, The conversion from proto::CompleteState to Object is different in serde_yaml = 0.9 compared to serde_yaml = 0.8. Tags are created.
The used serde_yaml::to_value inside the impl TryFrom<proto::CompleteState> for Object of object.rs does not use the recursive map parser. So, I have written custom code and invoked the serde_yaml::with::singleton_map_recursive::serialize manually.
And the problem is fixed, now I get the whole CompleteState and parts again when using the ControlInterface.
I will prepare another branch containing the fix for the serde_yaml parsing. @krucod3: Then you can compare the two variants and decide if we use serde_yaml oder serde_json in the future.

After this bug fix we need a new release (v0.4.1).
Since there are already changes scheduled for v0.5.0 on the main branch, we need the concept of release branches. It could happen also in the future that we are still working on the next release and have already merged some PRs and after that we receive a bug ticket causing a new bug fix release. Because of the  issue #347 and the lifetime exceeded issues on the embedded test certificates we need a release containing the following:

bug fix of the CompleteState filtering
bug fix of grpc failing itests #350

I have created the bug fix for the recursive map serializer of serde_yaml based on the v0.4.0 release tag 349_fix_complete_state. In addition, to unblock the CI/CD pipeline because of the failing grpc itests, I have cherry-picked the commits of #350 into the branch. The commits have only covered the itest section not production code, so I decided to go with that (and only 2 commits fixing the certificates inside the itests). With this prepared branch I have tested to create a release in my private fork and it works. The CI/CD pipeline checks out what is tagged in case of a new release build. So, we can tag release branches and build a release based on those branches as well. And with the tag it is of course traceable.
I have verified that all system tests are executed in the pipeline and I have also pulled the binaries with the tagged repo to execute the control interface examples locally to check if the whole CompleteState is still retrievable.
We still wait with the release until @krucod3 has looked over all the analysis and bug fixes. I just tested the procedure if it is possible to do releases with release branches in our current ci/cd setup.

I would prefer not to have explicit mentioning of yaml in the production code.
We can fix the representation of the ControlInterfaceAccess objects during the compilation of the proto derived Rust files.
Unfortunately the allow and deny rules were not wrapped in extra objects so I cannot properly fix the filtering of only allow and deny rules. This means that when one requests the allow rules only, also an empty deny rule would be provided. To fix this, an interface change in proto would be required, which we cannot do for a patch version update.
I opened a PR: #359

I would prefer not to have explicit mentioning of yaml in the production code. We can fix the representation of the ControlInterfaceAccess objects during the compilation of the proto derived Rust files.
Unfortunately the allow and deny rules were not wrapped in extra objects so I cannot properly fix the filtering of only allow and deny rules. This means that when one requests the allow rules only, also an empty deny rule would be provided. To fix this, an interface change in proto would be required, which we cannot do for a patch version update.
I opened a PR: #359

Sure, I agree, then it is also consistent with the other field attributes and so passed by attribute annotations.
I have already tested and reviewed the PR. Can be merged.

The problem described in the issue is fixed now, but we still don't have full-fledged automated test coverage for the filtering.
On the one hand manually written utests are hard to maintain and develop, on the other fuzz-testing is not deterministic.
As the topic is out of the scope of the current issue (related to future bug prevention), I created a separate issue: #360


HBASE-27720 TestClusterRestartFailover is flakey due to metrics assertion

This test has been flakey. When it is rerun by surefire, the underlying minicluster isn't completely cleaned. specifically, the metics system isn't reset. The result is an otherwise successful re-run is failed because there's 8 or 12 SCPcounts instead of the 4 that a single run of the test would otherwise produce.


Search for the FF's profiles along all the subfolders of Mozilla/Profiles/

core/pull-requests/1233


Integration test cases with chocolatey

Task Description
Strongbox has Nuget support which is why we are indirectly supporting choco.
We need to add some integration tests to strongbox/strongbox-web-integration-tests which cover all of the commands you would need to use in order to work with choco.
Tasks
The following tasks will need to be carried out:

[ ] Go over the documentation (PR preview; will be merged soon into master)
[ ] Write integration test for the commands:

[ ] Save API key in choco
[ ] Add Strongbox repository to Chocolatey package sources
[ ] Create a package
[ ] Make chocolatey Nuget package
[ ] Push Nuget package into Strongbox repository
[ ] Search for packages in Strongbox repositories
[ ] Install package using Strongbox source
[ ] Delete package

[ ] Update landing doc page to include Choco (after Nuget V2) in the Layout Providers section.

Task Relationships
This task:

Depends on: #1709
Relates to: #1685 #1113, strongbox/strongbox-docs#78

Useful Links

s-w-i-t with nuget
how to write tests
Chocolatey commands
hello-choco-example

Help

Our chat
Points of contact:

@carlspring
@fuss86
@sbespalov
@steve-todorov

This is temporarily on hold while I fix some issues around strongbox/ci-build-images#8 hopefully within the next few hours. @TheCakeIsNaOH would you be interested in helping out with writing the integration test case and finalizing strongbox/strongbox-examples#15 (if there is anything more to be done there) ? Sorry for the long delay, but we were simply blocked by chocolatey. :)

May I pick this.?

Please, proceed!


Integration with github and gitlab pages

I'm not sure how it works with GitHub, but on GitLab we have to create a docker image on which the CI of CadQuery projects will run.
Resources:

github pages doc
gitlab pages doc

Documentation should describe all required steps.

fixed in 0b84711