Bug: archivebox update and archivebox list are slow to start due to unnecessary calls to merge_links() when reading from disk

Describe the bug
On a large archive, archivebox update or archivebox list starts right away some CPU intensive process that takes a long time to complete.
Steps to reproduce

Have a large archive (I have 1800+ links)
archivebox update or list

Screenshots or log output
docker-compose run archivebox update
[i] [2022-12-21 23:02:15] ArchiveBox v0.6.2: archivebox update
    > /data

[▶] [2022-12-21 23:05:07] Starting archiving of 1847 snapshots in index...

The user is simply left waiting, and for the lack of an explanation, the user is asked of his faith. Fans are screeching, it could be an infinite loop. If the user waits, they would wait 3 minutes. If they kill the program, the suspicion of an infinite loop would be wrong: (who could blame them)
Traceback (most recent call last):
  File "/usr/local/bin/archivebox", line 33, in <module>
    sys.exit(load_entry_point('archivebox', 'console_scripts', 'archivebox')())
  File "/app/archivebox/cli/__init__.py", line 140, in main
    run_subcommand(
  File "/app/archivebox/cli/__init__.py", line 80, in run_subcommand
    module.main(args=subcommand_args, stdin=stdin, pwd=pwd)    # type: ignore
  File "/app/archivebox/cli/archivebox_update.py", line 119, in main
    update(
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/main.py", line 788, in update
    matching_folders = list_folders(
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/main.py", line 929, in list_folders
    return STATUS_FUNCTIONS[status](links, out_dir=out_dir)
  File "/app/archivebox/index/__init__.py", line 411, in get_indexed_folders
    links = [snapshot.as_link_with_details() for snapshot in snapshots.iterator()]
  File "/app/archivebox/index/__init__.py", line 411, in <listcomp>
    links = [snapshot.as_link_with_details() for snapshot in snapshots.iterator()]
  File "/app/archivebox/core/models.py", line 127, in as_link_with_details
    return load_link_details(self.as_link())
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/index/__init__.py", line 348, in load_link_details
    existing_link = parse_json_link_details(out_dir)
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/index/json.py", line 110, in parse_json_link_details
    return Link.from_json(link_json, guess)
  File "/app/archivebox/index/schema.py", line 246, in from_json
    cast_result = ArchiveResult.from_json(json_result, guess)
  File "/app/archivebox/index/schema.py", line 97, in from_json
    info['end_ts'] = parse_date(info['end_ts'])
  File "/app/archivebox/util.py", line 114, in typechecked_function
    return func(*args, **kwargs)
  File "/app/archivebox/util.py", line 157, in parse_date
    return dateparser(date, settings={'TIMEZONE': 'UTC'}).replace(tzinfo=timezone.utc)
  File "/usr/local/lib/python3.9/site-packages/dateparser/conf.py", line 89, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dateparser/__init__.py", line 54, in parse
    data = parser.get_date_data(date_string, date_formats)
  File "/usr/local/lib/python3.9/site-packages/dateparser/date.py", line 421, in get_date_data
    parsed_date = _DateLocaleParser.parse(
  File "/usr/local/lib/python3.9/site-packages/dateparser/date.py", line 178, in parse
    return instance._parse()
  File "/usr/local/lib/python3.9/site-packages/dateparser/date.py", line 182, in _parse
    date_data = self._parsers[parser_name]()
  File "/usr/local/lib/python3.9/site-packages/dateparser/date.py", line 196, in _try_freshness_parser
    return freshness_date_parser.get_date_data(self._get_translated_date(), self._settings)
  File "/usr/local/lib/python3.9/site-packages/dateparser/freshness_date_parser.py", line 159, in get_date_data
    date, period = self.parse(date_string, settings)
  File "/usr/local/lib/python3.9/site-packages/dateparser/freshness_date_parser.py", line 88, in parse
    self.now = apply_timezone(_now, settings.TIMEZONE)
  File "/usr/local/lib/python3.9/site-packages/dateparser/utils/__init__.py", line 115, in apply_timezone
    new_datetime = apply_dateparser_timezone(date_time, tz_string)
  File "/usr/local/lib/python3.9/site-packages/dateparser/utils/__init__.py", line 103, in apply_dateparser_timezone
    if info['regex'].search(' %s' % offset_or_timezone_abb):
KeyboardInterrupt

Long story short, I snooped after setting up the dev enviroment. Before telling the user anything, archivebox is iterating on every matching link, of 1847. But that's not what's slow! It's a particular function, merge_links() that, ran 1847 times, adds up to a lot of waiting.
Merge_links() is called by load_link_details(), seemingly to combine the information from disk about the link we're currently processing, and prettify it too. So far so good, but why are we doing in this in bulk? For example, archivebox list is going to iterate on each link, to print it, so why not "merge" as you roll? Do we really need a complete list of merged links before doing anything? Perhaps I'm not seeing the entire picture though...

The merge is done before the list in order to dedupe them, as sometimes there are duplicate snapshots from an older install or other archive getting merged in.
I'll likely improve this in the future, but it might require splitting out the import/dedupe step into an explicit user-run command in order to unblock the performance changes.


Run ddp_spawn dataloader checks on windows

What does this PR do?
Fixes current failing windows tests
The fix is removing the on_windows skip. I don't know why we were skipping it.
Before submitting

[x] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[n/a] Did you make sure to update the documentation with your changes? (if necessary)
[n/a] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[n/a] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

@carmocca does this affect Horovod somehow?

@carmocca does this affect Horovod somehow?

No, the Horovod failure is unrelated. Probably another flaky test


:bug: Remove $http.success and $http.error

The $http legacy promise methods success and error have been deprecated. Use the standard then method instead.

Closed via #82


v0.1 Tabular model name refinement discussion

For v0.1 release, we are looking to refine and improve the readability of the model names created by Tabular.
Below is a list of proposals. We are looking for community feedback, so please add your thoughts and suggestions!

Weighted ensemble renaming:
Current -> weighted_ensemble_k0_l2
Proposed -> WeightedEnsemble_l2
Reasoning: All other models are in CamelCase format except for the weighted ensemble. Additionally, k0 is a remnant from before we were able to handle duplicate model names. (k0 represented the special case = kfold 0, aka train with all the data and use train data as validation data as well).
Alternative 1: 'Weighted_l2'

Stacker model renaming:
Current -> {model_name}_STACKER_l1
Proposed -> {model_name}_S_l1
Reasoning: Stacker model names often become very long, such that they are difficult to read and type in general usage. Since we now store information that identifies a model as a stacker elsewhere (such as in .leaderboard or .info), we don't need to be as verbose in the model name itself.
Alternative 1: 'STACK'
Alternative 2: 'B' or 'BAG' due to 'S' being defined when level != 0 (level > 0 and S is redundant together), and additionally _STACKER_l0 is technically misleading, as the model is not a stacker but actually a bagged ensemble.
Alternative 3: No suffix, since the state of the stack and bag can be found through other means.

Classifier / Regressor suffix removal:
Current -> RandomForestClassifier
Proposed -> RandomForest
Reasoning: Classifier / Regressor suffix is a remnant from when we did not have such information available elsewhere. Since it is generally well known by the user what problem type they are dealing with, the extra info added to the model names is generally redundant and complicates the code, especially when handling duplicated model name resolution.

Stack level rework:
Current -> {model_name}_l{n}
Proposed -> {model_name}_l{n+1}
Reasoning: level=0 is confusing to users due to an l2 model being the head of a model stack which is 3 layers deep. With this change, an l2 model would be the head of a model stack which is 2 layers deep.

FULL rename:
Current -> {model_name}_FULL
Proposed -> {model_name}_F
Reasoning: Same reasoning as Proposal 2 in that it contributes to very long model names.

DSTL rename:
Current -> {model_name}_DSTL
Proposed -> {model_name}_D
Reasoning: Same reasoning as Proposal 5.

CatBoost rename:
Current -> Catboost
Proposed -> CatBoost
Reasoning: To align with the casing used by the CatBoost project, and to keep consistency with other models casing.

KNN rename:
Current: KNeighbors
Proposed -> KNearestNeighbors
Reasoning: Missing Nearest.
Alternative 1: KNNeighbors
Alternative 2: KNN

NeuralNet:
Current: NeuralNet
Proposed: ???
Reasoning: NeuralNet might be a strange name to have as we add more and more neural network type models into AG.

My personal recommendations are the following:

Accept (Main)
Accept Alternative 2 (BAG)
Accept
Accept
Reject
Reject
Accept
Accept (Main or Alternative 1)
Undecided

With this, the following changes would occur to the example model name:
Current: RandomForestClassifier_STACKER_l0
Proposed: RandomForest_BAG_l1

CamelCase is easier to type, but underscores are easier to read [1], I'd vote for underscores, but with capital L on level since lowecase l is in many cases confused with 1: weighted_ensemble_L2.
BAG is better because we don't really have stacking in the L0.
Accept. Alternative proposal: can we do camel_case for variations of the algorithm? This reads better:
Accept, but with capital L to avoid confusion of l with 1 or |.
Reject, FULL looks good
Reject, DSTL looks good
Accept: official project is using CatBoost
Accept (main). Alternatively k_nearest_neighbors if underscore_case is an option
Proposal: tabular_neural_network

Here's example how my options will look like:
weighted_ensemble_L2
random_forest_gini_FULL_BAG_L0
random_forest_entr_FULL_BAG_L0
neural_network_FULL_BAG_L0
random_forest_entr_BAG_L0
extra_trees_gini_BAG_L0
extra_trees_entr_BAG_L0
extra_trees_entr_FULL_BAG_L0
extra_trees_gini_FULL_BAG_L0
k_nearest_neighbors_unif_FULL_BAG_L1
k_nearest_neighbors_unif_BAG_L1
k_nearest_neighbors_unif_FULL_BAG_L0
k_nearest_neighbors_unif_BAG_L0
k_nearest_neighbors_dist_BAG_L1
k_nearest_neighbors_dist_BAG_L0
k_nearest_neighbors_dist_FULL_BAG_L1
k_nearest_neighbors_dist_FULL_BAG_L0

[1] https://whatheco.de/2013/02/16/camelcase-vs-underscores-revisited/

Default/majority on 1-8
As for 9: Are there not already two tabular neural nets present, one the original one, and the other from FastAI?

@gradientsky Re CamelCase: I feel that CamelCase is better because it is fewer characters and makes the logical distinction between the qualifiers that use underscores (level, _FULL, _BAG, _DSTL, etc.) and the model name itself. This clarifies which part of the name is the model itself and which is the qualities of the model.
Re capital L for level, I am ok with this, the reasoning for lowercase l would be that it generally takes less space than uppercase L, but this is likely not that big of a concern.

I vote for CamelCase as well and I also vote that level suffix should be capitalized to be more readable (_L1 instead of _l1).

Accept (Main, WeightedEnsemble_l2). Another alternative could simply be Ensemble since we refer to bagged/stack-ensembles in other ways.

I vote to get rid of _S and just let the level information _L1 inform the user when a model is stacker.  To indicate a bagged model, I'm fine if we use either _BAG or _B, depending on how long our typical model-names end up after all these changes.

Accept

Accept. I also think docs need to clarify that _L1 are base models that operate directly on the data features, and feed into stackers.

I'm fine with either _FULL or _F. I guess it should match what suffix we choose for bagged models.

I'm fine with either _DSTL or _D, to match whatever suffix we choose for bagged models.

Accept CatBoost.

Accept KNearestNeighbors.  Or ifKNearestNeighbors looks unnecessarily long with the suffixes, I'm fine to just keep current KNeighbors. KNeighbors turns up more correct results in Google search than KNNeighbors.

For our current 2 neural nets, I vote we keep these basic models named as NeuralNetMxnet, NeuralNetTorch, and use more specific names for more advanced NN architectures that we later add (keeping these names vague seems good to me, since these NN models are not exactly a standard architecture).

Alternatively, could name them: TabularNeuralNetMxnet, TabularNeuralNetTorch (but may be too long), or  alternatives to TabularNeuralNet* could include: TabularNN*, MLP*, MLPNeuralNet*.

A major thing is our documentation/tutorial needs to provide a glossary of what all the possible model names & suffixes mean.  Also the hyperparameters argument keys should be overloaded to either take the name-abbreviations or the full model names discussed here.

Given we are changing _STACKER suffix, I have another proposal to add:

Level suffix in non-bagged models:
Current: CatBoost
Proposed: CatBoost_L1
Reasoning: If we later intend to add a form of multilayer blend stacking to AutoGluon, this will be required to maintain clarity. Additionally, it helps to standardize the level scheme such that any model regardless of if it is bagged will have a level suffix.
Alternative 1: L1 (base) models never have a level suffix.

My personal recommendation is 10. Accept (Main)

I agree with 10, but this only with stacking activated correct? With no stacking, I don't think there should be _L1 as it's confusing.

All accepted proposals have been implemented into v0.1 release.


☔ [JS] Langchain.js instrumentation

Add Langchain.js instrumentation
Core functionality

[x] completions
[x] retrieval
[x] tool calling
[x] streaming
[x] agents

Additional functionality

[x] #356
[x] tracing suppression

Docs

[x] documentation
[x] examples

Scaffolding

[x] set up langchain js instrumentation project

Cleanup

[x] - update / test examples
[x] - remove dotenv setup for tests
[x] - add tests back to typecheck
[x] - test on lower version of langchain/core

Closing as all tasks are complete


feat: add a new REST API ('/v5-5/assignments/users/{username}') to Pe…

…rmissionsRESTController that returns permissions assignments based on a username

Checklist

[x] the individual contributor license agreement is signed
[x] commit message follows commit guidelines

Description of change

This API is a more sensible way for non-uPortal clients to obtain information about uPortal permissions assignments.
The existing API is /assignments/principal/{principal}.json, which expects you to know the principal string of the user or group you're interested in.  (A  principal string is a dot-separated concatenation that contains an internal datanase id.)

Some changes applied.  I plan to merge this one when the CI finishes.


[docs] Update Distributed Counter example app

Description
Describe the changes or additions included in this PR.
Test plan
How did you test the new or updated feature?

Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

[ ] Protocol:
[ ] Nodes (Validators and Full nodes):
[ ] Indexer:
[ ] JSON-RPC:
[ ] GraphQL:
[ ] CLI:
[ ] Rust SDK:

@ronny-mysten I added a link to the source code, but not sure if the link is up to standard. Please let me know and I can update it as needed, thanks!

@ronny-mysten I added a link to the source code, but not sure if the link is up to standard. Please let me know and I can update it as needed, thanks!

Looks good. Thanks for doing that. We can use it for the rest of the code, too, but it's a little more involved. I'll do that part in a later PR so I can fix/adjust the plugin, if needed.


Is event support planned?

The web3 api has event functionality, is this currently planned in the scope of moonbeam's grant?

This was outside of the grant but is currently being implemented also by moonbeam.
Current progress can be followed in this PR: https://github.com/paritytech/frontier/pull/103


Parameterize generic KeyValueStorage segment generation.

With future storage needs in mind change cratePruningStorage to a
more generic getStorageBySegmentIdentifier method where the segment
identifier is the argument.  For now that's just pruning storage as
world state needs a specific return type.
Signed-off-by: Danno Ferrin<EMAIL_ADDRESS>
[X] I thought about the changelog and included a changelog update if required.

more Bonsai prep


Not allowable C function for Go

close #287

This change is

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


added vertices function

Codecov Report

Merging #30 (9d23e14) into master (2e35678) will increase coverage by 2.50%.
The diff coverage is 100.00%.

:exclamation: Current head 9d23e14 differs from pull request most recent head 1c0f97c. Consider uploading reports for the commit 1c0f97c to get more accurate results

@@            Coverage Diff             @@
##           master      #30      +/-   ##
==========================================
+ Coverage   71.14%   73.65%   +2.50%
==========================================
  Files          16       16
  Lines         357      391      +34
==========================================
+ Hits          254      288      +34
  Misses        103      103

Flag
Coverage Δ

unittests
73.65% <100.00%> (+2.50%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

cordex/cmor/__init__.py
100.00% <ø> (ø)

cordex/cmor/utils.py
90.47% <100.00%> (+4.51%)
:arrow_up:

cordex/core/domain.py
80.50% <100.00%> (+1.22%)
:arrow_up:

cordex/tables/__init__.py
93.33% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e35678...1c0f97c. Read the comment docs.

implements #14


Mask systemd<EMAIL_ADDRESS>as it's flaky and doesn't apply in containers

What this PR does / why we need it:

Masks systemd<EMAIL_ADDRESS>as it's flaky in CI and doesn't apply in containers
Which issue(s) this PR fixes:

SC-113140
Does this PR require a test?

NONE
Does this PR require a release note?

NONE

Does this PR require documentation?

NONE

Merging as this doesn't affect tests yet.


Regression: Cannot delete default admin account

Bug description
Probably a regression of #1921. I cannot delete the default admin account from the UI even if I'm myself an admin.
Expected behavior
Being able to click the checkbox to then further delete the user.
Portainer Logs
probably no interesting logs there because nothing could be done
Steps to reproduce the issue:

Setup Portainer instance with default admin
Setup OAuth and create a admin account which will be further used with Portainer
With the OAuth admin try to delete original admin
See blocked checkbox (blurred other usernames for privacy):

Even in the user settings the delete button is blurred out:

Technical details:

Portainer version: 2.9.2
Docker version (managed by Portainer): 20.10.5
Platform (windows/linux): linux
Command used to start Portainer (docker run -p 9443:9443 portainer/portainer): docker-compose up
Browser: Mozilla Firefox 94.0 (64-bit) on Linux
Use Case (delete as appropriate): Using Portainer in a Commerical setup.
Have you reviewed our technical documentation and knowledge base? No

Additional context

I can change the user from being admin to being a normal user, that works.
We only want to use our OAuth because of security reasons. We have no one with a username admin.

@Zocker1999NET
This is by design, Portainer does not allow to delete that user.
I believe it is intended as a break-glass account in case portainer is unable to reach the external provided authentication source.

Yes, for user 0 you can only change the Administrator toggle; that way they're at least no longer an admin.


Wrong SourceBranch and SourceVersion with variables in workspace mapping (sever path)

Agent version and platform
Agent version: 2.117.2
OS: Windows
VSTS type and version
On-premise TFS 2017 Update 2 RTM (15.117.26714.0)
What's not working?
When using TFVC repository and using variables in workspace mapping (Sever Path) both Build.SourceBranch and Build.SourceVersion do not have the expected values.
Given the following scenario:

2 branches called "$/MyTeamProject/branch1" and "$/MyTeamProject/branch2"
Latest changeset in "$MyTeamProject" is 1000
Latest changeset in "branch1" is 1001
Latest changeset in "branch2" is 1002

Create one build definition with a variable called "BranchName" with an empty default value and mark "Settable at queue time". In "Get Sources" set the "Server Path" to "$/MyTeamProject/$(BranchName)",
Queue a build from this build definition and set variable "BranchName" to "branch1".
Expected: during build the predefined variables "Build.SourceBranch" and "Build.SourceVersion" should have these values: "$/MyTeamProject/branch1" and "1001"
What actually happens: predefined variables "Build.SourceBranch" and "Build.SourceVersion" have these values: "$/MyTeamProject/" and "1000"
It's important to notice that, the "Get Sources" task works correctly (gets the latest version from "branch1"). Also, if I set the variable "BranchName" to "branch1" in the build definition (not during queue time) everything works as expected.

Same issue here. This caused almost a big problem here. Detected it looking at the build details.... Imagine you have a Main and Develop branch. In my case the code of develop branch was always used even when i specified the Main to use.... so almost released develop versions as main to customers.... ;-(
2018 update 2 should be this month right ?

We are experiencing the same issue.  We are using TFS 2017 On Premise Update 3.0
I also found that this bug was identified in VSTS here
https://developercommunity.visualstudio.com/content/problem/201376/build-definition-queue-time-variables-not-applying.html
A workaround is to keep updating the value in the build template and saving every time you want to build from a different branch.  Obviously not ideal and results in a bloated template history, but we are making it work.
Should we expect an update/fix to TFS 2017? Or will we have to upgrade to 2018?

@yaananth - please update the dev community issue since this is a server side issue.  Closing the agent issue since it’s not an agent issue

whoever comes to this thread should realize that you should refer to . I was using Build variables in my release pipeline hence it was picking up the wrong one. Using release pipeline just works fine.
https://docs.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch


Disallow negative pagination

I noticed on one of the demos and in working with this directive that it's possible to see negative pages by clicking the back button, even though it appears disabled. Please see the screenshot below:

This PR fixes the problem. Please let me know any feedback and thanks again!

Thanks for the commit!


AzureRmWebAppDeploymentV4 tests temporary fix

Task name: AzureRmWebAppDeploymentV4
Description: Test fix for MSAL
Documentation changes required: (Y/N) N
Added unit tests: (Y/N) N
Attached related issue: (Y/N) N
Checklist:

[X ] Task version was bumped - please check instruction how to do it
[ X] Checked that applied changes work as expected

Waiting @onetocny to confirm if task is working as expected.


Attempt to expose non-public symbol

slalpha5 rejects this program:
class A {
}

class B {

    public function foo() returns A {
        return new;
    }

}

with the error  attempt to expose non-public symbol 'A'.
Strangely, the error goes away if I change returns A to returns A?.

Here's another case:
type M record {};

public type C record {
    M m;
};

This goes away if I change M from record {} to int;

Here's another case:
type M record {|
    readonly record {} k;
    boolean? isEmpty = ();
|};

type MT table<M> key(k);

public type C record {|
    MT mt;
|};

Strangely, if I make M public but not MT, the error goes away.


Preparing of resource path placeholders

I made a resource path preparing before routing.
The issue was found when I tried to implement lambda for working with URL /bay/{bay_id}/status.
The router was not able to find a correct handler.
I made preparing of the path before routing. Now, the resource path is being filled by path parameters before the routing, but if the parameters list is incorrect, the code still returns code 404.

@nabrosimoff did you get it solved or do you need any help?

I solved it and sent you a lot of pull requests, but almost all of them are closed because tests was failed. The last pull request contains fully correct code and tests finished successfully.
I already use code with the patch in my work project, but it would be nice to have possibility of installation patched lambdarest from repository.
Sorry for spamming by pull requests. It was my first experience of pushing a patch to public project.

The relevant pull request is here: https://github.com/trustpilot/python-lambdarest/pull/50


Support JSON5

JSON5 basically allows comments in JSON and some harmless leniency -
https://json5.org/
type: "json5"?

I do not think browsers natively can process JSON5.  That might be a better place to start.
But I wish both could happen.

 To me adding extra features to JSON is a plus (as long as they don't create a security risk).

 To me adding extra features to JSON is a plus (as long as they don't create a security risk).

Thumbs down doesn't mean hate!
I am concerned with interoperability and narrowing the attack surface. Besides, I believe it's very easy to convert JSON5 to JSON on the HTTP server or application server at the point of transmission or earlier. For a static HTTP website it might be necessary to convert from JSON5 to JSON at deploy time using a build tool, but again this should be very easy.
Is there a use case where it would be useful to support JSON5 in import modules, where it would also be difficult to convert to JSON before transmission?

I think JSON5 belongs in its own proposal.
For example, I would also want JSON.parse() and JSON.stringify() too
JSON.parse(str, { type: "json5" })
JSON.stringify(obj, { type: "json5" })

in addition to imports
import foo from "./foo.json" assert { type: "json5" };
import("foo.json", { assert: { type: "json5" } });

Why choose to support JSON5 rather than HJSON, HOCON, or YAML, which are also strict JSON supersets?

I don't see how one precludes the others.  I'd even like to see import string from './foo.txt' with { type: 'text', encoding: 'utf8' }; which would enable userland parsing for arbitrary formats.
I do see the value in minimizing the scope of this proposal though.


Amiga, Amstrad, Atari ST support ?

Hi,
It seems there is no support for Amiga, Amstrad CPC, Atari ST, ZX Spectrum etc.
Is that possible to add them ?
Thank for your help.

not yet probably

Computer systems are not incorporated yet, it will be great to include them in a future release.


Start final method statics

This begins the impl of static checks for final methods.
Motivation
To report errors statically.
Test plan
See included automated tests.
There are some things marked TODO in the tests. This is pending further future work.

cc @DarkDimius you might want to take a look at this too.
We're planning on landing the final static checks in stages to break up the implementation (rather than doing it all statically in one shot).


Type option seems to not be taken into account

As explained here https://github.com/rv-kip/express-redis-cache#content-type I tried to use the type config to force content type.
But it doesn't work : in fact I've searched the code to understand where it's used, and it seems to be never use. I think it should be use there : https://github.com/rv-kip/express-redis-cache/blob/master/lib/ExpressRedisCache/route.js#L186
Thanks a lot in advance,
Ghislain

#66


SPI begin transaction

I've seen that for AVR hardware SPI the driver doesn't call SPI.beginTransaction, therefore in case you are working with multiple SPI devices, transaction settings could be overriden by another device. I've tested this using Ethernet Shield and two ADS1118, and after each ethernet call the adc gets misconfigured. I've suggest to add the instruction into AVR blocks too.

Did not want to create new issue, and question is relevant to this. How can I use 2 ADS1118 modules with single SPI bus as is described in corresponding documentation with your library. There is no function in your library documentation or text explaining this, I can't deduce usage form .h and .cpp file, and afraid to overwrite it over your library because I can intervene in CS handling and just break everything. So can I (and how if able) use your library to interact with two ADS1118 modules via single SPI bus?


Pepolar unwarping: QwarpPlusMinus requires a value for input 'in_file'

Hi!
I'm trying to preprocess a dataset using the 'pepolar' (blip-up, blip-down) technique with fmriprep (using the Docker interface, version 1.1.6). It crashes with the following error message:
QwarpPlusMinus requires a value for input 'in_file'
My data passes the bids-validator checks, so I don't think the problem is caused by the data itself.
I don't know how relevant it is for solving this issue, but I'm including the meta-data for the functional run and the corresponding "topup" (*_epi.nii.gz) below.
Functional file .json:
"PhaseEncodingDirection": "j",
"EffectiveEchoSpacing": 0.00034538,
"ParallelReductionFactorInPlane": 1.51,
"MultibandAccelerationFactor": 4,
"WaterFatShift": 14.388,
"FlipAngle": 55,
"TotalReadoutTime": 0.01347,
"PulseSequenceType": "Multiband gradient echo EPI",
"TaskName": "wm"

Topup .json:
"PhaseEncodingDirection": "j-",
"EffectiveEchoSpacing": 0.00034538,
"ParallelReductionFactorInPlane": 1.51,
"MultibandAccelerationFactor": 4,
"WaterFatShift": 14.388,
"FlipAngle": 55,
"TotalReadoutTime": 0.01347,
"PulseSequenceType": "Multiband gradient echo EPI",
"IntendedFor": "func/sub-RE01_task-wm_acq-Mb4Mm2Tr1600_bold.nii.gz"

If I should post more information about the data, let me know!

Ah, I see the problem. Some inputs got renamed in an upstream refactor in nipype (nipy/nipype#2674).
Thanks for the report. I'll submit a fix and we can release today or tomorrow.

Hi @lukassnoek, are you able to verify that #1289 fixes the issue for you? The easiest way to check is to use the fmriprep-docker wrapper and patch in the working repository. Please let me know if you need any help.

Sure thing! I cloned your repository (https://github.com/effigies/fmriprep) and changed to the fix-branch (git checkout fix/qwarpplusminus) and then tried the following command:
 fmriprep-docker $proj_dir/bids_topup $proj_dir/preproc_topup --nthreads 10 --omp-nthreads 4 --output-space template --fs-license-file /usr/local/freesurfer/license.txt --participant-label $base_sub -f /home/lsnoek1/spinoza/MB2018/fmriprep

...which generated the following docker-command:
 docker run --rm -it -v /home/lsnoek1/spinoza/MB2018/fmriprep:/usr/local/miniconda/lib/python3.6/site-packages/fmriprep:ro -v /usr/local/freesurfer/license.txt:/opt/freesurfer/license.txt:ro -v /home/lsnoek1/spinoza/MB2018/bids_topup:/data:ro -v /home/lsnoek1/spinoza/MB2018/preproc_topup:/out poldracklab/fmriprep:1.1.6 /data /out participant --nthreads 10 --omp-nthreads 4 --output-space template --participant-label sub-RE02

... but this gave the following error:
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 7, in <module>
    from fmriprep.cli.run import main
ModuleNotFoundError: No module named 'fmriprep.cli'

Maybe I'm using the -f flag the wrong way?

Ah, yeah, now it works! At least, it starts running, but I get the following error:
traits.trait_errors.TraitError: The trait 'in_file' of a CopyHeaderInputSpec instance is an existing file name, but the path  'Qwarp_PLUS_WARP.nii.gz' does not exist.
Error setting node input:
Node: cphdr_warp
input: in_file
results_file: /tmp/work/fmriprep_wf/single_subject_RE02_wf/func_preproc_task_wm_acq_Mb4Mm3Tr550_wf/sdc_wf/pepolar_unwarp_wf/qwarp/result_qwarp.pklz
value: Qwarp_PLUS_WARP.nii.gz

I think I resolved that issue with nipy/nipype#2705. Could you try patching that in, as well?

Yes, seems to work!
Thanks for fixing this!

Great! You'll need to keep using this until the release. Given that we're going to need to depend on fixes to nipype, I think we'll put off the release until Monday. If we need it sooner, we can patch the interface here.

Thank you so much for the quick fix!


Git Bash integrated terminal odd behaviour

A few integrated terminal commands using Git Bash have some odd behavior right off the bat with this newest release of Insider.
The clear command results in:
$ clear
'bash.exe': unknown terminal type.

vim results in:
$ vim Vagrantfile
E437: terminal capability "cm" required
Press ENTER or type command to continue

Then after pressing enter, vim launches but all the spacing is messed up.
Also, backspace just adds spaces now.  And it seems like the up/down keys just keep appending previous commands rather than cycling through them.
Let me know if there is any testing needed on my end.

VSCode Version: Code - Insiders 1.9.0-insider (ee9d91c05fa571de56c41383cceee30b05b0a2f6, 2017-01-20T15:08:19.064Z)
OS Version: Windows_NT ia32 10.0.14986
Extensions:

Extension
Author
Version

vscode-instant-markdown
dbankier
0.2.1

python
donjayamanne
0.5.6

cpptools
ms-vscode
0.9.3

vscode-docker
PeterJausovec
0.0.11

vscode-icons
robertohuertasm
7.0.1

vim
vscodevim
0.4.11

vscode-todo-highlight
wayou
0.3.3

Steps to Reproduce:

Set C:\Program Files\Git\bin\bash.exe as integrated windows terminal
Launch integrated terminal
Type commands to view odd behaviour

This should be fixed in next Insiders, in the meantime typing this into the terminal should fix it on a per-session basis:
TERM=xterm

@Tyriar Not sure if this is a separate issue or maybe is resolved through the more permanent fix, but the clear command does not act properly even with the
TERM=xterm

The cursor moves back to the top but doesn't clear any of the text

@antmerlino the new build is out in ~4 hours I think, could you test it after then?

@Tyriar Surely, I'll let you know

@Tyriar Fixed 👍

Awesome 😄

Any fix for this issue? I am still facing it with backspace adding spaces and clear being an unknown terminal type.

@antmerlino @Tyriar What build fixed this issue? I just used the latest git bash and vscode.

@adil-waqar this was closed back in January. Do you have the latest version installed?


Fix page chooser not rendering proper get_admin_display_title

On a project, I noticed a couple of places in the page chooser where the get_admin_display_title function was not showing the correct title.
This was happening both when the title appeared in the chooser breadcrumbs and when the page title appeared in the top "parent page" section. It would also show the incorrect title in the value section of a filled chooser.
This can be reproduced by adding a page with an overridden get_admin_display_title function, then adding another two layers of pages underneath it.
As an example, let the tree be:
- Overridden page
    \- Middle page
        \- Leaf page

Using the page chooser, select the middle page
Click "Choose another page" on the same chooser
The parent page entry for Overridden page at the top of the chooser will show the incorrect title
Using the page chooser, descend under the middle page
The breadcrumb entry for Overridden page will show the incorrect title

This PR adds testing for both the parent page section and the breadcrumbs at the level of the rendered HTML (as extracted from the JSON response), and adds two calls to .specific - one in the code for chooser.browse and one in the template for chooser_breadcrumb - the ensure the override version of get_admin_display_title is called.

Merged in fa5c69ee006f9c701c1868fa7e13a3ac3adcd021 - thanks @haydnnewport!


Non Georeferenced Records

A species that has no georeferenced occurrences in the county can appear in "Browse all species", "List Species" eg Ranunculus baudotii with no attendant information.
This is confusing and/or superflous without the attendant information. In this case it is a record that has not yet been accepted by the County Recorder. The species does not appear on the current Flora Spreadsheet.
Is it possible for there to be non georeferenced entries in the NBN?

I've not seen any records without a grid ref. Can check this

Thanks @DanWrench, hopefully this one is a red herring.

I dont recall records without a grid reference either.


Problem with Adel Kalbani (Taraweeh 1437H)

Assalamu 'Alaykum waRahmatullah.
Actual links to the audio files are https://download.quranicaudio.com/quran/adel_kalbani_taraweeh_1437/<SURAH_NO>.mp3 but UI is configured for https://download.quranicaudio.com/quran/adel_kalbani_1437/<SURAH_NO>.mp3.
Maybe you have renamed the folder on server-side but have forgotten to update the UI?
Jazzakumullah.

wa3laikum alsalam wa ra7matullahi wa barakatuh,
nice catch - jazakumAllah khairan, fixed.
walsalam 3alaikum


Total test all over india not shown

Describe the bug
Total number of cases all over the india is not shown.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://www.covid19india.org/'
Click on 'all states
Scroll down to 'Map'
It shows a "-" on total number of test done

Expected behavior
It will be great if it shows the total test done.

Thanks for reporting the issue, @Dhrumit8596
It's duplicate of https://github.com/covid19india/covid19india-react/issues/1013


Roll Flutter Engine from 94023d711db3 to f271fd0726a9 (2 revisions)

https://github.com/flutter/engine/compare/94023d711db3...f271fd0726a9
2024-06-25<EMAIL_ADDRESS>Roll Skia from 04133a5bf21f to ea8ec1a618b0 (3 revisions) (flutter/engine#53569)
2024-06-25<EMAIL_ADDRESS>Roll Skia from da1ea4eb0270 to 04133a5bf21f (2 revisions) (flutter/engine#53566)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jimgraham@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Show source for type aliases

Hey there! Thanks for your work on mkdocstrings 🙏
I'm working on a project that defines some type aliases, e.g.:
CSSPathType = Union[
    str,
    PurePath,
    List[Union[str, PurePath]],
]
"""Valid ways of specifying paths to CSS files."""

This alias is used elsewhere to annotate variables.
In my project docs, CSSPathType shows up, it is documented as "Valid types for CSS file paths", and other variables that are annotated with CSSPathType will link to it.
However, I will not get the most important piece of information I need, which is the actual types in the type alias!
Even when show_source is set to True.
Do you have any idea how I could get this information shown?
The image below shows the issue. What would be really helpful was the actual definition of the attribute CSSPathType.

Hey @rodrigogiraoserrao, thanks for the detailed report :slightly_smiling_face:
Seems to be working fine on my end:

I believe you're using custom templates: maybe you forgot some part of the original template for attributes?


Refactoring the format asset directory function

Changelog Description
This PR is to refactor the format_asset_directory function so that users can have more freedom to customize their directories for loading the assets.
Additional info
n/a
Testing notes:

Build and install the unreal addon with this branch
Launch Unreal
Change your loaded asset template to your own preferences in ayon+settings://unreal/loaded_asset_dir in ayon setting
Load Assets

I have did changes to the asset directory preset but when using
{product[type]} it was not functional and gave me an error but once using this format:

its working when using {product[productType]} so now the question is wheter we shall make it work with a common format like {product[type]} instead.

We should follow the rules and use {product[productType]} . I will change the script:)


Allow direct messages

Is there a way to send a direct message to another user in your app? If it is not implemented, please consider implementing this feature.

This is implemented in #11. The readme has been updated with directions but you can now use ctrl-u to access the list of users and send them a direct message.

Consider to support direct messages with multiple users


Requirements for merged kata runtime

Here is a draft of requirements (working in progress).
https://docs.google.com/document/d/109pxj-90Ly58ma8CoeRKcMoPWBD0G911E53MeK2zhhA/edit?usp=sharing
Currently, it listed some of the working scenarios for the runtime, and more sections should be added.
Once we have agreement on the requirement, the doc should be put into repo as a markdown doc.

I added @tallclair @WeiZhang555 @sameo in the doc with edit permission, and you can invite others.

@gnawux Maybe it's a good idea to start discussing about those requirements here instead?
Here is my initial proposal:
[DRAFT]
OCI compatibility
The Kata Containers runtime MUST implement the OCI runtime specification and support all the OCI runtime operations.
runc CLI compatibility
In theory, being OCI compatible should be enough. In practice the a Kata Containers runtime should comply the latest stable runc CLI. In particular, it MUST implement with the following runc commands:

create
delete
exec
kill
list
pause
ps
start
version

CRI support
The Kata Containers runtime MUST provide an interface (CLI, API) that is compatible with the CRI specification.
In other words, any CRI server should be able to use the Kata Containers runtime to implement the CRI specification.
Multiple hardware architectures support
The Kata Containers runtime MUST NOT be architecture specific. It should be able to support multiple hardware architectures and provide a pluggable and flexible design for adding support for additional ones.
Multiple hypervisor support
The Kata Containers runtime MUST NOT be tied to any specific hardware virtualization technology, hypervisor or virtual machine monitor implementation.
It should support multiple hypervisors and provide a pluggable and flexible design for adding support for additional ones.
Nesting
The Kata Containers runtime MUST support nested virtualization environments.
Performance optimizations
One of the compelling aspects of containers is its minimal overhead compared to bare metal applications. A container runtime should strive for keeping that overhead to a minimum in order to provide the expected user experience.
As a consequence the Kata Containers runtime implementation should be optimized for:

Minimal workload boot and shutdown times
Minimal workload memory footprint
Maximal networking throughput
Minimal networking latency
Optimized I/O

Secure hardware resources access
Containers consumers typically want to get access to some host hardware resources, sometimes by making security compromises.
As the Kata Containers project main feature is about providing better container security and isolation, it should provide access to host hardware resources in a secure and efficient way by typically leveraging IO virtualization technologies.
Testing
The Kata Containers runtime MUST pass a set of container related tests:

Unit tests: (runtime unit tests coverage ?)
Functional tests: The entire runtime CLI and APIs
Integration tests: Docker and Kubernetes.

Runtime API
Some CRI implementations (e.g. Frakti) may rely on the runtime API instead of the CLI. The Kata Containers should provide a runtime API definition and a runtime library to support those cases.

cc @jessfraz @sboeuf @jodh-intel @egernst @devimc @amshinde @mcastelino @grahamwhaley

@sameo you could just edit the google doc, it is not easy to collaborate writing a document in an issue.

@gnawux Your suggestion about opening a PR for adding this as a document would be good. Folks can then comment on it and it can be rebased to take input into account.

@sameo feel it is a bit early to put a PR, others may input contents directly on the google doc.
However, if you think it is the time, I could file a PR and merge both our drafts into it.

@gnawux Since this is going to be part of our documentation, using a google doc for this sounds like an avoidable additional step. If this document would for sure not be merged to our documentation then a google doc would make sense, but here a PR makes more sense imho.

@sameo Though I think it is better to make the doc looks ready before convert to a PR, It's not a big issue. I will submit the PR firstly for discussion.

@sameo and all
Do you guys think the requirement docs should be put into docs directory of this repo, or the document repo? If the latter, do we have a dir hierarchy of the repo yet?

I kinda think the last part of that doc is very accurate, can we remove as many shim layers as possible and try to keep it simple :)

Can you clarify the section on CRI support? It sounds like it's not saying that the runtime should be a complete implementation of the CRI, but rather that an implementation should be possible using it? Is OCI not sufficient for that?

I think the document is trending towards describing architecture rather than defining requirements (ie, OCI compliance, provide an API/library suitable for a CRI like frakti, device hotplug, etc.).  If/when I get permissions for editing I can help make some edits/add clarity.

@jessfraz yes, this is what we do in production, and we should keep the scenario working in kata.
@egernst :+1: permission granted.

@tallclair Yes, that's what I was trying to say indeed. OCI is enough for CRI implementations like CRI-O or cri-containerd but e.g. Frakti does not rely on the OCI interface but rather on the runV API. The current docker-shim also does not rely on OCI.
So OCI is sufficient but it's not necessary.

@sameo @tallclair while OCI can implement CRI support, its runtime spec may not be efficient enough for Kata Containers due to lack of proper storage description. When Kata Containers support cri-containerd/CRI-O via OCI (aka. the runc cli interfaces), it relies on 9pfs (which itself is slow and problematic -- we even have to hack 9pfs kernel module to reach POSIX compliance) to map local storage to the guest, and there is no description for remote storage in the spec.
OTOH, the runV library API is more native for VM-based containers and favors CRI from design. In fact it was designed together with CRI. In that sense, runV is compatible with the OCI spec and provides extended APIs to better suit the need of CRI and VM-based containers.
So to amend the last paragraph of @sameo 's requirements list, I would suggest we change the requirement of Runtime API from

Runtime API
Some CRI implementations (e.g. Frakti) may rely on the runtime API instead of the CLI. The Kata Containers should provide a runtime API definition and a runtime library to support those cases.

To
"
Runtime Library API
While CRI-O and cri-containerd rely on runc compatible CLI, some CRI implementations like frakti rely on the runtime library API instead. The Kata Containers MUST provide a runtime library API favoring CRI design to support them.
"
And it should be moved up right after the CRI support section, instead of being put at the last of the list, which might give the impression that it is a minor requirement that can be dismissed.

@tallclair as we mentioned CRI things, I would like to provide a detailed research of potential integration options of Kata and CRI shims in next sig-node meeting.
Let's put all these problems on the table and see how it can be fixed.

@bergwolf

When Kata Containers support cri-containerd/CRI-O via OCI (aka. the runc cli interfaces), it relies on 9pfs (which itself is slow and problematic -- we even have to hack 9pfs kernel module to reach POSIX compliance) to map local storage to the guest.

That is not entirely correct I'm afraid. With cc-runtime we do hotplug block based devices as local storage when the container overlay filesystem allows for it. So 9pfs is a fallback, not the default.

there is no description for remote storage in the spec.

Yes, there is no such description in the OCI spec. But out of curiosity, is that specified in the current CRI spec? I don't see it but I may very well be missing something.

So to amend the last paragraph of @sameo 's requirements list, I would suggest we change the requirement of Runtime API from

Done, thanks for the input. I did not change the order because I think all those requirements are mandatory. Instead I created an Optional requirements section to explicitly state which ones are mandatory and the runtime API is part of it.

@sameo, thanks for updating the list! I still think we can put all CRI related requirements together instead of scattering them all over the doc. But that can be done in future updating.

That is not entirely correct I'm afraid. With cc-runtime we do hotplug block based devices as local storage when the container overlay filesystem allows for it. So 9pfs is a fallback, not the default.

Well, it's true that you can hotplug block devices if they are specified in theSpec.Linux.Devices section of the OCI spec. That is true for any pluggable devices. And what you present to the container process is the device itself rather than any file system directory.
The problem is that with an OCI spec, rootfs and volumes are specified in Spec.Root and Spec.Mount. Then you do not know if the rootfs and volumes are block based devices or not, not to mention which device you need to hotplug to the guest.

But out of curiosity, is that specified in the current CRI spec? I don't see it but I may very well be missing something.

That is not defined by the CRI spec but can be supported via flexvolume. And there is ongoing change to support it in the CSI spec.

@bergwolf

I still think we can put all CRI related requirements together instead of scattering them all over the doc.

Yes, that makes sense. I've merged the runtime api into the CRI support section. Please let me know how it looks now.

Well, it's true that you can hotplug block devices if they are specified in theSpec.Linux.Devices section of the OCI spec. That is true for any pluggable devices. And what you present to the container process is the device itself rather than any file system directory.
The problem is that with an OCI spec, rootfs and volumes are specified in Spec.Root and Spec.Mount. Then you do not know if the rootfs and volumes are block based devices or not, not to mention which device you need to hotplug to the guest.

Sorry for the confusion, I should not have mentioned the hotplugging side of things here. We do vmm hotplug, but only for efficiency reasons. But cc-runtime and virtcontainers dynamically detect if a rootfs is block based device or not and present it as a disk inside the VM or as a 9pfs mount point respectively. As you pointed out, the performance and posix compatibility are significantly different between the 2. So my point was that you can do proper block based IO with or without the current OCI spec help.

@egernst @tallclair I've updated the CRI support section. Hopefully it reads better now.

I read this whole thread and I think there are some requirements none of you mentioned.

first big part is "docker API" support, though "docker API" is also going through "runc compatible CLI " API, there're still some tricks to make "kata-runtime" work from docker command line. One example is "how to compose a POD",  CRI is using specific labels to classify POD or Container, but docker isn't. And "K8S->docker->kata" is still an important scenario though we already has CRI-O as a replacement.

K8S Ecosystem support:
2.1 CNI network support
2.2 Monitoring: how to use cAdvisor to monitor kata container resource usage.
2.3 Logs: K8S monitor container logs. As I know, it's using volume as log transfter channel, if so kata-runtime should support this natively, but I can't be sure

These are also important for working with K8S, and I know both cc and runv have experience on there. It's better to add some illustrations for them.

@WeiZhang555

And "K8S->docker->kata" is still an important scenario

Do you mean we want to support the dockershim CRI implementation, i.e. "K8S->dockershim->kata" ?

Looks good. I'm a little unclear about the OCI compat / runc compat sections, specifically why those particular runc commands are listed but not some others. For example, delete isn't mentioned, nor is state. There other runc commands, but they do not form part of the OCI runtime spec.
Another point - runc has particular options (like --console-socket and --pid-file) which are not part of the OCI spec, so should those also be listed?
Also, how about adding something about logging / problem determination / debugging? Specifically the ability to determine easily that a problem originates in the runtime rather than one of the other components.
We might benefit from some input from @chavafg for thoughts on Testing requirements.

@sameo dockershim could be enough as it will be formal implementation in K8S.
@jodh-intel

Also, how about adding something about logging / problem determination / debugging? Specifically the ability to determine easily that a problem originates in the runtime rather than one of the other components.
We might benefit from some input from @chavafg for thoughts on Testing requirements.

Good point! This is also important part I really care about, and can be strong guarantee for quality and give more confidence to kata container developers and users.

@jodh-intel delete was already there, and state was just an unintentional omission. I added it now.
I also added the 2 options you mentioned.
Could you elaborate a little more on the logging/debugging ?

@WeiZhang555 dockershim is moving out from kubelet, but I think a docker support is still a valid requirement.

Thanks @sameo.
Regarding logging, ideally the runtime would use structured logging (as provided by https://godoc.org/github.com/sirupsen/logrus, for example), such that one of the log fields would specify "runtime" or "kata-runtime" to allow consumers of the system log to determine that an error was being generated by the runtime (as opposed to the shim/proxy/agent/hypervisor).
Also, every time an OCI command is called, it would be extremely useful if the runtime could also log its version and the commit it was built with. That shouldn't necessarily be required for every log call but since an OCI command delineates a block of log calls, that should be sufficient.
In summary: a quick sudo journalctl -a | grep level=error should be sufficient to establish:
a) if there were any errors.
b) if errors occurred, which component first detected them.

@gnawux It seems that dockershim is indeed moving out, but afaik CRI is now mandatory for kubelet so the legacy Docker runtime is no longer supported.
So I'm fine with supporting dockershim, but not Docker itself.

@sameo what's the difference between dockershim and docker from kata's view?
One more thing, I think one of the most significant differences between different client could be the networking part. @WeiZhang555 could you have some input on the networking related requirements?

@egernst yes, at least, we need CNI at firstly, and 2-5 looks no problem.

@egernst This looks fine.
CNI should be more important for K8S integration.
From command line interface point of view, I saw that every "smart" or "auto-detect" implementation for CNI failed in cc and runv, it's hard to support both CNM and CNI with one implementation.
So CNI first, as a suggestion, we can use a more direct way to support CNI, just provide something like kata-runtime interface/route command to manipulate network interfaces and routes for a lite VM, and provide a new CNI plugin binary as a example in kata.  This has been proven to be efficient.
We can discuss more about networking part later, to find a good way for doing this.

@WeiZhang555 yes, for kubernetes faced scenarios, we don't need CNM. And all the existing CNM implementations for runV or CC could be summarized as "workaround" so far.

@gnawux

@sameo what's the difference between dockershim and docker from kata's view?

From kata's view it's almost identical as the kata runtime ends up being called as a Docker runtime. But from a development/integration perspective it's different: dockershim is a CRI implementation and by adding hardware virtualization awareness to the spec, we can have dockershim taking it into account and eventually calling Docker with e.g. the right annotations.

@egernst @WeiZhang555 @jodh-intel I think I captured all your input, please let me know if that's not the case yet.

Thanks @sameo - lgtm

Adding to documentation repo -further review should happen there.  Please see https://github.com/kata-containers/documentation/pull/17


Allow the user to record mic audio too

Hi I want to work on this feature . Can you guide me ?

Yep

Can you guide me?

Are you live on any other social platforms? If so share the link

Your DM isn't open please dm me so
https://twitter.com/heysagnik

done

Some update about this feature?

hey if this is open can I try?

Just wanted to confirm that when screen is recording - sound is not recording - are we talking about screen recording sound here or user MIC audio ? - I tried to record youtube song audio didnt get captured. Can you be specific on what is needed. Thanks

Progress: Was able to achieve MIC input, currently Testing the changes will share status tomorrow :)

Created the feature. Please Test :)


Webhook not working ...

Or, I am doing it wrong.
I am trying to webhook into HomeAssistant when a new device is on the network for notification to my phone.
I have fought for days to get the webhook working thinking it was a Home Assistant setup issue.
But, I just tried sending to webhook.site and nothing appears when using the Config|Settings Notification Button.
I am pasting the link into the Shoutrrr URL settings box and pressing the button after saving settings and restarting.  Any recommendations?

Or, I am doing it wrong.
I am trying to webhook into HomeAssistant when a new device is on the network for notification to my phone.
I have fought for days to get the webhook working thinking it was a Home Assistant setup issue.
But, I just tried sending to webhook.site and nothing appears when using the Config|Settings Notification Button.
I am pasting the link into the Shoutrrr URL settings box and pressing the button after saving settings and restarting. Any recommendations?

Look at Shoutrrr guide, please: https://github.com/containrrr/shoutrrr/blob/main/docs/services/generic.md
The link should be like:
generic+https://example.com/api/v1/postStuff

Grrrr.  I swear I tried that.  I have certainly been to the Shoutrrr githum many times.
Seems to be woking now.
Thanks.

I now have it sending to Home Assistant using generic+URL:port/api/webhook/-WebHookID
But, the payload is not JSON formatted even though it does go with the Content-Type: application/json.
The payload looks like:
UNKNOWN HOST IP: '<IP_ADDRESS>', MAC: '18:56:80:c7:dd:1e', Hw: 'Intel Corporate'
And, I believe it needs to be like:
{"Title":"UNKNOWN HOST","IP":"<IP_ADDRESS>","MAC":"18:56:80:c7:dd:1e","Hw":"Intel Corporate"}
When I send it this way to Home Assistant using curl, the notification works fine. Using the top payload, Home Assistant throws a JSONDecodeError.
I can't find a way to get Home Assistant to just put out the payload directly. It seems to need it JSON formatted.


Feature request: sprite support

There is an issue when using this tool with url like https://domain.com/sprite.svg#icon-foo
It will include the all sprite into the current svg element.

Changes accepted in #29.


Can't get acquireTokenSilent to work

I'm having problems to get acquireTokenSilent to work. The following code with acquireTokenPopup works fine but I would like to be able to fetch a token without the user having to manually approve every time.
state.userAgentApplication = new Msal.UserAgentApplication(state.applicationConfig.clientID, null,{cacheLocation: 'localStorage'});
state.userAgentApplication.acquireTokenPopup(state.applicationConfig.graphScopes).then(function (token) {

    commit("setGraphToken", token);

  }, function (error) {
        console.log(error);
  });

I have tried the sample code you provide but with no luck. I can get the loginPopup to work but then it runs acquireTokenSilent with no errors and no success.
I also have a second issue with the working code. Like I said it works fine but only for my add-in when I'm using the web-version of Outlook. When I run the desktop application there is no popUp dialog at all. What am I missing here?

@Ideline For the first issue, are you calling the acquireTokenSilent()  after loginPopup() like this ?
userAgentApplication.loginPopup(applicationConfig.graphScopes).then(function (idToken) {
//Login Success
userAgentApplication.acquireTokenSilent(applicationConfig.graphScopes).then(function (accessToken) {
//AcquireToken Success
updateUI();
}, function (error) {
//AcquireToken Failure, send an interactive request.
userAgentApplication.acquireTokenPopup(applicationConfig.graphScopes).then(function (accessToken) {
updateUI();
}, function (error) {
console.log(error);
});
})
}, function (error) {
console.log(error);
});
The samples that we have on github, works fine. Are you running the same sample and it doesn't work? Can you share your code?

Hi.
I have tried this exact code with no success:
getAccessToken({ state, commit, dispatch }) {
  state.userAgentApplication = new Msal.UserAgentApplication(state.applicationConfig.clientID, null, { cacheLocation: 'localStorage' });

  state.userAgentApplication.loginPopup(state.applicationConfig.graphScopes).then(function (idToken) {

    state.userAgentApplication.acquireTokenSilent(state.applicationConfig.graphScopes).then(function (accessToken) {
      updateUI();
      commit("setGraphToken", accessToken);
    }, function (error) {

      state.userAgentApplication.acquireTokenPopup(state.applicationConfig.graphScopes).then(function (accessToken) {
        updateUI();
        commit("setGraphToken", accessToken);
      }, function (error) {
        console.log(error);
      });
    })
  }, function (error) {
    console.log(error);
  });
},

I get the login popup and then it does the aquireTokenSilent but it doesn't get to
commit("setGraphToken", accessToken);
and it doesn't try to aquireTokenPopup. I'm not getting any errors either.
My add-in is for Outlook and so when the user starts it, they have already logged in to their Outlook account. That's why I would like to avoid loginPopup aswell.
I also had trouble getting the aquireTokenPopup to work at first. Once I have done a window.Office.initialize the aquireTokenPopup doesn't work. So currently I'm acquiring the token before the Office initialization.

Try using this for inner scope  https://github.com/AzureAD/microsoft-authentication-library-for-js/issues/278
state.userAgentApplication.loginPopup(state.applicationConfig.graphScopes).then(function (idToken) {
   // not state.userAgentApplication
   this.acquireTokenSilent(state.applicationConfig.graphScopes).then(...)
   this.acquireTokenPopup(state.applicationConfig.graphScopes).then(...)
})

I have tried this now but "this" is undefined while state.userAgentApplication is not undefined.
To clarify ... if I don't use acquireTokenSilent but instead only use acquireTokenPopup the authentication works and I will receive the graphToken when using the web Outlook. My problem is that it doesn't work for the desktop Outlook and acquireTokenSilent doesn't work at all.
It seems as the promise returned by acquireTokenSilent never resolves or rejects?

Take a look at #341
But I think this is a problem with the sample application and not with the library itself. I use it in a Vue.js app and userAgentApplication.acquireTokenSilent always resolves.

I still haven't found a solution to my problem. My add-in is for Outlook and at the moment I can not get a graph token while using the desktop client since neither the silent or popup function works, and I can only get one while using the popup in the web client. If anyone has another suggestion for what I can look into I would really appreciate it.

@Ideline Can you please share your repo or sampel code where we can reproduce this issue? Noone has ever reported this issue and we confirmed that sample works fine. There is something else in your code that is causing this issue.

I actually have exact same issue. That call just never comes back with either success or error. If I'm already logged in and refresh the page then it works, if I just logged in and get redirected back and try to get the token it doesn't do anything. It's really odd.
Since this is in Vue I'm wondering if Vue is doing something that makes acquireTokenSilent never come back.

Here's what I'm thinking -
new Msal.UserAgentApplication has some async bits that might not be done by the time acquireTokenSilent is called or something along those lines. That's why in some situations the object exists but doesn't work properly.

@vladkosarev I've just published my project at https://github.com/Tekki/ivoc4, take a look at it. Maybe the difference is that I don't get the token immediately, but first I log in, then load a new component and from here call acquireTokenSilent.

Thank you for chairing.
Looking at your code I think it's kind of what I do. I will keep trying.
Yesterday I tested the code that I have which is working in Chrome and Firefox. The one without the acquireTokenSilent and just the acquireTokenPopup. It's not working in Edge or Explorer.
In Explorer I get the log in popup but it doesn't close after signing in and it never resolves or rejects.
And in Edge the popup is just an empty window. I get the error message "Permission denied".

@Ideline  IE/Edge support is an open issue curently. Glad to know that it worked on chrome and firefox. We already have an issue for IE/Edge for tracking purpose. I am closing this issue for now.


"Your credit card is about to expire!" banner shows falsely.

When I login, I see red "Your credit card is about to expire!" banner on top of the page. If I follow the link in it, I can see that my card is due to be expired in 2 years.

Hmm ... thanks for the report, @YorikSar.

!m @rohitpaulk
Can you confirm, @YorikSar?

I've logged in to gratipay and saw that message once again. I followed the link, verified that the card there won't expire for more than a year, and... I don't see that message again.

Yes, true. We mark notifications as 'read' when you click on the link in them/close them. The root cause has been fixed, so you shouldn't see this anymore :) (hopefully)


clash-meta: 1.16.0 -> 1.18.1

Description of changes
https://github.com/MetaCubeX/mihomo/tree/Alpha
https://github.com/MetaCubeX/mihomo/compare/v1.16.0...v1.18.1
Mutually exclusive: https://github.com/NixOS/nixpkgs/pull/271658
As an optional replacement of ^ since there seems nobody wanna merge that.

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

#271658 looks good, what's stopping it from being merged?

#271658 looks good, what's stopping it from being merged?

As the repository's main branch and renaming were somewhat designed to evade censorship, some individuals believed that the downstream should remain unchanged, even though the binary have been renamed.

some individuals believed that the downstream should remain unchanged, even though the binary have been renamed.

Which as AUR did https://aur.archlinux.org/packages/clash-meta

I'll rebase that PR after merging this.

Result of nixpkgs-review pr 290873 run on aarch64-darwin 1

1 package built:

clash-meta


Memorize target device when VS Code is quit

Whenever I start VS Code, I have to remember to select the appropriate target since it always falls back to Nano S.
Would it be possible to persist this information and remember the last target used?
Thank you for the great extension!

We already have an available setting: Default Device.
Would it be sufficient?

Fixed in v0.7.0


What's the best way to get notified when a link is drawn between 2 nodes?

Hello,
I am missing an event that notifies me when a new link is drawn between 2 nodes.
I could override the CanAttachTo method of PortModel and assume that if this method returns true, a new link is created. Still looks a bit of a hack to me... Is there a better way? Or could you add an event for that?

Hello,

When a link is added (not attached), an Added event in the Links layer is fired, use Diagram.Links.Added.
When a link is (re)attached, an TargetPortChanged event in the BaseLinkModel is fired, use Link.TargetPortChanged. Check if the target port is set.

Hope this helps!

That works, thanks!


Add QR for configuration

Offer the possibility to scan a QR code to automatically configure app.
Also give the option to display current config as QR code to scan for other users.

Implemented in devel


Can I use ranger22 and cosine annealing strategy together?

Hello, first of all, I'm very glad to see the emergence of ranger2022, which I'm most looking forward to. Thank you for your work.
I've been using your Ranger optimizer before, and its effect is very good, but I've been using mmdet to do some work on target detection recently. I don't know how to register Ranger's strategy into mmdet, so I want to ask whether ranger22 and cosine strategy can be used to achieve the effect。

Hello, to use cosine annealing you only have to define the Scheduler as:
scheduler = Scheduler21(optimizer, len(train_loader), epochs, warmdown_type="cosine")
Don't forget to call scheduler.step() after each optimizer.step()!
You can check the evolution of the learning rate with the following snippet:
import pylab as plt
learning_rates = []

for epoch in range(1, epochs + 1):
  for x, y in train_loader:
    ## your code here

    optimizer.step()
    scheduler.step()

    for group in optimizer.param_groups:
      learning_rates.append(group['lr'])

plt.semilogy(learning_rates)
plt.show()

Note that additional tweaks might be necessary if you use different param groups, but that's not the default user's behavior.

Hello, to use cosine annealing you only have to define the Scheduler as: scheduler = Scheduler21(optimizer, len(train_loader), epochs, warmdown_type="cosine") Don't forget to call scheduler.step() after each optimizer.step()!
You can check the evolution of the learning rate with the following snippet:
import pylab as plt
learning_rates = []

for epoch in range(1, epochs + 1):
  for x, y in train_loader:
    ## your code here

    optimizer.step()
    scheduler.step()

    for group in optimizer.param_groups:
      learning_rates.append(group['lr'])

plt.semilogy(learning_rates)
plt.show()

Note that additional tweaks might be necessary if you use different param groups, but that's not the default user's behavior.
Thank you very much for your reply, but in fact, this is the case
lr_ config = dict(
_ delete_= True,
policy='CosineAnnealing',
min_ lr_ ratio=0.01,
warmup='linear',
warmup_ iters=2000,
warmup_ ratio=0.001)
runner = dict(type='EpochBasedRunner', max_epochs=25)
optimizer = dict(
_ delete_= True, type='AdamW', lr=0.0001, weight_ decay=0.05,
paramwise_ cfg=dict(
custom_ keys={
'pos_ embed': dict(decay_mult=0.),
'norm': dict(decay_mult=0.),
'bias': dict(decay_mult=0.)
}))
This is a section in a config file in mmdet. I don't see any change methods for setting training strategies. I can only register Ranger with the optimizer。like this
lr_config = dict(
delete=True,
policy='CosineAnnealing',
min_lr_ratio=0.01,
warmup='linear',
warmup_iters=2000,
warmup_ratio=0.001)
runner = dict(type='EpochBasedRunner', max_epochs=50)
optimizer = dict(type='ranger22')

I am not familiar with mmdet. Perhaps you should ask in their repo how to use a custom scheduler.

I am not familiar with mmdet. Perhaps you should ask in their repo how to use a custom scheduler.
OK, thank you for your generous answer

custom scheduler.

By the way，Avoid my misunderstanding of ranger22，I want to ask，Is this the right way to use it
from ranger22 import Ranger22
optimizer = Ranger22(model.parameters())
and i just adjust the lr in the ranger22.py。
because I haven't figured out the most correct way to use Ranger, haha。

If you want to use the optimizer and the scheduler, the correct way to do so would be:
from ranger22 import Ranger22
from scheduler22 import Scheduler22

optimizer = Ranger22(model.parameters(), lr=5e-3)
scheduler = Scheduler21(optimizer, epochs, len(train_loader))

for epoch in range(1, epochs + 1):
  for x, y in train_loader:
    # Forward pass

    optimizer.step()
    scheduler.step()

If you want to use the optimizer and the scheduler, the correct way to do so would be:
from ranger22 import Ranger22
from scheduler22 import Scheduler22

optimizer = Ranger22(model.parameters(), lr=5e-3)
scheduler = Scheduler21(optimizer, epochs, len(train_loader))

for epoch in range(1, epochs + 1):
  for x, y in train_loader:
    # Forward pass

    optimizer.step()
    scheduler.step()

I'm very sorry. I thought it could be solved, but I can only ask you for help again.
https://github.com/facebookresearch/mae/blob/main/main_finetune.py
I want to try to add Ranger to some existing finetune projects, such as the MAE finetune above.
I lived according to your premise
optimizer = Ranger22(model.parameters(), lr=5e-3)
scheduler = Scheduler22(optimizer, args.epochs, len(data_loader_train))
for args.epoch in range(1, args.epochs + 1):
    for x, y in data_loader_train:
        # Forward pass

        optimizer.step()
        scheduler.step()

but it doesnot work，Can you help me see how to modify it, at your convenience.
by the way，If I only import
optimizer = Ranger22(model.parameters(), lr=5e-3)
it can work，but The result is not as good as the original adamw。

You will have to be more specific: can you please report the error you have?

My modified version is as follows，https://github.com/wudizuixiaosa/finetune/blob/main/.gitattributes
when i run it，it will be stuck after outputting the network structure and will not continue

In fact, I feel that I failed to call scheduler successfully, which made the process unable to proceed, but I don't know how to change it, so I hope you can see the code I modified if it's convenient. I just imported and modified the parameter name in your way, but it can't work


Persisting user preferences & control values

When a user visits a webviz app, settings of different controls are currently not remembered on the "next visit" to the app.
By using Dash functionality in version >= 1.3, we can make the app remember user selected values.
Reference: https://dash.plotly.com/persistence

After #260 and #243, webviz-config plugins can now use Dash persistence out of the box.


Made tools directory and updated .toml

Change Summary
Completed these two action items from ISSUE #75. Organized already committed files into 'tools' directory.
- We need to add pandas, openpyxl to poetry dependency list
- Made a new directory called 'tools' under the 'imap_processing', as well as 'tools/xtce_generation' which will house the files to make xtce

Updated Files

pyproject.toml

pandas = ">=2.0.0,<3" & openpyxl = ">=3.0.7,<4"

The precommit is going to fail because you didn't update the poetry.lock file- once you've fixed that and everything is green, this looks good to me!


Include Bogota and Colombia in the tool

I added Colombia as region, as well as Bogota, which has more specific items for the capital.

Hello @angoca ! Thanks for your contribution !
Could you give me write access to your repository so I can finalize this PR ?
Thanks.

Hi Jonathan,
I gave you write access.
You can proceed.
Andres Gomez

On 5/11/2021, at 3:00 AM, Jonathan Beliën @.***> wrote:
Hello @angoca https://github.com/angoca ! Thanks for your contribution !
Could you give me write access to your repository so I can finalize this PR ?
Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/osmbe/osm-welcome-tool/pull/133#issuecomment-961694014, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACH3LJTD2W4QJSIJRQTBPDUKOFK3ANCNFSM5HNA34VA.
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Deployed 🚀

https://welcome.osm.be/colombia/list
https://welcome.osm.be/bogota/list


[FEATURE] Kubevirt GPU/vGPU Configuration

Kubevirt has support for provisioning GPUs and other host devices according to the docs
There is also an existing Nvidia plugin which will discover GPUs and vGPUs on nodes so they can be added to VMs.
It would be great if the Harvester UI could display available GPUs and add the necessary configuration into VirtualMachineInstances.
I've done some experimenting with assigning GPUs to Harvester-provisioned VMs which I documented in this gist
Edit: Include GPU support for VMs and K8s workloads.

Added a line to the description above: “Include GPU support for VMs and K8s workloads.”

Adding "hookscripts" that allow a script to run during different phases of the VM lifecycle would be beneficial when dealing with binding, unbinding and cleaning up the VFIO-PCI drivers.

Hi,  @tlehman  should this function include UI related changes?  If so, please help to add require/ui label.  thanks

The version of this feature we are targeting for release 1.1.0 will be based on #992 and use PCI Passthrough. The driver for your specific GPU will need to be installed in a VM, but the VM will have direct access to the underlying PCI device by using PCI passthrough.

The vGPU work is being tracked in #2764


Scale mutation queue with multiple shards

Tables with an incrementing value in the lower bits of the primary key are notoriously hard to scale because all the writes will go to a single server.  This PR allows the queue to scale by adding write support for multiple queues.
Depends on: read by high/low watermarks #1045
Part of: retry-able mutations #1044

Updated and nice and green. PTAL.
Happy to walk through the code with you

PTAL

Thanks very much for the review!


Add VVenC Support

With FFᴍᴘᴇɢ 7.1 classifying VVC decoding as now being stable, FFᴍᴘᴇɢ having optional support for VVC encoding via VVenC, apps like Shutter Encoder enabling VVC encoding via a GUI, and VVC being even more efficient than AV1 with regard to the video file sizes necessary to achieve similar visual quality, I hope ab-av1 also gains support for calculating which QP to use to achieve VMAF 95.
My hard drives are crying out for an even more efficient way to store archival video encodes.

Any clients supporting this? :D

If ffmpeg can use an encoder then ab-av1 should also be able to with -e. Is there anything else this vcodec would need from this project rather than ffmpeg?

afaics there is no work needed here since we already support any ffmpeg vcodec. Let me know if there are any tweaks required for this vcodec.


added lottie widget

@sibteali786 you need to resolve your build issues. Are you using pnpm to install your dependencies? The pnpm-lock.yaml changes should be commited also.


Release notes for Mx4PC version 2.10.2 (planned release date 17 March)

We've prepared a patch version of Mx4PC - containing bugfixes and small improvements. We're planning to release this on March 17.

Merged after confirming with @zlogic


Serialization.rst: Fix typo

Just fixing a typo in docs/Serialization.rst.

LGTM

@swift-ci smoke test and merge


Release v0.1.0

Problem
With the completion of v0.1.0 a release needs to be deployed representing a stable version
Solution
For now follow a manual release process

[x] Create a release/0.1 branch off of main Ensure github test actions run
[x] Tag as v0.1.0-rc1

[x] git tag
[x] Deploy docker image version

[ ] Integration Testing

[x] Deploy tagged version
[x] Manual testing against Integration
[x] Run acceptance tests (if yet available)
[ ] Run performance tests (if yet applicable)

[ ] Previewnet Testing

[x] Deploy tagged version
[x] Manual testing
[ ] Run acceptance tests (if yet available)
[ ] Run performance tests (if yet applicable)

[ ] Testnet Testing

[x] Deploy tagged version
[ ] Manual testing
[x] Run acceptance tests (if yet available)
[ ] Run performance tests (if yet applicable)

[ ] Tag as v0.1.0

[x] git tag
[x] Deploy docker image version
[x] Write up release notes and changelist

Let bake
[ ] Mainnet Testing

[ ] Deploy tagged version
[ ] Manual testing
[ ] Run acceptance tests (if yet available)
[ ] Run performance tests (if yet applicable)

Any bugs or missed features found should see a new ticket opened, addressed in main and cherry-picked to release/0.1 with a new rc version tagged and docker image deployed
Alternatives
Automate process

v0.1.0 saw successful release to previewnet.
At this point v0.2.0 was deemed a priority and focus was shifted.
Remaining items are not applicable and learnings have been incorporated into next version


Updating and fixing multi-select for attributes

Component references were simply ignored for multi-select situations.  Now the default and only way is to reference an array of all selected objects.  This prevents accidental creation of attributes that only support single selection in the future.
OnEditorChange has been updated to properly dispatch callbacks to all selected objects instead of just throwing tantrum.  AutoFind also had to be changed, since it also used the component reference, but AutoFind was left to be a component that only works when single-selecting.
To test:

[ ] Make sure both OnEditorChange and AutoFind still work properly!
[ ] Make sure OnEditorChange correctly dispatches callbacks to multiple selected objects

Yep! Awesome.


Fix ProblemSolver::maxIterPathPlanning

@florent-lamiraux this might have had impacts on some time-consuming constraint resolution.


[nextra-theme-blog] Allow for a draft post

Currently, you can get around this by putting a _ as part of the .mdx file although, I think it would be helpful to have a metadata field like:
---
type: post
draft: true
---

That way, it won't display in the posts list although still have a link that you can send out to people to review that will remain the same after publishing.

@rgoomar

you can get around this by putting a _ as part of the .mdx file although

Do you mind sharing more details on how to make it work? Thx!

@YikSanChan Basically, if you have an _, it ignores it based on this line https://github.com/shuding/nextra/blob/core/packages/nextra-theme-blog/src/index.tsx#L126
So you can link to it as a post, but doesn't appear in the list.


Fix build warning

From the pipeline output:

lets create targeted ones as we're in much better shape now


Select contains jsonb array

Bug report
Describe the bug
I'm not sure if this is a bug or if I am missing something. I have a jsonb array column containing object (Player) converted in json. I'm trying to fetch all rows whose jsonb array column contains a specific object.
The insert method works fine doing
await dbClient.from('polls')
  .insert([{
    ...
    "items" : poll.items.map((i) => i.toJson()).toList(),
    ...
  ])
  .execute()

and getting something like
[
  {
    "votes": 0,
    "player": {
      "id": "409",
      "name": "INSIGNE",
      "team": "Napoli"
    },
    "description": ""
  }
]

The select method gives me different errors and I can't fetch no rows.
To Reproduce
// Method 1
  .contains('items', poll.items.map((i) => i.toJson()).toList())

I get following error:
[GETX] Error: DbRequest Instance of 'DbRequest' invalid input syntax for type json

// Method 2
  .contains('items', jsonEncode(poll.items))

I get following error:
[GETX] "Error: DbRequest Instance of 'DbRequest' malformed array literal: "[{"player":{"id":"409","name":"INSIGNE","team":"Napoli"},"votes":0,"description":""},{"player":{"id":"2530","name":"IBRAHIMOVIC","team":"Milan"},"votes":0,"description":""}]""

Expected behavior
Given a list of objects, I want to select from table all rows that partially match (contains at least one element) with the element stored in jsonb array column.
Thank you in advance.

Maybe .contains('items', json.encode(poll.items.map((i) => i.toJson()).toList())) ?

Maybe .contains('items', json.encode(poll.items.map((i) => i.toJson()).toList())) ?

unfortunately it doesn't work, I get same error
"Error: DbRequest Instance of 'DbRequest' malformed array literal: "[{"player":{"id":"409","name":"INSIGNE","team":"Napoli"},"votes":0,"description":""},{"player":{"id":"2530","name":"IBRAHIMOVIC","team":"Milan"},"votes":0,"description":""}]""

Maybe .contains('items', poll.items.map((i) => json.encode(i.toJson())).toList());?

I get the same error, I don't know why in this way seems to be a map instead as a list
"Error: DbRequest Instance of 'DbRequest' malformed array literal: "{"{"player":{"id":"409","name":"INSIGNE","team":"Napoli"},"votes":0,"description":""}","{"player":{"id":"2530","name":"IBRAHIMOVIC","team":"Milan"},"votes":0,"description":""}"}""


This PR updates the present ITs w.r.t gw redundancy and adds new ones

This PR is outdated. closing.


MAINTAINER NEEDED!

hi everyone,
my heart is bleeding when I see several-month-aged issues here, but currently I really can't find time to support x-editable-yii. Also my work is not related with yii already for a long time and it's became difficult for me to find and understand problems.
To not let this project get frozen I would be very gracefull if someone will be ready to maintain and support it. Mainly the work is porting features from root x-editable to this extension.
I'm continuing to maintain root x-editable and will provide any personal help needed for x-editable-yii.
Thanks a lot!

Hello, this can support yii2?


Update Scalafmt

I don't really like the extra newlines but whatever.

Codecov Report

Merging #1331 into master will increase coverage by 0.29%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #1331      +/-   ##
==========================================
+ Coverage   85.77%   86.06%   +0.29%
==========================================
  Files          74       74
  Lines        2537     2534       -3
  Branches      108      108
==========================================
+ Hits         2176     2181       +5
+ Misses        361      353       -8

Impacted Files
Coverage Δ

.../core/shared/src/main/scala/io/circe/ACursor.scala
71.42% <ø> (ø)
:arrow_up:

.../core/shared/src/main/scala/io/circe/Encoder.scala
85.93% <ø> (ø)
:arrow_up:

.../core/shared/src/main/scala/io/circe/Printer.scala
94.33% <ø> (ø)
:arrow_up:

...in/scala/io/circe/testing/ArbitraryInstances.scala
100% <ø> (ø)
:arrow_up:

...les/core/shared/src/main/scala/io/circe/Json.scala
78.78% <ø> (+0.6%)
:arrow_up:

...re/shared/src/main/scala/io/circe/JsonObject.scala
96.21% <ø> (ø)
:arrow_up:

...o/circe/generic/util/macros/DerivationMacros.scala
77.35% <ø> (ø)
:arrow_up:

...ain/scala/io/circe/literal/JsonLiteralMacros.scala
61.17% <ø> (ø)
:arrow_up:

...core/shared/src/main/scala/io/circe/CursorOp.scala
53.12% <ø> (ø)
:arrow_up:

...jawn/src/main/scala/io/circe/jawn/JawnParser.scala
100% <ø> (ø)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0bbf35...76050a9. Read the comment docs.


Which props should i use to change the border colour for the selector

There is so many props to customise style, and yet i cannot change the border colour by using style props, any idea how to change the border colour for the selector?

The modal is instantiated from a RN Modal.  Whether its border can be customized depends on what RN offers.
https://github.com/peacechen/react-native-modal-selector/blob/master/index.js#L299
You are welcome to submit a PR if you find a way to customize the modal the way you like.


chartType updated

There was an issue (https://github.com/schme16/Chart.js-RangeSlider/issues/7). That was fixed

Thank you


Amazon Order History Report Stuck

Hi. I tried to use the software like I do every month and I can't get Amazon to process the report. They get stuck on Processing (0%). Am I the only one to have that issue? Hard to tag is the raw material is not available...

Hi. I have the exact same issue. Even when creating reports manually. It is stuck at "0% processing". It seems like an error on Amazon's end.

Thank you both for the report! I am seeing the same thing (unfortunately).
Please report back if your report completes! Note: If you rerun the tagger on the same day (or select the same range of dates), it will reuse the already processed report instead of requesting another one. However, it will require a few rounds, as it will block on items, then orders, then refunds.

Glad to know I am sane. I called amazon bu they weren't helpful. I think we need to get a technical ticket somewhere but unsure where to ask. Any ideas?

https://www.amazonforum.com/s/question/0D56Q00008LqV79SAF/order-history-reports-csv-download-

You have to wait a day or two for the report to process. I manually triggered the reports, then came back two days later and they were ready. Oddly, I did one test that was only 5 days and that one is still running.

I did tried for 3 weeks now. Eventually they all failed and no data was released. It seems the problem happens a lot but I gave up calling amazon. Ask 5 times to notify the technical team and they always fail to send the email with the ticket number...

I've had some intermittent success as well. Keep trying @ovitch (maybe try smaller date ranges?). However, there clearly is an issue/bug on Amazon's side; I imagine this will clear up sometime in January on "it's own".  But today by some kind of miracle it finally worked. Did it in 2 sec for 2021 (all year). Hopefully this works with the tagger!

@ovitch I have noticed the same thing! Please give the tagger another go and reopen this issue if it happens again or file a new one if you run into a new problem. Thanks for the early initial alert!


initial 0.18.16

aws-crt-cpp 0.18.16
Destination channel: defaults
Links
{ticket_number}
Upstream repository
dependency for aws-sdk-cpp 1.10.55 -> arrow-cpp 14.0.1 -> pyarrow 14.0.1 which would address CVE

Linter check found the following problems:
ERROR conda.cli.main_run:execute(124): `conda run conda-lint /tmp/abs_9b52siw6g1/clone` failed. (See above for error)
The following problems have been found:
===== ERRORS =====

clone/recipe/meta.yaml:26: patch_must_be_in_build: patch must be in build when source/patches is set.
===== Final Report: =====
1 Error and 0 Warnings were found


chore(main): release 1.9.0

:rocket: New release
1.9.0 (2023-09-13)
:sparkles: Features

add support for custom date formatting (09a204c)

:bug: Bug Fixes

logo not displaying in readme, closes #106, PR (#107) (7042eaf)

:hammer: Housekeeping

update list of websites using this theme (#110) (366a762)
update post title (e0abd99)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/hugo-sid/hugo-blog-awesome/releases/tag/v1.9.0 :sunflower:


CI testing

[ ] enable Travis at https://travis-ci.org/cyberFund (@litvintech, need your help)
[ ] add .travis.yml with language: java to the top (it will pick up gradle automatically)
[ ] add status badge to README.md

You can also use default gradle wrappers.
gradlew.bat build or ./gradlew  build will download, cache and run gradle build.


API GatewayV2 GetApi with an empty string: not implemented

Hi, I am trying to test the use case where I try to get a REST API with an empty string.
Using the AWS API, I receive a 200 OK with no content:
import boto3
import json

client = boto3.session.Session().client("apigatewayv2")
resp = client.get_api(ApiId="")

print(json.dumps(resp, default=str))

Response:
{
    "ResponseMetadata": {
        "RequestId": "1234567890",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
            "date": "Tue, 13 Aug 2024 12:06:06 GMT",
            "content-type": "application/json",
            "content-length": "14951",
            "connection": "keep-alive",
            "x-amzn-requestid": "1234567890",
            "x-amz-apigw-id": "1234567890"
        },
        "RetryAttempts": 0
    }
}

When using moto, I get a 404, Not yet implemented response:
import boto3
import moto

with moto.mock_aws():
    client = boto3.session.Session(region_name="us-east-1").client("apigatewayv2")

    resp = client.get_api(ApiId="")

I've been looking through the code to have a go at adding this myself, but I am a bit stuck, so I'd appreciate some guidance. I don't think I have enough to open a draft PR, that's why I opened this as an issue.
I changed the URL regex in apigatewayv2/urls.py to be * instead of + , to make my test case get a bit further:
"{0}/v2/apis/(?P<api_id>[^/]*)$": ApiGatewayV2Response.dispatch,

but the uri_to_regexp still returns a regex that will not match the uri of the request when the ApiId is an empty string.
Any feedback will be appreciated!

Hi @juanitosvq, thanks for trying to fix this in Moto! That is quite odd behaviour from AWS - I would expect them to return a 400 here instead.
The dispatch-method uses the URL's supplied by botocore, and botocore seems to also always expect the api_id to have some value.
Instead of fighting the url-matcher, and challenging/changing all the assumptions that the dispatch-method makes, I would suggest adding a custom method to handle this use-case. It is an outlier - I don't think I've seen anything else like this from AWS - so I'm not opposed to having a custom handler to solve this.
diff --git a/moto/apigatewayv2/responses.py b/moto/apigatewayv2/responses.py
index 18d6b1b83..553b1b8e7 100644
--- a/moto/apigatewayv2/responses.py
+++ b/moto/apigatewayv2/responses.py
@@ -20,6 +20,9 @@ class ApiGatewayV2Response(BaseResponse):
         """Return backend instance specific for this region."""
         return apigatewayv2_backends[self.current_account][self.region]

+    def get_api_without_id(self, *args):
+        return 200, {}, "{}"
+
     def create_api(self) -> TYPE_RESPONSE:
         params = json.loads(self.body)

diff --git a/moto/apigatewayv2/urls.py b/moto/apigatewayv2/urls.py
index 72f4922d6..013550a8e 100644
--- a/moto/apigatewayv2/urls.py
+++ b/moto/apigatewayv2/urls.py
@@ -9,6 +9,7 @@ url_bases = [

 url_paths = {
     "{0}/v2/apis$": ApiGatewayV2Response.dispatch,
+    "{0}/v2/apis/$": ApiGatewayV2Response.get_api_without_id,
     "{0}/v2/apis/(?P<api_id>[^/]+)$": ApiGatewayV2Response.dispatch,
     "{0}/v2/apis/(?P<api_id>[^/]+)/authorizers$": ApiGatewayV2Response.dispatch,
     "{0}/v2/apis/(?P<api_id>[^/]+)/authorizers/(?P<authorizer_id>[^/]+)$": ApiGatewayV2Response.dispatch,

If you could just add a small comment inside the new method to explain this usecases this serves, that would minimize future head scratching. :slightly_smiling_face:
Note that I haven't linted this yet, so make sure to run make lint at the end to make sure our CI is happy - see https://docs.getmoto.org/en/latest/docs/contributing/installation.html#installing-moto-locally


backup 空表时会一直SNAPSHOTING

备份到阿里云的oss库，如果表里有空表，会一直在SHAPSHOTING状态
Steps to reproduce the behavior (Required)

CREATE TABLE IF NOT EXISTS lineorder_flat (
LO_ORDERKEY int(11) NOT NULL COMMENT "",
LO_ORDERDATE date NOT NULL COMMENT "",
LO_LINENUMBER tinyint(4) NOT NULL COMMENT "",
LO_CUSTKEY int(11) NOT NULL COMMENT "",
LO_PARTKEY int(11) NOT NULL COMMENT "",
LO_SUPPKEY int(11) NOT NULL COMMENT ""
) ENGINE=OLAP
DUPLICATE KEY(LO_ORDERKEY)
COMMENT "OLAP"
DISTRIBUTED BY HASH(LO_ORDERKEY) BUCKETS 192
PROPERTIES (
"replication_num" = "1",
"colocate_with" = "groupxx1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);

Expected behavior (Required)
BACKUP SNAPSHOT  test.lineorder_flat TO test_repo ON (lineorder_flat);
Real behavior (Required)
show backup;
State: SNAPSHOT
StarRocks version (Required)
版本 2.0

hi, thank you for your issue.
I use starrocks 2.0-GA to reproduce the process. I can get a success in backup the empty table.

I think you can take a look at the fe.log to check whether there is any warning or error logs.
Maybe there is something wrong with your broker?