Add my school

@phuongttc file/lib/domains/vn/edu/fst.txt: this request has an incorrect file path format, please ensure that you are familiar with     this repository rules located at the bottom of this page and change     your request accordingly. If you don't know how to change request, close it and open a new one.     If you are not able to make the pull request according the instructions, please send us an email to<EMAIL_ADDRESS>Please mention this request URL in the message.

@phuongttc First stage of verification passed, official domain is huit.edu.vn as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision. Proof of domain ownership: https://foodtech.huit.edu.vn/Introduction.aspx?c=3

@phuongttc Thank you for your patience while waiting for a review and processing your request. Pull request merged.


The branch-planner does not clean up after running

User story
As a WeGO user who’s working with Terraform resources, I would like the Terraform CRs I create to be cleaned up, so that I don’t have to manually remove orphaned resources.
Acceptance Criteria

[ ] The Controller can delete every TF resource generated by the branch planner, and there are no orphaned resources.
[ ] When the user isn’t happy with a PR and closes it, the branch planner clears up all affected objects.
[ ] When no resources are referring to the Git repo object that points at the PR branch then and only then does it delete the Git repo source.

Notes About Implementation

Establish a 1:1 relationship between source and object. This is based on an assumption that it’s already 1:1 but not working/set up that way.
Test and guarantee this 1:1 property.
Write test case to generate the multiple objects and source and check the number to guarantee it’s working. If you create five objects you have to have five sources as well.

When we have the branch planner installed and enabled, the additional Terraform CRs for branches are created, but never cleaned up, leaving orphaned resources that need to be manually removed:
kc get terraforms.infra.contrib.fluxcd.io -A
NAMESPACE     NAME                                                                                READY     STATUS                                                                                AGE
flux-system   example-wge-workshops-dynamodb-table                                                 True      No drift: v4.38.0-v1alpha11@sha256:6033f3b9fb6458dda7f432ea5ce1a8dbb9560c2388d612f0d1036ba03cf2e063   5d17h
flux-system   irsa-role-as-demo-dev-01-external-dns                                                Unknown   Terraform Planning                                                                                4d19h
flux-system   irsa-role-as-demo-dev-01-external-dns-delete-clusters-branch-7i71ib-33               False     error running Plan: rpc error: code = Internal desc = exit status 1...                                116m
flux-system   irsa-role-as-demo-dev-01-external-dns-delete-clusters-branch-cuy98h-31               False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-cuy98h-31' not found             118m
flux-system   irsa-role-as-demo-dev-01-external-dns-delete-clusters-branch-h066yf-37               False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-h066yf-37' not found             21m
flux-system   irsa-role-as-demo-dev-01-external-dns-delete-clusters-branch-h3h05j-29               False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-h3h05j-29' not found             16h
flux-system   irsa-role-as-demo-dev-01-external-dns-gitopssets-pullrequests-19                     False     Source 'GitRepository/flux-system/flux-system-gitopssets-pullrequests-19' not found                   35h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-1ltgan-36                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-1ltgan-36' not found                  46m
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-2d0th-25                     False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-2d0th-25' not found                   19h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-4ft00a-18                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-4ft00a-18' not found                  39h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-78sved-26                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-78sved-26' not found                  17h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-91djwn-38                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-91djwn-38' not found                  14m
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-bm0vu8-20                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-bm0vu8-20' not found                  23h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-cp0rpj-23                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-cp0rpj-23' not found                  19h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-epjgcg-28                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-epjgcg-28' not found                  17h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-m38e4t-17                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-m38e4t-17' not found                  39h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-nfe4lf-14                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-nfe4lf-14' not found                  41h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-nnuxlf-22                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-nnuxlf-22' not found                  19h
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-qy94gf-35                    False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-qy94gf-35' not found                  51m
flux-system   irsa-role-as-demo-dev-01-external-dns-wge-create-branch-wkbgx-27                     False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-wkbgx-27' not found                   17h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-delete-clusters-branch-cuy98h-31   False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-cuy98h-31' not found             118m
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-delete-clusters-branch-h066yf-37   False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-h066yf-37' not found             21m
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-delete-clusters-branch-h3h05j-29   False     Source 'GitRepository/flux-system/flux-system-delete-clusters-branch-h3h05j-29' not found             16h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-gitopssets-pullrequests-19         False     Source 'GitRepository/flux-system/flux-system-gitopssets-pullrequests-19' not found                   35h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-1ltgan-36        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-1ltgan-36' not found                  46m
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-2d0th-25         False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-2d0th-25' not found                   19h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-4ft00a-18        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-4ft00a-18' not found                  39h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-78sved-26        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-78sved-26' not found                  17h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-91djwn-38        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-91djwn-38' not found                  14m
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-bm0vu8-20        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-bm0vu8-20' not found                  23h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-cp0rpj-23        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-cp0rpj-23' not found                  19h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-epjgcg-28        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-epjgcg-28' not found                  17h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-m38e4t-17        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-m38e4t-17' not found                  39h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-nnuxlf-22        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-nnuxlf-22' not found                  19h
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-qy94gf-35        False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-qy94gf-35' not found                  51m
flux-system   secret-store-irsa-as-demo-dev-01-external-secrets-wge-create-branch-wkbgx-27         False     Source 'GitRepository/flux-system/flux-system-wge-create-branch-wkbgx-27' not found                   17h

related to https://github.com/weaveworks/tf-controller/issues/898

I tested this with the latest version 0.16.0-rc.3 and this will only clear up correctly for 1 terraform resource being planned by the  branch planner.  If you have more than 1 resource then the closing of the PR results in:
1 - delete the first terraform resource - completes correctly.
2 - deletes the git repository source referring to the PR branch
3 - Now all other TF resources are stuck as now there is no valid git repo object, blocking deletion of any other TF resources.
Workaround is to remove the finalizer and delete manually.
Workflow we would need:
1 - delete each terraform resource that was generated by the branch planner.
2 - When no resources are referring to the git repo object pointing at the PR branch, then and only then, delete the Git repo source.

If you have more than 1 resource then the closing of the PR results in

Why does a branch has more then one PR? Or did I misunderstood that?

Next step is to test it again with the recent version, then can conclude it.

I don't think the latest release is going to fix this issue. The server iterates through all of the planner objects here and deletes both the plan object and source object at the same time for each object. If those plan objects share a source object, the first plan object will be successfully deleted still, but subsequent attempts to delete the Terraform objects will bail here due to the 404 returned by the k8s API.

If those plan objects share a source object

When can this happen? Right now (as far as I know) the branch planner creates a clone from the source for each and every branch planner Terraform object. That was (kind of) my question back in Oct, how can two separate PR Terraform object refer to the same Source object, or how can a single PR have more than one planner Terraform resources, or basically any combinations like that.
The planner uses a very simple flow:

Find the source TF resource
Find its Source
Clone Source (change some values like branch)
Clone TF resource (change some values, like source ref, only plan, etc)
When the PR is closed, delete the Source and the TF Resource.

@chanwit (iirc) had a patch around a month ago to reverse the deletion order of the Source and TF resources so the Terraform resource is deleted first.

The case I ran into was having multiple Terraform resources which referred to the same source but had different paths. In that case,all of the Terraform resources referencing the original source are duplicated while only one instance of the source is created.
My company currently uses a TF mono repo for our multiple environments so we need the ability for multiple Terraform resources to reference a single source and be cleaned up properly.

I think that's an issue with the planner creature, it should clone all resources always, no matter how many TF references you have to the same Source, if you have 5 PRs across all TF resources, you should have 5 separate clone of the Source object, same or different doesn't matter. Can you please verify the issue is there in recent version too?

I'm seeing that the root path is special cased in the reconciliation code so I should add that due to a migration my team is still working on, we have both the root of the repo and subdirectories in the repo configured as Terraform resources.
On Thu, Nov 16, 2023, at 7:40 AM, Danny Seymour wrote:

I deployed 0.16-rc.3 when I tested and did not get multiple copies of the source object. This issue does still exist.
On Thu, Nov 16, 2023, at 12:26 AM, Balazs Nadasdi wrote:

I think that's an issue with the planner creature, it should clone all resources always, no matter how many TF references you have to the same Source, if you have 5 PRs across all TF resources, you should have 5 separate clone of the Source object, same or different doesn't matter. Can you please verify the issue is there in recent version too?
—
Reply to this email directly, view it on GitHub https://github.com/weaveworks/tf-controller/issues/923#issuecomment-1813988023, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYDZ4QKBCENL5K7XVIBRQLYEXE3NAVCNFSM6AAAAAA4M2YWF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJTHE4DQMBSGM.
You are receiving this because you commented.Message ID: @.***>

Looking through the code, it looks like the source for a Terraform resources related to a PR is named only based on the original source name and the PR id: https://github.com/weaveworks/tf-controller/blob/main/internal/server/polling/terraform.go#L145
If multiple Terraform resources are cloned for planning for a single PR, the name of the source object would always be the same for each Terraform resource. The controller may attempt to reconcile the source multiple times, but would only create it once due to the name collision.
On Thu, Nov 16, 2023, at 7:46 AM, Danny Seymour wrote:

I'm seeing that the root path is special cased in the reconciliation code so I should add that due to a migration my team is still working on, we have both the root of the repo and subdirectories in the repo configured as Terraform resources.
On Thu, Nov 16, 2023, at 7:40 AM, Danny Seymour wrote:

I deployed 0.16-rc.3 when I tested and did not get multiple copies of the source object. This issue does still exist.
On Thu, Nov 16, 2023, at 12:26 AM, Balazs Nadasdi wrote:

I think that's an issue with the planner creature, it should clone all resources always, no matter how many TF references you have to the same Source, if you have 5 PRs across all TF resources, you should have 5 separate clone of the Source object, same or different doesn't matter. Can you please verify the issue is there in recent version too?
—
Reply to this email directly, view it on GitHub https://github.com/weaveworks/tf-controller/issues/923#issuecomment-1813988023, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYDZ4QKBCENL5K7XVIBRQLYEXE3NAVCNFSM6AAAAAA4M2YWF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJTHE4DQMBSGM.
You are receiving this because you commented.Message ID: @.***>

I went through all possible cases I can think of and finally I understand the use-case and the issue.
Use-case:

There are 4 Terraform resources (t1, t2, t3, t4)
There is one Source resource (s1)
t1 uses s1 with path to /team-a.
t2 uses s1 with path to /team-b.
t3 uses s1 with path to /team-c.
t4 uses s1 with path to /team-d.

If we create a PR that changes /team-a and /team-b at the same time, the system will try to create t1-clone and t2-clone with s1-clone and s1-clone. As it's the same PR, the name of the two generate source IDs will be the same therefore they will share that resource.
If EnablePathScope is false (default as I remember), the system will try to create all four (t1-clone, t2-clone, t3-clone, t4-clone).
@thefirstofthe300: Am I on the right track with this?

Solution:

On user side: turn on EnablePathScope to reduce the issue and create smaller, scoped PRs. That's useful in general as you don't want to create new resources for a branch where you have no changes. (it still doesn't resolve the issue if a PR contains changes in different scopes)
Add original terraform name (or hash of it) to the name of the generate source to avoid conflicts. I think it's much easier and cheaper than always checking if there are resources, add an incremental number, and add extra labels to track which on is the original.

@chanwit: Any idea, notes, or concerns?

@yitsushi You're on the right track. One special aspect of our environment is that we not only utilize paying for Terraform resources but we also use the root of the repo for a Terraform resource.
In other words, given three resources tf1, TF2, and tf3, tf1 has a path of ./, tf2 a path of ./tf2 and tf3 a path of ./tf3.
I know, it's a terrible layout that I wouldn't recommend to anyone, but it's what I inherited.

Thanks for the clarification, now I understand the issue and the use case better.


Fixes/transaction type

Pull request type

Please check the type of change your PR introduces:

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Other (please describe):

What is the current behavior?

Missing Transaction type in some of the class.

Asana URL: https://app.asana.com/0/1201929997897801/1202621864500255
What is the new behavior?

added transaction type in missing class files

Does this introduce a breaking change?

[ ] Yes
[x] No

Related changes

Other information

closing this branch, wrong branch


Haha


Fix error handling in x509_lu.c

can you please re-confirm?

Merged to master as c70e2ec, and 1.1.1 as 5d16346.  Thanks!


Delete last line on datasets/simple-join/output.jsonl

issue#128
Hi marcenacp,
last line on datasets/simple-join/output.jsonl deleted, please check it out

recheck

Hi @viktoriussuwandi, thanks for the PR!
You changed the test. Now:

You would need to change the code that takes care of the join: https://github.com/mlcommons/croissant/blob/main/python/ml_croissant/ml_croissant/_src/operation_graph/operations/join.py
You could add unit tests in the corresponding test: https://github.com/mlcommons/croissant/blob/main/python/ml_croissant/ml_croissant/_src/operation_graph/operations/join_test.py

Hi @viktoriussuwandi, thanks for the PR!
You changed the test. Now:

You would need to change the code that takes care of the join: https://github.com/mlcommons/croissant/blob/main/python/ml_croissant/ml_croissant/_src/operation_graph/operations/join.py
You could add unit tests in the corresponding test: https://github.com/mlcommons/croissant/blob/main/python/ml_croissant/ml_croissant/_src/operation_graph/operations/join_test.py

Hi marcenacp,
Actually, I love to contribute on your repo, but I have no idea how to do these requests,
would you please let me know step by step ?
Appreciate your guidance

Hi @viktoriussuwandi, sorry for this! Here are some more detailed steps:

Read how the repo is designed in the README. In particular, this explains what "operations" are.
Read the description of the failing dataset: https://github.com/mlcommons/croissant/blob/main/datasets/simple-join/metadata.json. It joins two sets:

a list of publications
with a list of authors.

So the problem comes from the Join operation. After investigation, I think you would have to invert source and references to fix the issue: https://github.com/mlcommons/croissant/blob/main/datasets/simple-join/metadata.json#L99-L100
If you can, add a unit test with an example of a similar join between two pd.DataFrames in the corresponding test: https://github.com/mlcommons/croissant/blob/main/python/ml_croissant/ml_croissant/_src/operation_graph/operations/join_test.py
Update your code until the command pytest . (running pytest tests) is green.

Please, tell me if it's not clear. Thanks a lot!

@viktoriussuwandi The problem was tricky... I submitted https://github.com/mlcommons/croissant/pull/146, which better documents how to do joins.
I'll close this PR for now. Thanks again!


Refactor cpu to cores (resolves #220)

Failure doesn't look like it's related to this change. Transient?
Jenkins, test this please.

Some of the comments and strings were mangled by this change. A simple context-unaware search-and-replace just doesn't cut it in this case.
As it is, I would not want to merge this

This is mostly plural vs. singular. Like it should be "core usage" and not "cores usage".
I would also keep the changes to gridengine.py minimal. 1) because John and Alden are working on those modules and 2) we currently don't have good tests that would detect if this change breaks anything.

Jenkins, test this please.


edgedb: 3.4.0 -> 4.0.2

update default.nix src
update Cargo.lock

Description of changes
I wonder if Cargo.lock can be de-vendored, since it built with ${src}/Cargo.lock in a flake of mine as well.
The discussions in https://github.com/NixOS/nixpkgs/pull/221716 go into some depths, but I couldn't find a quick answer.
nix shell .#edgedb
edgedb -V
edgedb project init
# ...
edgedb
select "foo";
# {'foo'}
\quit
edgedb instance destroy -I nixpkgs --force
rm -rf dbschema
rm edgedb.toml

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Unlike what ofborg suggests, ${src}/Cargo.lock comes into existence on my machine. I'm not a "nix-pert" enough to say whether it was realized from other builds or so.
$ nix-store --query --referrers-closure /nix/store/f1qwbypqxv2xvkqpqf25p4h49j6caaj3-source
/nix/store/f1qwbypqxv2xvkqpqf25p4h49j6caaj3-source
/nix/store/ajfzjh6wlq4dl2l58mzc6dbn7xk412va-cargo-vendor-dir

I guess I revert 00dcab4 and then it's good to go? @kirillrdy @ranfdev

Unlike what ofborg suggests, ${src}/Cargo.lock comes into existence on my machine. I'm not a "nix-pert" enough to say whether it was realized from other builds or so.
$ nix-store --query --referrers-closure /nix/store/f1qwbypqxv2xvkqpqf25p4h49j6caaj3-source
/nix/store/f1qwbypqxv2xvkqpqf25p4h49j6caaj3-source
/nix/store/ajfzjh6wlq4dl2l58mzc6dbn7xk412va-cargo-vendor-dir

I guess I revert 00dcab4 and then it's good to go? @kirillrdy @ranfdev

yeah sorry I forgot to follow up,
"${src}/Cargo.lock" will trigger import from derivation which is disabled in hydra/ofborg to speed up eval time.
you can test this locally by adding
          allow-import-from-derivation = false

into your nix.conf
so yes, revert devendering of lock file

I'm sorry, I can't follow this. Could you remove me from the maintainers list? Thanks

maintainers list

Are the thumbs up a vote for emtpy maintainer list or for @kirillrdy to step up? Not sure if an empty list is acceptable ;)

maintainers list

Are the thumbs up a vote for emtpy maintainer list or for @kirillrdy to step up? Not sure if an empty list is acceptable ;)

empty list for existing package is acceptable. @ahirner if you are actively using edgedb then I would recommend you to add yourself as maintainer.  I am happy to be added as well. ( if you add yourself or not )

using edgedb ... happy to be added as well

Sounds great! I know the package but not enough of nix(pkgs).

@ahirner can you either squash commits, eg all maintainer related commits squashed.
as for hash change can be own commit ( or squashed ) just need to update commit message so that it matches contribution guidelines

apologies, i kinda told you a wrong thing

maintainers: add ahirner
edgedb: 3.4.0 -> 4.0.2
edgedb: update maintainers


VertSplit is deprecated in favor of WinSeparator

VertSplit is now deprecated on nightly. Linking WinSeparator to VertSplit should be enough to fix the issue.
https://github.com/neovim/neovim/blob/66f1563c7a48d76f99c89e32de030e57af2abfb4/runtime/doc/deprecated.txt#L80

It works, thanks!


Regression on title usage in need body

.. need:: Test with title
   :id: FOO_1234

   Title
   *****

   xxx

The following snippet works with at least sphinx_needs 2.0.0, but not anymore in sphinx_needs 4.0.0. It fails with
CRITICAL: Unexpected section title.

Is it expected or just a regression ?

Heya, yes it is expected:
it did not warn in v2, but that did not mean it worked since nested headings are fundamentally incompatible with the sphinx/docutils documentation model: https://gist.github.com/chrisjsewell/0c5827add50074fef0937e2543e955b4

Thanks you for the answer.  I will update our documents accordingly so (and probably try to fix place where we use internally nested_parse_with_tiles.


[Lua language def] Built in keywords are now listed properly

Removed commas and added spaces at the end of each concatenated string so these keywords are recognised as built-ins.

Well caught, thanks!


Add multigraph option toPyDiGraph.to_undirected

Closes #369.
Instead of relying to the default graph api that would keep the last edge data among the parallel edges we see, i added an optional weight_combo_fnargument that combines all edge data. Do note here that ifweight_combo_fn` is omitted, the behavior is the same as the default graph api.

Pull Request Test Coverage Report for Build 958212261

37 of 37 (100.0%)  changed or added relevant lines in 2 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.007%) to 97.52%

Files with Coverage Reduction
New Missed Lines
%

src/lib.rs
1
99.32%

Totals

Change from base Build 957466827:
0.007%

Covered Lines:
8140

Relevant Lines:
8347

💛  - Coveralls


Add harvester-release.yaml file to track versions

Create a file to track component versions. e.g.,
harvester: f6bc010-dirty
harvesterChart: 0.0.0-dev
os: Harvester dfd4a9d-dirty
kubernetes: v1.21.5+rke2r1
rancher: v2.6-638b29a687f3818e7de676a573ec9b9156f193a9-head
monitoringChart: 100.0.0+up16.6.0

The file is stored in /etc/harvester-release.yaml and the release ISO.

harvester: The harvester version. Formal release tag or commit hash of harvester/harvester project.

harvesterChart: The harvester chart version, which is tracked in harvester ManagedChart.

os: The installer version. The version part (without Harvester  prefix) is identical to harvester for a formal release. We record the version in PRETTY_NAME option in the os /etc/os-release file as well.
PRETTY_NAME is the only thing that can indicate the OS version in Kubernetes nodes. It's exposed in .status.nodeInfo.osImage of a node:
# kubectl get node node1 -o yaml | yq e '.status.nodeInfo' -
architecture: amd64
bootID: 3f84cc62-5f0f-4105-9306-df7a56807b34
containerRuntimeVersion: containerd://1.4.9-k3s1
kernelVersion: 5.3.18-59.34-default
kubeProxyVersion: v1.21.5+rke2r1
kubeletVersion: v1.21.5+rke2r1
machineID: 9a47446a1c3842e6c9bbaac661a60ff2
operatingSystem: linux
osImage: Harvester dfd4a9d-dirty               <--- PRETTY_NAME
systemUUID: 2ffb9b8a-c6c7-4d65-9273-1a22da475e83

We can use the info to track if a node's OS is upgraded.

rancher: Embedded Rancher server version.

monitoringChart: Monitoring chart version, which is tracked in rancher-monitoring-crd and rancher-monitoring ManagedCharts.

Any reason for using the yaml format? Personally speaking, I prefer to align it with /etc/os-release

Any reason for using the yaml format? Personally speaking, I prefer to align it with /etc/os-release(or append those variables directly to /etc/os-release?)

It's easier to be parsed by yq in job scripts and unmarshalled in go codes.


Missing: TV - Production Countries

I'm not shure if this is only missing in your API or also in the TMDB API, but the website list "Production Contries" for tv shows (it's not documented in Apiary).
Can you check that and add it to your API if TMDB does support this inforamtions in the API?

Do you have an example?

https://www.themoviedb.org/tv/1399-game-of-thrones/edit?active_nav_item=production_information
(need to be logged in to see the list)

Can't find it anywhere in the docs or the output. They have origin_country and production_companies - but not production_countries

Ok, i have pushed an old thread on TMDB forum:
https://www.themoviedb.org/talk/5427067ac3a3680b1b000472

Thanks. I also wrote on their google page, but never received a response. Iirc.
-----Original Message-----
From: "DanCooper"<EMAIL_ADDRESS>Sent: 02-11-2015 09:36
To: "LordMike/TMDbLib"<EMAIL_ADDRESS>Cc: "Michael Bisbjerg"<EMAIL_ADDRESS>Subject: Re: [TMDbLib] Missing: TV - Production Countries (#153)
Ok, i have pushed an old thread on TMDB forum:
https://www.themoviedb.org/talk/5427067ac3a3680b1b000472
—
Reply to this email directly or view it on GitHub.

https://tmdb.lighthouseapp.com/projects/83077-api/tickets/513-add-production-countries-to-tv-series-response

Finally TMDb implemented this information to the database (see https://www.themoviedb.org/talk/5427067ac3a3680b1b000472?page=1)

bump :)

Oh shoot - would you have the ability to make a PR with the fields?

PR?

Pull request :)

lol...  :D
No sorry, I don't know enough about C # to do a reasonable PR.

@DanCooper this should be live on Githubs package feed soon.
I haven't yet worked up docs on how to use the github feed, but if you're able to, you can use it directly. :)
I think it'll be 1.7.7-alpha.11


[5.1] Fixed deprecation notice

This method was only added to the repository class in 5.1.28, so can't have been deprecated before it existed.

I guess it was always deprecated via __call though.


Launch new Mage AI docs

Summary
This PR enables the initial documentation setup for Mage, currently available at docs.mage.ai

🚀 Setup
Simply merge in this PR and your documentation will be connected!
👩💻 Development
Install the Mintlify CLI to preview the documentation changes locally. To install, use the following command
npm i -g mintlify

Run the following command at the root of your documentation (where mint.json is). In your case, the docs directory.
mintlify dev

You may need to run mintlify install the first time if the docs don't run automatically.
😎 Publishing Changes
Changes will be deployed to production automatically after pushing to the default branch.
😄 Community
Join our Discord server for help with the Mintlify CLI, feature suggestions, or good vibes.

@tobias-mintlify It looks like the mage_ai folder got merged into the docs folder on your branch/PR. Can you take another look and keep the mage_ai folder separate? This would break everything if we changed the folder structure and path for the mage_ai folder.

I think I'd only expect to see the readme files replaced/updated/moved.

@johnson-mage should be good now!


appimage-run: fix missing libatk-bridge-2.0.so.0

Motivation for this change
Fix error while loading shared libraries: libatk-bridge-2.0.so.0 when trying to run Beaker Browser >= 0.8.0.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

cc @tilpner

This is Probably Fine™, but we may want to check the library list against https://github.com/AppImage/AppImages/blob/master/excludelist in the future instead of adding what seems necessary.

@kamilchm My opinion is that if in ~a week you don't have a definitive answer in there, this can be merged. Upstream is broken in various ways and we already repair it, if we have more libraries than required by the AppImage “spec”, the only thing we'll miss will be a somewhat bigger closure. I just ran nix path-info -S $(nix-build --no-out-link . -A appimage-run, it goes from<PHONE_NUMBER> ~= 1596M to<PHONE_NUMBER> ~= 1597M… no big deal.

@fusion809 This one will add libatk-bridge to the defaults.

I guess we should go for it now :smile:


doc: fix broken link to environment.h

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 66.95%. Comparing base (346fcb9) to head (5fea556).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #108   +/-   ##
=======================================
  Coverage   66.95%   66.95%
=======================================
  Files          35       35
  Lines        3465     3465
  Branches      606      606
=======================================
  Hits         2320     2320
  Misses        826      826
  Partials      319      319


feat(sidekiq-prometheus): Added a quickstart for Sidekiq Prometheus m…

…etrics
Summary
A quickstart dashboard to monitor Sidekiq Prometheus metrics. It uses a third-party sidekiq prometheus exporter that exposes a '/metrics' endpoint which is then scraped by the New Relic Prometheus Agent.
The owners of this repo are not experts in the subject matter of the quickstarts. We review for the quickstart to be functional and for security risks. If you are seeking feedback on the content of the quickstart, please seek out a subject matter expert. If you are not an internal NR contributor, we can do our best to connect you with a content reviewer.

Pre merge checklist

[x] Did you check you NRQL syntax? - Does it work?
[x] Did you include a Data source and Documentation reference?
[x] Are all documentation links publicly accessible?
[x] Did you check your descriptive content for voice and tone?
[x] Did you check your descriptive content for spelling and grammar errors?
[x] Did you review your content with a subject matter expert? (e.g. a Browser agent quickstart is reviewed with a member of the Browser Agent team)

Dashboards

[x] Does the PR contain a screenshot for each of your dashboards?
[x] Do your screenshots show data?
[x] Has the sanitization script been run on each dashboard?

Alerts

[ ] N/A - Did you check that your alerts actually work?

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Motion planning doesn't avoid obstacles

Motion-planned arm movement will not avoid walls in an interactive scene. Do walls have a body ID like floors that should be appended to the MotionPlanningWrapper's obstacles list?

This is still an issue, and a significant one with the amount of data I'm generating. Are there known issues with the MotionPlanningWrapper class that would cause it to ignore static objects when planning arm trajectory?

We can try to help with this - first of all, can you check if all wall objects (they have a category of walls) are in your set of obstacles?
wall_ids = set(bid for wall in scene.objects_by_category["walls"] for bid in wall.get_body_ids())
assert len(wall_ids) > 0, "No walls found in scene"
assert wall_ids.issubset(obstacles), "Some walls not listed in obstacles"

@cgokmen I appreciate the help. I have confirmed that the assert statements in your example do not fire, so all of the wall IDs are in the MotionPlanningWrapper's mp_obstacles list. The body IDs of the objects I've spawned are also in the list, however the arm moves into them as well.

@cgokmen I've been looking at this more. I'm curious, should len(wall.get_body_ids()) == 1? I'm seeing a single body id from this list. Makes sense if all of the walls are a single entity, but figured it was worth asking. The more I look at this, the more I think there's something in the MotionPlanningWrapper that's nullifying the "avoid obstacles" effect

Sorry I missed your last message. It is indeed expected that the walls have a a single body ID. Are you running the wrapper with full_observability_2d_planning=True and collision_with_pb_2d_planning=True?  (It has an integrated visualizer)

I'm using plan_arm_motion, but I can try a quick debug script with base motion. FWIW, I have full_observability_2d_planning=False, collision_with_pb_2d_planning=False, visualize_2d_planning=False, and visualize_2d_result=False. Setting them all to true did not open a planning window like in the base movement demo

@cgokmen I dropped a table into the standard motion planning example, and the motion planner did not avoid it, instead driving straight through it. Here's the modification: https://gist.github.com/CharlesAverill/8152b93f0ecf041a4befba6ae418adc7 and here's a video:

Hi,
sorry, I'm not completely sure I understand the issue, or, if I understand, they are separate issues because the arm and base motion planners are independent, and very differently implemented. The arm motion planner will move the robot in pybullet and query the simulator for collisions (expensive). The base motion planner uses A PRECOMPUTED MAP of the environment and does 2D motion planning (fast). The execution of the base motion plan is not a real physically simulated execution; that would require passing the plan to a trajectory generator (e.g., interpolator, splines generator...) and execute them with some controllers. Instead, we just "teleport" the robot following the found path for visualization purposes.
For your last issue, if you load an object the precomputed map for the base motion planning is wrong. Then, the paths found are not guaranteed to be collision free, as in your example. To solve that, there is a version of the 2D motion planner that you could use:
collision_with_pb_2d_planning = True
This should use pybullet to check for collisions, so you should be able to add any objects
For the arm motion planner, yes, the flags you were discussing are only for the 2D base motion planner. There is no flag to visualize the arm planner.
Can you check in the arm motion planning code that the wall is in the list of obstacles?
Thanks

I understand, wasn't aware that the environment collisions were precomputed.
As for the separate issues, I just wanted to see if my arm issues were caused by something I had done, so I went to the motion planning example to see if the same issue would happen. I'm not concerned with base motion, just the arm.
In the arm motion planning code, I can confirm that the wall, and all objects spawned after the scene was initialized, are in the list of obstacles.

Hi @CharlesAverill, thanks for the reply!
Can you send me some example code to reproduce the issue? if the walls are in the list of obstacles, it is really strange the plan goes through it, it shouldn't happen.

Setting a breakpoint within the collision function could also help - are collisions being checked for those body IDs? What is pybullet returning?

@roberto-martinmartin It's a bit of a behemoth, but here it is: https://gist.github.com/CharlesAverill/6ae16bfcc53d544092bd4b1b38a619ef. I've confirmed that the motion planner will collide with the spawned table and objects, and the wall when it spawns close enough.

Thanks for the code snippet, @CharlesAverill . Could you tell me what is the expected behavior and how it is failing? thanks!

I may not understand what you are trying to do or the problem you are facing. is the provided example your use case?
is the problem that the robot's end-effector collides with the table and objects? This is expected as we use the end-effector to interact. We deactivate collision checking for the end-effector (https://github.com/StanfordVL/iGibson/blob/master/igibson/utils/motion_planning_wrapper.py#L459). You can deactivate this behavior by setting the allow_collision_links to be []. But in that case, I think your code will often fail, as you are requesting the robot's end-effector to move to the position of a randomly picked object, this position is the position of the object's reference frame, that could be inside the object, or for small objects, very close to the table, and the frame you request to move there is the robot's end-effector frame that I think it is defined inside the robot's hand. So the final configuration is often in collision and the planner will return failure.
Again, if you explain what is the behavior you are trying to obtain and the lines in the code that should get to that, it will be much easier.
Thanks!

Hi @roberto-martinmartin. The intended behavior is as follows:

The robot and table spawn in a location with no collisions
Objects spawn atop the table
The robot picks a random object on the table and attempts to push it slightly, preferably without pushing other objects but it's alright in moderation.

What's actually happening is that the motion planner doesn't avoid the table or objects (or walls when the robot spawns close enough) causing the arm to collide with many more objects than it should, which is generating poor training data for our model.
The provided example is our use case. The end-effector should collide with objects, but only minimally with objects that are not the selected one.
I'll update the snippet I linked to above with some more descriptive comments.

This is where the motion planning starts. The motion planning functions in the Fetch class are already fairly documented. You can see I remove the target object to push from the obstacles list already, so your allow_collision_links solution could work, I'll try it tomorrow.

Great! Let me know if that helps.
Another option is that you plan a motion over the object, for example to (x,y,z) - (0,0,0.1) and then you just manually go down. this is similar to what we did to push objects.

@roberto-martinmartin The "pick point above and move downwards" method works. Picking the point above can be done with just IK, and then motion planning can be used to find a path downwards. However, for added realism, it might be worthwhile to add an allow_collision_links_override option to the arm movement planning function so that motion to the above point can be executed through the physics engine.

you mean instead of teleporting? that requires a controller (or better a motion generator, for example some kind of spline generator) so that the trajectory of via point in joint space is executed correctly. That would be great to have, but we do not have it right now. If you implement a trajectory controller, that would be awesome! Feel free to submit a PR and we would def include it
I'm going to close this issue, I think we covered it?

I am still teleporting the joints to each point on the trajectory. I'd like to add in a trajectory controller, but I have to put the research first. Perhaps later in the summer I'll be able to put time into it.
As for the change I suggested, I was just referring to how the plan_joint_motion function has no user-accessible way to force collision checking for the hand and fingers of Fetch, and maintaining an internal edited version of iGibson is messy. I'll create a PR for it, it's a tiny change.
And yep, I think we've hit the closing point. Thank you so much for your help through this!

oh, the "flag" for activating collision checking for hand+fingers is a great idea! thanks for that!

@roberto-martinmartin
Hey I do have one follow up question. I have trained the locomotion controller for the Locobot and wanted to learn in addition to that some other skills in a hierarchical fashion. However, transferring the policy to the Fetch robot seems not to work, since they have very different Diff-drive behaviors. Therefore, my questions is, if it is possible to use the motion-planner for the Locobot arm as well ? (I did not go into much detail there, perhaps its just the arm-wrapper class)
But Thanks for any information :)

Hi @sycz00 ,
I'm not sure I understand 100% your question. You say you train locomotion and try to transfer from locobot to Fetch and your policy suffer because it is low level control and changes with the changes in the robot base. But then you mention something about the arm of the locobot? I get a bit lost there, sorry.
You can train using the base planner as action space abstraction and that should facilitate transferring between embodiments because an action of "move x,y meters wrt your current location" would be the same for both robots but implemented differently by the planner+controller.
Is that your question?

Hey, @roberto-martinmartin
Okay sorry, I see your point of confusion... :D
Those are two different things I was talking about.
First, I was talking about the lack of transferring a RL-policy from the locobot to the Fetch robot. The policy has been trained in iGibson 2.0.3. (in the newer version, a lot of changes have been made in the controllers etc.. so training it in the newer version might solve this ?)
A general observation is, that the controller (both differential drive) work the same but in practice, the policy on the fetch robot behaves very different. (I adjusted linear and angular velocities, but still...).
The second and more important questions to me was: Since transferring the policy is not working as expected, I want to work with the arm of the Locobot. But all examples so far are using the Fetch arm only. My question is, if there's a minimal working example or at the functionality in general to use the same stuff for Locobot's arm.
Greetings,
Fabi

Very late reply, I'm sorry, but I hope it still helps.

locobot to fetch. Policy trained in iG 2.0.3. Both with differential drive.
If I understand this (correct me if I'm wrong), you are training a policy in one robot with diff drive and transferring it to another. This will fail because, even though the action spaces are the same, the dynamics are very different, leading to different behavior. For example, a speedy curve trajectory that may work with one robot can make the other robot tip over. Think of it like this: is like you were driving two cars using the same commands (pedal push, wheel turn) but both cars would be different, or the same car but one empty and the other completely loaded. If you want your policy to generalize to different robots you could 1) train with that, or 2) use an action space that takes care of the difference. Continuous control will be hard.

examples of motion planning with the arm, or in general, control of the arm of a locobot? no, we do not have those examples, you would have to modify the existing ones for Fetch

I hope this is still helpful!

Hey
yea a little late but still thanks for your reply. I kind of got along with the way you set the positions of the agent for the base.
For the grasping, I spawned a cabinet into the iGibson scene and I reduced the scale of it. The Idea is to spawn plates and other stuff inside the cabinet, and the agent needs to find them. Now I am using parts of the motion_planning example to figure out the fixed offset for the agent's base and the grasping point. That is still somehow buggy (the gripper tips move away in some of the cases). Then I found the function : from igibson.utils.grasp_planning_utils import get_grasp_position_for_open. But there's no meta-data available for the cabinet. Do you have any suggestions using this functionality ?

@cgokmen could you provide some info about this functionality?

@sycz00 Hi, this feature should let you sample a grasp pose such that if executed, it will let you open the cabinet. You can see an example of how this is used here. What metadata seems to be missing for the cabinet, are you getting a particular error?

@cgokmen Hey:)
Thanks for your reply. When using this method, the hitting positions are always way off. I tried it with doors (the ones which are already in each scene), with fridges and cabinets. In all situations, the grasping positions were not correct. Anyways, as far as I can see in the code snippet, these functionalities are only available for the behviour robot. Since Iam using the Fetch, I assume this won't work at all ?
Thanks in advance :)

Ah, yes - these positions currently are intended for the BehaviorRobot.


Update to serde 1.0

Hi there! serde now has a stable release we can target which should mean rainbows forever.
See the release notes here.
I'm sort of on a Great serde Upgrade Crusade for the next few days so am happy to add bincode to the pile, but it might be worth studying the changes a bit to get the most out of them.

Looks like you're already on top of it in #150 :)


Update civetweb to 1.10

I've updated the civetweb dependency to version 1.10 in order to fix the compile error on macOS introduced in version 1.9.
This release deprecated the uri field in the MgRequest structure but this didn't seem to affect either the tests nor the crates.io tests.
Fixes #12

Just to clarify, I'm getting the linking error I mentioned in #12 on this branch as well.  Rust is linking with cc which reports its version as  cc (Ubuntu 7.2.0-8ubuntu3) 7.2.0.

I think this is due to the switch to cmake.  If I use the old build script from 0.9.1 (git checkout ec2341c2 civet-sys/build.rs) then this branch builds and passes tests on my machine.

I'm able to get this building by switching to the cc crate, which should work on Windows as well.  Would you give it a test @kureuil?
diff --git a/civet-sys/Cargo.toml b/civet-sys/Cargo.toml
index 950633f..81c7863 100644
--- a/civet-sys/Cargo.toml
+++ b/civet-sys/Cargo.toml
@@ -14,4 +14,4 @@ name = "civet_sys"
 path = "lib.rs"

 [build-dependencies]
-cmake = "0.1"
+cc = "1"
diff --git a/civet-sys/build.rs b/civet-sys/build.rs
index 5c8cd08..2f4a06b 100644
--- a/civet-sys/build.rs
+++ b/civet-sys/build.rs
@@ -1,14 +1,8 @@
-extern crate cmake;
-
-use cmake::Config;
+extern crate cc;

 fn main() {
-    let mut dst = Config::new("civetweb")
-                         .define("CMAKE_BUILD_TYPE", "Release")
-                         .define("BUILD_TESTING", "OFF")
-                         .define("CIVETWEB_ALLOW_WARNINGS", "ON")
-                         .build();
-    dst.push("lib");
-    println!("cargo:rustc-link-search=native={}", dst.display());
-    println!("cargo:rustc-link-lib=static=civetweb");
+    cc::Build::new()
+        .file("civetweb/src/civetweb.c")
+        .include("civetweb/include")
+        .compile("civetweb");
 }

Closed. See #14.


Cannot resolve background-image when running npm run unit

When I run npm run dev all is ok, but when trying to run unit test npm run unit I get this error:
ERROR in ./~/css-loader?{"minimize":false}!./~/sass-loader/lib/loader.js?{}!./src/assets/styles/main.scss
Module not found: Error: Can't resolve './assets/img/bg_network_purple.svg' in '/Users/wladimircoka/Documents/Develop/logahub/src/assets/styles'
 @ ./~/css-loader?{"minimize":false}!./~/sass-loader/lib/loader.js?{}!./src/assets/styles/main.scss 7:209572-209617
 @ ./src/assets/styles/main.scss
 @ ./src ^\.\/(?!main(\.js)?$)
 @ ./test/unit/index.js
03 03 2017 01:20:52.238:INFO [karma]: Karma v1.5.0 server started at http://<IP_ADDRESS>:9876/
03 03 2017 01:20:52.240:INFO [launcher]: Launching browser PhantomJS with unlimited concurrency
03 03 2017 01:20:52.251:INFO [launcher]: Starting browser PhantomJS
03 03 2017 01:20:53.135:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: Connected on socket zQeZT4atqRJZ8DNgAAAA with id 75929886
PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
  Error: Cannot find module "./assets/img/bg_network_purple.svg"
  at webpack:///src/assets/styles/main.scss:7:0 <- index.js:16980

PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 0 ERROR (0.187 secs / 0 secs)

The problem is with the background-image at my main.scss but only when testing not in dev:
.background-login {
  background-image: url(assets/img/bg_network_purple.svg);
}

I created this repo to be able to reproduce the problem.
Basically its the original template with unit test activated and I additionally only did this:

At App.vue I added this

At Hello.vue I added a

I created main.scss with just this:

.background-login {
  background-image: url(assets/img/bg_network_purple.svg);
}

Related to https://github.com/vuejs-templates/webpack/issues/563#issuecomment-283876151

@LinusBorg @yyx990803 any chance you could give me some quick hint of where I might need to check to fix this?

I have the same problem, but with postcss :(

I solve my problem making the pattern less permissive since it was checking not only test files but all files and this was causing me the problem

I change this in test/unit/index.js
const srcContext = require.context('../../src', true, /^\.\/(?!main(\.js)?$)/)

instead of this
const srcContext = require.context('../../src', true, /^\.\/(?!main\.js$).+\.(js|vue)$/i)

and works good !!!! :smile:


Enable bypassing the cached CodeQL bundle via a feature flag

In the case that a broken bundle makes its way through the release process, and we only make a rollback decision after that bundle has already made it into the toolcache, this gives us a mechanism to easily perform that rollback in a timely way.  See the internal issue for more details.
Merge / deployment checklist

[x] Confirm this change is backwards compatible with existing workflows.
[x] Confirm the readme has been updated if necessary.
[x] Confirm the changelog has been updated if necessary.

Could we get pr-checks/sync.py to add in TEST_MODE to the environment for all the tests so we don't have to remember to add it on every individual one? Or is there some tests for which it shouldn't be set? Also, I would weakly prefer it to be set at a higher level on the whole job rather than in individual steps, although again I guess this might not be possible if some steps shouldn't have it.

Hmm, we could modify the workflow generator to look for patterns like uses: ./../action/init and add a TEST_MODE env variable.  It would be nice if that was a bit more robust though.
We have one workflow that explicitly doesn't set TEST_MODE, but it's not autogenerated.
My main concern with setting TEST_MODE across the whole workflow is that this is a fairly generic environment variable name, and could potentially change the behaviour of other actions.  Perhaps we should look at renaming TEST_MODE to something more specific like CODEQL_ACTION_TEST_MODE first?

I'll hold off on merging this until we've deployed the feature flag and I can do a final manual test in our internal testing org.

Manual testing 👍


Testcafe does not open primeng dropdown

What is your Scenario?
Testcafe could not open primeng dropdowns for primeng 13 and primeng 15. This is the same issue as #7602 - I also encountered this after we upgraded to Angular 15.
What is the Current behavior?
Dropdown does not expand when testcafe clicks on it with t.click().
For primeng version 13, the dropdown still works (see my example code). For versions 14 and 15, not anymore.
What is the Expected behavior?
Dropdown expands, dropdown items become visible.
What is your public website URL? (or attach your complete example)
Primeng demo website:

version 13: https://www.primefaces.org/primeng-v13-lts/#/dropdown
version 14: https://www.primefaces.org/primeng-v14-lts/dropdown
version 15: https://www.primefaces.org/primeng-v15-lts/dropdown

What is your TestCafe test code?
See this repository. The test code:
import { fixture, Selector } from 'testcafe';

fixture('primeng example');

test.page('https://www.primefaces.org/primeng-v15-lts/dropdown')('primeng v15 example', async (t) => {
  const basicDropdown: Selector = Selector('dropdown-basic-demo p-dropdown');
  const dropdownItems: Selector = basicDropdown.find('p-dropdownitem');

  await t.click(basicDropdown);

  await t.expect(dropdownItems.count).gt(0, 'No dropdown list items'); // fails
});

test.page('https://www.primefaces.org/primeng-v14-lts/dropdown')('primeng v14 example', async (t) => {
  const header: Selector = Selector('h5').withExactText('Basic');
  const basicDropdown: Selector = header.nextSibling('p-dropdown');
  const dropdownItems: Selector = basicDropdown.find('p-dropdownitem');

  await t.click(basicDropdown);

  await t.expect(dropdownItems.count).gt(0, 'No dropdown list items'); // fails
});

test.page('https://www.primefaces.org/primeng-v13-lts/#/dropdown')('primeng v13 example', async (t) => {
  const header: Selector = Selector('h5').withExactText('Basic');
  const basicDropdown: Selector = header.nextSibling('p-dropdown');
  const toggleExpandBtn: Selector = basicDropdown.find('div[role="button"]');
  const dropdownItems: Selector = basicDropdown.find('p-dropdownitem');

  await t.click(basicDropdown);

  await t.expect(toggleExpandBtn.getAttribute('aria-expanded')).eql('true'); // passes
  await t.expect(dropdownItems.count).gt(0, 'No dropdown list items'); // passes
});

Your complete configuration file
module.exports = {
    "src": "tests"
}

Your complete test report
No response
Screenshots
No response
Steps to Reproduce
Run test:
npx testcafe chrome
TestCafe version
2.6.0
Node.js version
v16.15.1
Command-line arguments
npx testcafe chrome
Browser name(s) and version(s)
Chrome <IP_ADDRESS>
Platform(s) and version(s)
Windows 11
Other
Does not occur in native automation mode. Native automation mode has other issues, e.g. no firefox support and issues with RequestMock and RequestLogger.

Hi @imma90
Thank you for the shared example. I've reproduced the issue.

This looks very similar to the issue I noticed a couple of months ago, after updating to Angular 14 and then to 15, although the project uses Angular Material, not primeng - the clicks handler sometimes do not trigger in tests, although the ripple activates (in the OP video, the select focuses). Happens in both NA and the regular modes. Might be an Angular bug or something in my app for all I know. As a workaround, I've been using this client function instead of t.click:
export const clickWithKeyboard = async (element: Selector, key: 'space' | 'enter' = 'enter') => {
  await t.expect(element.exists).ok();
  await t.eval(
    () => {
      // @ts-ignore
      return (element() as HTMLElement).focus();
    },
    { dependencies: { element } },
  );
  await t.pressKey(key);
};

https://github.com/DevExpress/testcafe/assets/2667662/ddc63136-7556-4933-ba78-e57c622967b8

https://github.com/DevExpress/testcafe/issues/7713
Hi @Klaster1,
Thank you for sharing the workaround. As for the fix, we will update this thread once we have any news.


animation on clear

would be nice to see some animation when the toast clears

Yeah, I'll might work on it at weekend


Create a login page

Create the Whales Manager login page (design and logic)

Done in commit feat (frontend): add bad credentials management and handle redirects to main page (d70d9f4)


Transaction costs calculation results in leveraging

Due to the fact that cash levels can be negative in QsTrader, we have leveraging effect when transaction costs are included. For example, we have 100 000 USD in initial equity and I want to buy 1000 of shares with price of 100 and transaction costs of 1$ per share. QsTrader will let you buy shares for 100 000 USD = 1000 * 100 and with transaction costs of 1000 = 1000 * 1$. As a result, cash level will be: 100000(initial equity) - 100000(shares) - 1000 (transaction costs) = - 1000 USD


Management commands has less documentation than deprecated ones

File: engine/reference/commandline/container_run.md, CC @mistyhacks
In comparison with the docker run page the docker container run page is missing:

the extended description
the many examples

For a new comer to docker it's hard to see that docker run is deprecated. The only information I can find about this and what Management commands are under Docker CLI docs is
DOCKER_HIDE_LEGACY_COMMANDS When set, Docker hides “legacy” top-level commands (such as docker rm, and docker pull) in docker help output, and only Management commands per object-type (e.g., docker container) are printed. This may become the default in a future release, at which point this environment-variable is removed.
Also there is an asterisk after docker container * in the navigation tree on the left, but I can't find where the * is explained.
Let me know if I need to split this ticket into multiple ones.

This content is all managed upstream in docker/cli repo. The docs have a build process that takes the ## Description and ## Examples sections from files in https://github.com/docker/cli/tree/master/docs/reference/commandline files and adds them to the docs. If you feel like examples should be moved from, for example, run to container run, you can submit a PR to do that. 👍 Be careful about using terms like "deprecated" though, as the docker run command is certainly not deprecated.
In a perfect world, there would be the ability in the command parser (Cobra) to recognize command "aliases" and consolidate the docs for them all, but I do not believe this is currently possible.  cc/ @dnephin @thaJeztah for confirmation.
As for the *, it is just a placeholder that shows that if you expand that section, you will find all the sub-commands. So the docker container * section expands to show all of the sub-commands like docker container run and docker container stop.
As this specific issue is about content that is actually in a different repo, I'll close the issue here. Thanks for your feedback!


Suggestion: reorganize navigation menu in the document (show Links/Functions etc.)

Hi, I am new to Chainer, I think the concept is great.
Just a little bit confused when reading the online document of chainer.
As a regular user, what I want to know most is what kind of layers (Links) and functions is available in Chainer. But currently there is no such list available in the document, instead there are classified menus such as "learnable connections" and "loss functions" which is not intuitive for me.
It will be surely helpful if you put all the links and functions right inside the navigation, and try to remove those redundant upper layers. Since the references are heavily used during the entire usage of chainer, it's really important to make a good navigation without too deep wrapping. (Tutorials are only for beginners, and CuPy is more for developers, thus they are not so heavily used.)
A good example for me is http://keras.io/layers/core/
What do you think?

Thank you for the suggestion. I agree with you; the navigation is not good, indeed. Is it enough to just show all Links/Functions directly in the side menu, or do we still need some (better) mid-level organizations like the numpy document does?


when I uese hybrid_parallel, and set the enable_fused_normalization = True. I can't run the code, here are some error: RuntimeError: Failed to replace input_layernorm of type LlamaRMSNorm with FusedRMSNorm with the exception: Please install apex from source (https://github.com/NVIDIA/apex) to use the fused RMS normalization kernel. Please check your model configuration or sharding policy, you can set up an issue for us to help you as well. However, I have install the apex, it will still occur. How can i solve it?

🐛 Describe the bug
raise RuntimeError(
RuntimeError: Failed to replace input_layernorm of type LlamaRMSNorm with FusedRMSNorm with the exception: Please install apex from source (https://github.com/NVIDIA/apex) to use the fused RMS normalization kernel. Please check your model configuration or sharding policy, you can set up an issue for us to help you as well.
Environment
    plugin = HybridParallelPlugin(
        tp_size=8,
        pp_size=1,
        num_microbatches=None,
        microbatch_size=1,
        enable_fused_normalization=True, #
        enable_jit_fused=True,
        enable_flash_attention=True,
        check_reduction=True,
        gradient_as_bucket_view=True,
        find_unused_parameters=True,
        zero_stage=0,
        precision="bf16",  # fp32
        initial_scale=1,
    )

Hi, Please install apex from https://github.com/NVIDIA/apex, or set enable_fused_normlization to False.

Hi, Please install apex from https://github.com/NVIDIA/apex, or set enable_fused_normlization to False.
I have installed it, but it is not working.

Maybe the version of apex is not correct, can you have a try that "from apex.normalization import FusedRMSNorm"

Me too !!
RuntimeError: Failed to replace input_layernorm of type LlamaRMSNorm with FusedRMSNorm with the exception: No module named 'fused_layer_norm_cuda'. Please check your model configuration or sharding policy, you can set up an issue for us to help you as well.

And I saw this prompt in  examples/language/llama2/scripts/benchmark_70B/3d.sh
# TODO: fix this
echo "3D parallel for LLaMA-2 is not ready yet"

Does it mean , even if I deployed apex correctly, I won't be able to use hybrid_parallel properly ?

And I saw this prompt in examples/language/llama2/scripts/benchmark_70B/3d.sh
# TODO: fix this
echo "3D parallel for LLaMA-2 is not ready yet"

Does it mean , even if I deployed apex correctly, I won't be able to use hybrid_parallel properly ?

hybrid parallelism can normally work, Could you run Python and then execute from apex.normalization import FusedRMSNorm to see if it runs successfully?

And I saw this prompt in examples/language/llama2/scripts/benchmark_70B/3d.sh
# TODO: fix this
echo "3D parallel for LLaMA-2 is not ready yet"

Does it mean , even if I deployed apex correctly, I won't be able to use hybrid_parallel properly ?

Hybrid parallelism can normally work now, Could you run Python and then execute from apex.normalization import FusedRMSNorm to see if it runs successfully?

Yes, python -c "from apex.normalization import FusedRMSNorm"    runs successfully.

No module named 'fused_layer_norm_cuda'

https://blog.csdn.net/iteapoy/article/details/117389407, please try this.

And I saw this prompt in examples/language/llama2/scripts/benchmark_70B/3d.sh
# TODO: fix this
echo "3D parallel for LLaMA-2 is not ready yet"

Does it mean , even if I deployed apex correctly, I won't be able to use hybrid_parallel properly ?

Hybrid parallelism can normally work now, Could you run Python and then execute from apex.normalization import FusedRMSNorm to see if it runs successfully?

Yes, python -c "from apex.normalization import FusedRMSNorm"  runs successfully.

https://blog.csdn.net/iteapoy/article/details/117389407 , please try this.

Can you share your pip list and your cuda version?


Failed to build installable wheels for some pyproject.toml based projects (spark_dsg)

Hi,
When i try to run pip install -e . I keep getting these error messages (this is a copy-pasted part of a long error message):
 CMake Error at CMakeLists.txt:17 (find_package):
        By not providing "Findnlohmann_json.cmake" in CMAKE_MODULE_PATH this
        project has asked CMake to find a package configuration file provided by
        "nlohmann_json", but CMake did not find one.

        Could not find a package configuration file provided by "nlohmann_json"
        with any of the following names:

          nlohmann_jsonConfig.cmake
          nlohmann_json-config.cmake

        Add the installation prefix of "nlohmann_json" to CMAKE_PREFIX_PATH or set
        "nlohmann_json_DIR" to a directory containing one of the above files.  If
        "nlohmann_json" provides a separate development package or SDK, be sure it
        has been installed.

      -- Configuring incomplete, errors occurred!

note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building editable for spark_dsg
Failed to build spark_dsg
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (spark_dsg)

I've tried to re-install spark_dsg and retrace my steps but have not come to a solution. Hope someone can help.

Did you install prerequisites of spark_dsg described in https://github.com/MIT-SPARK/Spark-DSG?

I was trying to do so, doesn't seem like it worked. Also using a Mac so not being able to use apt might be part of the problem. Are there any workarounds for using this project on Mac?

I try to install "nlohmann-json3-dev", and it works for me.


Helm chart CI linting

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
This PR adds helm chart testing and linting to the CI process using the
"chart-testing" project:

https://github.com/helm/chart-testing
https://github.com/helm/chart-testing-action

Which issue(s) this PR fixes:
None.
Does this PR have test?
Yes.
Special notes for your reviewer:
Does this PR introduce a user-facing change?
NONE

/ok-to-test

/ok-to-test
Can you remove the WIP tag?

/ok-to-test

Seems you need to rebase

Whoops. I think I forgot to merge main prior to my previous rebase.

I updated the workflow name and added CI permissions like the other workflows.
The main reason I created a separate workflow config was to prevent merge conflicts. I am happy to combine it with .github/workflows/test.yml if desired.

/lgtm
/approve

/retest

/retest


Add react error overlay (based on #2900)

Issue: #2900
What I did
Merged to this branch inside the organization for an easier fix of the middleware conflicts.

Not sure what should be done with the stories that throw the error on purpose (storyshots wise) + there is some code block in comments.

Not sure what should be done with the stories that throw the error on purpose (storyshots wise)

Let's just ignore those

@Hypnosphi , something else here?

https://github.com/storybooks/storybook/pull/2900#issuecomment-362947703
https://github.com/storybooks/storybook/pull/2900#issuecomment-362961929

@Hypnosphi both comments refer to a merged PR, can this be merged as well?
I can't determine what is blocking this PR.

@ndelangen it's merged into this branch (add-react-error-overlay), not into master
You can run official storybook and reproduce both issues

@Hypnosphi something I can do here?

I'm not sure we really want it on our side, given that our error handling is now framework-agnostic

OK, so should we close?


how to trigger the calendar with button

I'm Submitting a ...
[] Bug report
[ ] Feature request
[x] Support request
I want to open the calendar with a button when click. How to do that?
thank!

Seems like something that could be solved with the following:
// In the component
toggleCalendar() {
  this.setState({ calendarOpen: !this.state.calendarOpen });
}

render() {
  return (<div>
    <Button onClick={toggleCalendar} />
    <Datetime open={this.state.calendarOpen} />
  </div>
}

Note that this is psudo-code, but it should work all the same.

Thank @Keith-Ball I'll try.

@Keith-Ball You are right i have click in button time picker is open but blur event not triggering in click outside of box.


ADR001: Adopt ABCI++ Early

Description
ADR that describes the options we have to adopt ABCI++ early. The whole point of this ADR is to emphasize that we are in a difficult situation. ABCI++ enables us to fix most of our issues in a non hacky way, but it is also not finished yet. We could implement hacky solutions instead of waiting for ABCI++, only to reimplement the same things in a less hacky way as soon as ABCI++ comes out.
rendered
closes: #156

Over break, I've been thinking about slightly revising this ADR.
I still think we should work on all the things that don't need ABCI++ first, such as fixing the cli and tracking WirePayForMessages, but after we would help push ABCI++ over the finish line instead of adopting prematurely.

I think we can close this PR, as we should no longer have to go out of our way to adopt ABCI++ early


Migrate away from "@fbsource//tools/build_defs/apple:flag_defs.bzl"

Summary:
We still have usages of "fbsource//tools/build_defs/apple:flag_defs.bzl" in react-native-github. But this should get us closer towards not using the fbsource cell. Hopefully, this is enough to unbreak the  test_docker CircleCI build.
Changelog: [Internal]
Reviewed By: fkgozali
Differential Revision: D26289304

D26289304 was landed.


Deprication Notice for yaml.load(fmatter)

venv\lib\site-packages\frontmatter_init_.py:37: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
"attributes": yaml.load(fmatter)

Seems like this is fixed in master but pypi still has the old version any idea when it will be updated?

I've also reproduced the above error in my codebase.
  /usr/local/lib/python3.9/site-packages/frontmatter/__init__.py:37: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
    "attributes": yaml.load(fmatter),

After reviewing master, I believe this is the offending line in frontmatter
https://github.com/jonbeebe/frontmatter/blob/accd644b15a5ac7adaab261655eb552065020821/frontmatter/__init__.py#L37
According to the documentation at https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation, this should be replaced with from PyYAML 5.1+ which is already supported:
        "attributes": yaml.full_load(fmatter),


simplify app_base path'ing

Remove APP_BASE from systemjs config
Set APP_BASE to '' only for desktop build (relative path)

Confirmed this works in all modes:
-web (dev and prod)
-desktop
-test

million times better 👍


Issue 2272: Fix git-commit-id-plugin

Motivation
Fixes #2272.
We are using git-commit-id plugin for generating a properties file containing all the build information.
However you have to skip git-commit-id-plugin when the source code is not in a git repo.
Changes
add <failOnNoGitDirectory>false</failOnNoGitDirectory> to git-commit-plugin

merged as 046e500 in branch-2.1


formatting tweaks; augmented model info; augmented debugging e.g.

Ref #405
@lee1043 caught a couple of formatting inconsistencies, and augmented model info and debugging example - nothing major

Perfect, that's looks good enough to me as a first pass - nbviewer.
We can think about whether ESGF, HiLAT or other logos are worth adding next week


Release: can't find riscv64 binary in releases page

Description
I noticed that distribution has already adapted riscv64 in #4444
But I can't find it in the docker hub and release page (https://github.com/distribution/distribution/releases/tag/v3.0.0-beta.1)
Can we add it?

Yeah the beta release hasnt had the riscv platform change, but the next release will do. See https://github.com/distribution/distribution/pull/4444#issuecomment-2314596839

Thank you for your reply and hark working :)


Added new entry for new plugin named 'top' from your friends at ECS Team

Hi @kkellner
Changes look good. Please ensure the CFF has your CLA on file and wait for us to be notified, or if ECS has a corporate CLA that covers employees using the GitHub organization, make your membership to that organization public in your profile.
Cheers,
Dies Koper
CF CLI PM

ECS Team has a corporate CLA and my profile (for kkellner) has ECSTeam organization association public.

@chipchilders Reopening to see if that kicks the bot into submission.


Supporting hostapd-wpe

This is a very interesting project and I liked how the API was used in WiFiPhisher.
I also use hostapd in my project, but in my case I just write the hostapd.conf file and call hostapd-wpe with Popen.
I'm not sure you know this but hostapd-wpe (wireless pwning edition) is a patch for hostapd that facilitates Evil-Twin attacks on enterprise networks (WPA-EAP and such). The patched version is being actively maintained by the authors of aircrack-ng.
Since this project was mainly to developed for WiFiPhisher I think this would be a great addition to the project.
Also I would like to use this library in my project but it needs to support WPA-EAP.

@Esser420 thanks for opening the first issue : )
The reason why we don't use hostapd-wpe and hostapd-mana are as following:

Since we need a good python binding so we decide to patch the hostapd by ourself.

If you have saw the implementation of roguehostapd, you'll discover that we don't use Popen to fork the hostapd process and instead we build the hostapd as the shared library and use ctype to bind the c-functions we're interested in.

I guess @sophron also plans about the enterprise network attack : )

[BTW]
Currently if you want to use KARMA attack in wifiphisher. You need to do pip install roguehostapd first;otherwise, wifiphisher will still lunch the system hostapd.

Hi @Esser420,
Thanks for reaching out. Yes, we definitely plan to implement Evil Twin against Enterprise WLANs. That was one of the reasons behind the creation of roguehostapd (#569).
roguehostapd is open-source and you are welcome to use it on your project. Note that we are currently dealing with some distribution issues that we should solve soon.


Add required files for Pong game

Fix to  Issue #192
===============================
This Python Pong game is an implementation of the classic Pong arcade game. The game features two paddles and a ball that bounces between them.
===============================
Checklist

[x] I read the contributing guidelines
[x] I structured my project as stated in the contributing guidelines
[x] My project uses dependencies
[x] I starred the repo
[x] I had fun contributing!

Project reviewed and merged.
Thank you @shadowranger07
Feel free to star the repo and add more projects. :)


I cannot use this app after put command

Describe the bug
I cannot use this app after put command, always show "Forbidden" error message.
eg. build website to find user's life list, using php, vue.js
eg. build website with vue.js

Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-r3sflf
Steps to reproduce

put text on the command line
press enter button to submit the command
show me error message

Expected behavior
I will get response from server.
Screen Recording / Screenshot
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

https://github.com/stackblitz/bolt.new/issues/77#issuecomment-2395503410

Hi! this issue should be resolved now!
Let us know if you're still having problems

we've just shipped a fix for the forbidden / trouble connecting to server in from certain location errors, please confirm it now works!


Identify nofification scan as type in execution hint #1055

With these changes and the changes in apache/accumulo#972, the following accumulo config should run notification scans in their own executor.
config -s tserver.scan.executors.background.threads=1
config -t stresso -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
config -t stresso -s table.scan.dispatcher.opts.executor.fluo-ntfy=background

@mikewalch  I think the breakage is related to the change in apache/accumulo-maven-plugin#3 and the fact that the Accumulo version is 2.0.0-alpha-2


⚠ Hartman Foundation has degraded performance

In 9e61be7, Hartman Foundation (https://gordonhartman.com/) experienced degraded performance:

HTTP code: 200
Response time: 2588 ms

Resolved: Hartman Foundation performance has improved in b37ffd6.


chore: proper exports inside chain-adapters package

fixes #462

@pbteja1998 merged - can you please update the web PR with the new published packages in a few minutes after CI publishes them to npm? ty :)

@0xdef1cafe sure, will update it.


Do not expand the process environment variables

Expanding variables from the tmt process could reveal sensitive
information such as user tokens. Let's substitute only explicitly
provided variables in the plan and the --environment option.

@FrNecas, please have a look as well. For a bit more context see also the following discussion.

/packit test

/packit test

/packit test


[logs]Suricata

to:
cc: @airbnb/streamalert-maintainers
related to:
resolves:
Background
Reason for the change
Changes
Add suricate log filter
Issues
https://github.com/airbnb/streamalert/issues/319
https://github.com/airbnb/streamalert/issues/99

Coverage remained the same at 95.62% when pulling c3f2d5f242fefcdeb4cca31c6d6335436eb750b0 on gavinelder:suricate into 2162c64f269afa47710b8adb4d922b2865e179be on airbnb:release-3-1-0.

Hi @gavinelder is it possible to add testing events as well? Similar to trendmicro testing events. https://github.com/airbnb/streamalert/blob/a9f985184bcdac27bed97f93c5e9a4d59870083f/tests/integration/rules/trendmicro/trendmicro_schema.json#L1-L11
Since we don't have rule based on suricate, you can set
  "validate_schema_only": true,
  "trigger_rules": []

Will run Suricata for a few days hopefully capture some alert logs.

Will re-open when I am back in the office and can grab the needed logs.


Add information about manually forcing retention

Description
This question came up in the community slack, and was not otherwise addressed by the documentation.
Merge requirements
Please take into account the following non-code changes that you may need to make with your PR:

[ ] CHANGELOG entry for user-facing changes
[x] Updated the relevant documentation

Closing this. The concept of needing docs for this lives on in https://github.com/timescale/promscale_extension/issues/312. The underlying doc has been moved to the docs page anyway, so this PR ain't doin' nothin'.


Merged main dev 1.28

Maintain the current and upcoming dev branch for Kubernetes 1.28 release.
Release deadlines
From sig-release:

Description
Date

Docs deadline — Open placeholder PRs
Thursday 16th March  2023

Docs deadline — PRs ready for review
Tuesday 21st March 2023

Docs complete — All PRs reviewed and ready to merge
Tuesday 4th April 2023

v1.27.0 released
Tuesday 11th April 2023

For More Information
cc: Kat Cosgrove (@katcosgrove / Slack: @katcosgrove), Tania Duggal (@taniaduggal / Slack: @Tania Duggal),  Vibhor Chinda (@VibhorChinda / Slack: @Vibhor Chinda), Michael Levan (@AdminTurnedDevOps / Slack: @Michael Levan)

PS: This would need to merge before I update the config.toml, now hugo.toml for 1.28.

Closing in favour of https://github.com/kubernetes/website/pull/41169


[SPARK-39942][PYTHON][PS] Need to verify the input nums is integer in nsmallest func

The input parameter of nsmallest should be validated as Integer. So I think we might miss this validation.
And PySpark will raise Error when we input the strange types into nsmallest func.
What changes were proposed in this pull request?
validate the input num is integer type only.
Why are the changes needed?
PySpark will raise Error if we not limit the type.
>>> df = ps.DataFrame({'A': [1, 2, 3, 4], 'B': [3, 4, 5, 6]}, columns=['A', 'B'])
>>> df.groupby(['A'])['B']
<pyspark.pandas.groupby.SeriesGroupBy object at 0x7fda5a171fa0>
>>> df.groupby(['A'])['B'].nsmallest(True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/spark/spark/python/pyspark/pandas/groupby.py", line 3598, in nsmallest
    sdf.withColumn(temp_rank_column, F.row_number().over(window))
  File "/home/spark/spark/python/pyspark/sql/dataframe.py", line 2129, in filter
    jdf = self._jdf.filter(condition._jc)
  File "/home/spark/.pyenv/versions/3.8.13/lib/python3.8/site-packages/py4j/java_gateway.py", line 1321, in __call__
    return_value = get_return_value(
  File "/home/spark/spark/python/pyspark/sql/utils.py", line 196, in deco
    raise converted from None
pyspark.sql.utils.AnalysisException: cannot resolve '(__rank__ <= true)' due to data type mismatch: differing types in '(__rank__ <= true)' (int and boolean).;
'Filter (__rank__#4995 <= true)
+- Project [__index_level_0__#4988L, __index_level_1__#4989L, B#4979L, __natural_order__#4983L, __rank__#4995]
   +- Project [__index_level_0__#4988L, __index_level_1__#4989L, B#4979L, __natural_order__#4983L, __rank__#4995, __rank__#4995]
      +- Window [row_number() windowspecdefinition(__index_level_0__#4988L, B#4979L ASC NULLS FIRST, __natural_order__#4983L ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS __rank__#4995], [__index_level_0__#4988L], [B#4979L ASC NULLS FIRST, __natural_order__#4983L ASC NULLS FIRST]
         +- Project [__index_level_0__#4988L, __index_level_1__#4989L, B#4979L, __natural_order__#4983L]
            +- Project [A#4978L AS __index_level_0__#4988L, __index_level_0__#4977L AS __index_level_1__#4989L, B#4979L, __natural_order__#4983L]
               +- Project [__index_level_0__#4977L, A#4978L, B#4979L, monotonically_increasing_id() AS __natural_order__#4983L]
                  +- LogicalRDD [__index_level_0__#4977L, A#4978L, B#4979L], false

Does this PR introduce any user-facing change?
No
How was this patch tested?
input non-Integer type will raise AssersionError during calling nsmallest func

cc @itholic @xinrong-meng @zhengruifeng FYI

I think this is a more general problem:
def nsmallest(self, n: int = 5)

this method already hint the input type, but given a boolean it will fail with a AnalysisException (which should be a TypeError).
I guess it could accept a floating number like 5.5, and run successfully.
I did a quick investigation of typing, there is an annotation @typing.runtime_checkable, but only works for class typing.Protocol
is there some method to enforce the input type checking?   @zero323

is there some method to enforce the input type checking? @zero323

There exist some 3rd party attempts, but as far as I'm aware none is particularly popular and well maintained (pydantic included validate_arguments on provisional basis, but I haven't seen it used in the wild).
Runtime checking is officially out-of-scope for core typing (@typing.runtime_checkable is a bit different beast) so it wouldn't expect any solution emerging there.
Personally, I'd say that the current behavior is acceptable ‒ there is no significant divergence compared to Pandas:
>>> df = pd.DataFrame({'A': [1, 2, 3, 4], 'B': [3, 4, 5, 6]}, columns=['A', 'B'])
>>> df.groupby(['A'])['B'].nsmallest(5.5)
Traceback (most recent call last):
...
TypeError: cannot do positional indexing on Int64Index with these indexers [True] of type bool

and the error message is meaningful enough.

Thank you @zero323 for detailed explaination!

Can one of the admins verify this patch?

is there some method to enforce the input type checking?

I did some try on strict type check for pyspark in last year (long long ago): https://github.com/Yikun/annotation-type-checker/pull/4
We could easy to add decorator to enable type check without any 3rd lib introduced.
But if we enable this, the type check will be the first priority validation, this might bring some different behavior with pandas finally.

If it's a common mistake, we might want to add this fix, but for this patch, I personally think this example seems a little too extreme. The user could also found the error by seeing due to data type mismatch: differing types. @bzhaoopenstack Or you have any other plus?

maybe we still need type checking in this PR:
In [1]: import pandas as pd

In [2]: import pyspark.pandas as ps

In [3]: df = pd.DataFrame({'A': [1, 2, 3, 4], 'B': [3, 4, 5, 6]}, columns=['A', 'B'])

In [4]: df.groupby(['A'])['B'].nsmallest(5.5)
...
TypeError: cannot do positional indexing on Int64Index with these indexers [5.5] of type float

In [5]: df = ps.DataFrame({'A': [1, 2, 3, 4], 'B': [3, 4, 5, 6]}, columns=['A', 'B'])

In [6]: df.groupby(['A'])['B'].nsmallest(5.5)

A
1  0    3
2  1    4
3  2    5
4  3    6
Name: B, dtype: int64

In [7]: df.groupby(['A'])['B'].nsmallest("5.5")
Out[7]:
A
1  0    3
2  1    4
3  2    5
4  3    6
Name: B, dtype: int64

ps runs successfully with 5.5 and "5.5", whilepd throws a TypeError

I'm not sure about @Yikun 's type_checker annotation, but it seems simpler than manually adding isinstance checking for each parameter.

I'm not sure about @Yikun 's type_checker annotation, but it seems simpler than manually adding isinstance checking for each parameter.
The code looks checking the definition of annotation, as make it as a decorator and using typing to checking during runtime.

If it's a common mistake, we might want to add this fix, but for this patch, I personally think this example seems a little too extreme. The user could also found the error by seeing due to data type mismatch: differing types. @bzhaoopenstack Or you have any other plus?

That's just a demo. We can not image the developers how to type in the python session and provide a confused err-msg. Try to image when you are the developer and try to use the function in the first time, you want to try the borderline of a func.  I compare the pandas codeline with pyspark, and try the best to leverage the validation gap between pandas and PySpark.
From discussion and my experience about this kind issue, the validation and early validation are necessary. The key point now I think is finding a good way to support validate during runtime. That would be good that community could consider a good/acceptable way to make this happen, due to the codetree of PySpark Pandas already contains so many sperated valiations already. So I think that would be a good improvement for PySpark Pandas. ;-)

already contains so many sperated valiations already

ok, if you wanna add checking here, please follow https://github.com/apache/spark/blob/master/python/pyspark/pandas/frame.py#L735-L741 to raise a TypeError

Kindly asking, any update on this PR ?
IMHO, the most important goal of the pandas API on Spark is to match all behaviors, including error types and messages, to avoid confusion for existing pandas users.
So, I think at least we should raise TypeError just same as pandas, like suggested in https://github.com/apache/spark/pull/37369#issuecomment-1205991647.


fix: 修复存在字段标记的 icon 时, 紧凑模式列宽计算错误出现省略号的问题

👀 PR includes

🐛 Bugfix

[x] Solve the issue and close #0

📝 Description

问题: 计算列宽时只考虑了文本的宽度, 未考虑 icon 的场景, 导致出现省略号
该问题在 1.x 中 https://github.com/antvis/S2/pull/2673 已修复, 同步代码时漏掉了相关逻辑
🖼 Screenshot

Before
After

🔗 Related issue link

🔍 Self-Check before the merge

[ ] Add or update relevant docs.
[ ] Add or update relevant demos.
[ ] Add or update test case.
[ ] Add or update relevant TypeScript definitions.

Codecov Report
Attention: Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 80.81%. Comparing base (ecdc9c8) to head (fac357b).
Report is 444 commits behind head on next.

Files with missing lines
Patch %
Lines

packages/s2-core/src/facet/pivot-facet.ts
90.00%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             next    #2920      +/-   ##
==========================================
+ Coverage   75.77%   80.81%   +5.04%
==========================================
  Files         257      180      -77
  Lines       11994    10425    -1569
  Branches     2464     2362     -102
==========================================
- Hits         9088     8425     -663
+ Misses       1398      553     -845
+ Partials     1508     1447      -61

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:tada: This PR is included in version @antv/s2-v2.0.0-next.29 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Mocking Autobahn for offline unit tests

.. should be easy;)

we are doing this in unit tests now


Tab: Visual border adjustment

problem: the border of the active tab looks broken
https://www.figma.com/file/xZ0IW0MJO0vnFicmrHiKaY/Components-Post?type=design&node-id=11253%3A51917&mode=design&t=GsxSZb9ZX24YsT0s-1
possible solution:

Won't do for v1


Xamarin.iOS port

I've ported Fusuma to Xamarin.iOS. If anyone is interested you can grab it here: https://github.com/Cheesebaron/Chafu

@Cheesebaron Cool! Let me add your library to the readme 👍


[feature] plain_io

There are still some TODOs...

I have improved the header_kind and added snippets also to the reader.


Do you have any kind of tooling around how to manage those hashes in the network

We need an internal data catalog
https://www.youtube.com/live/AO8oJUijINU?feature=share&t=371

I think something similar to https://github.com/bacalhau-project/bacalhau/issues/2318#issuecomment-1504849965 could work.
The hashes are there, a Bacalhau job could crawl it every day/hour (GH Action/Cron/..) and publish it back to IPFS and somewhere else like GitHub. Not fully decentralized but it allows progressive decentralization when needed!
PS: Would be awesome to visualize the Bacalhau DAG over time, if that's not yet a thing.

@aronchick what is this about?

closing in favour of this unified https://github.com/bacalhau-project/bacalhau/issues/4187


Update jdk20 with jtreg7

related #3965
Signed-off-by: Sophia Guo<EMAIL_ADDRESS>

Depends on https://github.com/adoptium/TKG/pull/357 and job rerun https://ci.adoptopenjdk.net/view/all/job/test.getDependency/

https://ci.adoptopenjdk.net/view/all/job/test.getDependency/1563/ done


fix: more ci conflicts

Had to rebase and force push here: https://github.com/ethereum-optimism/optimism/pull/11617.


backend-starter ultimate vesion

A lot of unnecessary stuff. Look at what already exists in the backend starter (e.g. user model, login controller) and try to replicate it for the new routes.


Changes for displaying image path on /status page.

Changes for displaying image path on /status page.

Pull Request Test Coverage Report for Build 5827

0 of 4 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.004%) to 43.399%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

azkaban-common/src/main/java/azkaban/imagemgmt/dto/ImageVersionMetadataResponseDTO.java
0
2
0.0%

azkaban-web-server/src/main/java/azkaban/imagemgmt/services/ImageVersionMetadataServiceImpl.java
0
2
0.0%

Totals

Change from base Build 5826:
-0.004%

Covered Lines:
16017

Relevant Lines:
36906

💛  - Coveralls

Pull Request Test Coverage Report for Build 5827

0 of 4 (0.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.004%) to 43.399%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

azkaban-common/src/main/java/azkaban/imagemgmt/dto/ImageVersionMetadataResponseDTO.java
0
2
0.0%

azkaban-web-server/src/main/java/azkaban/imagemgmt/services/ImageVersionMetadataServiceImpl.java
0
2
0.0%

Totals

Change from base Build 5826:
-0.004%

Covered Lines:
16017

Relevant Lines:
36906

💛  - Coveralls


DR-1069 Prevent form to be submitted more than once

Background
On activation page, if the user clicks on the confirmation button several times, it generates several API calls
Changes done

Add new debounced-ng-click directive in charge of preventing new submits if a promise is in progress
Adjust submitActivation function so it returns a promise

Pending to be done
If this approach is approved we should consider using this directive in every element that causes an API call. In those cases, we should adjust controller code so the involved functions returns a promise

❤


Issue with using "--retain_unpaired" with the "FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE" module

Description of the bug
if you pass --retain_unpaired to FASTQC_UMITOOLS_TRIMGALORE:TRIMGALORE, then ALIGN_STAR:STAR_ALIGN breaks by returning segmentation fault. I believe the issue is that four files are passed to STAR, two paired and two unpaired! (slack disscussion)
@drpatelh suggested the use of a discriminator in the TrimGalore module to determine the type of read
Command used and terminal output
No response
Relevant files
No response
System information
No response

This should be fixed in https://github.com/nf-core/rnaseq/pull/811 @m3hdad if you want to give -r dev a test drive.


Retrieving data from Oracle seems really slow

System information:

Windows 10 Enterprise 20H2
21.0.4
None

Connection specification:

Oracle
Default drivers
No tunnels or proxies

Describe the problem you're observing:
Steps to reproduce, if exist:

Open a SQL editor and run a SELECT query that returns at least several hundred rows.
The first hundred rows are populated in the data grid.

Scroll down in the data grid to the bottom of the results and then page down to retrieve the next hundred rows.
The next hundred rows are returned in about 5 seconds.

Repeat step 2.
Each additional 100 rows fetched adds at least a couple of seconds to the retrieval time.

I repeated the query from 1. and then issued a Fetch all rows. It took nearly 21 seconds to retrieve all 873 rows of a two column table.
Essentially retrieving rows from Oracle seems really slow.
Just tried this on a MariaDB database and each additional 100 rows fetched seems to take just over 1 second consistently.

@bobf32 you can try to change some settings related to performance in Connection configuration and check if it helps or not.

@uslss I tried enabling each of the four options individually and none made any difference.

is it a simple select * from table or a more complex query?

@p91paul Is straight select *. No clobs.

@bobf32 are your oracle and mariadb instances on the same network?
I constantly use Oracle and never noticed anything like this.

@p91paul Not my area of expertise, but I would say ultimately yes, although the two databases are in different countries. Oracle is UK, MariaDB mainland Europe.

What I was suggesting is that maybe your connection to Oracle has an higher network latency than the one to MariaDb. However seeing they are both remote that might not be the case.

Seems to be a network configuration specific issue. Please check it with your system administrator.

there is no update on the issue for a long time.
If it is still actual for you - feel free to reopen it with additional info