OWASP Considerations

Hello
I'm currently attempting to deploy a TUS enabled application into Azure behind a WAF and am running into various issues with how the default OWASP rules are detecting the TUS protocol.  I will be referencing Azure here, but under the hood Azure appears to be using ModSecurity along with the standard OWASP CoreRuleSets. Other cloud providers will all have very similar implementations.
Request Body Size Limits
The default policy for azure is to limit request bodies to 128kb and file uploads to 100mb.  This limit comes into effect if the WAF is configured to do request body scanning.
File uploads are detected if the request is a multipart/form-data with a part that has a filename attribute.
Azure does have a way to increase these limits, but it's an additional cost feature
TUS uploads files in such a way they get treated as normal requests, and as such they are limited to a chunk size of 128kb.. It's not the end of the world, but equally, not very efficient either.
Content Type for uploads
TUS uploads files using the application/offset+octet-stream, which is not part of the default OWASP ruleset for allowed content types.
https://github.com/coreruleset/coreruleset/blob/v3.1.2/crs-setup.conf.example#L390-L400
application/octet-stream is in the default list of allowed content types.
As far as I can tell, Azure only allows this rule to be enabled or disabled, we can't update the underlying regex string. Personally I am of the opinion limiting content types to an arbitrary subset has limited security benefit but non the less it is what it is.

I accept that changing the TUS protocol so it doesn't fall afoul of these detections is going to be near impossible at this stage, however I wanted to shine a light on the issues so they are at least known about and can be taken into consideration in the future.

Note: I transferred this issue to the repo covering tus v1, as this issue seems to be talking about this version. The tus-v2 repo was not the correct repo because tus v2 works a bit different.

Thank you for these insights! They are always helpful with discovering the actual issues that arise when deploying tus services.

File uploads are detected if the request is a multipart/form-data with a part that has a filename attribute.

tus never uses the multipart format and, as such, this requirement will never be met. We have decided not to use multipart because for our use cases it does not provide any benefits and would only cause additional complexity.

TUS uploads files using the application/offset+octet-stream, which is not part of the default OWASP ruleset for allowed content types.

Ok, that seems overly restrictive. But again, the content type is now an integral part of the tus protocol.

I accept that changing the TUS protocol so it doesn't fall afoul of these detections is going to be near impossible at this stage, however I wanted to shine a light on the issues so they are at least known about and can be taken into consideration in the future.

Again, thank you for these insights! AFAICT, you are able to work around these issues. tus v2 will not have a pre-defined content type, and therefore the second issues should not be present. However, tus v2 will also not use multipart, so the first issues still needs a workaround.
I hope that helps!

yeah I expected that would be the response, but it's good to know the content type is not fixed for the v2 protocol.
I'm going to close this issue as there isn't a path to resolve the multipart form data use case. Having to restrict the chunks to 128kb is a pain but it is what it is.

Having to restrict the chunks to 128kb is a pain but it is what it is.

I just want to let you know that this will likely cause bad upload performance for your application. Such a small chunk size is never a good option.


Remove "Store" from input fields, add "Brands"

The "Store" field is not very important, we should prioritize which input field we display:

Product name
Brands
Quantity
...

@herau @itchix Brand is more important than the store (for the web version) so we need it more than the Store

Ideally the user should be able to fill any fields of a product... without any limitations regarding the web version. Maybe we have to find a UI to expose all the possibilities

The pencil button plays a temporary solution. If it could load in a Chrome View instead of Chrome, that would improve usability until a better native solution is ready.

d0650ddb3b2a772d6a4075749b37e5d7ce5e102b


dbt --single-threaded rpc does not work

Describe the bug
The dbt executable supports a flag called --single-threaded which is intended to bypass multithreading when building nodes (eg. models, seeds, whatever). The --single-threaded argument previously worked with the dbt rpc command, but it appears that there is now a regression!
Steps To Reproduce
Run the rpc server in single-threaded mode:
dbt --single-threaded rpc

Run a query against the server (this one does select 1 as id):
curl --location --request POST 'http://localhost:8580/jsonrpc' --header 'Content-Type: application/json' --data-raw '{
	"jsonrpc": "2.0",
	"method": "run_sql",
	"id": "123",
	"params": {
		"sql": "c2VsZWN0IDEgYXMgaWQ=",
        "name": "my_query"
	}
}'

This returns the following error:
{
    "error": {
        "code": -32000,
        "message": "Server Error",
        "data": {
            "type": "Exception",
            "message": "'Manifest' object has no attribute 'root_project'",
            "tags": null,
            "logs": [
                {
                    "timestamp": "2021-03-30T21:35:40.781048Z",
                    "message": "Acquiring new postgres connection \"rpc.debug.my_query\".",
                    "channel": "dbt",
                    "level": 10,
                    "levelname": "DEBUG",
                    "thread_name": "MainThread",
                    "process": 76144,
                    "extra": {
                        "request_id": "123",
                        "method": "run_sql",
                        "context": "request",
                        "addr": "<IP_ADDRESS>",
                        "http_method": "POST",
                        "run_state": "internal"
                    }
                },
                {
                    "timestamp": "2021-03-30T21:35:40.791030Z",
                    "message": "uncaught python exception",
                    "channel": "dbt",
                    "level": 14,
                    "levelname": "ERROR",
                    "thread_name": "MainThread",
                    "process": 76144,
                    "extra": {
                        "request_id": "123",
                        "method": "run_sql",
                        "context": "request",
                        "addr": "<IP_ADDRESS>",
                        "http_method": "POST",
                        "run_state": "internal"
                    },
                    "exc_info": "Traceback (most recent call last):\n  File \"/Users/drew/fishtown/dbt/core/dbt/rpc/task_handler.py\", line 94, in task_exec\n    result = self.task.handle_request()\n  File \"/Users/drew/fishtown/dbt/core/dbt/task/rpc/sql_commands.py\", line 162, in handle_request\n    node = self._get_exec_node()\n  File \"/Users/drew/fishtown/dbt/core/dbt/task/rpc/sql_commands.py\", line 123, in _get_exec_node\n    add_new_refs(\n  File \"/Users/drew/fishtown/dbt/core/dbt/task/rpc/sql_commands.py\", line 38, in add_new_refs\n    manifest = manifest.deepcopy()\n  File \"/Users/drew/fishtown/dbt/core/dbt/contracts/graph/manifest.py\", line 729, in deepcopy\n    selectors=self.root_project.manifest_selectors,\nAttributeError: 'Manifest' object has no attribute 'root_project'"
                }
            ]
        }
    },
    "id": "123",
    "jsonrpc": "2.0"
}

Expected behavior
I expect the request to succeed and return a successful response. I also expect the --single-threaded command to block until the query returns, then return a complete success response. This is (if I am recalling correctly) the intended behavior of the --single-threaded flag. This is a different behavior than the default which is to return a "task_id" immediately which can be polled for a response.
Screenshots and log output
Attached above
System information
Which database are you using dbt with?

[x] postgres
[ ] redshift
[ ] bigquery
[ ] snowflake
[ ] other (specify: ____________)

The output of dbt --version:
0.19.0

Using current develop branch at commit 17e57f1e0b3
The operating system you're using: macOS
The output of python --version: 3.8.6
Additional context
Should this be a command-line flag (--single-threaded?) or should it be an option in the RPC request method like sync: true? Just a thought :)

This is a regression due to changes for #2693. We don't have any test cases for the rpc single threaded option, and that's the only place that Manifest.deepcopy code is called.

This was fixed a few months back.


onAdFailedToLoad: 3

Even when I create the key around 2 month ago, I still geting the anoying onAdFailedToLoad: 3
What I can do about that?

@oscarshaitan Please provide  a small self-contained app that demonstrates the problem. You can include small examples inline if they're enclosed by backticks, or just provide a public link to a github "gist" - https://gist.github.com/.

I have the same problem, i put everything like i see on the example of the package

It looks as though you're trying to show an advert before it's loaded, therefore nothing can show. Try the below:
bannerAd = createBannerAd();

// Wait for the banner to load
await bannerAd.load();

// Once loaded, show it
bannerAd.show();

I'll close this one for now, please let me know (with a code example) if it still fails to load and I can reopen.
Thanks


deadlock detected

I'm using Fonduer to construct knowledge base from Chinese corpus,
Spacy doesn't support Chinese, So I train a Chinese model by myself , instead of downloading model from Spacy. And my Chinses model cannot split sentences, so the files in the directory which is assigned in TextDocProcess are already splitted by my script.
But when I  run featurizer or labeler applys, it's too slow to bear,
Something wrong is recorded in logfile of PostgreSQL:
Not only applying Labeler, but also applying featurizer, deadlock is detected
ERROR: deadlock detected
DETAIL: Process 31694 waits for Sharelock on transaction 71392072; blocked by process 31697.
        Process 31697 waits for Sharelock on transaction 71392071; blocked by process 31694.
        Process 31694: INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_5', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_2', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_3', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;
        Process 31697: INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_5', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_2', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes

HINT: see server log for query details
CONTEXT: while inserting index tuple(1,66) in relation "label_key"
STATEMENT:INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_5', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_2', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;INSERT INTO Label_key(name, candidate_classes) VALUES('subAction_3', ARRAY['sub_action']) ON CONFLICT(name) DO UPDATE SET name=exclued.name, candidate_classes=exclued.candidate_classes;

I don't know why that happen.

I set parallelism one when I apply featurizer or labeler.  And I can bear that speed.
But another problem happens:
The speed of training discriminative model, such as LR, or LSTM, is too slow.
the only reason is DataLoader which fetches the next step data costs almost 7s .
And when I wanna set num_workers 4 or other number, error occurs like this:
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
THCudaCheck FAIL THCCachingAllocator.cpp line=507 error=3: initialization error.
Runtime Error: Cuda runtime error(3): initialization error at THCCachingAllocator.cpp:507
the last sentence of error stack is:
X.cuda()                  classifier.py
there is the same question in pytorch 0.4 or 1.0
I don't know how to speed up the train

@SenWu @lukehsiao assign this deadlock issue to me. I have an idea and am working on it.

@yzj19870824 THCCachingAllocator.cpp error is different from the dead lock. I'd suggest to create another issue.


Add dispatching metrics in clusterchecks module + fix resolution

What does this PR do?

Fix the ClusterCheck attribute being lost during template resolution
Add the following prometheus metrics (more to come):

# HELP cluster_checks_configs_dangling Number of check configurations not dispatched.
# TYPE cluster_checks_configs_dangling gauge
cluster_checks_configs_dangling 0
# HELP cluster_checks_configs_dispatched Number of check configurations dispatched, by node.
# TYPE cluster_checks_configs_dispatched gauge
cluster_checks_configs_dispatched{node="moi6"} 0
# HELP cluster_checks_nodes_reporting Number of node agents reporting.
# TYPE cluster_checks_nodes_reporting gauge
cluster_checks_nodes_reporting 1

Motivation
What inspired you to submit this pull request?
Additional Notes
Anything else we should know when reviewing?

Codecov Report

Merging #2786 into master will increase coverage by 0.7%.
The diff coverage is 100%.

@@           Coverage Diff            @@
##           master   #2786     +/-   ##
========================================
+ Coverage    57.1%   57.8%   +0.7%
========================================
  Files         383     355     -28
  Lines       24556   19879   -4677
========================================
- Hits        14022   11491   -2531
+ Misses       9610    7562   -2048
+ Partials      924     826     -98

Impacted Files
Coverage Δ

...g/clusteragent/clusterchecks/dispatcher_configs.go
91.66% <100%> (-0.53%)
:arrow_down:

pkg/clusteragent/clusterchecks/metrics.go
100% <100%> (ø)

pkg/clusteragent/clusterchecks/dispatcher_nodes.go
91.83% <100%> (+0.92%)
:arrow_up:

pkg/autodiscovery/configresolver/configresolver.go
84.88% <100%> (+1.39%)
:arrow_up:

pkg/clusteragent/clusterchecks/stores.go
91.42% <100%> (+2.85%)
:arrow_up:

pkg/collector/runner/runner.go
53.15% <0%> (-15.27%)
:arrow_down:

pkg/metadata/resources/resources.go
75% <0%> (-8.34%)
:arrow_down:

pkg/quantile/key.go
46.15% <0%> (-7.18%)
:arrow_down:

pkg/metadata/host/host_tags.go
74% <0%> (-6%)
:arrow_down:

pkg/util/kubernetes/clustername/clustername.go
57.89% <0%> (-5.75%)
:arrow_down:

... and 243 more


BUG: Multiple callouts get merged V1.11.0

Multiple callouts become condensed into one when the 'empty line around blockquotes' setting is on. The expected result after running the linter would be for the '>' not to be inserted into the document. This new bug essentially removes blank lines between multiple callouts and was introduced with the latest update (V1.11.0)
Before Linter
## HEADING 2
> [!TLDR] Titel
> blah blah blah

> [!FAQ] Title
> blah blah blah

> [!NOTES] Title
> blah blah blah

After Linter
## HEADING 2
> [!TLDR] Titel
> blah blah blah
>
> [!FAQ] Title
> blah blah blah
>
> [!NOTES] Title
> blah blah blah

I remember fixing this prior, but maybe I did not catch a case for (#672 ). Could you provide your data.json so I can see about reproducing the issue?

My bad, I didn't see that issue and there was an update sitting there that I had not installed.

No worries

Hi, this issue appears to be back. It appears to be associated with the empty line around the blockquote setting. I have just disabled this for now.

I will see if I can reproduce this sometime later, but I just tried with the example on this ticket and could not reproduce. Is it happening for the example provided above?

Forgive me for the weird way of taking notes, it might be the reason I have encountered this issue. I sometimes use callouts to draw extra attention to something when I am studying. I have provided a new example of the scenario which cause the issue. Please let me know if this helps. I can provide my data.json file if you would like that.
> [!INFO] Title
> Content

> [!IMPORTANT] Point to emphasise
>

> [!EXAMPLE] Title
> Content

> [!TLDR] Title
> Content

> [!FAQ] Title
>

> [!NOTES] Title
> Content

So the issue is resolved for the most part... But in this example, the 'point the emphasise' callout will be broken and get bunched together like so:
> [!INFO] Title
> Content

> [!IMPORTANT] Point to emphasise
>
> [!EXAMPLE] Title
>
> Content

> [!TLDR] Title
> Content

> [!FAQ] Title
>
> [!NOTES] Title
>
> Content

I believe that I have a fix for this. I will update the title of this bug and create a PR to fix the issue.

Looks like the change I made creates some problems for some other rules, so I need to double check what is going on and  then once things are working get that in. It will likely need to wait until Friday before I can come back to this.

I believe I have the fix for this. It does not seem too bad to implement.


Fixed issue with multiple VLANs

When multiple VLANs are specified the script does not split the provided name and it uses the whole string to check if that VLAN exists (e.g. using "--cvlan Vnet-01,Vnet-02" will return an error stating that "Vnet-01,Vnet02" does not exist).
The proposed change fixes this behavior: the script now checks if every specified VLAN exists in the Datacenter.

Good catch! Thank you for fixing it.


BoxDecoration of editor

Is there an existing issue for this?

[X] I have searched the existing issues

Use case
I ran into problem of styling the quilleditor: i see no styles in parameters and try to do it with parent Container like this:
Expanded(
  child: Container(
    height: 200,
    decoration: BoxDecoration(
      color: Colors.white,
      borderRadius: BorderRadius.circular(8),
      border: Border.all(
        color: field.hasError
            ? Colors.red
            : _focusNode.hasFocus
                ? Colors.grey.shade500
                : Colors.grey.shade100,
      ),
    ),
    child: QuillEditor(
      focusNode: _focusNode,
      ),
  )
)

problem is that focusNode does not work as expected and is buggy – please advise how to overcome it.
Proposal
Add a Decoration parameter to style the quill container's border colors

Never mind me: add listened with a setState to initstate / dispose:
_focusNode.addListener(() => setState(() {}));


doc(Localization): The document seems to have a problem with display.

Document describing which component
Getting Started -> Localization

@twizardp 感谢，我马上修复一下，帮忙项目点个 star


Bump the KCEF dependency version

@DatL4g Sorry to push, I'm a bit blocked by this

@SamC-Apadmi even though I have repository access, I just review desktop request/changes and @KevinnZou approves and merges them

@KevinnZou Apologies I hadn't realised there was a klint job, I've fixed this now :) Should be ready to re-run

@KevinnZou Can this be merged?

❤ ❤ ❤

so when are you planning to release version with this fix to maven?

thanks! it works

@KevinnZou lets bump it to dev.datlag:kcef:2<IP_ADDRESS>
This version supports loading jcef from the jdk, so apps build with the Jetbrains Jcef sdk won't need to download anything anymore.


Allow eq() and any() with UUID

Bug-Url: https://github.com/nhaarman/mockito-kotlin/issues/17
Signed-off-by: Laszlo Hornyak<EMAIL_ADDRESS>

Thanks.
Instead of explicitely fixing the issue for the UUID class, I have chosen to implement a more general fix.
See #19.


python3Packages: fix some cross compilation

Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

if these aren't critical now, I would prefer to target staging. 500+ rebuilds

The rebuilds are mainly Home Assistent tests, that's why I'm targeting master.

sounds good then


rename canopen node and adjust diagnostics

as discussed with @ipa-mdl - follow-up of #434

renamed the node name
updated contains tags of the canopen components in diagnostic_analyzer for robots using canopen components

FR: OK


button group disabled state not working

Is it possible to disable one (or few) buttons in button group?
I tried [disabled] - it is not working.

prior to upgrading to Angular 2.1.2, ng2-bootstrap 1.1.16, [disabled]=condition worked for btnRadio.
I believe it was ng2-bootstrap 1.0.6.
I have a rest page with:
dropdownToggle: [disabled] functions as expected
btnRadio: [disabled] fails
btnCheckbox: [disabled] fails
with the checkbox and radio the attribute 'ng-reflect-disabled="true"' is added however the 'disabled' attribute is not added to the DOM tag (button, label, div)
In the dropdown-toggle directive I noticed it has @HostBinding('class.disabled') which is not in the button directives...

ping

Thanks, we'll look at this.

any update on the fix?

Waiting for merge :-|

still waiting....

still.. waiting.

waiting....

still waiting...this merge is taking forever apparently :)

Hi!  I'm running into the same issue.  Any idea when the code will be merged?
Thanks!
Rob

+1, any updates on this?

Is there any updates to this issue? For anyone looking for a workaround, you might be able to do something like this:
<ng-container *ngFor="let option of eventFilterMode; last as isLast">
  <button
    *ngIf="!someDisableCondition"
    [(ngModel)]="myModel"
    [btnRadio]="option.id"
  >
    {{ option.name }}
  </button>
  <button
    *ngIf="someDisableCondition"
    disabled
    [(ngModel)]="myModel"
    [btnRadio]="option.id"
  >
    {{ option.name }}
  </button>
</ng-container>

I don't think this is actually solved (for the OP and others):
Adding disabled does in fact disable the button, while [disabled]="true" (or applying any other expression that evaluates to true) does not, visually. The click event is suppressed, but the button still looks like it is enabled.
Apparently in this case setting the disabled property to true does not add the disabled attribute.

Still an issue

@SergeyKuryatnick @IlyaSurmay please see https://ngx-bootstrap-4am3wr.stackblitz.io


--upgrade-all does not work correctly on Windows

It appears to be unable to remove all of the files present in the folder (in particular, if the package is a Git repository).
We should probably optimize this so that if there's a Git repository already there, we just update cached version and then do "git fetch --alll; git reset --hard " in the repo.

Fixed in 25283a4e0858e8492383ea2f318b0b75d737350b.


[Bug]: node group update fails when VPC CNI addon is enabled

Welcome to Amazon EKS Blueprints!

[X] Yes, I've searched similar issues on GitHub and didn't find any.

Amazon EKS Blueprints Release version
4.0.4
What is your environment, configuration and the example used?
Terraform version:
Terraform v1.1.8
Provider version(s):
Terraform v1.1.8
on linux_amd64
provider registry.terraform.io/gavinbunney/kubectl v1.14.0
provider registry.terraform.io/hashicorp/aws v4.14.0
provider registry.terraform.io/hashicorp/cloudinit v2.2.0
provider registry.terraform.io/hashicorp/helm v2.5.1
provider registry.terraform.io/hashicorp/kubernetes v2.11.0
provider registry.terraform.io/hashicorp/local v2.2.3
provider registry.terraform.io/hashicorp/null v3.1.1
provider registry.terraform.io/hashicorp/time v0.7.2
provider registry.terraform.io/hashicorp/tls v3.4.0
provider registry.terraform.io/terraform-aws-modules/http v2.4.1
What did you do and What did you see instead?
I've found some bug in EKS blueprint when VPC_CNI addon is turned on. Editing a node group (for example adding a new one) fails with the timeout because the role assigned to the nodes has policy attachment removed.
How to reproduce
The code is taken from
https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/
and as this tutorial says
terraform-aws-eks-blueprints/examples/eks-cluster-with-new-vpc example is taken as a start
then getting started tutorial is executed but skipping cleanup section - the EKS is working
Steps to reproduce:
execute steps from https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/ but skip cleanup section
so now there is a cluster up and running
then:
add the following
mg_test = {
          node_group_name = "managed-test"
          instance_types  = ["m5.large"]
          min_size        = "1"
          subnet_ids      = module.aws_vpc.private_subnets
        }

in that part:
  # EKS MANAGED NODE GROUPS
  managed_node_groups = {

then execute terraform apply
the very first step terraform does is to remove policy attachment from IRSA role:
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.eks_blueprints_kubernetes_addons.module.aws_cloudwatch_metrics[0].module.helm_addon.module.irsa[0].aws_iam_role_policy_attachment.irsa[0]: Destroying... [id=aws001-preprod-dev-eks-cloudwatch-agent-irsa-2022051909043063440000000e]
module.eks_blueprints_kubernetes_addons.module.aws_vpc_cni[0].module.irsa_addon[0].aws_iam_role_policy_attachment.irsa[0]: Destroying... [id=aws001-preprod-dev-eks-aws-node-irsa-2022051909043068410000000f]
module.eks_blueprints_kubernetes_addons.module.aws_vpc_cni[0].module.irsa_addon[0].aws_iam_role_policy_attachment.irsa[0]: Destruction complete after 1s

( Note that the removed policy is arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy )
then terraform tries to create a new node group
 module.eks_blueprints.module.aws_eks_managed_node_groups["mg_ski"].aws_iam_role.managed_ng: Creating...
but it times out after 20 minutes
│ Error: error waiting for EKS Node Group (aws001-preprod-dev-eks:managed-ski-20220519091022118900000005) to create: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: 1 error occurred:
│ 	* i-06576ff0ac5971553, i-0849e995e20cbf19b: NodeCreationFailure: Instances failed to join the kubernetes cluster

Node group gets "Create failed" status and nodes have "Not ready" status
Instances are up and running but:
aws-node pod is restarting with
{"level":"info","ts":"2022-05-19T09:25:57.154Z","caller":"entrypoint.sh","msg":"Retrying waiting for IPAM-D"}
and ipamd.log shows
tail -n 3 /var/log/aws-routed-eni/ipamd.log
{"level":"error","ts":"2022-05-19T09:26:17.788Z","caller":"ipamd/ipamd.go:456","msg":"Failed to call ec2:DescribeNetworkInterfaces for [eni-0721b5e0b27ea7275]: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 0e88218f-037c-4838-a347-ee4fa254f495"}
{"level":"error","ts":"2022-05-19T09:26:17.903Z","caller":"ipamd/ipamd.go:456","msg":"Failed to call ec2:DescribeNetworkInterfaces for [eni-0721b5e0b27ea7275]: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: b8133cd9-a889-4e5c-b191-5d970f0dc7cf"}
{"level":"error","ts":"2022-05-19T09:26:18.013Z","caller":"aws-k8s-agent/main.go:27","msg":"Initialization failure: ipamd init: failed to retrieve attached ENIs info"}

I confirmed that all above happens due to arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy removal from IRSA role used by aws-node pods
Additional Information
This issue doesn't exist when VPC_CNI addon is turned off (aws-node pods don't use IRSA role then)
enable_amazon_eks_vpc_cni = false

Thanks @szkbury . Have you seen this  issue #559? I have added a background about this issue and a solution.
We found out recently that it's an issue with depends_on = [module.eks_blueprints.managed_node_groups] line in our examples.
We had to add this depends_on  to ensure all the resources are deployed in one TF APPLY. This is not a recommended approach for production. You probably need to split this deployment with three TF Apply (VPC, EKS Cluster with Node groups and Kuberentes-Addons) and this approach may not require module level dependencies.

Related to #559

alright, I will test it once again without depends_on and create new issue if any problems occur.

Thanks!

@vara-bonthu can you clarify what you meant:

This is not a recommended approach for production.

is removing depends_on not recommended? If that's the case, what is the recommended way of deploying a cluster & add-ons? If it's to apply in stages where should that be documented?


Noob need to know where is the file path

Hi there
I really like to test this player but I have no idea where is the file path so i can play the music?

Hi @adetheprince
You can't play the music because it's a prototype.
The main goal is coding an UI proposal, skipping some basic Android details and focusing on transition and animation approach, as in the article.


CSS to change a theme

I am looking for a way to add dark theme for certain websites.
One feature that is very important for me is to be able to change the theme of the website based on my system settings.

Totally agree.


Update wlo/olo version 1.3.2 for Liberty on AKS

Task 675 Update version of OLO/WLO to 1.3.2 for liberty-on-aks offer


support save helm chart

Describe what this PR does / why we need it
Fixes #966

This is preliminary work, finish store helm chart.
I will improve it latter, e.g: error handles


Update README for Clarity and Comprehension

This pull request addresses issue #15 by enhancing the README file to improve its clarity and comprehensibility.

Thanks!


GZip compression for REST endpoints

As there is no cache upgrade policy, poll mechanisms download the full state of a folder and folder tree every iteration.
The amount of data downloaded from the server can be quite big.
Requirements

Enable gzip compression for recursive REST endpoints that consume considerable bandwidth


make multipliers configurable

I want to use "times" instead of "dot" as the multiplier symbol, so I created this pull request.

Thank you for your contribution!


Remove vendored dependencies

Now that we have switched to Go modules¹, there's no need to commit dependencies in the vendoring directory to the repository, so this PR removes them.

Which Traffic Control components are affected by this PR?

Documentation
Traffic Control Cache Config (T3C, formerly ORT)
Traffic Control Health Client (tc-health-client)
Traffic Ops Client (Go)
Traffic Monitor
Traffic Ops
Traffic Stats
Grove
CDN in a Box
Automation (GHA, anything that relies on Go)

What is the best way to verify this PR?
Make sure everything still works and there are no license issues.
PR submission checklist

[ ] This PR has tests
[x] This PR has documentation
[ ] This PR has a CHANGELOG.md entryhave a changelog entry.
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY

Kinda

Vendoring prevents the build from breaking in the future, when people delete their github repos that TC depends on.
Internet resources disappearing is a common problem, and it makes it impossible to build older versions, if we don't vendor and include everything necessary to build without the internet. See: left-pad.
It also makes it possible to build if someone doesn't have an internet connection, or is in an area where one of the dependency URLs is blocked by a firewall.
I really think we should vendor, for those reasons.

Yeah, this seems like it needs a mailing list discussion. There are pros and cons to keeping the vendored dependencies in the repo (@rob05c pointed out a big one). Another pro of vendoring dependencies is that it helps to understand the amount of code changed whenever a dependency is upgraded/added/removed.

this seems like it needs a mailing list discussion

Alright, no problem. I really just didn't expect anyone to come to the defense of vendoring :P


When User Not Logged In We Can See Logout and profile button

i can contribute to this assign it to me. i can fix it.

@nishant0708 I want to work on it, Can you assign it to me?


Create attribute with null value

Can extensions generate an empty attribute, e.g. ...?
If not, could that be enabled?
I tried these:
mdi.renderer.rules.link_open = function(iTokens, iIdx, iOptions, iEnv, iSelf) {
   iTokens[iIdx].attrs.push(['download', null]); // yields download="null"
   //                       ['download']            yields download="undefined"

   return iSelf.renderToken(iTokens, iIdx, iOptions);
};

https://github.com/markdown-it/markdown-it/blob/master/lib/renderer.js#L177
You may override attrs renderer to support empty values.

Understood, thanks.
Could you consider making a null attrs value yield an empty HTML value in 11.x?

That's strange - nobody asked such feature for 5 years.  But i have no principal objections.

Could you consider making a null attrs value yield an empty HTML value in 11.x?

Alternative suggestion would be to represent empty values as attrs.push(['download', true]).
I think many people would expect attrSet('download', undefined) to unset the existing attribute rather than set it to empty value.

I'd suggest use notation attrSet('download'), it's clear. Also, i'd avoid use attrSet for unsetting and any kind of unsetting in our api at all.

I don't think any change is required to support boolean attribute
Your "empty attribute" is canonically named "boolean attribute".
The following work:
attrSet("download", "")

attrSet("download", "download")

I'm afraid the discussion here reveals a few potential bugs of markdown-it
https://github.com/markdown-it/markdown-it/blob/6e2de08a0b03d3d0dcc524b89710ce05f83a0283/lib/token.js#L170-L181
attrGet() uses null to indicate the absence. Thus, both null and undefined should be reserved. But attrSet() and renderAttrs() don't validate it:
https://github.com/markdown-it/markdown-it/blob/6e2de08a0b03d3d0dcc524b89710ce05f83a0283/lib/token.js#L153-L167
https://github.com/markdown-it/markdown-it/blob/6e2de08a0b03d3d0dcc524b89710ce05f83a0283/lib/renderer.js#L168-L185

Besides, the access control on Token should be reinforced.
At least, the following should be marked as either "internal" or "deprecated", because they allow users to add attributes that may never take effect, or even destroy the attribute records.

attrs
attrPush()

The following can be replaced by clearer things. You can take JavaScript Map as a reference.

attrIndex()

HTML Standard
2.3.2 Boolean attributes:

The presence of a boolean attribute on an element represents the true value, and the absence of the attribute represents the false value.
If the attribute is present, its value must either be the empty string or a value that is an ASCII case-insensitive match for the attribute's canonical name, with no leading or trailing whitespace.

<IP_ADDRESS> Attribute name state:

if there is already an attribute on the token with the exact same name, then this is a duplicate-attribute parse error and the new attribute must be removed from the token.


S3 build cache

Hey there!
Thanks a lot for the work you've been putting on this project. It's awesome.
I've recently written some small scripts to get my front-ends to use a build cache, instead of re-building the application every-time. I did that so that I could deploy to production without consuming at least one of the cores building the application.
Since it's more or less neatly integrated into the project, I thought I'd propose it as a feature and hear what you think.
So what I did was, first of all, script bin/docker-build.sh, which uses a very slightly modified version of the Dockerfile in this repository to build the application:
#!/bin/bash -e
sha=`git rev-parse HEAD`
app="myapp"
image="$app:$sha"

echo ">>> Checking if image exists ($image)"
if docker images | awk '{print $1 ":" $2}' | grep $image; then
  echo ">>> Image exists ($image)"
else
  echo ">>> Image doesn't exist ($image)"
  docker build -t $image .
  docker tag $image $app:latest
fi

echo ">>> Creating temporary container ($image)"
id=$(docker create $image)
build_location="build-$sha"

echo ">>> Copying assets ($id -> $build_location)"
rm -rf build-$sha
docker cp $id:/opt/este/build $build_location
docker cp $id:/opt/este/webpack-assets.json $build_location/
echo $sha > $build_location/commit
date -u +%s > $build_location/created_at

echo ">>> Removing temporary container ($id)"
docker rm -v $id

echo ">>> Querying S3 for build ($build_location)"
if aws s3 --region us-east-1 ls s3://$app | grep $build_location; then
  echo ">>> S3 build exists ($build_location)"
else
  echo ">>> Uploading assets to S3 (s3://$app/$build_location)"
  aws s3 --region us-east-1 sync $build_location s3://$app/$build_location
fi

# if you want to keep images in AWS ECR
# aws_image=""
# echo ">>> Uploading image ($aws_image)"
# eval $(aws ecr get-login --region us-east-1)
# docker tag $image $aws_image
# docker push $aws_image

Then there's a way to pull the build from S3, in bin/pull-build.sh:
#!/bin/bash -e
sha=`git rev-parse HEAD`
build_location="build-$sha"
app="myapp"
s3_build_location="s3://$app/$build_location"

echo ">>> Querying S3 for build ($s3_build_location)"
if aws s3 ls --region us-east-1 s3://$app | grep $build_location; then
  echo ">>> S3 build exists ($build_location)"
else
  echo ">>> S3 build doesn't exist"
  exit 1
fi

echo ">>> Pulling build from S3 ($s3_build_location -> $build_location)"
mkdir -p $build_location
aws s3 sync --region us-east-1 $s3_build_location $build_location

mkdir -p build
mv $build_location/* build/
rmdir $build_location
mv build/webpack-assets.json .

Finally, instead of just building, webpack/build.js was modified to:

Check if git is dirty

function gitIsDirty() {
  try {
    const output = childProcess.execSync('git status -s').toString();

    if (output.split('\n').length > 1) {
      return true;
    }

    return false;
  } catch (err) {
    gutil.log('[beijaflor]', 'Failed to check if dirty. Assuming dirty.');
    return true;
  }
}

If it's not run that script
If the script fails (there's no cached build), move on

So a CI server (in my case GitLab CI) comes in and runs the first script somehow. And then you have a cached build.

In the end you run gulp build and get:
$ gulp build
[15:09:19] Requiring external module babel-register
[15:09:20] Using gulpfile ~/program/github.com/beijaflor-io/website/app/gulpfile.babel.js
[15:09:20] Starting 'env'...
[15:09:20] Finished 'env' after 77 μs
[15:09:20] Starting 'build-webpack'...
[15:09:20] [beijaflor] Trying to get build from S3
bash ./bin/pull-build.sh
>>> Querying S3 for build (s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0)
                           PRE build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/
>>> S3 build exists (build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0)
>>> Pulling build from S3 (s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0 -> build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0)
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/3fb6388cd20e6f07a4032a75a83cab67.png to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/3fb6388cd20e6f07a4032a75a83cab67.png
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/9988747877cafafc5ab2316e74f17b58.svg to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/9988747877cafafc5ab2316e74f17b58.svg
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/959ad79962510cba8822625386ce4baf.png to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/959ad79962510cba8822625386ce4baf.png
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/aaf86d9c9eaa60cd052170a184f4b60c.png to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/aaf86d9c9eaa60cd052170a184f4b60c.png
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/69673b82cc91cdac0e3b566dfb60aae9.svg to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/69673b82cc91cdac0e3b566dfb60aae9.svg
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/commit to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/commit
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/bb5975966442b9df3b1081532ccc472a.png to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/bb5975966442b9df3b1081532ccc472a.png
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/created_at to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/created_at
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-javascript.ff73e3c68d2d2113f5af.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-javascript.ff73e3c68d2d2113f5af.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/df72f3e3cb11bca726cdd9cb12b010f0.svg to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/df72f3e3cb11bca726cdd9cb12b010f0.svg
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/d31d488d2171c1668c15962fc6cf1ef7.otf to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/d31d488d2171c1668c15962fc6cf1ef7.otf
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/dffb2b80813d195d26fa4d2aad48b059.otf to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/dffb2b80813d195d26fa4d2aad48b059.otf
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-xml.93129cdd66acebf31ba2.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-xml.93129cdd66acebf31ba2.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror.9fccf619f977ebfe9d74.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror.9fccf619f977ebfe9d74.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-jsx.aea4c88f284f5be0e8a9.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/codemirror-jsx.aea4c88f284f5be0e8a9.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/webpack-assets.json to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/webpack-assets.json
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/babel.1d42d347b8f51c097c66.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/babel.1d42d347b8f51c097c66.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/highlight.18ab482b5e73eec39319.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/highlight.18ab482b5e73eec39319.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/app.526afda93ce17c650187.js to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/app.526afda93ce17c650187.js
download: s3://beijaflor.io/build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/app.526afda93ce17c650187.css to build-fff397599f0da7bd60ad0c1aa84a517eb2d79ee0/app.526afda93ce17c650187.css
[15:09:28] [beijaflor] Build restored
[15:09:28] Finished 'build-webpack' after 7.57 s
[15:09:28] Starting 'build'...
[15:09:28] Finished 'build' after 17 μs

Which is a good 8s, low-CPU way of running npm install on a production server, then just reloading a pm2 process running the server-side bit. You could also just upload all assets to the same S3 directory, in which case rollbacks would be just about restoring the webpack-assets.json file from the current commit.

Does it belong here?

In the end; if you want a CDN in front of S3, you wouldn't really have a problem other than uploading / restoring webpack-assets.json. But I guess this is something that could be neatly integrated; to actually be able to deploy to a DigitalOcean/AWS box and be done with it.

I don't know what should be done. Feel free to send PR.


Update MLOps stacks project layout

Signed-off-by: Mingyu Li<EMAIL_ADDRESS>stacks layout design doc https://docs.google.com/document/d/1BTFOzxiVzCJ2uKN0f9N3id85pL32Gof6c0cD1ao7fNo/edit#bookmark=id.gklrrat3qlf8
It's a big change. Let's focus on code correctness for now and fix/improve docs later.

Decisions

Create one more variable for the root folder.
We'll just ask user what's the root folder name. Add to instructions that if user intends to use polyrepo, they should keep the root folder name same as project name.
Put terraform folder inside the project
Don't create a separate folder for projects, just add them to the root folder.

Layout
non-project related folders

projects

.azure

devops-pipelines
…

.github

workflows
…

.mlops-setup-scripts

cicd

…

terraform

…

README.md

docs

images

...

...

README.md
pytest.ini
requirements.txt
test-requirements.txt
_params_testing_only.txt

project related folders

projects

my_project1

terraform/bundle yaml.
development

training
feature_engineering

validation
deployment
monitoring
data

sample.parquet

tests

deployment
development

training
feature_engineering

my_project2
my_project3
utils

Test
All tests are passing
Sanity check of generating project
run Train notebook

run batch inference

Not ready for review

Ready for review

Ready for review again:) @vladimirk-db @zhe-db @arpitjasa-db

Tested running notebook and batch inference in databricks workspace.

@arpitjasa-db I added you to my repo. It's the refactor2 branch. https://github.com/mingyu89/test-repo1/tree/refactor2
Please note that the root folder name is not visible because it's also the git root folder.

@vladimirk-db I feel it better too to have one less layer for users. I made the change and tested by generating project and run the training and inference on databricks.


UIElements Fix for 2019.1

UIElements are not experimental anymore.
This gets it right for 2019.1 and 2018.3

Wow, it got released yesterday and already a PR for this, love it! ⭐


Tkakar/cat 946 map ensemble ids

This PR maps ensemble-gene-ids to gene-symbols when no gene symbols are provided in the data.
Fixes CAT-946

Background

Change List

Checklist

[ ] Have tested PR with one or more demo configurations
[ ] Documentation added, updated, or not applicable

@keller-mark thanks for reviewing and refactoring. The marker genes on the scatter plot now show ensembile-ids.

In which config? I am not seeing it on my end for http://localhost:3000/?dataset=test-config

Thanks for testing. I'm using the same test-config, other views are showing. as long as it's working, should be good.
Will remove the testConfig and merge.


A few questions

What is the target column which we will be scored by?  Is it 'ans_simple'? Do we also need to provide the equations?
What is the evaluation metric?
Which datasets can we use to train our model?
Will we be provided with a test set consisting of only the raw text questions?
When will we get that set? How many rows are in it?
Will there be an option to submit our answers before the end of the competition to find out how well we are doing?

We join the questions

you are scored by ans_simple order doesn't matter we expect for you to write the evaluation code\
The evaluation metric is hard accuracy based on ans_simple we will also ask you to present to me the model and approach you took to ensure that you didn't over fit the test set or cheat
You can use any dataset as long as it doesn't contain any values in the test set
The test set has already been provided see the attached notebook
There is no evaluation tool but you can come find me and I will be happy to guide, provide advice and tell you how you are doing


[BUG] - missing libpam-cap package to install as mandatory

Describe the bug
After hardenize, pam complains about missing library:
Apr 28 17:11:19 fcpa-fc-cz saslauthd[737]: PAM unable to dlopen(pam_cap.so): /lib/security/pam_cap.so: cannot open shared object file: No such file or directory
Seems that pam_cap is referenced in hardening template common-auth.j2 but it is not installed as mandatory package
Expected behavior
add libpam-cap to required debian packages

Added in https://github.com/konstruktoid/ansible-role-hardening/commit/6e3695ec3151605e769d314c87a4a81a20118bd3

Thank you!


Web redirects from apps does not set up the correct wallet and has no ability to switch the wallet

Problem
Web redirects from apps does not set up the correct wallet and has no ability to switch the wallet
Expected Behavior
The redirect should set up the correct wallet AND user should be able to switch the signing wallet.
Steps to reproduce

The user has multiple wallets, the last wallet used is wallet_1
He logs in the app with wallet_1 (for example, https://ecosystem.pixeldapps.co/cryptoheroes)
He switches in the wallet interface to the wallet_2
During the interaction with the app and when it is requested to sign a transaction, the user is redirected to the wallet
The wallet displays as an active wallet wallet_2 instead of wallet_1 with inability to switch the wallet

Here's the screenshot:

The URL is the following: https://wallet.near.org/sign?transactions=CAAAAGFzMi5uZWFyAI1rFJM6rTTaPFh4BKcB%2Bw7DGa6KT4fiFa3IODd%2BaiJYVml%2B6g01AAAPAAAAcGl4ZWx0b2tlbi5uZWFySiTUi1PV9b7fCAsDShAAi8hurggsGIJTgrP0QDO40sgBAAAAAgsAAABjaF9idXlfaXRlbVAAAAB7InRva2VuX2lkIjoiMTIzNjk4Iiwib3duZXIiOiJyb3pib3IubmVhciIsImJ1eWVyIjoiYXMyLm5lYXIiLCJwcmljZSI6IjEwMDAwMDAifQBgt5hsiAAAAQAAAAAAAAAAAAAAAAAAAA%3D%3D&referrer=Pixelpets+Dapp%3A+Pixelpets+Dapp&callbackUrl=https%3A%2F%2Fecosystem.pixeldapps.co%2Fcallback%3Fpage%3Ditem_bought

+1
Some dapps like rainbowbridge expect the signer to be the connected account (wallet_1)

@esaminu is working on this here: https://github.com/near/near-wallet/issues/2402 :)


Question about different models and GPU usage (i.e., VSSM, VSS, and VSSMD)

Hi! Thank you all for making this amazing work public! It is really helpful!
I had a question about the different models/files. What is the difference between VSSM, VSS, and VSSMD? Are they the same thing or should I use one versus the other?
Another question I had is about actually using the models. I'm loading the model like this:
def load_model():
    model = VSS(depths=[2, 2, 9, 2], dims=96)
    model.head = nn.Sequential(
        nn.Linear(768, 1024, bias = True),
        nn.ReLU(),
        nn.Dropout(p=0.5),
        nn.Linear(1024, 1024, bias = True),
        nn.ReLU(),
        nn.Dropout(p=0.5),
        nn.Linear(1024, 1000, bias = True)
    )
    # model = torch.nn.DataParallel(model) #make parallel

    model.train()
    model = model.cuda()
    return model

However, I find that when I run inference or attempt to train the model, it uses an absurd amount of GPU memory. I am using 512*512 pixel images, and I can only support a batch size of 16 given the configuration described. The model itself only takes 500 MB of VRAM, but each image takes around ~2 GB of VRAM. Am I doing something wrong?

Some more information: I'm using a GPU with 32 GB VRAM. Surely, I should be able to have a larger batch size, no? I can do so with ResNet and ViT. Even without parallelization, I can run batch sizes of up to 64/128.

Than you for your attention.
We use the model VSSM. The hyper-parameter depths=[2,2,9,2], dims=96 is exactly the standard of vssm_tiny.
We have batch size 32 each card for classification with resolution 224x224, which cost 7000+MB gpu memories.
We have batch size 2 and resolution 512x512 in upernet framework for ADE20k segmentation, which only cost 6000+MB GPU memories, even with extra big modules in upernet framework. see https://drive.google.com/file/d/17nh9_hdF9QQxyqj81U86HoGUnMxZQ4nN/view for details.


Version 5.2.1 on PyPI, not on GitHub

Currently, PyPI shows 5.2.1 as the latest version of python-arango, but the releases page here on GitHub only has 5.2.0.
PyPI: https://pypi.org/project/python-arango/#history
GitHub: https://github.com/joowani/python-arango/releases
Is current master actually version 5.2.1, just untagged & unreleased here on GitHub?

Hi @themowski,
Thanks for pointing this out. I forgot to make a new release. It is now created.
And yes as of this writing the master is version 5.2.1 (you can check by looking at version.py).


android: CameraAnalyzer.Decode frame disposes fastArray even though its still referenced

I've been running into infrequent (but persistent) crashes with null object references and/or references to deleted objects in OnPreviewFrame (which leads to a crash).
I noticed that on android, CameraAnalyzer.Decode disposes the fastArray that's passed in. But OnPreviewFrame (which invoked OnPreviewframeReady -> Decode) passes this fastArray to AddCallbackBuffer.
Removing the dispose() in Decode() makes these problems go away (at least, so far). The code flow definitely looks suspect -- disposing the object in Decode(), but continuing to use it after the Invoke().

Turns out its not as straightforward (is threading every straightforward?).
it depends on the timing if there will be a problem. The task will dispose the fastArray, which could happen before its used in AddCallbackBuffer, but probably not. However, by adding the data back as a callback buffer, the camera is now free to reuse it before our background task is complete. All of these race conditions will lead to some unpredictable results in our listener.
Ownership of the data buffer (and the fastArray that wraps it) should be managed by the task (which will limit us to a single listener event, since the event callback will have to manage returning the buffer back to the camera and disposing the fastBuffer).


Fix flaky integ test caused by latency of undeploy model

Description
For ml-commons plugin, after model undeploy returns, the max interval to update model status is 3s in ml-commons CronJob. This can cause flaky integ test in neural-search plugin. Add 3s sleep to fix this.
example: https://github.com/opensearch-project/neural-search/actions/runs/6321926472/job/17166662817
Issues Resolved
[List any issues this PR will resolve]
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@zane-neo does this change needs to be backported?

@zane-neo does this change needs to be backported?

Yes, let's backport this.


Supporting changes for WRK items 1 and 2

Removed most of the locking section but left a paragraph about concurrency control and how that is device specific.

Reviewed on the call on the 13th of December 2022.


feat(dataproc): generate library

Fixes #8182
The service config json has different retryable codes per RPC. I went with the strictest list for each service, and discounted kDeadlineExceeded. This left everything as kUnavailable.
The endpoint is interesting. I did not see it in documentation, but if you send a request with a region, the service responds that you should be using the regional endpoint. Other client libraries do the same thing.
I enabled the Dataproc API and created two clusters in region "us-central1" using the UI. (as described in this quickstart).
# Command
bazel run //google/cloud/dataproc/quickstart:quickstart -- dbolduc-test us-central1

# Output
cluster-6f37
cluster-8d27

This change is

Our publish-docs build fails because <tbody> is not handled by Doxygen. See https://www.doxygen.nl/manual/htmlcmds.html
I decided to just change the generator in this PR to comment out the tag. See the "fix doxygen" commit.
PTAL


Improve number assignment when annotating an exam to avoid hole (e.g 1 followed by4 )

First improvement here #245


Updating many-to-many relationship values in model.

Problem:
When updating many-to-many relation via this package there is a situation where there is a nessecity to pass specific structure of data. For example:
POST /books
{
    "name": "Example name",
    "published_at": "2022-08-08",
    "tags": [
          {
              "id":1
          },
          {
              "id":2
          }
     ]
}

Above example works but if we add a little twist (for example when front application want to pass returned parameters in request) like this
{
    "name": "Example name",
    "published_at": "2022-08-08",
    "tags": [
          {
              "id":1,
              "name": "Horror"
          },
          {
              "id":2,
              "name": "Sensation"
          }
     ]
}

There is an exeption raised by mysql that field name is not in pivot table.
Resolution:
Check if using Pivot models that extendsIlluminate\Database\Eloquent\Relations\Pivot class can be working as filters for attributes that are not in pivot tables. Especially by guard attribute.

Solution:

add pivot model and attach it to relation,
add fillable attribute to pivot model and add all fields you want to edit in pivot table
now only the fields you set in fillable attribute will be saved, even if you pass extra params (without throwing an exception)

todo:

prepare some generic solution to make it easier...

Solution proposed by @mg10110 was discussed and approved.


Racecondition: grunt serve with auto rebuild

Hi,
if I serve an app using livereload there seems to be a racecondition when to reload the page.
I basically just use a starter project and serve it in chrome
grunt serve --consolelogs
If i now change something in the app folder, like the title in the index.html page everything is rebuild
>> File "app\index.html" changed.
Running "newer:copy:app" (newer) task

Running "copy:app" (copy) task

But the browser reloads too early, when basically not all files have been moved over to the www folder.
I then see errors like this in the console
0     282928   log      LiveReload protocol error (invalid command 'reload', only valid commands are: hello)) after receiving data: "{"command":"reload","path":
"myapp\\www\\bower_components\\ionic\\js\\angular\\directive\\modalView.js","liveCss":true,"liveJs":true,"liveImg":true}"..

Not sure if this is a specific issue with my setup - or just one in general an issue with ionic detecting the change not as a "set"?
I couldn't find an option for specifying the timeout or delay in the ionic.project file.
Any suggestions?
I'm using
<EMAIL_ADDRESS><EMAIL_ADDRESS>win 7

I have same issue. I even reinstalled whole stack (node, npm, ionic, cordova etc.). Clean, newest packages and LiveReload just choking. Thanks for any advice.
npm 2.14.4
node v4.1.1
Windows 10


How to update the excel cell as Number instead of text.

Hi ,
I am trying to save some values in the excel sheet as numbers that will later be used in Formulas. I am using SpreadsheetDecoder in Dart . The problem is , I am able to save the data , but the data saved is in Text format and so it being text i am not able to use in Formula that need Number. The code I am using is provided below.
import 'dart:io';
import 'dart:typed_data';
import 'package:path/path.dart';
import 'package:spreadsheet_decoder/spreadsheet_decoder.dart';
File file = File('FileToUpdate.xlsx');
List requiredRow;
SpreadsheetDecoder decoder;
Uint8List bytes;
/*
Row number (rowNum) for various values in excel sheet
'aValue':14 , int
'bValue':15 int
'cValue':16 double
*/
int rowNum = 14;
int newValue = 5;
void main(List args) {
bytes = file.readAsBytesSync();
decoder = SpreadsheetDecoder.decodeBytes(bytes, update: true);
final table = decoder.tables['SheetName'];
requiredRow = table.rows[rowNum];
var requiredValue = requiredRow[0] as String;
print(requiredValue);
setValue(newValue);
final valueStored = getValue();
print(valueStored);
}
void setValue(int value) {
decoder.updateCell('SheetToUpdate', 1, rowNum, value);
File(join('FileToUpdate.xlsx'))
..createSync(recursive: true)
..writeAsBytesSync(decoder.encode());
}
String getValue() {
String returnValue = requiredRow[1].toString();
return returnValue;
}

This feature isn't available yet.
The code that manage this is here.
https://github.com/sestegra/spreadsheet_decoder/blob/master/lib/src/xlsx.dart#L517
I won't have time to implement now.

@sestegra according to you is probably more related to https://github.com/sestegra/spreadsheet_decoder/blob/master/lib/src/xlsx.dart#L558 ;)
I am trying to implement the feature but I struggle to find the right structure excel demands.
Can you send me the source you used to look it up? I used https://docs.microsoft.com/en-us/dotnet/api/documentformat.openxml.spreadsheet.celltype?view=openxml-2.8.1.
My solution atm is adding:
final cellType = value.runtimeType == String ? 'InlineString' : 'CellValue';
 XmlAttribute(
       XmlName('r'), '${numericToLetters(columnIndex + 1)}${rowIndex + 1}'),
      XmlAttribute(XmlName('t'), cellType),

This produces:

1234567890.0

(also omitting the type assuming number is the default did not work)
But excel can not handle it.
Any help would be appreciated.

I just found http://officeopenxml.com/SScontentOverview.php I think that will help me :D

Hey @sestegra could you throw me a 🦴 and send me into right direction. Unfortunately I am pretty stuck right now 😕
Thx

@AnkuK @sestegra @DerBasler
Facing same issue...
Any solutions?

Hey @MalikSamiAwan no unfortunately I had no time to dig in any further but let me know if you have more luck.


Use a larger runner for Linux test jobs

From 8 to 16 cores, 32 to 64 GB RAM for a 2x per minute cost increase.
As in:

#5874
#5873

From 4m to 2m.


Add cloudflare support

Add cloudflare support according to https://support.cloudflare.com/hc/en-us/articles/229373388-Cache-Content-by-Device-Type-Mobile-Tablet-Desktop-

Hi, @nachogarcia
Thank you for the PR!
According to the reference, headers property has lower cased keys.
Does your code work correctly on your environment?
https://nuxtjs.org/api/context/#-code-req-code-a-href-https-nodejs-org-api-http-html-http_class_http_incomingmessage-em-http-request-em-a-
https://nodejs.org/api/http.html#http_message_headers

You're completely correct. Looks like I have setup my testing environment wrong.

I'll check this and merge if everything looks good.

Let me know if more changes are required.

No need more changes.  I'll merge this after check.

Thanks!

No problem :)
Will you release it soon?

@nachogarcia
I've released just now!


Create Django models to store data from the NOVELCovid API, JHUCSSE section

Create Django models to store data from the NOVELCovid  API, JHUCSSE section.
We can start with storing Data from the /v2/historical get API.
The data comes in the form:
    "country": "Afghanistan",
    "province": null,
    "timeline": {
        "cases": {
            "1/26/20": 0,
            "1/27/20": 0,
            "3/18/20": 22,
        },
        "deaths": {
            "1/26/20": 0,
            "1/27/20": 0,
            "1/28/20": 0,
        },
        "recovered": {
            "1/26/20": 0,
            "1/27/20": 0,
            "3/5/20": 0,
            "3/6/20": 0,

        }
    }

I suggest we go ahead and create a model that stores this in the form:
country = models.CharField
province = models.CharField
timeline = models.TextField

Fixed by #9


[TRAC-951] RDS usage report

This PR :
- Changes the architechture of usage reports
- Adds cost detail for EC2 reports
- Adds metrics statistics and cost detail on RDS reports
It adds for each RDS instance :

CPU Average
CPU Peak
Minimum free space
Maximum free space
Average free space
Cost detail

- Adds a monthly RDS usage report
As for EC2, it creates a detailled report based on billing data for the previous month.
- Changes the endpoint /rds
There is a new query arg: date to specify on the request
- Adds the endpoint rds/unused
As for EC2, it returns an array of unused instances ordered by cost based on query params:

accounts: list of aws accounts separated by , (Optionnal: default: all)
date: date of the wanted report
count: number of instance in the array (Optionnal: default all)

@simonmeyerrr If we merge this PR to stg and deplpoy it, are we sure that the currently implemented monthly cost report will not take down the nomad cluster ?
Since you seemed to have trouble running it on large accounts locally


Terminal text shifts to the left when scrolling up

VSCode Version: 1.2.0
OS Version: Windows 10 Pro Insider Preview build 14352.rs1_Release.160522-1930

I am using angular-cli's ng--help function for this case.
Steps to Reproduce:

Launch VS Code
Open Terminal
ls
dir
ng --help
Scroll up

Result:  All the text in the terminal shifts to the left, and the terminal becomes stuck that way.  In my gif, i somehow got it back, but once i scrolled up it broke again.

Here is my dev tools console at startup.  It does not change when this error occurs.
/C:/Program Files (x86)/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:92 [Extension Host] Spell and Grammar checker active...
/C:/Program Files (x86)/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:92 [Extension Host] .NET Core Debugger tools already installed
/C:/Program Files (x86)/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:92 [Extension Host] rejected promise not handled with 1 second
/C:/Program Files (x86)/Microsoft VS Code/resources/app/out/vs/workbench/workbench.main.js:118 Unexpected token /: SyntaxError: Unexpected token /
    at Object.parse (native)
    at C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\out\assets.js:35:42
    at tryCatcher (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\util.js:26:23)
    at Promise._settlePromiseFromHandler (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\promise.js:507:31)
    at Promise._settlePromiseAt (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\promise.js:581:18)
    at Promise._settlePromises (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\promise.js:697:14)
    at Async._drainQueue (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\async.js:123:16)
    at Async._drainQueues (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (C:\Users\NickA\.vscode\extensions\ms-vscode.csharp-1.0.11\node_modules\bluebird\js\main\async.js:15:14)
    at tryOnImmediate (timers.js:534:15)
    at processImmediate [as _immediateCallback] (timers.js:514:5)

Duplicate of https://github.com/Microsoft/vscode/issues/6738


How to disable buttons?

This is not related to this package.

@puzrin  It's related to that package
For some reason you can disable images but not buttons why
const md = MD({ html: true, typographer: true, breaks: true, linkify: true }).disable([
  "image"
]);

What BUTTONS are you speaking about? Please provide example via online demo.

@puzrin Here is an image

All enable/disable are about markdown components only. If you enabled raw hml - it's only your responsibility how to sanitize output.


Support $NNN bind parameters for sqlite

The sqlite driver for sqlx currently supports only the ? and ?NNN bind parameter formats. However, sqlite supports additional formats as specified here: https://sqlite.org/c3ref/bind_blob.html. With this change, I propose adding support for $NNN, which is a subset of $VVV. This will make it easier for sqlx users to write queries that are supported by both sqlite and postgres.

@nitsky I would like to at least see a test for this to make sure it works as expected.

Excellent, thanks!


What happen to ICQ?

hi folks,
what happen to the icq client in franz 5?
Greetings

It will be aded again by the end of this week.

This repository is only for the Franz client. Please use this form for service requests or check out the guide to create your own service integration.


Steps are outdated

[Enter feedback here]
Steps are outdated in this section: Create a function app
Options of "Operating system" and "Plan" in Step 4 are moved to Step 3.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 12720fc3-2b5d-301d-c5d7-42878a7ee212
Version Independent ID: b992b5d7-052e-5095-2ab4-1dcae7e563da
Content: Create an Azure Functions Premium plan in the portal
Content Source: articles/azure-functions/create-premium-plan-function-app-portal.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@gwu70ca
Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate.

@ggailey777
Can you please check and add your comments on this doc update request as applicable.


Why are AppendIdentityWhereCondition and AppendRowsAffectedWhereCondition abstract and rest is virtual?

Why are AppendIdentityWhereCondition and AppendRowsAffectedWhereCondition abstract and rest is virtual?
In FirebirdClient I don't have any code path where these could be used (but I do use other parts from UpdateSqlGenerator like i.e. AppendInsertCommandHeader), yet I still have to override these and have a dummy code there.
Would making these virtual brake anything?
Maybe even, given these are very much SQL Server specific (although I can imagine other providers can fit in), these could go into EFCore.SqlServer (but that would be breaking change).

@AndriySvyryd This looks like update pipeline; can you answer @cincuranet's questions?

@cincuranet These methods are required to be implemented for correct behavior and there's no standard SQL that could be placed in the base implementation as far as I can tell. What would you suggest to be generated that could reasonably be expected to work across different relational databases?

I would just make them virtual instead of abstract. Then these still can be overridden, yet if you don't use them, you can safely ignore them, as other virtual methods in UpdateSqlGenerator.

@cincuranet If they are virtual they need to have an implementation, one that has reasonable behavior when not overridden.

Empty body or exception (neither is going to be breaking ATM) is good for me. Because there's no good general implementation every provider is either going to use skip it or override it, SqlServer provider included.

@cincuranet It seems that you are overriding a lot of methods in UpdateSqlGenerator. Are there any that you actually leave as is? Would it make more sense for your provider to implement IUpdateSqlGenerator directly?

I override only AppendXxxOperation. The rest I (re)use (like AppendInsertCommandHeader, AppendValuesHeader, AppendValues, ...). So it doesn't make much sense to implement all this from scratch to use IUpdateSqlGenerator directly.

Ok, we will consider extracting another base class that only has the more useful methods, but we need more feedback from another provider writer to make sure that the above is a good set.

@cincuranet This issue is quite old; is this still something you need?

Yes.

@cincuranet I'm sorry - I intended to get this done for 6.0, but it is now too late to get this in (we are now in principle only doing bug fixes). Part of the reason this got delayed, is that there's some probably related work on using the new ADO.NET batching API in the update pipeline (see #18990) - I'll likely work on both of these together at the beginning of the EF 7 work cycle.

@roji OK. :)


Order of accessing related lazy loaded relations on closed entity manager

Bug Report

Q
A

BC Break
no

Version
2.6.2

Summary
Probably easier demonstrated via test case. Basically, if you have a loaded entity (a) and close em and it's relations(b,c) are not loaded yet and (b) relates to (c) then accessing(loading) methods on (c) first would break the internal (proxy?) state of (b). Accessing in order of (b),(c) however would work just fine
Current behavior
Order of operations matters
How to reproduce
https://gist.github.com/sserbin/bd0c685b90d8b73f5d04cbc2307c07bd
Expected behavior
Order of operation does not matter, internal object state is not lost/broken
upd: let me know if I should create a pull request with the test case (not sure on the process)

@Ocramius yes, although the behaviour is slightly different:
# both are null, both assertions fail
        $this->assertEquals('the organization', $inv->organization->name); # null
        $this->assertEquals('john doe', $inv->account->name); # null

# account->name is null, organization->name is correct
        $this->assertEquals('john doe', $inv->account->name); # null
        $this->assertEquals('the organization', $inv->organization->name); # correct

here's the full test file: https://gist.github.com/sserbin/1904116d9cd241af7af06f68569f7cad


Fix Rails 5 deprecation warning for to_prepare

Fixes #1233
This pull introduces/changes:

Switch to use ActiveSupport::Reloader.to_prepare to fix Rails 5 deprecation warning

Pings:
@cheerfulstoic
@subvertallchris

I think there is more work to do in updating the reloading code in general, but this at least fixes the deprecation warning and doesn't seem to break anything.

I think that this is fixed already in the rails5 branch:
https://github.com/neo4jrb/neo4j/compare/rails5
That branch is actually pretty old.  @subvertallchris is it alright if I merge 7.0.x into rails5?
@bloomdido so that you're in the loop, @subvertallchris has been trying to get the rails5 branch specs running so that we can cut a 7.1.0 release to support Rails 5, but he's been having trouble with some weird error.  To quote him:
"I emailed Travis about the remaining JRuby issue. Looks like it might be some incompatiblity with bundler, certain Linux distros, and Docker!? ugh"

The rails5 branch was based off of master, so if you're going to merge anything into there, it should probably be that. The rails-5-neo4jrb-7.0.x branch is what you'd expect from the name and what I'm using as the basis for #1233, which targets 7.1.x, like this PR. I'm fine with merging this into the 7.1.x branch or cherry-picking from rails5, but aside from this, I'd prefer minimal changes on this branch until the Rails 5 stuff is merged in and stable.
I'm going to leave a comment on my PR regarding the remaining failing tests over there.

Yeah, I stopped using the rails5 branch in favor of the rails-5-neo4jrb-7.0.x months ago. Just noticed a to_prepare fix never made its way into that branch, and since @subvertallchris was getting a PR ready to merge it into 7.1.x, figured I'd add one. Didn't know which branch to submit the PR to, so I just synced it with 7.1.x and let you guys figure it out.

[Current coverage][cc-pull] is 97.22%

Merging [#1237][cc-pull] into [7.1.x][cc-base-branch] will decrease coverage by 0.02%

@@              7.1.x      #1237   diff @@
==========================================
  Files           168        168
  Lines         10406      10466    +60
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          10120      10176    +56
- Misses          286        290     +4
  Partials          0          0

Powered by Codecov. Last updated by [f9b28f0...90564ea][cc-compare]
[cc-base-branch]: https://codecov.io/gh/neo4jrb/neo4j/branch/7.1.x?src=pr
[cc-compare]: https://codecov.io/gh/neo4jrb/neo4j/compare/f9b28f01f678326c0776d583a5166b95cff8b52d...90564eae54e166a1eaaf4a20c483077153ae31c4?src=pr
[cc-pull]: https://codecov.io/gh/neo4jrb/neo4j/pull/1237?src=pr

This is perfect. Gonna merge and once Travis passes one more time, we'll do the release!

We'll probably get a conflict on this because of this already being fixed on master

I thought the old fix was on the rails5 branch, which never got merged into master, right? At least, I don't see it fixed on master...

Yeah, I think you're right.  I was thinking that the rails5 branch was the one to use, but I don't think we're going to merge it.  We can just merge 7.0.x -> 7.1.x -> master and be up to date


[FEATURE REQUEST] Prevent cache upload (to S3) if cache file/folder is empty/non-existent

In one of my CI jobs, I noticed that the plugin was able to upload an empty/non-existent file/folder to S3.
[2023-06-27T23:17:34Z] [typescript-cache] - 🔍 Locating cache:  my_project/some_cacheable_folder
[2023-06-27T23:17:34Z] tar: my_project/some_cacheable_folder: Warning: Cannot stat: No such file or directory
upload: ./cacheable_folder-v1-checksum123.tar.gz to s3://bucket/caches/cacheable_folder-v1-checksum123.tar.gz

This has consequences for CI jobs that have the same cache key because those jobs end up downloading and using an empty cache. For example, this might lead to slower build times (e.g. imagine a bundler like Webpack trying to read from an empty cache which causes bundling time to be slower compared to not using a cache).
Ideally the build would complete but not upload the cache if the cache file is not found. Maybe we can add an option to do this.
e.g.
nienbo/cache#v2.4.14:
    id: node-modules-cache
    backend: s3
    ...
    paths:
      - "my_project/some_cacheable_folder"
    upload:
      - "always"(default) | "skip-if-tarfile-empty" | "skip"

We could do something like this:
  if [ -d "$TAR_TARGETS" ]
  then
  	if [ "$(ls -A $TAR_TARGETS)" ]; then
       info "Caching... (hit)"
  	else
      warning "Root exist but empty (exist-but-empty)"
  	fi
  else
  	echo "Root dir not found (non-existent)"
  fi

However, this will work on your example above but will ultimately fail on most users because many of them uses multiple paths. Globs will be fine, however, giving multiple dirs. to cache can fail. One of them could be empty but other is not.
So, this is still doable, but we need to think better how to properly implement those checks.
Probably making a loop and excluding paths that's empty or non-existents then cache what remains.

In case when there are multiple paths and some empty, some non-empty, it's still ok to cache as cache will have smth at least. In case when everything's empty, it doesn't make sense at all to create an empty cache.
PS Not sure if using multiple paths that common as well, as usually there is one cache per tool (e.g. lint cache for linter, prettier cache for prettier and so on)


Better support for more logging levels (WARN, ERROR, FATAL)

Strange as it may seem, some folks will want to hide WARN level logging (not my cup of tea, but a general-purpose library should cater to all tastes).  Some extremists might even want logic for ERROR or FATAL levels.
In particular, BaseLogger should have functions: isWarnEnabled(), isErrorEnabled(), isFatalEnabled()

Thanks, will do.

Done.


chore: update to fastify v5

Update fastify to v5, and all the fastify plugins to their new compatible v5 version

Nice! We'll have to do this upgrade alongside a larger upgrade to node 20 and drop support for node 18. Fastify v5 will only support node 20.


[magnum]: do not use deprecated Magnum functions

Update for latest upstream Magnum...

shouldn't we merge this to `devel' and keep the master to a specific version of Magnum?
Merging this will break all the installations/containers that did not update Magnum yet, I guess...

We need to first have a devel branch.. ;)
I think that I like more the following: master is development, other branches or tags for stable things. I can create one for your needs. What do you think?

Yes, this works for me too, but we need to keep a "release x" branch in that case :)

Good! I'll first make a PR to remove (the dead now) travis and use GitHub actions for CI. Then I'll create separate release branches..

@jbmouret I'll merge this and let me know if things will break for your dockers/setups. Of course the easiest solution is just to use latest Corrade/Magnum etc, but once we make the 1.0.0 release we'll provide installation instructions for specific commits of Corrade/Magnum etc for easy portability and avoid issues.


make hostdb.round_robin_max_count configurable

@SolidWallOfCode @PSUdaemon could you please review this one?

[approve ci]

@SolidWallOfCode updated. Thanks for the suggestion.

@SolidWallOfCode pushed another commit related to round_robin_max_count. Could you review it?

[approve ci]

@SolidWallOfCode ping. Looking forward to feedbacks from an ATS expert like you.

[approve ci]

@zizhong There are still build issues for this PR

@bryancall That's weird. The building is okay on my local. The Details link is broken now. Can you [approve ci] again?

[approve ci]

Thanks, @scw00 . Can you please trigger the build again?

[approve ci]

format fixed.

[approve ci]

@SolidWallOfCode Fixed.

[approve ci]

[approve ci]


Implement toString / fromString on VST3 Parameter

IThe wrapper doesn't call param::value_to_text anywhere or text_to_value
So what we need to do is on Vst3Parameter add an overload of
    virtual void toString (ParamValue valueNormalized, String128 string) const;
    virtual bool fromString (const TChar* string, ParamValue& valueNormalized) const;

then implement to_string as _pluigin->_paramextension->value-to_text.
Since the VST3 Parameter object doesn't have a reference to the plugin, the idomatic way seems to be as follows

Add a pair of std::functions to the VST3 Parameter std::function<std::string(float)> valueToString and std::function<float(const std::string &)> valueFromString
Call those in the overloads with a null check. Deal with the TCHar/Widechar thing on windows properly
Have them implemented to bind the plugin and clap id. Have them call through to the to string from string ids
Test etc

implemented in IEditController overloads.


Add Ramsey model

Add a Ramsey model.

only covers frequency scans since time scans are just a sinusoid so not much point using a Ramsey model
the heuristic here only covers the case where w_0 and phi are the unknowns. e.g. we don't have a heuristic for the Ramsey delay / pulse duration. It wouldn't be too hard to add something to cover these, but I can't think of any contexts where we don't have a good initial guess for them / don't know them accurately so it doesn't feel worth the effort of writing, testing and maintaining that code.

Credit: this is largely just pushing code written by @AUTProgram

Merging this as is. If anyone has feedback (e.g. about parameter names) we can always change it later, but I don't want this to bit rot.


Fix import statement for meraki_networks_vlan_profiles

upstream: https://github.com/cisco-open/terraform-provider-meraki/issues/120

fixed upstream


SSL configuration is Running - [More than 8 hours]

App Id: d19lms38rip60s
Region: us-east-1
Step: SSL configuration
Status: Running

Note: Do not include information that is sensitive in nature such as your domain name, company etc.

Issue/question
Application is stuck in SSL configuration for more than 8 hours. we are using google domain to host our application and CNS records has been updated and verified at google side.
Error message
If there is an error message, please include it here.
Note: Be sure to check the message for sensitive information.
Additional information
Please add any other relevant information. Please feel free to include screenshots.

Refer to our documentation to troubleshoot domain setup.
https://docs.aws.amazon.com/amplify/latest/userguide/custom-domain-troubleshoot-guide.html
https://docs.aws.amazon.com/amplify/latest/userguide/custom-domain-troubleshoot-guide.html#i-get-a-cnamealreadyexistsexception-error
Let us know if you're still facing issues.

Refer to our documentation to troubleshoot domain setup.
https://docs.aws.amazon.com/amplify/latest/userguide/custom-domain-troubleshoot-guide.html
https://docs.aws.amazon.com/amplify/latest/userguide/custom-domain-troubleshoot-guide.html#i-get-a-cnamealreadyexistsexception-error
Let us know if you're still facing issues.

We are going to close out this issue, but please feel free to re-open if you still need help.


Add badges

Is your feature request related to a problem? Please describe.
I want to add badges in readme. Please assign under jwoc

@khushimarothi Thank you for showing your interest. But this feature is not required at the moment.

@khushimarothi You may go ahead

No I feel that it will make the readme look more attractive

@khushimarothi Cool. You can work on it


[AKS] Fix several live test cases in aks-preview

This PR aims to fix several live test cases in aks-preview by removing redundant "--service-principal xxxx" and add necessary "--yes" option in test file. Fixed test cases are listed below:

azext_aks_preview.test_aks_create_with_virtual_node_addon
azext_aks_preview.test_aks_create_with_ingress_appgw_addon_with_deprecated_subet_prefix
azext_aks_preview.test_aks_enable_addon_with_openservicemesh
azext_aks_preview.test_aks_disable_addon_openservicemesh
azext_aks_preview.test_aks_byo_appgw_with_ingress_appgw_addon
azext_aks_preview.test_aks_byo_subnet_with_ingress_appgw_addon
azext_aks_preview.test_aks_create_with_ingress_appgw_addon

General Guidelines

[ ] Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
[ ] Have you run python scripts/ci/test_index.py -q locally?

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

AKS


entity issue

Deno Code and Error Message : https://nekobin.com/kubohoxuyo
Tg Discussion : https://t.me/gramjschat/43683

Seems like this problem is still persisting, and might be related to this: https://github.com/grmjs/grm/blob/main/src/sessions/memory_session.ts#L214

Note: This is a bot-only issue.


no enums are being generated when running docker-generate edge

enums.py doesn't contain any classes when running docker_generate.py edge. Running generate against a FP10.x instance generates a whole bunch of enum classes, but this appear to be happening when running against a PF11 instance.

Hi Ed,
As of PF11, Ping has removed the reference to any definitions of enumerations in their Swagger file. Therefore, the class generation for enums.py needs to be enhanced.
A feature branch has been created that allows for testing this functionality (https://github.com/Py-Ping/py-ping-fed-sdk/tree/feature/enums).

thanks Ted, this branch has fixed the issue

Awesome, I'm glad to hear. I will create a Pull Request to move this enhancement to master.


Add docker dev

PR Checklist
[X] The commit message follows our guidelines: https://github.com/intershop/intershop-pwa/master/CONTRIBUTING.md
[No ci] Tests for the changes have been added (for bug fixes / features)
[X] Docs have been added / updated (for bug fixes / features)
PR Type
[X] Feature
[X] Documentation content changes
[X] Application / infrastructure changes
What Is the Current Behavior?
No docker for development
What Is the New Behavior?
Provide a docker file for development in docker
Does this PR Introduce a Breaking Change?
[X] No
Other Information
This PR is providing a docker file for development purpose. The strategy of this Dockerfile is to be as simple as possible and creating image compatible with any version of PWA. For that purpose the image generated by the dev docker file is not containing the PWA code nor the npm dependencies installed. Please read the documentation included in the PR.

Hi @LeoBenoist,
sorry for the really late reply. Now I had a chance to look at it.
Working with the PWA is as easy as:

install NodeJS LTS
npm install
ng serve

To me, the idea of wrapping it in containers does not really give additional benefit with this rather lengthy list of commands to setup everything. If it works for you, fine. But I'd rather not include it into the standard code.

Hi @LeoBenoist,
sorry for the really late reply. Now I had a chance to look at it.
Working with the PWA is as easy as:

install NodeJS LTS
npm install
ng serve

To me, the idea of wrapping it in containers does not really give additional benefit with this rather lengthy list of commands to setup everything. If it works for you, fine. But I'd rather not include it into the standard code.


Redundant / added in router while using strict routing

I tried using SvelteKit's handler with ultimate-express, while serving the website on a different base path (paths: { base: '/ui' }).
Initially, it was presenting an infinite redirect because the / was being stripped from the path.
app.set('strict routing', true) solved that, and all good there.
However, this introduced another issue: After enabling it, it was 404'ing. After further inspection, seems like an extra slash is being added to the URL when it should not be there.
https://github.com/dimdenGD/ultimate-express/blob/b34c5b15ba16e5ad1963d4c45d5528360014f220/src/router.js#L454-L456
This whole piece of code feels redundant since the / is not stripped at all anywhere, not sure why you'd want the URL to become //. Commenting this out fixes my issue.
Keep in mind it works in express without using app.set('strict routing', true) at all.
TL;DR:

express: GET /ui/ -> req.pathname is /ui/
ultimate-express, without strict routing: GET /ui/ -> req.pathname is /ui
ultimate-express, with strict routing: GET /ui/ -> req.pathname is /ui//

I couldn't reproduce your issue, but I noticed that req.path was implemented wrong and fixed it. Please try and check if it fixed it for you: npm install https://pkg.pr.new/dimdenGD/ultimate-express@61150dd
Also I assume you meant req.path? You have req.pathname in your message which doesn't exist in Express.
If it wasn't fixed, please provide a simple reproduction code. Also as for infinite redirect, if it still happens with new version, please open a second issue and provide reproduction code for it too.

Fix now released on NPM (1.3.5)


[WIP] Read cache for FASTER

Currently, FASTER supports read caching only by an optional parameter that copies reads to the tail of the hybrid log. However, this results in write amplification in the log, which is undesirable. Further, due to the log growth, it necessitates log compaction even when the workload is read-only.
To address this, we are adding a "read cache" to FASTER. The idea is to have a separately configured circular buffer in memory that will hold recently read items. The FASTER hash index will point to this cache entry, and the entry will in turn point to records in the actual hybrid log.
For simplicity, we will allow at most one cache entry per hash-table entry, and the cache entry will point to the original hybrid log address (tail of record linked-list). This means:
(1) Any upsert or RMW on the same hash entry will result in invalidating the read cache entry and point the hash entry to the newly added hlog entry
(2) A read for a colliding key will either replace the currently cached key or not be cached itself (tbd).
Cache addresses are distinguished from hlog addresses using the most significant bit of the address. When checkpointing the hash table, we will have to replace addresses to cache entries with original addresses.
Adding @peterfreiling  and @gunaprsd for ideas and comments.

New iteration of code pushed.


Build precompiled extension for Python on Linux w/ UCS2

cc @jtattermusch @nathanielmanistaatgoogle
cc #5280

I don't know if it is relevant, but PEP 513 spells out how to build (and upload to PyPI) manylinux1 wheels, including separate names for UCS-2 vs UCS-4 builds.

@tseaver Aye, relevant and something we'll get around to: #6036.

The only folks who I've seen have problems with this appear to have custom builds of Python, which implies that they have the necessary tools available to them to build gRPC Python from source. We can punt on this for now (and it will become easier to address following our use of manylinux1).

Fixed in #6280.


Issue #1539 : Fixing dynamodb filtering (contains, begins with)

Currently contains and begins with are not respecting the given filter value

Coverage decreased (-0.003%) to 92.522% when pulling 09ac77d979d455dccba6bb73fad73b0b0cd8de85 on sawandas:master into 56f29a0e6ed65ba197f6e005d86cd56e66db0731 on spulec:master.


Test spec for Document Packager getHtmlData failed on Windows due to \r\n problem

The test should work fine in Travis since by default git will commit our file into the using \r\n newline
However, when it is cloned to Windows, \n will be used regardless of what was being committed into the repo.
  Document Packager
    getHtmlData
      ×should handle generation of TOC
        expected '
Heading
\n

Heading

Sub heading 1

Sub heading 2

\n
Sub heading 1
\n
Some nice text
\n
Sub heading 2
\n
Some even nicer text
\n' to equal '
Heading
\r\n

Heading

Sub heading 1

Sub heading 2

\r\n
Sub heading 1
\r\n
Some nice text
\r\n
Sub heading 2
\r\n
Some even nicer text
\r\n'

One possible solution to this problem is to normalize the output / input \r\n before performing the expect  testing. e.g. replace all \r\n to \n in both the input and output strings before doing the comparison.

Bug has been solved! Filtered output.txt files before comparing with getHtmlData. #240


fix(pagehero): feedback styles too much space between the title and the content

Summary

Feedback on styles of the page hero, adjust the top padding of the title and top align all the pagehero content

Related Issues

Closes IBM/carbon-addons-iot-react#157

Acceptance Test (how to verify the PR)

Check that the Page Hero stories look good

:tada: This PR is included in version 1.38.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.38.0 :tada:
The release is available on npm package (@release-1.x dist-tag)
Your semantic-release bot :package::rocket:


Double bond geometry loss on calling removeHs

SMILES Strings with explicit hydrogens present and double bond geometry specified lose the double bond geometry when importing to RDKit ROMol object with default sanitization.
e.g. If we use, where we wish to keep explicit hydrogens:
RWMol mol = RWMol.MolFromSmiles(smiles, 0, false);
mol.findSSSR();
String canonicalSmiles = mol.MolToSmiles(true);
then the value of canonicalSmiles is as expected.
But, if we use the following, where we would like to remove hydrogens:
RWMol mol = RWMol.MolFromSmiles(smiles);
mol.findSSSR();
String canonicalSmiles = mol.MolToSmiles(true);
Then, the resulting SMILES has lost the stereochemistry supplied has been lost, for example:
[1*]/C(Cl)=C(/[H])C([H])([H])[H]		--> 		[1*]C(Cl)=CC
[1*]C([H])([H])/C([H])=C(\[H])C([H])([H])[H]	-->		[1*]CC=CC
[1*]/C(Cl)=C(\[H])C([H])([H])[H]		--> 		[1*]C(Cl)=CC
(In each case, the SMILES on the left has been generated from a previous ROMol#MolToSmiles(true) call))
Steve

In all your cases, you are relating stereochemistry to the Hydrogen.  As a test, I have done it to the heavy atom with expected results:
m=Chem.MolFromSmiles("[1*]/C(Cl)=C\C")
Chem.MolToSmiles(m, isomericSmiles=True)
'[1*]/C(Cl)=C\\C'

So the hydrogen issue smells like a bug to me,  I expect the hydrogens are being removed and unexpectedly removing the chiral information.  We can perhaps validate this here:
m = Chem.MolFromSmiles("[1*]/C(Cl)=C(/[H])C([H])([H])[H]", sanitize=False)
Chem.SanitizeMol(m, rdkit.Chem.rdmolops.SanitizeFlags.SANITIZE_ALL)
Chem.MolToSmiles(m, isomericSmiles=True)
# so far so good
'[1*]/C(Cl)=C(/[H])C([H])([H])[H]'
mh=Chem.RemoveHs(m)
Chem.MolToSmiles(mh, isomericSmiles=True)
# chirality is gone
'[1*]C(Cl)=CC'

So, indeed, this is a bug.  Hopefully the python snipped above will give you a work around!

@sroughley : The bug is fixed and the fix is queued for review.


Add checkbox beside each row of "table"

Is your feature request related to a problem? Please describe.
Add checkbox beside each row of "table" visualization type to allow user to select from the list and reflect the rows to another chart(such as line diagram). ie, If user select 3 rows out of 10 records, only the selected rows will be on the line chart.
Additional context
Current "table" is not so powerful. As a cornerstone to data presentation, the most important missing feature IMHO is to have a way to interact with user in order to provide a self-service platform.

You want to implement these features, which means you need to drill down and so on

Should this be extended from the legacy package or plugin-* packages? I saw the recent discussion still around the legacy one. Is it because the plugin-* packages is not mature/stable enough to extend?
@superset-ui/plugin-* packages are newer and has higher quality in general. A key difference that they do not depend on viz.py (which contain visualization-specific python code) and interface with /api/v1/query/ instead: a new generic endpoint instead meant to serve all visualizations. Also should be written in Typescript.

Also what's the start point/approach to implement these? Any hint or document is helpful. Thanks

I suggest looking at the "emit filter events" option in the "Customize" menu for the table viz. It might already have what you're looking for or just require a bit of tweaking.

@suddjian Thanks for the info. I found this issues on ui-plugin repo might be useful. will take a look:
https://github.com/apache-superset/superset-ui-plugins/issues/125