Vtable analysis doesn't seem to work on VC++ binaries

Given the following program:
struct Bar {
        virtual int a(int x) = 0;
        virtual int b(double x, double y) = 0;
        virtual int get() = 0;
};

struct Foo : public Bar {
        int a(int ax) override {
                x = ax;
                return x;
        }
        int b(double ax, double ay) override {
                x = ax;
                y = ay;
                return y;
        }
        virtual int get() {
                return y;
        }
        int x;
        int y;
};

int main() {
        Bar* b = new Foo;
        b->a(4);
        b->b(6, 7);
        return b->get();
}

Foo and Bar don't seem to show up in the ClassTypeInfo Tree.

Here's an x64 version compiled with VC++ 2019
Archive.zip

And a x86 version compiled with VC++ 2008
Archive 2.zip

So 8cc330d should have solved the problem with Foo and Bar not showing up in the tree. I was accidentally excluding struct and union rtti.
I did notice that some vtables were missing functions. Most notable was std::exception missing the what function. I'm not sure if it is only the last function being trimmed off or if they are more. I suspect this is due to an issue with Ghidra's Windows rtti analyzer but I'd like to investigate further before I go pointing any fingers.
For the x64 program vtable analysis is intentionally not being run. This is due to the presence of control flow guard (CFG). When CFG is present some vtables will have pointers to the control flow functions instead of the actual virtual functions and I have no idea how to resolve the destination. Running the analyses when they are present causes much more harm than good so the analysis is automatically disabled.


feature: Helm chart for Kubernetes deployment

Make sure these boxes are checked! 📦✅

[x] You have a nightly compatible version of rustfmt installed

rustup component add rustfmt --toolchain nightly

[x] You ran cargo +nightly fmt on the code base before submitting

What was wrong? 👾
The repository was lacking a Helm chart for Kubernetes deployment.
How was it fixed? 🎯
The Helm chart was added at charts/safe-client-gateway, a location adhering to Helm best-practises. The helper script at scripts/build-and-publish-helm-chart.sh can be used for releasing new versions of the Helm chart. Documentation of the chart is at charts/safe-client-gateway.
This chart is already used successfully to run the Safe infrastructure in Kubernetes.

I have read the CLA Document and I hereby sign the CLA

recheck

@tolbrino and @ausias-armesto thank you for your contribution.

The repository was lacking a Helm chart for Kubernetes deployment.

This is not something that we need for the project. My recommendation is to fork the repository and change it according to your needs.
At the moment we won't specify on how the service should be deployed and the respective resources (that is up to each team using it). This applies to the suggested kubernetes configuration.

@tolbrino and @ausias-armesto thank you for your contribution.

The repository was lacking a Helm chart for Kubernetes deployment.

This is not something that we need for the project. My recommendation is to fork the repository and change it according to your needs.
At the moment we won't specify on how the service should be deployed and the respective resources (that is up to each team using it). This applies to the suggested kubernetes configuration.

Thanks for the feedback. We are already working with our own forks, but that does not help out the Safe community as a whole since discovery of such additional changes is hard.
I understand you might not want to add these files into each repository. How about creating a new repository called safe-charts which holds all available charts for the various Safe infrastructure components? You may then also state that the repository is a community effort and not an official distribution, but at least its a single place for the community to find it.

Thanks for the feedback. We are already working with our own forks, but that does not help out the Safe community as a whole since discovery of such additional changes is hard.

We try to address the community needs including feature-discovery. However a discussion about those needs is also required. The introduction of this tool and respective deployment files assumes that this how the project should be deployed and these tools should be used. We do not make any assumptions on how the deployment should work in this repo and we give that choice to the developers who want to run our services themselves. We do provide a Docker image to make it easier to deploy but even then we do not assume that the image itself should be used (e.g. a developer/team might want to use a competing solution to Docker).

I understand you might not want to add these files into each repository. How about creating a new repository called safe-charts which holds all available charts for the various Safe infrastructure components? You may then also state that the repository is a community effort and not an official distribution, but at least its a single place for the community to find it.

If the suggestion is for it to live under the safe-global then this needs to be aligned internally. However, due to the open-source nature of the project, something like safe-charts can be created by any org. I cannot promise that this is something that will be done under safe-global (for the reasons that I mentioned).
Also, there was no discussion on why this change is necessary for the project and an evaluation on the community support behind it. Adding Helm to the repository assumes that Helm is the desired solution that we want for the community and so far that discussion did not take place. Since this repository mostly deals with the business logic for client-side rendering, the respective deployment of such service should in this case be considered on a separate repo (as you suggested).


Adds 'Required' to required input's label

There's already an "Optional" tag that's added to the fields by default, this PR adds a "Required" tag.

Thanks guys!


Home icon in testimonial page

There is no way to go back to home page from testimonial page . I will add home icon in the page which will re direct to homepage.
Please assign this issue to me under JWOC.

Also I can add similar navbar if u want

Sure, you can work on it! @S-ishita

Should I add navbar too or just home icon? @SSHSRN


SFDX force:source:push fails with latest version of sfdx-cli

Summary
SFDX force:source:push fails with latest version of sfdx-cli
Steps To Reproduce:

Create scratch org via VS Code command "Create Default Scratch Org"
In terminal, enter "sfdx force:source:push --json"

Expected result
Code should have been pushed to scratch org
Actual result
"Pushing v56.0 metadata to<EMAIL_ADDRESS>using the v56.0 SOAP API
ERROR running force:source:push:  Push failed. UNKNOWN_EXCEPTION: An unexpected error occurred. Please include this ErrorId if you contact support:<PHONE_NUMBER>-71171 (-1850168425)"
Received the following error, when --json turned on
PS C:\Source\pop-int-poc> sfdx force:source:push --json
{
"result": [],
"status": 1,
"name": "DeployFailed",
"message": "Push failed. ",
"exitCode": 1,
"actions": [],
"context": "Push",
"data": [],
"stack": "DeployFailed: Push failed. \n
at PushResultFormatter.getJson (C:\Users\CCB\AppData\Roaming\npm\node_modules\sfdx-cli\node_modules\@salesforce\plugin-source\lib\formatters\source\pushResultFormatter.js:37:27)\n
at Push.formatResult (C:\Users\CB\AppData\Roaming\npm\node_modules\sfdx-cli\node_modules\@salesforce\plugin-source\lib\commands\force\source\push.js:171:26)\n
at Push.run (C:\Users\CB\AppData\Roaming\npm\node_modules\sfdx-cli\node_modules\@salesforce\plugin-source\lib\commands\force\source\push.js:33:21)\n
at runMicrotasks ()\n
at processTicksAndRejections (node:internal/process/task_queues:96:5)\n
at async Push._run (C:\Users\CB\AppData\Roaming\npm\node_modules\sfdx-cli\node_modules\@salesforce\command\lib\sfdxCommand.js:102:40)\n
at async Config.runCommand (C:\Users\CB\AppData\Roaming\npm\node_modules\sfdx-cli\node_modules\@oclif\core\lib\config\config.js:298:25)",
"warnings": [],
"commandName": "Push"
}
Note: when trying to reproduce using the dreamhouse repo, issue not reproducible, but messaging
referenced v55.0 metadata, instead of v56 of repo encountering error.
PS C:\Source\dreamhouse-lwc> sfdx force:source:push
Pushing v55.0 metadata to<EMAIL_ADDRESS>using the v56.0 SOAP API
DEPLOY PROGRESS | ████████████████████████████████████████ | 89/89 Components
Updating source tracking... done
System Information

Which shell/terminal are you using? (e.g. bash, zsh, powershell 5, powershell 7, cmd.exe, etc.)

Powershell

If you are using sfdx

Run sfdx version --verbose --json
{
"cliVersion": "sfdx-cli/7.186.2",
"architecture": "win32-x64",
"nodeVersion": "node-v16.17.0",
"pluginVersions": [
"@oclif/plugin-autocomplete 1.4.2 (core)",
"@oclif/plugin-commands 2.2.4 (core)",
"@oclif/plugin-help 5.2.1 (core)",
"@oclif/plugin-not-found 2.3.17 (core)",
"@oclif/plugin-plugins 2.2.4 (core)",
"@oclif/plugin-search 0.0.11 (core)",
"@oclif/plugin-update 3.1.0 (core)",
"@oclif/plugin-version 1.2.1 (core)",
"@oclif/plugin-warn-if-update-available 2.0.23 (core)",
"@oclif/plugin-which 2.2.9 (core)",
"@salesforce/analytics 1.0.9 (user)",
"alias 2.1.19 (core)",
"apex 1.4.3 (core)",
"auth 2.3.12 (core)",
"community 2.1.5 (core)",
"config 1.4.23 (core)",
"custom-metadata 2.0.16 (core)",
"data 2.1.22 (core)",
"generator 2.0.17 (core)",
"info 2.3.5 (core)",
"limits 2.2.3 (core)",
"org 2.2.24 (core)",
"packaging 1.12.4 (core)",
"schema 2.2.3 (core)",
"signups 1.2.12 (core)",
"source 2.3.16 (core)",
"telemetry 2.0.7 (core)",
"templates 55.2.3 (core)",
"trust 2.3.0 (core)",
"user 2.2.1 (core)",
"@salesforce/sfdx-plugin-lwc-test 1.0.1 (core)",
"@salesforce/sfdx-scanner 3.6.2 (user)",
"salesforce-alm 54.8.5 (core)",
"sfdx-mohanc-plugins 0.0.205 (user)"
],
"osVersion": "Windows_NT 10.0.22621",
"shell": "cmd.exe",
"rootPath": "C:\Users\CB\AppData\Roaming\npm\node_modules\sfdx-cli"
}

Additional information
Feel free to attach a screenshot.

This error: UNKNOWN_EXCEPTION: An unexpected error occurred. Please include this ErrorId if you contact support:<PHONE_NUMBER>-71171 (-1850168425)"
means there was a server side error.  Support is best suited to help you with that error.  You might want to try another scratch org or maybe looking for more deployment details about the error within the scratch org UI.

I tried 8 different scratch orgs, with different configurations and options - all get the same error, even on the deployment status page in the orgs. I'm also seeing issues with mdapi deploy to our QA org, and that was working fine earlier this week. I do note that it's the eve of an SF release for my company's instance, which means all kinds of oddities may be occurring. I'll switch tracks and wait for the hubbub to subside, and see if it's resolved after the release is finished. If not, I'll follow up with SF support - thanks!

I see this in the server side error details: Invalid Descriptor Format: zipcode_filter
so maybe there is something wrong in one of the metadata files.  You could try setting the SFDX_MDAPI_TEMP_DIR env var and inspecting the contents of that dir.

That is apparently the beast, but it doesn't look or act like the usual suspects when there's a WaveTemplate error thrown - those are usually quite specific, down to the file/line, etc. Also, this was code that deployed to a scratch org just fine yesterday, before I updated my CLI version this morning. I really hope they haven't made changes to their error handling that obfuscate messages.

BTW - thanks for your help, @shetzel !


API permissions not working

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report
[ ] Performance issue
[ ] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

Browser:

[x] Chrome version 76.0.3809.132
[ ] Firefox version XX
[ ] IE version XX
[ ] Edge version XX
[ ] Safari version XX

Library version

Library version: 1.1.3

Current behavior

I have created an AAD app with Azure Service Management api permissions and the scope listed on the portals say no admin consent required.
https://management.azure.com/user_impersonation
Admin consent required
No
But when i run my sample app using msal and after successful login my sample app was redirected to a UI popup saying admin approval required, I am not sure which admin it is referring if its azure subscription admin (which I am ) and even tried login again with my credentials but it always redirected to the same UI popup
Expected behavior

Minimal reproduction of the problem with instructions

@srasool Is this in the Microsoft 1st party tenant? It sounds like this permission is restricted at the tenant level, and you would need a tenant admin to grant the permission.
https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-permissions-and-consent#admin-restricted-permissions
@manoj-rath Are you or someone from your team able to provide any additional insight?

@jasonnutter thanks for response. I might be wrong in my understanding for how it should work. Our scenario is basically we want to release an app to third party customers and they should be able to login using their AAD login and call Azure ARM apis (get azure resources list etc.) My understanding is we would create a multi-tenant AAD app in our tenant and use that app (client id basically ) in msal login flow, once user logged in using his creds there will be a new SP created in their tenant to access azure resources.

@srasool Have you tried setting this up in a custom tenant (as opposed to the Microsoft 1st-party tenant)?

@jasonnutter yes it worked on custom tenant. I reached out the internal teams and learned that i need to make my AAD app as 1P app. Thanks for the info

@srasool Thanks for the update, glad you were able to get it working!


[ZEPPELIN-906] Apply new mechanism to AlluxioInterpreter

What is this PR for?
This PR applies the new interpreter registration mechanism to Alluxio interpreter.
What type of PR is it?
Improvement
Todos

[x] - Remove static property definition code lines in AlluxioInterpreter.java and create interpreter-setting.json under alluxio/src/main/resources dir

What is the Jira issue?
ZEPPELIN-906
How should this be tested?

apply this patch
rm -r interpreter/alluxio and rm conf/interpreter.json
build source
bin/zeppelin-daemon.sh start
Run a paragraph with alluxio commands

Screenshots (if appropriate)
Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

Looks good to me.
Merging if there is no further discussion


getQueryParams not returning proper data type

Bug Report
zend-expressive v2.0.2
Summary
Do not know if this is a known issue or if this has already been fixed. The issue exists in zend-expressive v2.0.2. When accessing url query parameters via $request->getQueryParams(), integer values will be cast to string (e.g. 2 => "2"). I discovered this on parsing a query string encoded with http_build_query on an array.
Same thing happens when applying parse_str($_SERVER['QUERY_STRING']).
Current behavior
$request->getQueryParams() casts integers to strings
How to reproduce
$aggregates = [
            'aggregates' => [
                'number' => 1
            ]
        ];

$client->request('GET', $myServiceUrl . '?' . http_build_query($aggregates))
Sends sth. like [myServiceUrl]?aggregates%5Bnumber%5D=1 to your service
$params = $request->getQueryParams();
var_dump($params['aggregates']) // number: "1"

Expected behavior
1 stays 1 :sunglasses:
FYI @heiglandreas

Let's start with:

Same thing happens when applying parse_str($_SERVER['QUERY_STRING']).

and:

`$client->request('GET', $myServiceUrl . '?' . http_build_query($aggregates))

We'll start with a statement: query string arguments are presented as a string; hence the verbiage "query string". This is defined by the IETF specifications.
Now, when it comes to PHP, http_build_query() creates a string representation of the arguments passed to it. parse_str() parses the string pass to it, has zero idea what the types of the arguments are, and, as such, it resolves all of the values as strings.
This is true of the behavior of $_GET as well. If you were to build a script with the following:
<?php
http_response_code(200);
header('Content-Type: text/plain');

echo var_export($_GET, true), "\n";

and then pass it a URI with the query string "?aggregates%5Bnumber%5D=1", you'd get:
array (
  'aggregates' =>
  array (
    'number' => '1',
  ),
)

Essentially, this is something we cannot change.
It's up to the developer of the application to decide what the intended types are in this scenario, validate them, and, if necessary, cast them.


Unhardcode icon path in GNU/Linux laucher

Since the icon already is installed to a location compliant with freedesktop.org standards you don't need to specify the full icon path (neither the file type extension) in the launcher. The icon will be found anyway.
This facilitates icon theming.
Cf. this example .desktop launcher.

Thanks :+1:


[master] Upgrade to latest dependencies

Produced via:
./hack/update-deps.sh --upgrade
./hack/update-codegen.sh

/cc knative-sandbox/channel-wg-leads
/assign knative-sandbox/channel-wg-leads

I can't.
/cc @matzew @slinkydeveloper

/lgtm
/approve


Sandbox Mode

This would be extremely useful for integrators looking for a more restrained environment.
Some considerations:

[ ] A limited luaL_openlibs that doesn't expose the filesystem
[ ] Disallow require (except for Pluto's pseudo-modules such as crypto), maybe provide an easy way to make a custom resolver
[x] Hard execution time limit (alternatively, ILP but no way to get around it)

[ ] Considerations for os.sleep function

We do have ETL and content moderation now. Going to track the rest in separate issues.


[EXPERIMENTAL] docker build --parallel for multi-stage build

- What I did
Parallelized building multi-stage Dockerfile
- How I did it
Added --parallel to docker build.
This flag is experimental.
Code reading guide:

builder/dockerfile/builder.go:         The entrypoint. This calls ./parallel.go if --parallel is set, and returns the last stage image ID as the whole image ID.
builder/dockerfile/parallel.go:        Create *Builder instance for each of stages, and execute *Builder in parallel.
builder/dockerfile/parallel/*.go:      Parse Dockerfile and create DAG.
pkg/dag/*.go:                          Generic DAG utility
pkg/dag/scheduler/*.go:                Generic parallel DAG executor

Notes:

you can specify number of parallelism as --parallelism, but not functional yet (just set to the maximum parallelism)
if base images are not pulled in advance, they can be pulled redundantly by multiple child worker goroutines

- How to verify it
example Dockerfile
# usage: docker build --parallel --no-cache .

FROM busybox AS st0
RUN echo BEGIN: st0
RUN touch /ST0 && sleep 5
RUN echo END: st0

FROM busybox AS st1
RUN echo BEGIN: st1
RUN touch /ST1
RUN echo END: st1

FROM busybox AS st2
RUN echo BEGIN: st2
COPY --from=st0 /ST0 /ST2.A
COPY --from=st1 /ST1 /ST2.B
RUN echo END: st2

Build the example (st0 and st1 will be executed in parallel)
$ docker build --parallel --no-cache  .
Sending build context to Docker daemon  2.048kB
Prepared the experimental parallel builder.
 * Total stages: 3
 * DAG         : &{Nodes:[0 1 2] Edges:[{Depender:2 Dependee:0} {Depender:2 Dependee:1}]}
 * Schedule    : par{seq{0;par{2;}}seq{1;par{2;}}}
Note that this feature is not stable yet. Use carefully.
Notes:
 * you might need to pull base images manually before running this. Otherwise they would be puled redundantly by the child worker goroutines.
 * --parallelism is just ignored currently, and the maximum parallelism is forcibly used.

Building stage 1 with deps map[]
(parallel) Stage 1 Step 1/4 : FROM busybox AS st1
(parallel) Stage 1  ---> 00f017a8c2a6
(parallel) Stage 1 Step 2/4 : RUN echo BEGIN: st1
Building stage 0 with deps map[]
(parallel) Stage 0 Step 1/4 : FROM busybox AS st0
(parallel) Stage 0  ---> 00f017a8c2a6
(parallel) Stage 0 Step 2/4 : RUN echo BEGIN: st0
(parallel) Stage 1  ---> Running in 220d88ac8247
(parallel) Stage 0  ---> Running in 9373274f37ec
(parallel) Stage 1 BEGIN: st1
(parallel) Stage 0 BEGIN: st0
(parallel) Stage 0  ---> b27fc1f42bda
(parallel) Stage 0 Removing intermediate container 9373274f37ec
(parallel) Stage 0 Step 3/4 : RUN touch /ST0 && sleep 5
(parallel) Stage 1  ---> d53d4f649ebd
(parallel) Stage 1 Removing intermediate container 220d88ac8247
(parallel) Stage 1 Step 3/4 : RUN touch /ST1
(parallel) Stage 0  ---> Running in 0da400734d20
(parallel) Stage 1  ---> Running in 7ff4ad06f5fb
(parallel) Stage 1  ---> 68643f6dfc17
(parallel) Stage 1 Removing intermediate container 7ff4ad06f5fb
(parallel) Stage 1 Step 4/4 : RUN echo END: st1
(parallel) Stage 1  ---> Running in a512da2b14d7
(parallel) Stage 1 END: st1
(parallel) Stage 1  ---> 9807c27d2fca
(parallel) Stage 1 Removing intermediate container a512da2b14d7
(parallel) Stage 1 Successfully built 9807c27d2fca
Built stage 1 as sha256:9807c27d2fca18b3040c87baa849855390b03bc419f96ba71fee834c228125f8
(parallel) Stage 0  ---> 00ee7ce9540e
(parallel) Stage 0 Removing intermediate container 0da400734d20
(parallel) Stage 0 Step 4/4 : RUN echo END: st0
(parallel) Stage 0  ---> Running in 3ddc6d80085e
(parallel) Stage 0 END: st0
(parallel) Stage 0  ---> d63fef66d14b
(parallel) Stage 0 Removing intermediate container 3ddc6d80085e
(parallel) Stage 0 Successfully built d63fef66d14b
Built stage 0 as sha256:d63fef66d14b2e5d3fc41373b8c01ec32212edc5eef47c427c682fb47a7a2f42
Building stage 2 with deps map[st0:sha256:d63fef66d14b2e5d3fc41373b8c01ec32212edc5eef47c427c682fb47a7a2f42 st1:sha256:9807c27d2fca18b3040c87baa849855390b03bc419f96ba71fee834c228125f8]
(parallel) Stage 2 Step 1/7 : FROM sha256:d63fef66d14b2e5d3fc41373b8c01ec32212edc5eef47c427c682fb47a7a2f42 as st0
(parallel) Stage 2  ---> d63fef66d14b
(parallel) Stage 2 Step 2/7 : FROM sha256:9807c27d2fca18b3040c87baa849855390b03bc419f96ba71fee834c228125f8 as st1
(parallel) Stage 2  ---> 9807c27d2fca
(parallel) Stage 2 Step 3/7 : FROM busybox AS st2
(parallel) Stage 2  ---> 00f017a8c2a6
(parallel) Stage 2 Step 4/7 : RUN echo BEGIN: st2
(parallel) Stage 2  ---> Running in 1621c3210950
(parallel) Stage 2 BEGIN: st2
(parallel) Stage 2  ---> 268befe7322d
(parallel) Stage 2 Removing intermediate container 1621c3210950
(parallel) Stage 2 Step 5/7 : COPY --from=st0 /ST0 /ST2.A
(parallel) Stage 2  ---><PHONE_NUMBER>99
(parallel) Stage 2 Removing intermediate container 2af1a96e438e
(parallel) Stage 2 Step 6/7 : COPY --from=st1 /ST1 /ST2.B
(parallel) Stage 2  ---> 4d8f089bc457
(parallel) Stage 2 Removing intermediate container d5e979be13a4
(parallel) Stage 2 Step 7/7 : RUN echo END: st2
(parallel) Stage 2  ---> Running in 35e245f72668
(parallel) Stage 2 END: st2
(parallel) Stage 2  ---> ac31fb4dc308
(parallel) Stage 2 Removing intermediate container 35e245f72668
(parallel) Stage 2 Successfully built ac31fb4dc308
Built stage 2 as sha256:ac31fb4dc3082180adcc24496228e14e184e1d9e6110bbe0544bb516130f097b
Finished building 3 stages. The last stage is sha256:ac31fb4dc3082180adcc24496228e14e184e1d9e6110bbe0544bb516130f097b

- Description for the changelog

[EXPERIMENTAL] docker build --parallel for multi-stage build
- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Akihiro Suda<EMAIL_ADDRESS>

cc @tonistiigi
(sorry for very big PR 😅 )

I haven't taken the time yet to read through the code, but I'm curious to understand the theory behind this.
You say that in the example, st0 and st1 will be run in parallel. Does that include the RUN directives? What if those RUN directives read or wrote the same files? One simple example of when this could happen is if there are multiple RUN apt-get ... commands. Each invocation of apt-get will lock the apt database in order to make changes to the package management state. If you try to two apt-get commands at once, one will fail. Other tools won't have this file locking and will show unpredictable failure modes when you run them in random order or concurrently.

Thanks for the work on this!
It would be great to discuss through the design of big things like this to minimize wasted effort.
We definitely want this feature and it was hinted as the next step for multi-stage builds. And we want it by default, not behind a flag.
The DAG based execution engine should be the new core of the builder, not only provide concurrent stage execution but concurrent build jobs, more efficient processing and cache reuse, cache export/import for DAG branches etc. In the same time, this core should be separated from the frontend(Dockerfile) logic to provide more options for declaring the build definition and also provide more extension points for others to reuse this solver.
I'd like to put together a more concrete execution draft how we can start implementing everything required for this with the help of the community. From this PR I would recommend separating out a package that can do the parsing from Dockerfile into build stages(also assume that ARG is supported in the --from/AS). That lets us keep the problems of turning Dockerfile into a build-step-DAG and solving any build-step-DAG different. From that DAG next step would be to run the arguments expansion and image metadata calculation as a preprocessing step.

I wonder if #9656 is still an issue, and if this implementation has the same problem.
Have you timed the parallel runs? Is it actually faster than sequential ?

@tonistiigi
Thank you for looking into this.
I'm closing this PR, and open an github issue in a few days.
For generic DAG scheduling library, is there any existing implementation that fits our usecase?

Number of graph nodes would be less than 10.  So, we don't need any clever algorithm.
No distributed execution (we could scatter docker build over swarm, but it would be too much, I suppose)
Minimum addition to vendor/

If there is no neat implementation, I'll refactor the pkg/dag package in this PR, and split it as a small separate PR.
@dnephin
Thank you for the information.
A benchmark result:
Environment

GCE n1-standard-4
overlay2 (AUFS produces similar result)

Dockerfile (imitates Dockerfile of Docker itself)
# POC; not suitable for `docker run`
FROM golang:1.8 as docker
RUN apt update \
  && apt install -y btrfs-tools libdevmapper-dev \
  && git clone https://github.com/docker/docker.git /go/src/github.com/docker/docker \
  && go build -o /usr/local/bin/dockerd github.com/docker/docker/cmd/dockerd \
  && go build -o /usr/local/bin/docker github.com/docker/docker/cmd/docker

FROM golang:1.8 AS runc
RUN apt update \
  && apt install -y libseccomp-dev \
  && git clone https://github.com/opencontainers/runc.git /go/src/github.com/opencontainers/runc \
  && (cd /go/src/github.com/opencontainers/runc && make && make install)

FROM golang:1.8 AS containerd
RUN apt update \
  && apt install -y btrfs-tools \
  && git clone https://github.com/containerd/containerd.git /go/src/github.com/containerd/containerd \
  && (cd /go/src/github.com/containerd/containerd && make && make install)

FROM alpine AS dind
COPY --from=docker /usr/local/bin/dockerd /usr/local/bin/dockerd
COPY --from=docker /usr/local/bin/docker /usr/local/bin/docker
COPY --from=runc /usr/local/bin/runc /usr/local/bin/runc
COPY --from=containerd /usr/local/bin/containerd /usr/local/bin/containerd
COPY --from=containerd /usr/local/bin/containerd-shim /usr/local/bin/containerd-shim

Result

docker build --no-cache .: 2m12s
docker build --no-cache --parallel .: 1m54s

Unfortunately, it is not so fast I thought.

opened issue https://github.com/docker/docker/issues/32550

Ok hey sorry guys. Im back on android trying to get use to the functions


Prettier v2 formatting changes

This PR is the result of running npm run prettier -- --write after merging #606
Might need to test this against upgraded eslint dependencies too

We must carefully merge it, because it will create conflicts with all existing PR.

We must carefully merge it, because it will create conflicts with all existing PR.

Good point, I'll make sure to put this in after dealing with the existing PRs.

 Would probably result in nicer PRs if put it in before this one is merged.

Going to merge this now it looks like PR activity has subsided. Merging formatting changes now prevents them from polluting future diffs due to lint-staged, I found that annoying when I was working on other stuff.


query template cannot find its data

I'm trying to insert a line into a table. Because there is base64 encoded data in the request and that I have to decode it server side, I wrote a SQL query to insert a line into the table. I tested the query directly with psql, it works. Then I moved the query to queries/data/upload.write.sql and changed it include my 3 fields:
INSERT INTO data(meta, texte, audio) VALUES
    ({{.meta}}, '{{.texte}}', decode({{.audio}}, 'base64'));

Then I ran this:
curl -XPOST "http://localhost:3000/_QUERIES/data/upload" \
    -d '{"meta":1, "texte":"pouet pouet", "audio":"W29iamVjdCBCbG9iXQ=="}'

And here is the result:
"error": "could not execute sql pq: syntax error at or near \"\u003c\",
INSERT INTO data(meta, texte, audio) VALUES\n
 (\u003cno value\u003e, '\u003cno value\u003e', decode(\u003cno value\u003e, 'base64'));"

Obviously the template engine didn't find the data to insert and replaced it with . What did I do wrong ?

@tgirod  scripts support params only in query string.
curl -XPOST "http://localhost:3000/_QUERIES/data/upload?meta=1&texte=pouet%20pouet&audio=W29iamVjdCBCbG9iXQ==

This should work.

@tgirod  i will close but feel free to reopen if don't work

Just for confirmation i make some tests to prove what i said yesterday and i am correct.
The Queries feature works only with query string as you can see here https://github.com/felipeweb/prest-issue-299
I am ok with it.

OK. I guess it doesn't work with multipart/form-data either ?

multi/form-data not rest pattern (correct me if I'm wrong)

@avelino From my limited knowledge of the subject, I don't see why multipart/form-data wouldn't be a REST pattern - it is stateless and it transmits data through a POST request.
Someone asked the same question here
From the answers, it looks like multipart/form-data is the canonical way to send binary data to a REST endpoint.

https://github.com/prest/template/pull/4


OperationalInsights@2019-09-01: add status code 201 for QueryPacks_CreateOrUpdate

ARM (Control Plane) API Specification Update Pull Request

[!TIP]
Overwhelmed by all this guidance? See the Getting help section at the bottom of this PR description.

[!NOTE]
As of January 2024 there is no PR assignee. This is expected. See https://aka.ms/azsdk/pr-arm-review.

PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

 Click here to see the details of Step 1
Breaking changes review (Diagram Step 1)
If the automation determines you have breaking changes, i.e. Step 1 from the diagram applies to you,
you must follow the breaking changes process.
IMPORTANT This applies even if:

The tool fails while it shouldn't, e.g. due to runtime exception, or incorrect detection of breaking changes.
You believe there is no need for you to request breaking change approval, for any reason.
Such claims must be reviewed, and the process is the same.

 Click here to see the details of Step 2
ARM API changes review (Diagram Step 2)

For help with Step 2 / "ARM review", see https://aka.ms/azsdk/pr-arm-review.

 Click here to see the diagram footnotes
Diagram footnotes
[1] See ARM review queue (for PR merge queues, see [2]).
[2] public repo merge queue, private repo merge queue (for ARM review queue, [1])
The ARM reviewer on-call engineer visits the merge queue twice a day, so the approximate ETA for merges is 12 - 24 hours.

Purpose of this PR
What's the purpose of this PR? Check the specific option that applies. This is mandatory!

[ ] New resource provider.
[ ] New API version for an existing resource provider. (If API spec is not defined in TypeSpec, the PR should have been generated using OpenAPI Hub).
[ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix OpenAPI spec quality issues in S360.
[ ] Other, please clarify:

fixes #19603

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[x] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[x] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to the diagram Step 2, "ARM API changes review", for this PR.

Additional information

 Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
suppressions guide to get approval.

Getting help

First, please carefully read through this PR description, from top to bottom. Please fill out the Purpose of this PR and Due diligence checklist.
To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
For help with PR workflow diagram Step 2 (ARM review), see https://aka.ms/azsdk/pr-arm-review.
If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

@wuxu92 any chance we can reopen this one?

Labels were wrong: this PR is not ready for ARM review yet. Always use this comment for guidance: it is updated as your PR moves through the process: https://github.com/Azure/azure-rest-api-specs/pull/27524#issuecomment-1909426449.


Concise specification for on-Ledger transaction display

Explain how transaction fields should be shown on Ledger.

This can be found here: https://github.com/cosmos/ledger-cosmos/blob/master/UISPEC.md


Test that an error is raised when staging to the elements build directory

In GitLab by @tristanvb on Oct 22, 2017, 08:57
I had to fix the recent branch staging sources to the build directory.
The branch was checking this for every source, which is wrong, it needs to check once while staging all sources of an element to a non-empty element build directory (top level only), this is fixed.
After this I found that:

A test was added to tests/sources, poking around in Source APIs, please never do this, that directory has to go away, and tests should be only using the CLI fixtures, with only a few exceptions, for instance for testing the _yaml module
Deadcode was added to buildstream, files were added to tests/frontend/project, which look like they may be useful for testing this exact thing - however, these added files were unused. Please never add deadcode.

In GitLab by @tlater on Oct 23, 2017, 15:40
created branch 124-test-that-an-error-is-raised-when-staging-to-the-elements-build-directory

In GitLab by @tlater on Oct 23, 2017, 15:59
mentioned in commit af3a8e5874777923ecccfabd4502a3862ddc7f99

In GitLab by @tlater on Oct 23, 2017, 16:12
mentioned in commit 35d7051a07451e0647f60963d0485908b1d0139a

In GitLab by @tlater on Oct 23, 2017, 16:25
mentioned in merge request !121

In GitLab by @tristanvb on Oct 24, 2017, 08:04
closed via merge request !121


messages+messagesgui app/library

https://github.com/espruino/BangleApps/blob/55e52be7ddec9830b8b1a364e464958b2b7fa3f6/apps/messages/README.md?plain=1#L38-L51
This statement seems in contradiction with how its implemented.  No event is emitted and:
Bangle.load((msg && msg.new && msg.id!=="music") ? "messagegui.new.js" : "messagegui.app.js"); is ran.
So I assume one could write their own gui by installing an app on top of messagesgui that uses the same file name messagegui.app.js in their app? Or is the event emitting work in progress?

Yes, that doesn't seem quite right. I believe you can do require("messages").openGUI() and that's fine.
But Bangle.load(...."messagegui.app.js"); is called from messagegui's lib.js listener fn, which is called from boot.js with Bangle.on("message", (type, msg) => require("messagegui").listener(type, msg));
So actually the only difference is it should say Bangle.on("message", (type, msg) and not Bangle.on("messageGUI", message=>{  ?

https://github.com/espruino/BangleApps/blob/55e52be7ddec9830b8b1a364e464958b2b7fa3f6/apps/messages/lib.js#L109-L120C3
This comment says emit but no emit eg.

But Bangle.load(...."messagegui.app.js"); is called from messagegui's lib.js listener fn
Ah, right I didn't see this pathway.  Do the pathways sit well with each other, hm.

The only thing I don't understand is where is the openGUI event emitted from?

There is no openGUI event? It's just the method... Where did it say there was a openGUI event?

Here:
https://github.com/espruino/BangleApps/blob/55e52be7ddec9830b8b1a364e464958b2b7fa3f6/apps/messages/lib.js#L110
Thats the part that had me confused, hm.

Ok - yes, it looks like when everything was split into apps the docs weren't updated. I've updated the docs.
Basically if writing a gui you provide your own messagegui lib which has a open method that gets called


[REF-1464] Handle requirements.txt encoding

All Submissions:

[x] Have you followed the guidelines stated in CONTRIBUTING.md file?
[x] Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

New Feature Submission:

[x] Does your submission pass the tests?
[x] Have you linted your code locally prior to submission?

Changes To Core Features:

[ ] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your core changes, as applicable?
[ ] Have you successfully ran tests with your changes locally?

After these steps, you're ready to open a pull request.

Previously the handling of requirements.txt in user local environment assumes utf-8 the default encoding used in builtin open method. If user has a different encoding, we open and append will cause the file to have inconsistent encoding on different lines and make part of the content unreadable using any given encoding
Add UT to cover the non utf-8 files.
For hosting: now that we re-check/re-generate requirements.txt when user deploys, it is safe to simply pin the reflex version instead of >=.

@masenf thumbs up one more time please

sorry, thought this already merged

No worries. I think our branch is set up to dismiss approvals if there's any new changes. I needed to merge upstream


New Feature: Add support for changing SkyQ stb settings

Would it be possible to add functionality to enable changes to the settings on the SkyQ box?  It's be really useful to change 'Picture resolution', 'Digital audio output HDMI', bluetooth pairing, etc as part of a 1 click automation and without interrupting viewing.
I'm not sure if the end points are known for any of these?  If not I was thinking you could do a certain amount with button push combos as the menu layout seems to be static for all STBs.  It would be a bit clunky and only really works for settings that are an on/off toggle, but would be a start.
If you think there may be undiscovered endpoints for the settings that and no-one's already looked then let me know and I'm happy to go on the hunt.  Hopefully not a wild goose chase!
Otherwise just to say this is great work here that benefits many and is much appreciated 👍

Thanks for thinking of this. I'm not aware of any other api endpoints, though I'm happy for you to do some research and I'll try to incorporate if I can. To be honest, what I would anticipate to be available are endpoints that achieve what the remote can do (send button presses) and what any official Sky App can do. I son't use the apps a lot, but I don't think they can configure settings.
You can create your own command sequence using the (custom) sources options, which will play back the configured set of buttons - https://github.com/RogerSelwyn/Home_Assistant_SkyQ_MediaPlayer#sources
You could the potentially write an automation to run the sequence whenever it sees the SkyQ box change to a certain channel. However, of course this would interrupt viewing since it is just imitating what you would do with the remote.

Closing, since no known api.

Yeah soz Rog, I got nowhere with this one.


The :imagesdir: directive is not supported

I have an asciidoc document with images located in a subdirectory images, where all images are stored. It seems that AsciidocFX doesn't support the directive  :imagesdir: and shows broken links in place of images. Here's a fragment of my document (the image fig_1_01.png is not dislayed):
:imagesdir: ./
== Chapter 1. Writing Your First Program
Some text goes here
=== Installing Java on Your Computer
To start programming in Java you need to download Java Development Kit (JDK)
[[FIG1-1]]
.Selecting the Java SE version
image::images/fig_1_01.png[]

Hi,
Here are the cases that shows how imagesdir works with the mentioned image dirs:
:imagesdir: images/

image::subimages/image0001.png[]
image::image001.png[]

or you could use as follows:
:imagesdir: .

image::images/subimages/image0001.png[]
image::images/image001.png[]

or
:imagesdir:

image::images/subimages/image0001.png[]
image::images/image001.png[]

Thank you!
:imagesdir: . fixed the issue in my file.


Adding cluster type in getworkflowrun api

Signed-off-by: Raj Babu Das<EMAIL_ADDRESS>
Proposed changes
Adding cluster type in getworkflowrun API response
Types of changes
What types of changes does your code introduce to Litmus? Put an x in the boxes that apply

[ ] New feature (non-breaking change which adds functionality)
[x] Bugfix (non-breaking change which fixes an issue)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices applies)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[x] I have read the CONTRIBUTING doc
[x] I have signed the commit for DCO to be passed.
[ ] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ ] I have added necessary documentation (if appropriate)

Dependency

Please add the links to the dependent PR need to be merged before this (if any).

Special notes for your reviewer:

Merging this PR. Tested it locally.


Error using /database/import endpoint

I am exporting and then importing back the Examples database by using Superset's REST API.
I am able to use the same process to export/import back charts, dashboards and datasets, but not databases.
Expected results
Expect to be able to import back.
Actual results
{"message":{"datasets/examples/video_game_sales.yaml":{"params":["Not a valid mapping type."]}}}

How to reproduce the bug

curl  --cookie "session=<cookie>" https://<hostname>/api/v1/database/export/?q=[1] --output database_1.yaml
curl -X 'POST' --cookie "session=<cookie>"  'https://<hostname>/api/v1/database/import/'   -H 'accept: application/json'   -H 'Content-Type: multipart/form-data'   -F<EMAIL_ADDRESS>-F'overwrite=true'  --header "X-CSRFToken: <csrf_token>" -H "Referer: https://<hostname>/api/v1/database/import/"

Environment
(please complete the following information):

superset version: 1.0.0
python version: 3.8

Checklist
Make sure to follow these steps before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
[x] I have reproduced the issue with at least the latest released version of superset.
[x] I have checked the issue tracker for the same issue and I haven't found one similar.

Closing as I have not had this issue with non-examples databases.


Virtualenv/py version backport

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Yeah, well. That wasn't ready to submit and was submitted against the wrong branch. Sorry for the noise.


Bug: When TRACKER_SCRIPT_NAME is set, the tracking code still refers to script.js

When TRACKER_SCRIPT_NAME is set, the tracking code in the Edit panel still shows sript.js (see image below).
However, this is just a cosmetic issue, but unexpected nevertheless. The tracking script is also available at the expected location, after you've fixed the envvars in v2.0.1.

Nice, thank you


Cannot find module 'mongodb'

@gadicc, Really excited for this.
Currently using Meteor 1.3-modules-beta.7, as suggested, and I follow the instructions, yet I get this error when I run meteor.

Any suggestions? Thanks!

Our first issue :)
It's a really strange error, but now that we're in atmosphere, you can delete all the hot-related stuff from your app's packages dir and just run it like a regular package.  I think it will fix this too.  Let me know.

Or have you already done that?  I might be a bit rusty with Meteor error messages, but the 3rd line from the bottom made suspect it's in your app.
I'm assuming you've never experienced this with other packages that require things from npm?

No, I am only referencing the ecmascript-hot package as described in the README. Here are my Meteor packages and package.json file in case that helps. https://gist.github.com/tgoldenberg/57c68e47380e9c4c8f8b

This is a super strange one... I'm requiring mongodb exactly according to the package spec, and in the same way that the package is getting meteor-babel, etc.  And assuming of course that the regular ecmascript is working.  I also can't reproduce on my end.
The only thing I can really think of is I noticed that your package.json (thanks for sending!) also includes mongodb and maybe the version constraint is blocking it from using a lower version.  Can you try change it from 2.1.7 to 2.1.4 and see if this makes any difference?  If this is the issue and you need 2.1.7, I'll bump our deps on the next release.
What OS are you on?

Hi @gadicc, unfortunately same error with that change. I might try taking a stab with a fresh project, but it's too bad because our current project could really use it.

So I got it running fine on a new project, but can't figure out how to integrate it with the old. At least it works though :)

Also, maybe a separate issue, but I find that it doesn't reload when I add HTML to my react component. For example, if I add another  tag within my render function, it doesn't appear at all.

Haha, ok, thanks.  Is there any chance you can share a repo if your current code base, so I can try figure this out?

Hi @gadicc, it's for a company so I unfortunately cannot. I'm going to try to take apart the pieces and see if I can debug it a bit today, so I'll report back my findings

@gadicc Finally got this going, but hard to pinpoint what went wrong. Maybe because I tried to do the configuration pre-atmosphere? Could also be that the particular branch I was on didn't have the right Meteor version. Anyhow, very grateful for this!

Thanks for reporting back!  Glad you're up and running, hopefully the issue won't recur.  It's possible from using the pre-atmosphere versions, but I think unlikely... it's a case that Meteor should have figured out, not really sure where things were going wrong.  Let me know if this happens again, and happy you're enjoying :)


⚠ Bofrost.de has degraded performance

In e5a40c6, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1205 ms

Resolved: Bofrost.de performance has improved in 493f051 after 1 hour, 13 minutes.


Add the beginnings of lib.sys.traits.

This starts lib.sys.traits with just Unconst, Unshared, and Unqual. Both the tests and documentation have been beefed up from what std.traits has, with many more examples provided to try to make the exact semantics crystal clear, whereas the previous explanation in the docs was pretty sparse. The actual semantics are the same though.
This isn't what I would have chosen for traits first, but they're used heavily in unit tests, so they got moved up in the list.
I used version (StdDdoc) for the ddoc version, since that's what we do normally in Phobos. I don't know if we'll want or need to change that later given than Phobos v3 technically isn't Std, but all that really matters there is that it's a specific version identifier that we use for building Phobos' documentation and which no other projects would use. And since Phobos v3 currently is set up to build with dub rather than Phobos' normal build system, I expect that the documentation generation is completely busted right now anyway. So, that'll need to get fixed at some point.

__traits
core.internal.traits
lib.sys.traits

Why does the language need 3 different ways to consume traits?
What problem do these modules even solve?

__traits is just what's built into the language and does not cover everything that we need for the standard library or all of the kind of traits that your average D program needs. And most of what's in __traits can't be done without the compiler's help, whereas there are plenty of other useful traits that can be built on top of those or built using templates - which is the kind of thing that std.traits - and thus lib.sys.traits - does.
core.internal.traits is not intended for public consumption. As the module name indicates, it's internal to druntime, and no programs should be using it directly. It has a limited number of traits that were needed by druntime, most of which came from std.traits, but druntime couldn't use those, because Phobos depends on druntime, but druntime does not depend on Phobos. Unconst and Unqual in std.traits - and here - alias the versions from std.traits simply because it avoids duplication. They were actually defined in Phobos first.

To me this module is unnecessarily, just like the meta one
It is forcing phobos 3 to embrace template soup all over again, solutions to imaginary problems
What code in phobos 3 justify to Unshared!T something?
Perhaps design phobos 3 modules so nobody wants to want to Unshared!T something

I expect that you're very much in the minority with such an opinion. While not everything in std.traits and std.meta has been valuable, and some of it definitely needs to be reworked, as a whole, they have been invaluable for template constraints and metaprogramming in general. Phobos v2 uses them heavily as does a lot of code in the wild. The problems that they solve are not at all imaginary. They are critical for working with templates. And yes, Phobos v3 is going to use templates very heavily, because that's one of D's strengths, and it's required for ranges and the generic algorithms that use them. And that means using traits to test types and other templates like you find in std.meta to manipulate AliasSeqs where appropriate.

Besides, phobos 3 now depend on internal druntime code, that's not good, phobos 3 should be an opportunity to have a better foundation that is future proof

Phobos has always depended on druntime. That's nothing new. And since it's the standard library, it shouldn't be at all surprising if it takes advantage of that connection where it makes sense. In this case, for a number of years now, Unconst and Unqual have shared their implementations with druntime's core.internal.traits just to save code and reduce the number of template instantiations in the cases where anyone uses those traits on types that druntime uses them on (and the number of types that they get used on may actually be increasing with the work that's being done to templatize the druntime hooks for stuff like array operations). We could copy that code into Phobos and not depend on that code in druntime, but it would just mean code duplication and additional template instantiations. And the fact that those symbols are being aliased in Phobos is irrelevant to the actual API. If we ever do decide that we don't want Phobos to be aliasing Unqual and Unconst from core.internal.traits, we can change Phobos without breaking anything. But having those aliases has not caused problems.

To me this module is unnecessarily, just like the meta one

meta is nessery if only for seq and staticmap; template hell algorithms are different from range algorithms and are going to be narrower, uglier and worse to work with, and depended on by alghorthims (zip/chain/choose needs tuple and seq)

@ryuukk @crazymonkyyy I think I should clarify something at this point. V3 was never intended to be a ground-up redesign of Phobos. The primary purpose of this project is to remove auto-decoding, and the natural downstream consequence is that a bunch of range stuff is going to be redesigned, but in this case what you're really asking for is language changes, not library changes. These two modules are used by almost every other module, so getting them ported over is required to proceed with anything higher-level. V3 development will not be waiting on the language changes that would be required to not have these the features that these modules provide to the rest of the library.


COPYING is a license file, shouldn't need to be treated as additional legal file

In short, I don't think Licensed::Dependency::LEGAL_FILES needs to match anything that Licensee::ProjectFiles::LicenseFile::FILENAME_REGEXES matches because licensee will provide texts matching the latter.
I don't think this will cause any behavior change.

JRuby is a good edge case, I suppose:
https://github.com/jruby/jruby/blob/master/COPYING

It's not really an edge case, there are lots of projects that include third party info in their LICENSE, COPYING, or similar files. Licensed will get the contents of those files from Licensee whether the latter can positively detect a license from them or not, thus Licensed doesn't need to collect them, thus this PR. 😄


[SPARK-11583] [Core]MapStatus Using RoaringBitmap More Properly

test cases

1.1 sparse case: for each task 10 blocks contains data, others dont
sc.makeRDD(1 to 40950, 4095).groupBy(x=>x).top(5)
1.2 dense case: for each task most block contains data, few dont
1.2.1 full
sc.makeRDD(1 to 16769025, 4095).groupBy(x=>x).top(5)
1.2.2 very dense: about 95 empty blocks
sc.makeRDD(1 to 16380000, 4095).groupBy(x=>x).top(5)
1.3 test tool
jmap -dump:format=b,file=heap.bin
1.4 test branches: branch-1.5, master

memory usage

2.1 RoaringBitmap--sparse
Class Name                                                                        | Objects | Shallow Heap | Retained Heap
org.apache.spark.scheduler.HighlyCompressedMapStatus| 4,095    |  131,040          | >= 34,135,920
my explaination: 4095 * short[4095-10] =4095 * 16 * 4085 / 8 ≈ 34,135,920
2.2.1 RoaringBitmap--full
org.apache.spark.scheduler.HighlyCompressedMapStatus| 4,095     | 131,040           | >= 360,360
my explaination：RoaringBitmap(0)
2.2.2 RoaringBitmap--very dense
org.apache.spark.scheduler.HighlyCompressedMapStatus| 4,095      | 131,040           | >= 1,441,440
my explaination：4095 * short[95] = 4095 * 16 * 95 / 8 = 778, 050 (+ others = 1441440)
2.3 BitSet--sparse
org.apache.spark.scheduler.HighlyCompressedMapStatus| 4,095       | 131,040          | >= 2,391,480
my explaination：4095 * 4095 =16,769,025 + (others = 2,391,480)
2.4 BitSet--full
org.apache.spark.scheduler.HighlyCompressedMapStatus| 4,095       | 131,040         | >= 2,391,480
my explaination：same as the above

conclusion
memory usage:( Retained Heap)
RoaringBitmap--full < RoaringBitmap--very dense < BitSet---full = BitSet--sparse < RoaringBitmap--sparse

In this specific case, for  RoaringBitmap--sparse, use RoaringBitmap to mark non-empty blocks instead of empty ones. In this way, memory usage can stay low in all cases.
This is better than my former pr ---- https://github.com/apache/spark/pull/9559

Can one of the admins verify this patch?

The problem is that this brings back roaringbitmap, and one of the reasons we removed it was, separately, that kryo wasn't serializing it properly. You'd have to address that too.
What is your conclusion though? why is this better than your alternative/

@srowen There is still some analysis left to be done, but I think there is a growing case that roaring is actually the right way to go, with some minor tweaks.  So I'll do more reviewing here, assuming the analysis works out the way we expect, but this is still pending some final details from the discussion on the jira.

test cases
sparse case: 4085 empty ---    S
sc.makeRDD(1 to 40950, 4095).groupBy(x=>x).top(5)

dense case: 95 empty   ----    D
sc.makeRDD(1 to 16380000, 4095).groupBy(x=>x).top(5)

test result
1 without runOptimize or  trim

case
Class Name
Objects
Shallow Heap
Retained Heap

S
HighlyCompressedMapStatus
4,095
131,040
>= 34,135,920

D
HighlyCompressedMapStatus
4,095
131,040
>= 1,441,440

2 with runOptimze and trim

case
Class Name
Objects
Shallow Heap
Retained Heap

S
HighlyCompressedMapStatus
4,095
131,040
>= 687,960

D
HighlyCompressedMapStatus
4,095
131,040
>= 687,960

3 with runOptinize and trim, also isSparse

case
Class Name
Objects
Shallow Heap
Retained Heap

S
HighlyCompressedMapStatus
4,095
131,040
>= 687,960

D
HighlyCompressedMapStatus
4,095
131,040
>= 687,960

conclusion

with runOptimize,  roaing saves a lot, especially in  S
after runOptimize,  separate sparse case is not necessary

For my questions in my last comment
continuous
scala> import org.roaringbitmap._
import org.roaringbitmap._

scala> val r = new RoaringBitmap()
r: org.roaringbitmap.RoaringBitmap = {}

scala> for (i <- 1 to 2000000)  r.add(i)

scala> r.runOptimize
res1: Boolean = true

scala> r.contains(2000000)
res2: Boolean = true

bits
original size
opitimized size

200K
33,056
328

2M
255,472
1,768

uncontinuous
scala> import org.roaringbitmap._
import org.roaringbitmap._

scala> val r = new RoaringBitmap()
r: org.roaringbitmap.RoaringBitmap = {}

scala> for (i <- 1 to 2000000) if(i%10==0) r.add(i)

scala> r.runOptimize
res1: Boolean = false

scala> r.trim

scala> r.contains(2000000)
res2: Boolean = true

bits
original size
opitimized size
trimed size

200K
255,472
255,472
254,072

2M
2,516,688
2,516,688
2,516,272

conclusion
In uncontinuous, runOptimize failed and the size after r.trim neither became smaller as  in continuous
I guess my separating sparse case idea is still necessary for roaringbitmap to add lesser bits incase of runOptimize failure

@yaooqinn thanks for the additional analysis, but I would really prefer this is written up and attached to the jira for reference -- extended discussions in the PR are harder to keep track of later on.  Also include the code and the additional cases, in particular the worst case analysis, from alternating bits.  If you would prefer that someone else puts those details together (maybe me or @lemire), we can help, but it would be great if you could go that last extra mile :)
I don't think the analysis of the "uncontinuous" case and whether to track empty / non-empty blocks is correct.  I wouldn't say that runOptimize "fails" in any of the cases -- there are sometimes when it doesn't decrease the size, but that doesn't mean that another approach would make the size significatnly smaller.  The key question is: does the memory usage change if we track empty or non-empty blocks?  The main case we should be worried about is when the bitset is very full.  We might be better off inverting all the bits.  Here's a test for that by taking a bitset of 200k elements, where all the elements are set except for every 1000th element.  We compare the size (a) after adding all elements (b) after calling runOptimize (c) after flipping the bits.  Also, just to be extra-sure, I also compare by directly adding the bits in their flipped state, just to make sure that there isn't a size difference from calling flip at the end:
import org.roaringbitmap._
val n = 2e5.toInt
val orig = new RoaringBitmap()
val flip1 = new RoaringBitmap()
(0 until n).foreach { idx =>
   if (idx % 1000 == 0) {
     flip1.add(idx)
   } else {
     orig.add(idx)
   }
}
val flip2 = orig.clone()
flip2.flip(0,n)
val origOptimized = orig.clone()
origOptimized.runOptimize()

scala> orig.getSizeInBytes
res1: Int = 31374

scala> flip1.getSizeInBytes
res2: Int = 432

scala> flip2.getSizeInBytes
res4: Int = 432

scala> origOptimized.getSizeInBytes
res6: Int = 844

This shows that the flipped version is a bit better than the the version w/ just runOptimize (as daniel had indicated in earlier comments).  However, the difference is pretty small.  IMO thats not worth worrying much about.
There are two other reasons we might want to use the flipped bits: (1) memory used while building the bitset and (2) time taken to build the bitset.  However, both of those reasons are much less important.  And in either case, we could only improve things in all cases by making two loops through the array -- one to count the non-zeros and decide which version is sparser, and then a second pass to create the bitset.  This is way less important, so I wouldn't really worry about it now.  Certainly if we just stick to always tracking the empty blocks its no worse than what we have now or what was in 1.5 and before.

@yaooqinn This PR looks good to me overall. Since your patch basically revert #9243, I'd like to revert that first, then rebase this pr, to make the history more clear. We still need some changes to address SPARK-11016.
@srowen Does this sound good to you?
@lemire @squito @srowen Thanks to your all valuable discussion.
cc @rxin @JoshRosen

SGTM.

@davies,
I've worked on the issue SPARK-11016.
The fix is available in this PR: https://github.com/yaooqinn/spark/pull/1

@davies I don't necessarily think we have to revert first. The original change did fix SPARK-11016 and, eh, sort of fixed one type of issue described by SPARK-11271 but really introduced a different related potential problem. This is a further improvement that happens to walk back part of the change.
Keep in mind the original PR is in branch 1.6 so I want to make sure we leave 1.6 in a good state if it's going to be cut soon.
This PR does not include a fix for SPARK-11016. Adding it would technically cause that problem to occur again. I think it would have to include some of the changes proposed by Charles Allen in the JIRA. Not hard.

@srowen If we are going to use RoaringBitmap, then #9243 does not really fix SPARK-11016. It introduce a performance regression for common workload (when only a few blocks are empty, the memory consumption of MapStatue is bigger than before), so I don't think branch 1.6 is in healthy state now.
Since SPARK-11016 is a bug fix, it's still OK to fix even 1.6 preview is published.

It did fix the problem, but only by making it a moot point. If roaring bitmap comes back then it just needs a real fix. I'm not arguing against this change of course just making sure it doesn't take a step backwards for two forwards.

I talked to @davies offline just now -- he thinks it'd be more clear for the diff of this patch if it doesn't need to include the "revert" of the last patch. I don't have a super strong opinion, but given he is willing to spend the time, I'd say he should make the call since nobody else feels super strongly about it.

And I think we should merge it into 1.6 too. I was busy when we were reviewing the previous patch but I did have some concerns about changing the data structure of a fundamental part of the shuffle code path. And that's why I never explicitly LGTMed the old patch.

Sound good on all counts as long as the serialization fix can be worked in too.

@yaooqinn #9243 is reverted, could you rebase this one master, hopefully we will merge it soon (also into 1.6).

$ git push https://github.com/yaooqinn/spark.git mapstatus-roaring:test
Counting objects: 5581, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1765/1765), done.
Writing objects: 100% (3957/3957), 908.68 KiB | 0 bytes/s, done.
Total 3957 (delta 1701), reused 3171 (delta 1016)
efrror: RPC failed; result=22, HTTP code = 403
atal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
Everything up-to-date
after git rebase, i can't push it to my own repo, failed with 403
while my backup of this branch before still working?
Is there something wrong with master branch?

@yaooqinn I rebased and didnot have any problem, could you try<EMAIL_ADDRESS>?

@yaooqinn I had fix the conflict for you (you still got the credit when merging it), also upgrade RoaringBitmap to 0.5.11.

@davies thanks.

@yaooqinn can you close this now that we have #9746?


fix bug samsung s8+

I have to try to fix this issues
https://github.com/YummyLau/PanelSwitchHelper/issues/144

thx for your pr


Failed to load template

Failed to load template bootstrap/match.tpl.html, bootstrap/choices.tpl.html, bootstrap/select.tpl.html when using reference and referenced_list. Dependencies of ui-select?
ng-admin version: 0.8.0 installed from npm

which version of ng-admin are you using: ng-admin-only.js or ng-admin.js?
Also, this doesn't seem to happen on the master branch, try installing from npm but using marmelab/ng-admin to get the HEAD.

Same issue here, using "ng-admin.min.css" and "ng-admin.min.js" found here: https://github.com/marmelab/ng-admin/tree/master/build.
Related code:
 var user = nga.entity('user');
 user.listView().fields([
    nga.field('id'),
    nga.field('username'),
    nga.field('email')
]);
 user.editionView().fields([
    nga.field('username'),
    nga.field('email'),
    nga.field('user_permissions', 'reference_many')
        .targetEntity(permission)
        .targetField(nga.field('name'))
 ]);
 admin.addEntity(user);

@fzaninotto , using<EMAIL_ADDRESS>marmelab/ng-admin), same code example with @TheBlusky .

@fzaninotto how can I build a non-minimized version?

I rebuilt the css and js files and pushed them to master. Can you try again?
@hbprotoss you can't. Use the sourcemaps instead.

Bump. Any updates?

No news for several months, closing.

Just had the same issue.
In case it helps: the problem was because I had set up a HTTP request interceptor.
Here is a quick workaround from my code :
 .factory('httpInterceptor', [ 'qnConf', function(qnConf) {
                var httpInterceptor = {
                    request: function(config) {
                        if(config.url.indexOf('.html') < 0) { // <-- prevent updating URL when requesting .html files
                            config.url = qnConf.url+'?'+config.params.operation;
                            delete config.params.operation;
                        }
                        return config;
                    },
                    response: function(response) {
                        if(typeof response.data.message == 'undefined') {
                            response.data.message = '';
                        }
                        return response;
                    }
                };
                return httpInterceptor;
            }])


Add a Gitter chat badge to README.md

DifferentiableUniverseInitiative/jax_cosmo now has a Chat Room on Gitter
@EiffL has just created a chat room. You can visit it here: https://gitter.im/DifferentiableUniverseInitiative/jax_cosmo.
This pull-request adds this badge to your README.md:

If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

Let's go!


feat: enable fx pools

displays Xave's fxpools on pool list
displays fx pool data and stats
links to Xave's interface to add/remove liquidity

Some data is still not correctly displayed, like the pool swap fee apr, pool owner, and pool attribute immutability. Using the swap interface also does not route through these pools.
However, these inaccuracies where also present in the frontend-v2 UI.

resolves #973

@0xnook what's the state here? Would be great to get this working. When I first checked I think something was required on the API side first?

hi @0xnook @garethfuller Xave team member here. If there's anything we can help with let me know.
Thanks for opening this PR!

I think it should be FX instead of Fx right?

I think so. Added the label


[New Tail Request: Kenmore Air]

Add new icon(s) to the library

Please fill out the table below with the desired information per icon. This information must be included with the icon request before it can be accepted.

see the table below for a list of available categories

category
description

alert
Icons used specifically to alert users as to the state of awareness

in-flight
icons reserved for 'day of travel' user experiences

interface
Icons used to create interface enhancements

payment
Icons specifically to be used in a transaction flow

shop
icons for use with shopping experiences

social
Icons for use with social media

terminal
Icons related to terminal experiences

partnership
Icons related to business partnerships and Alaska Airlines

logo
Logo SVGs, e.g. tails or company logos

Name: What is the name of the icon? Please update the name of the file to match. PLEASE reference other similar icons for a naming convention.
Category: What category of icons will this new icon be listed?
Title: An OPTIONAL description of the icon. ††
Description: Alternative text of the icon that will be read back to assistive technologies.

†† The title is only needed if there is text that you wish to appear in the native browser tooltip functionality. See an example with the external link icon in the interface section.

name
category
title
description

[KenmoreAir]
[Tail]
[Kenmore Air Tail]
[White airplane tail with a yellow and grey swoosh ]

Icon art

The icon(s) per this request, has the new art been completed?

[x] Yes
[ ] No

If YES, does the new icon art follow all the recommendations from the Auro Icon Design Guidelines?

[x] Yes
[ ] No

If YES, the icon(s) in the request, have they been added to the UiKit Abstract repo?

[x] Yes
[ ] No

Attach exported SVG files to this issue
DO NOT upload the icon(s) directly. ALWAYS compress the file(s) and upload the .zip file.
See the options below for attaching a file to this request.
KenmoreAir_final.svg.zip

Updated version with black stripe per Maggie's request


Running multiple instances of ShinyProxy has led to the unresponsiveness or failure of ShinyProxy Application Proxies.

ShinyProxy version - 3.0.1/3.0.2
For the purpose of replicating the problem locally, I've set up a configuration similar to the one provided at containerized Kubernetes config,utilizing WSL/Minikube.
While operating a single ShinyProxy instance, everything appears to be functioning adequately. However, upon expanding the ShinyProxy instances to two or more, I consistently encounter the following issues:

The message "This app has been stopped, you can now close this tab."
JSON data: {"status":"fail", "data":"app_stopped_or_non_existent"}
Sluggish rendering of Shiny Proxy apps, often requiring multiple attempts.

I am utilizing the IP from the minikube service  to access ShinyProxy.
Our infrastructure is primarily composed of Azure Kubernetes Services, the Nginx Ingress Controller, custom ShinyProxy code, and other components.
I would appreciate it if you could assist me with this issue, as I have no intention of transitioning to or utilizing the shinyproxy-operator.

Hi
You don't need the operator for running multiple proxies, however, you'll have to run ShinyProxy in High Availability mode. This requires a Redis server, which is used for persisting session information and also for communicating between the replicas.
So basically the steps are:

deploy a Redis server (or use one provided by a cloud service)
configure ShinyProxy to use "session persistence" and "app persistence" https://shinyproxy.io/documentation/configuration/#session-and-app-persistence

Although you don't want to use the operator, you can have a look at the examples in the operator repo, especially these parts: https://github.com/openanalytics/shinyproxy-operator/blob/master/docs/deployment/overlays/1-namespaced/resources/shinyproxy.shinyproxy.yaml#L11-L22
The operator does not do anything with Redis, so you can use the same config actually.

I appreciate your reply. I inadvertently omitted to mention that I am utilizing Redis along with ShinyProxy session persistence, but not App persistence. I have also activated Application recovery. I will now attempt the approach involving Application persistence.

I followed the recommended approach, and it worked wonderfully. I'm pleased to report that my problem has been resolved. Thank you so much for your prompt assistance.


Step over not working in v0.9.9

Step over is working totally fine in v0.9.8
https://github.com/EpicGamesExt/raddebugger/assets/29519295/e2439575-44e1-4ff0-ad07-a663bdf2e6ff
While in v0.9.9, the step over doesn't seem to be working, sometimes the target is stuck and status in raddbg says running.
https://github.com/EpicGamesExt/raddebugger/assets/29519295/7556dd5b-b071-4e67-8ccc-c4fdd991f916

Tried deleting all the RADDBGI files, no luck.

It's not the for loop.
It's doing this in my hobby project and not able to repro it on work project.
Here is the project in case: https://github.com/MohitSethi99/ArcGameEngine
Branch: dx12
Run: scripts/GenerateSolution.bat
PS: It will require dotnet 7 components from VS installer

Yes, always failing at the same location.

Since this is a hobby project, can you just send me a build? I am running into issues almost ASAP getting it set up, and I think it would just be faster to have a prebuilt binary I can test with locally.

Definitely, will send it as soon as I reach home. (1~2hrs)

Here you go: https://drive.google.com/file/d/1lCLeUtlpSvuQvlkNH1gyEkV4jfPE2d4j/view?usp=sharing
Exe: Arc-Editor.exe
Upon launch it will ask for project to open, simply clicking cancel should open the editor with no project loaded.
A good place to check out is while loop in ArcEngine::Application::Run()

Thanks! I'll take a look.

Turns out remedybg is behaving the same for "step into" and not taking to the correct location. I'll investigate further and will update here.

The issue is happening when I'm calling a function in a dll. specifically nethost.dll I use to enable scripting backend in my engine. It allows to interop with C#. More details here: https://learn.microsoft.com/en-us/dotnet/core/tutorials/netcore-hosting
Will create a small example by the weekend.

Here is the visual studio solution for the minimal reproduction: https://drive.google.com/file/d/1rFZjYVJ2vn9ABg3JkbCZ7uOAJgL4RElt/view?usp=sharing
Zip is bloated cos of dotnet files. output directory has "App.runtimeconfig.json" config file and ".dotnet" folder which contains the dotnet framework that will be loaded by the code in Main.cpp

According to Visual Studio output, an exception is thrown (presumably in dotnet library) when Line 39 is executed. Rest of the code execution is fine in VS and remedy. But raddbg's step_over and step_into breaks after Line 40.

Issue is observed after this particular exception:
Exception thrown at 0x00007FFE0252543C (KernelBase.dll) in Arc-Editor.exe: 0x04242420 (parameters: 0x0000000031415927, 0x00007FFD20B90000, 0x0000000C5270E670).

Also noticed that this is written to the RadDebugger's output view:
onecore\vm\dv\storage\plan9\rdr\dll\util.cpp(99)\p9np.dll!00007FF9B890F0CC: (caller: 00007FF9B89093B0) LogHr(1) tid(1bf8) C0000034     Msg:[?????????????????????????????????????????????????????/??????????????????????????????????????????????)]

Sorry, I've been a bit swamped with other stuff. Can you somehow resend that minimal repro to me? Google removed it due to "terms of service violations"... sigh...

I hope onedrive doesn't remove it.
https://1drv.ms/u/s!ApjsIRdEL5T4kxab7q9YlvPVLww_?e=n4jIby

I think #106 is related, not sure though.

I think probably not, but not clear... Just fixed that one again in 489ae56223210c3ceefe39a899c34f8b4c9ffb54, digging into this one now.

Well, that took way longer than I expected, and it took way longer than it deserved! This was being caused by an incredibly stupid state machine bug I introduced during a pass over the Demon layer between 0.9.8 and 0.9.9, and it was difficult to locate. This should be fixed in 435877824a7fcc56ad1d860c700d85685a7e944a.

Thanks a lot! It works now :)


backport of linkage fix to v1.21

https://github.com/golang-fips/go/pull/190 fixed a linkage issue in RH builds.
This PR aims to backport that fix to the go1.21-fips-release branch

@derekparker @dbenoit17 Please review and merge.

Sure.

@dbenoit17 Done.

@dbenoit17 I used 1.21.7 as go version. Could that be the reason the tests are failing ?

The test failure is due to an unrelated change in OpenSSL, just fixed in #195.  If you could rebase on top of the latest changes, we should see a green test run.  Thanks!

If you could rebase on top of the latest changes, we should see a green test run

Done


"Gerbil Error: Unsupported k" for k = 8

I'm getting the error Gerbil Error: Unsupported k for a k of 8, however the manual states that "Supported k currently range from 8 to 136". Is a kmer size of 8 not supported after all? Had no problem running with sizes of 12, 16, etc.

Could you please give us the exact command you tried to run?

Ok, I guess I see what's wrong. I updated the code. This bug shoud be fixed now.

Thanks @srechner !


Provider foreman controller cleanup

Rubocop -a
Whitespace and alignment
Other minor cleanup and deduplication

@jrafanie @Fryguy Please review

Might need to be darga/yes if backporting future things built on top of this becomes a pain, but we'll keep it darga/no for now.  cc @chessbyte

🎉


Add some values for each babysitter

Could you happen to add some values for each babysitter:

Payment method (options are check, cash, paypal)
Name on Check (text)
Pay rate (numerical value)

And all of these would be options that I as the admin would set.
Also, could we just display the number of hours for that month? Or is it easier to do per week?
If you can't do a financial timesheet, the most would be to just have

take number of hours total for that month and multiply by pay rate. display total amount to pay

could we differentiate between hired and nonhired sitters...only the hired would have a pay rate.


Visit the project on GitHub link is not linking to GitHub repo

Describe the bug
Visit the project on GitHub should take you to GitHub repo and not GitHub pages
To Reproduce
Steps to reproduce the behavior:
Click on visit the project on GitHub link
Expected behavior
Go to GitHub repo

@Akash-Nayak PTAL


/users/history header X-Pagination-Item-Count does not match the actual number of results

When calling https://api.trakt.tv/users/minas1/history/movies?page=1&limit=10000, the value of the response header X-Pagination-Item-Count does not match the actual number of returned items.
At the current time of testing, the value of the header was 593 but only 583 items exist in the array.
Happens with and without the Authorization header.

I'm late in responding to this, but I looked into this and the issue is this user watched some movies that were later classified as adult and and thus skipped from the result set. The header count isn't taking that into account and I need to see if there is a way to do that without a performance hit.

Thanks for taking the time to look into it.
That's an interesting issue. Keep us updated. :)


⚠ Turnos 24x7 has degraded performance

In 86aefe4, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 1067 ms

Resolved: Turnos 24x7 performance has improved in a339ecf.


Commit list wrong user name

I have a list of commits in my newly created GIT Repo.
In the list of commits, it shows me the system / pc name in place of the user who committed the code.

Hello,
i guess this issue is fixed by now since there was no further feedback provided.
For everyone else having this issue, check the configured username on your computer.
To do that open a terminal/cmd and use the following commands to check your username:
git config --global user.name
This will show you your global configured username.
git config user.name
This will show you any username that is configured for the repository itself.
Your username might be setup incorrectly.


registry_key resource doesn't recognize keys when passed as attribute

Description
The registry_key resource doesn't recognize the keys in the option values hash when passed as an attribute.
Chef Version
13.9.4
Platform Version
Windows 2016
Replication Case
Create an attribute with an array of hashes and then loop through it in code. Below is my ruby example.
attributes > default.rb
default['iis_wrap']['registry'] = [
  {
    key: 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NetFramework\v4.0.30319',
    values: [{ name: 'SchUseStrongCrypto', type: :dword, data: 1 }]
  }
]

recipe > default.rb
node['iis_wrap']['registry'].each do |reg|
  registry_key reg['key'] do
    key reg['key']
    values reg['values']
    recursive true
  end
end

The error I'm hitting is located here. I believe its related to the check looking for a symbol key called name but what gets passed is a string key called name. Both should work interchangably. I've verified what was getting passed by raising an exception and passing in the message the value of the values key with inspect and got output as shown:
RuntimeError
------------
TEST1 = [{"name"=>"SchUseStrongCrypto", "type"=>:dword, "data"=>1}]

Client Output
-----> Starting Kitchen (v1.21.2)
-----> Converging <default-windows-2016>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 7.0.2...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb

       ModuleType Version    Name                                ExportedCommands
       ---------- -------    ----                                ----------------
       Script     0.0        Omnitruck                           {Get-ProjectMetadata, Install-Project, install}
       chef installation detected
       install_strategy set to 'once'
       Nothing to install
       Transferring files to <default-windows-2016>
       Starting Chef Client, version 13.9.4
       resolving cookbooks for run list: ["pl_iis_wrap::default"]
       Synchronizing Cookbooks:
         - iis_wrap (0.1.0)
         - iis (7.0.0)
         - windows (4.3.1)
       Installing Cookbook Gems:
       Compiling Cookbooks...

       ================================================================================
       Recipe Compile Error in C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/cookbooks/iis_wrap/recipes/default.rb
       ================================================================================

       ArgumentError
       -------------
       Bad key name in RegistryKey values hash

       System Info:
       ------------
       chef_version=13.9.4
       platform=windows
       platform_version=10.0.14393
       ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x64-mingw32]
       program_name=C:/opscode/chef/bin/chef-client
       executable=C:/opscode/chef/bin/chef-client

       Running handlers:
       [2018-06-12T18:50:00+00:00] ERROR: Running exception handlers
       [2018-06-12T18:50:00+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2018-06-12T18:50:00+00:00] ERROR: Exception handlers complete
       [2018-06-12T18:50:00+00:00] ERROR: Exception handlers complete
       Chef Client failed. 0 resources updated in 04 seconds
       [2018-06-12T18:50:00+00:00] FATAL: Stacktrace dumped to C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/chef-stacktrace.out
       [2018-06-12T18:50:00+00:00] FATAL: Stacktrace dumped to C:/Users/vagrant/AppData/Local/Temp/kitchen/cache/chef-stacktrace.out
       [2018-06-12T18:50:00+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-06-12T18:50:00+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
       [2018-06-12T18:50:00+00:00] FATAL: ArgumentError: Bad key name in RegistryKey values hash
       [2018-06-12T18:50:00+00:00] FATAL: ArgumentError: Bad key name in RegistryKey values hash
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-windows-2016>.  Please see .kitchen/logs/default-windows-2016.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

you should probably just reg['key'].to_sym that

the reg['key' line works fine it's the second part reg['values'] that is where it breaks. the values option takes an array of hashes but inside expects symbol keys :name, :type, :data but when the key values are passed to it they are passed as "name" => "string" etc...
This line in code here https://github.com/chef/chef/blob/515d1683a6549da75905ff33ef3e2858d6d10fa7/lib/chef/resource/registry_key.rb#L87 is then raising an error unless [:name, :type, :data].include?(key) when checking the values option hashes for those specific keys.


Compatibility with Room v 1.1.1 ?

With this error:
constructor RoomOpenHelper in class RoomOpenHelper cannot be applied to given types;
required: DatabaseConfiguration,Delegate,String
found: DatabaseConfiguration,,String,String
reason: actual and formal argument lists differ in length
and by this initialization:
RoomAsset.databaseBuilder(context!!.applicationContext,                             MyDatabase::class.java, "appname.v4.db")                             .fallbackToDestructiveMigration()                             .build()
I think there might be a difference between the constructor in Room and the one this lib is using.
Thanks for the great work.

Works great, if set up correctly, which is not easy, as Room is quite complicated starting out.
Maybe follow this tutorial https://www.youtube.com/watch?v=ARpn-1FPNE4 and read the instructions on how to implement  RoomAsset!
And you know, that you must pass an actual class, right? Like this
RoomAsset.databaseBuilder(context!!.applicationContext, MyDatabase.class, "appname.v4.db") .fallbackToDestructiveMigration() .build()
And the Class musst be a Room Database (obviously).

Also you probably don't want to use .fallbackToDestructiveMigration() as it drops all rows, which is exactly not what you want fora prepopulated database, this should be closed.

HEllo there,
same issue found
Failed to resolve: com.github.humazed:RoomAsset:1.0.3 with Room v 1.1.1,
how can i resolve ??

Hi,
Room now supports using a pre-packaged database out of the box, since version 2.2.0
Check this answer https://stackoverflow.com/a/50383879/4146943


Issues using remote_env_bucket/file

Getting errors trying to use a s3 stored json file for environment variables.
[DEBUG]	2016-07-11T22:52:37.788Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Making request for OperationModel(name=GetObject) (verify_ssl=True) with params: {'body': '', 'url': u'https://s3-us-west-2.amazonaws.com/REDACTED/test.envs', 'headers': {'User-Agent': 'Boto3/1.3.1 Python/2.7.10 Linux/4.1.19-24.31.amzn1.x86_64 Botocore/1.4.35 Resource'}, 'context': {'client_region': 'us-west-2', 'signing': {'bucket': 'REDACTED'}, 'has_streaming_input': False, 'client_config': <botocore.config.Config object at 0x7f3a3d9dfc50>}, 'query_string': {}, 'url_path': u'/REDACTED/test.envs', 'method': u'GET'}
[DEBUG]	2016-07-11T22:52:37.788Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Event request-created.s3.GetObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f3a3d9dfb90>>
[DEBUG]	2016-07-11T22:52:37.788Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Event before-sign.s3.GetObject: calling handler <function fix_s3_host at 0x7f3a3e802ed8>
[DEBUG]	2016-07-11T22:52:37.789Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Checking for DNS compatible bucket for: https://s3-us-west-2.amazonaws.com/REDACTED/test.envs
[DEBUG]	2016-07-11T22:52:37.789Z	2926daa1-47ba-11e6-b896-69dad9b829bf	URI updated to: https://REDACTED.s3.amazonaws.com/test.envs
[DEBUG]	2016-07-11T22:52:37.789Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Calculating signature using hmacv1 auth.
[DEBUG]	2016-07-11T22:52:37.789Z	2926daa1-47ba-11e6-b896-69dad9b829bf	HTTP request method: GET
[DEBUG]	2016-07-11T22:52:37.789Z	2926daa1-47ba-11e6-b896-69dad9b829bf	StringToSign: GET   Mon, 11 Jul 2016 22:52:37 GMT x-amz-security-token:FQoDYXdzEDgaDJhqCi2nNztqi4HF0iLYAqspBYW526RPshtH7Jwsy/G0E7BYFpsoQIaGUsseG4s8ll+cJA9/cohhqegFeT9HSDFkv6v0mjvQ/9W247nSGY4UKg1N2IuRafEg5HXLzZPfKrNnkOcMTbdxiinZcALpN5Yb5VUOWSV+6TLugWyaG3ReKNnkKcuY2/UWr16dOeU/O8VpCi22UBtYyM5JRcg07d20TKPzimP7OVbw7LHGmTibrGlpgocuo33njavHs1UhaLbBFseQeP+ZUp74yExAA/acLFi16VHEV6xaPsOcQU6MCKfm9NEY4+HcSffeBPVhugFYMRnEfBjOZ+DJPV50m6oJVxkRa5FF7tscGm9OdbLo5woqIXPBFZ7izoDZO5HxGxKVv5sMi2SzWbli0TPVI5lqfXoMnFO3mntuXU8rXqdh99E82Ye6E2aRJy7Vbjm6RhUzub5xRTY4TIHmkrmukh2PcwyAl97lKPvBkLwF /REDACTED/test.envs
[DEBUG]	2016-07-11T22:52:37.791Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Sending http request: <PreparedRequest [GET]>
[INFO]	2016-07-11T22:52:37.793Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Starting new HTTPS connection (1): REDACTED.s3.amazonaws.com
[DEBUG]	2016-07-11T22:52:37.809Z	2926daa1-47ba-11e6-b896-69dad9b829bf	ConnectionError received when sending HTTP request. Traceback (most recent call last): File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/endpoint.py", line 174, in _get_response File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/vendored/requests/sessions.py", line 573, in send File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/vendored/requests/adapters.py", line 431, in send SSLError: [Errno 2] No such file or directory
[DEBUG]	2016-07-11T22:52:37.809Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Event needs-retry.s3.GetObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f3a3dce7110>
[DEBUG]	2016-07-11T22:52:37.809Z	2926daa1-47ba-11e6-b896-69dad9b829bf	retry needed, retryable exception caught: [Errno 2] No such file or directory Traceback (most recent call last): File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/retryhandler.py", line 266, in _should_retry File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/retryhandler.py", line 314, in __call__ File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/retryhandler.py", line 223, in __call__ File "/private/var/folders/xg/bg1n3znx2mz971xns5f2sx_00000gn/T/pip-build-GwP4n5/botocore/botocore/retryhandler.py", line 356, in _check_caught_exception SSLError: [Errno 2] No such file or directory
[DEBUG]	2016-07-11T22:52:37.810Z	2926daa1-47ba-11e6-b896-69dad9b829bf	Retry needed, action of: 0.688243944645

Lambda role has full s3 access(default that zappa created).  Not sure the issue here. Can't find much on SSLError: [Errno 2] No such file or directory for boto. Most people have a similar issues from requests, suggest turning verify_ssl to false.. but thats obviously not a good fix.
Running on OSX. Thought maybe SSL issues might be related to that(maybe a lib is being built there and isn't compatible), tried to copy my project over to an Amazon Linux box. Ran into issue trying to build there.
(venv)[ec2-user@ip-10-0-100-168 chief_brody]$ zappa update test
Packaging project as zip...
[Errno 17] File exists: '/tmp/1468280512'
(venv)[ec2-user@ip-10-0-100-168 chief_brody]$ ls /tmp
1468280511<PHONE_NUMBER>
(venv)[ec2-user@ip-10-0-100-168 chief_brody]$ ls /tmp/146828051*
/tmp/1468280511:
app  billcom_vendor_ids.txt  bs4-0.0.1-py2.7.egg  config.py  config.pyc  htmlcov  __init__.py  manage.py  migrations  __pycache__  remove.sh  run.py  setup.py  test.envs  tests  zappa_settings.json

/tmp/1468280512:
base58.py                  docutils                  hjson                       lambda_packages                        pbr                     pyOpenSSL-16.0.0.dist-info       six.py                      wheel
base58.pyc                 easy_install.py           idna                        lambda_packages-0.5.0.dist-info        pbr-1.10.0.dist-info    python_dateutil-2.5.3.dist-info  six.pyc                     wheel-0.29.0.dist-info
boto3                      easy_install.pyc          idna-2.1.dist-info          _markerlib                             pip                     requestlogger                    slugify                     zappa
boto3-1.3.1.dist-info      enum                      ipaddress-1.0.16.dist-info  mock                                   pip-6.0.8.dist-info     requests                         tqdm                        zappa-0.19.3.dist-info
botocore                   enum34-1.1.6.dist-info    ipaddress.py                mock-2.0.0.dist-info                   pkg_resources           requests-2.10.0.dist-info        tqdm-4.7.6.dist-info
botocore-1.4.35.dist-info  funcsigs                  ipaddress.pyc               ndg                                    pyasn1                  setuptools                       unidecode
concurrent                 funcsigs-1.0.2.dist-info  jmespath                    ndg_httpsclient-0.4.1-py2.7-nspkg.pth  pyasn1-0.1.9.dist-info  setuptools-12.0.5.dist-info      werkzeug
dateutil                   futures-3.0.5.dist-info   jmespath-0.9.0.dist-info    OpenSSL                                pycparser               six-1.10.0.dist-info             Werkzeug-0.11.10.dist-info

Pinging @collingreen

I was able to make this work by changing the boto3.session.resource('s3') in the hander to a client resource for s3 where i could set ssl verify false. My variable loads now. Not a long term solution.
Issue 1: can't build on linux, there is another Issue for this as well.
Issue 2: when build on OSX SSL requests seem to be broken, possibly from OSX based libs on a linux box.. but can't really verify due to issue #1.

Weird. Can you post your zappa_settings, and anything else that'll help replicate?

Sure, pretty simple one.
{
    "test": {
        "aws_region": "us-west-2",
        "s3_bucket": "REDACTED",
        "app_function": "app.app",
        "parameter_depth": 4,
        "http_methods": ["GET", "POST"],
        "exclude": ["*.gz", "*.pem", "*.ipynb", "*.db"],
        "remote_env_bucket": "REDACTED",
        "remote_env_file": "test.envs",
        "use_precompiled_packages": true,
        "delete_zip": false
    }
}

Does the same thing occur if aws_region is us-east-1?

Created a test2 with us-east1, new bucket homed in that region. Same SSLError. Except from zappa tail test2:
[1468347622313] (u'Could not load remote settings file.', SSLError(SSLError(IOError(2, 'No such file or directory'),),))

Was able to replicate with a new deploy (however, have existing older code functioning correctly, even after updating today). Not sure what is going on quite yet, but something isn't happy when running on the lambda side.
Essentially the same code as the zappa handler runs locally, both from the python interpreter and from the flask app. For example:
>>> import boto3, json
>>> session= boto3.Session()
>>> s3 = session.resource('s3')
>>> obj = s3.Object(bucket_name, file_name).get()
>>> content = json.loads(obj['Body'].read().decode('utf-8'))
>>> print(content))

I can't make sense of this then. Any theories?

To make it ignore ssl I just modified the handler to a s3 client vs session. Not super helpful, but incase someone else is blocked and wants to just move forward.
s3 = boto_session.client('s3', verify=False)
    try:
        remote_env_object = s3.get_object(Bucket=remote_bucket, Key=remote_file)

Does your bucket have a '.' in it, or any other unusual characters? Do you have requests[security] in your virtualenv? What OSX version are you using?

I do have requests[security], no '.', has a few '-'.  OSX El Capitan.

Hm, I'm still on Yosemite. I'm not sure why that would matter, however. Can you try with a bucket that doesn't have the '-' in it?

Tried with no hyphens: all one word bucket, in us-east-1. Same result:
68449260565] (u'Could not load remote settings file.', SSLError(SSLError(IOError(2, 'No such file or directory'),),))

I feel like its gotta be something with the SSL libs being used form OSX on Amazon Linux. If i could build everything on linux I'd be happy to try that and see if it works, but that other bug prevents the zipping phase.

@Miserlou Found the issue. the File not found error is what led me to it. Zappa is not packaging the ca bundle in botocore/vendored/requests/cacert.pem  (or any other lib like boto/requests that uses a pem).  by manually adding that to the zip it works.  Looks like the default exclude that does *.pem applies to not just your project files, but all the files in the dependencies as well, breaking them.

!!!
Excellent information! Removed cacert.pem from excludes and changed 'pem' in the example.