HBASE-26176 Correct regex in hbase-personality.sh

This is a rework of the original commit to enable
Hadoop 3 profile for the HBASE-25853 branch.

This is a commit that we don't want to merge back to the base (branch-2) branch once HBASE-25853 is ready.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console in case of problems.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 36s
Docker mode activated.

-0 :warning:
yetus
0m  5s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 17s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

2m  4s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests

uname
Linux 899d77e9d963 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
50 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 37s
Docker mode activated.

-0 :warning:
yetus
0m  6s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

2m 11s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests

uname
Linux ec296ec8ce3d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
51 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m 53s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  8s
Maven dependency ordering for patch

+1 :green_heart:
shellcheck
0m  2s
There were no new shellcheck issues.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

_ Other Tests _

+0 :ok:
asflicense
0m  1s
ASF License check generated no output?

3m 42s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests
dupname asflicense shellcheck shelldocs

uname
Linux c75ac2cee549 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
47 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/1/console

versions
git=2.17.1 maven=3.6.3 shellcheck=0.4.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

How about branch-2? This regex will only accept branch-2.x?

How about branch-2? This regex will only accept branch-2.x?

The regex will accept branch-2 and branch-2 from now on. The previous regex was faulty and allowed branch-1 too.

How about branch-2? This regex will only accept branch-2.x?

The regex will accept branch-2 and branch-2<anything> from now on. The previous regex was faulty and allowed branch-1 too.

I'm not an expert of regex in bash but skimmed the script, we also have this
if [[ "${PATCH_BRANCH}" = branch-1* ]]; then
    yetus_info "Setting Hadoop 3 versions to test based on branch-1.x rules."
    hbase_hadoop3_versions=""

Is this also wrong?

But also this
elif [[ "${PATCH_BRANCH}" = branch-2.* ]]; then
    yetus_info "Setting Hadoop 2 versions to test based on branch-2.3+ rules."

I'm a bit confused now...
What does the dot actually mean here? It means the dot itself or any char?

I'll fix these regexes as well.

What does the dot actually mean here? It means the dot itself or any char?

The dot means any character.

Using = or =~ is handled differently. The first uses pattern matching the latter uses regex.
bash-5.1$ export PATCH_BRANCH=branch-1; if [[ "${PATCH_BRANCH}" = branch-1* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-1.2; if [[ "${PATCH_BRANCH}" = branch-1* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-2; if [[ "${PATCH_BRANCH}" = branch-1* ]]; then echo "true"; fi
bash-5.1$ export PATCH_BRANCH=branch-; if [[ "${PATCH_BRANCH}" = branch-1* ]]; then echo "true"; fi

bash-5.1$ export PATCH_BRANCH=branch-1; if [[ "${PATCH_BRANCH}" =~ branch-1* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-1.2; if [[ "${PATCH_BRANCH}" =~ branch-1* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-2; if [[ "${PATCH_BRANCH}" =~ branch-1* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-; if [[ "${PATCH_BRANCH}" =~ branch-1* ]]; then echo "true"; fi
true

bash-5.1$ export PATCH_BRANCH=branch-1; if [[ "${PATCH_BRANCH}" =~ branch-1.* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-1.2; if [[ "${PATCH_BRANCH}" =~ branch-1.* ]]; then echo "true"; fi
true
bash-5.1$ export PATCH_BRANCH=branch-2; if [[ "${PATCH_BRANCH}" =~ branch-1.* ]]; then echo "true"; fi
bash-5.1$ export PATCH_BRANCH=branch-; if [[ "${PATCH_BRANCH}" =~ branch-1.* ]]; then echo "true"; fi

Based on this I can simplify the code. :)

I've modified the PR for master. Please take a look at #3567. I'll update this PR for the feature branch once master is merged.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console in case of problems.

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console in case of problems.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 32s
Docker mode activated.

-0 :warning:
yetus
0m  7s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 15s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

1m 58s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests

uname
Linux 2e81acf280cd 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
52 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 33s
Docker mode activated.

-0 :warning:
yetus
0m  7s
Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck

_ Prechecks _

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

_ Other Tests _

2m  3s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests

uname
Linux 226d7b6d6fea 4.15.0-151-generic #157-Ubuntu SMP Fri Jul 9 23:07:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
51 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console

versions
git=2.17.1 maven=3.6.3

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
1m  8s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ HBASE-25853 Compile Tests _

+0 :ok:
mvndep
0m 16s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  6s
Maven dependency ordering for patch

+1 :green_heart:
shellcheck
0m  2s
There were no new shellcheck issues.

+1 :green_heart:
whitespace
0m  1s
The patch has no whitespace issues.

_ Other Tests _

+0 :ok:
asflicense
0m  1s
ASF License check generated no output?

2m 52s

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/3568

Optional Tests
dupname asflicense shellcheck shelldocs

uname
Linux c63871027497 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
HBASE-25853 / 4fbc4c29f2

Max. process+thread count
46 (vs. ulimit of 12500)

modules
C:  U:

Console output
https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3568/2/console

versions
git=2.17.1 maven=3.6.3 shellcheck=0.4.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Tactic mode "escapes" sections for sections within namespaces

Prerequisites

[X] Put an X between the brackets on this line if you have done all of the following:

Checked that your issue isn't already filed.

Specifically, check out the wishlist, open RFCs,
or feature requests.

Reduced the issue to a self-contained, reproducible test case.

Description
Consider the following code:
namespace foo
section
parameter x : ℕ

def my_x : ℕ := x

lemma bar : my_x = my_x + 0
  := nat.add_zero my_x

lemma bar' : my_x = my_x + 0
:= begin apply (nat.add_zero my_x) end --error on this line

end
end foo

Actual behavior: Definition of bar' fails with the error
type mismatch at application
  nat.add_zero my_x
term
  my_x
has type
  ℕ → ℕ
but is expected to have type
  ℕ
state:
x : ℕ
⊢ my_x = my_x + 0

What is happening is that inside the tactic mode, the environment is behaving as if it were outside the section, and thus my_x takes an argument corresponding to the parameter x. If one removes the lines namespace foo and end foo, so that the section is no longer within a namespace, then the definition of bar' works.
Expected behavior: The environment should behave similarly in tactic mode as outside of tactic mode. That is, it should behave as it does behave for sections that are not within namespaces.
Versions
Lean HEAD from July 21, 2017, Lean (version 3.2.1, commit 667e1e1, Release)

The problem is that the my_x alias for my_x x is stored in the parser right now.  A fix for this will probably have to wait until we implement #1674.


Direct Path API need to support Oracle native datatypes, not strings only

OCI_DirPathSetEntry by default is able to handle string values only; DATE, TIMESTAMP, NUMBER values must be passed as C-like strings only, with further conversion.
That is not true for general case, such conversion can lead to data loss issues and has some performance drabacks.
Moreover - string conversion does not allow to convert/pass some values:

DATE value, which is presented by all zeroed bytes in DUMP(), those can be found in PL/SQL Profiler tables);
some DATE values which is produced by JDBC the same way (JDBC client passes all zeroes in buffer, but some conversion still happens);
TIMESTAMP WITH TIMEZONE values with textual timezone info, like Europe/Moscow - those values will be silently converted into absolute offset, -03:00, losing textual part, it's known OCI limitation

Main goal is to be able to SELECT DUMP(x, 16) values from source database, covert them from textual hexadecimal back to native binary format and to pass to Direct path API calls AS-IS - that way it will be possible pass practically any value which is valid from source database point of view.
Taking values from OCI_Date and OCI_Timestamp is not an option - Oracle converts DATE values when passes them to client, and for TIMESTAMP values it's not possible to lookup into raw buffer - only handles are visible.
As result there is no need to extend list of existing OCILIB functions. We can extend existing ones next backward compatible way:
OCI_DirPathSetColumn() - if maxsize will be set to 0 and format to NULL - that can set mode to proceed with binary/no-conversion way for further OCI_DirPathSetEntry() calls;
for DATE, TIMESTAMP WITH TIME ZONE, TIMESTAMP WITH LOCAL TIME ZONE and TIMESTAMP the
dbcol->sqlcode should be set to SQLT_DAT, SQLT_TIMESTAMP_TZ, SQLT_TIMESTAMP_LTZ and... suddenly, 180, instead of SQLT_TIMESTAMP (I don't know why but it works that way only), respectively
where
OCI_DirPathSetEntry() - will take value as void pointer which we can point it to source buffer, which was built from source hexadecimal DUMP(x, 16) database function output.
Please note - for DATE values the buffer is practically always 7 byte structure
For TIMESTAMP values it can be either 11 or 7 byte structure, it depends on part which encodes fractional part of second. The same is true for TIMESTAMP WITH LOCAL TIME ZONE
For TIMESTAMP WITH TIMEZONE value size is always 13 byte structure, as I see.
For NUMBER values buffer size varies, up to 22 bytes in total.
But size of exact value is derived from database, so OCI_DirPathSetEntry() must always take into account user specified size argument

As most of the time, direct path is used with text files, its support was made using string in OCILIB.
You're right indeed. It is a nice enhancement.

Also I think it's worse to be able to handle BINARY_FLOAT, BINARY_DOUBLE the same way, though I have not faced with those types yet:
https://support.oracle.com/knowledge/Oracle Database Products/1315200_1.html
and INTERVAL types - those do work fine with current textual conversion at the moment, but taking into account that such conversion is potentially buggy - I'd copy them using DUMP()ed values as well.

Final findings for today:

While DATE, TIMESTAMP values can be copied  using
memcpy(data, value, (size_t) size);

the NUMBER raw values need to be copied using
memcpy(data, value, (size_t) size + 1);

due subsequent
    dpcol->lens[row-1]  = size;

Also I was not able to find any documented example how to handle the TIMESTAMP values for Direct Path API properly, in terms of the sqlcode values - do we need to specify SQLT_ ones or actual DUMP() Types, probably it worse to debug SQL*Loader calls - gbd is able to set breakpoint on OCIAttrSet()

Accodring to the next link
https://ellebaek.wordpress.com/2011/02/25/oracle-type-code-mappings/
for NUMBER and DATE the SQLT_NUM, SQLT_DAT and DUMP()ed type codes are the same (2 and 12, respectively),
while for timestamps those differ, and each type has own behavior:
for TIMESTAMP WITH LOCAL TIMEZONE only 232 (SQLT_TIMESTAMP_LTZ) - fits, DUMP() shows 231 as result, if we initially specify 231 then OCI gives AV error.
for TIMESTAMP WITH TIMEZONE both 181 and 188 (SQLT_TIMESTAMP_TZ) work fine, DUMP() shows 181 as result but
for TIMESTAMP only 180 fits, while 187 code, i.e. SQLT_TIMESTAMP gives conversion result.
DUMP() shows 180
So there is no unambiguous way how to handle sqlcode for the TIMESTAMPs, unfortunately.

Closing this issue (7 years old) as it won't be addressed in a near future.


refactor(bump): refactor AutoBump to support more use-case more cleanly

The current AutoBump construct has outgrown its original intent.
For example:

It defaults scheduleExpression to cron(0 12 * * ? *). While this was ok for delivlib itself, if forces us to use a magic value of disable in aws-cdk since we manually trigger bump builds.

It defaults bumpCommand to /bin/bash ./bump.sh. While this is mostly ok, when we added the merge-back PR creation, since it doesn't actually bump anything, if forced us to again introduce this magic disable value.

In addition, the code seemed rather cluttered and hard to follow. Hopefully this PR will rectify that.
Notable Changes
AutoPullRequest construct
Extract most of the logic from AutoBump to a generic AutoPullRequest construct that exposes all the necessary features to implement various scenarios, for example:

commands: Which commands to run on the head branch.

This means we can leave it blank for merge-back, and use /bin/bash ./bump.sh for the bump.

exports: Key value mapping for exporting env variables that will be accessible after the command have executed.

For example: VERSION: node -p "require('./scripts/get-version')" will make the VERSION env variable available and its value will be the output of the command.

The existing AutoBump and AutoMergeBack constructs simply use AutoPullRequest and provide some relevant defaults.
Monocdk version upgrade
Monocdk version was upgraded to 1.44.0. This required changing a bunch of imports, which is why there are so many files changed in the PR. I'm not a 100% sure I did the imports correctly, would appreciate a validation.
Merge Back Stage
The AutoMergeBack build can now be configured as part of the pipeline, so we won't need to manually trigger it, it will simply by triggered after the Publish stage:
releasePipeline.autoMergeBack({
  versionCommand: `node -p "require('./scripts/get-version')"`,
  labels: ['pr/no-squash'],
  base: {
    name: 'master'
  },
  head: {
    source: 'release'
  },
  stage: {
    name: 'MergeBack',
    after: 'Publish'
  }
})

Breakages
I've also taken the liberty to change some of the existing API's in favor of reducing code duplication and clarity. So this release will trigger a major version bump.
BREAKING CHANGE: AutoBump API has significantly changed. Refer to the docstrings for guidance.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: e57232bc8235ae56239dc41c8f39c82955f377fb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: f4e0e854e7e86b36118eab4c99b2dbe5c93f352e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 88f0ad884403cdb6092ddba2a319c642ada6a70b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 046d68f6c8acd32ef88a789e6396120663e3d30c
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 6cd6e66971e1975b2a8a732bbc7f7a5ee3065707
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 6f0137c5499a0c51ef5592f55ac3b826b4210050
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 475b3c798d2a995d039e11f34ef98df0039393b8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 4bf2acaa61e0aa93e071bf67332d0d4ae99909a8
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: d70211789fd562f328b58e224ac5599702964903
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

To reviewers. Im still working out the integration tests and the pipeline deployment, but would appreciate a look at the code in the meantime.
Thanks!

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: e9405d3e13b751e2abb7b8fcc712d000774e8117
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: cebd44cdbc8637757e973dabb8311132c5ecc604
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 5e7b1ef73ffba5d318e2e09b683c474291955c0a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 3c7f02183b79713df6fb5bc5aa3c07d6cf4896cc
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 12a974a7cbc3c8bddc4a74b87efdeaac9f515ba9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: 62fbc7a1618159da20e04daaf09c040f607b4606
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: a018a40d437c254d901c3d541591ad69ac3deb95
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: c8c18325213108a875159ce59689b72d873ec22d
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: GitHubPipelineAutoBuildProj-eNGjrh4O6eRH
Commit ID: efbc862e3ba1fedd5bf461d0ca86b18f924af839
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Deployment error about websocket

hi! I am trying to deploy this project in Ubuntu 22.04.
Firstly, I used 'ws://localhost:8000' as origin docker-compose file. But the websocket not connected. I found that in container the localhost can't access server outside.
Then I used ws://backend:8000, the server name to access.
The http request by http://backend:8000/ worked BUT the websocket by ws://backend:8000 is not connected.
here is my docker-compose file:
version: '3.8'

services:
  db:
    build:
      context: .
      dockerfile: Dockerfile.postgres
    restart: always
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      PGDATA: /var/lib/postgresql/data/futurepreppers
    ports:
      - '5432:5432'
    volumes:
      - db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5

  backend:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    hostname: backend
    depends_on:
      db:
        condition: service_healthy
    env_file:
      - ./.env
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/postgres
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/status"]
      interval: 10s
      timeout: 5s
      retries: 5

  web:
    build:
      context: ./client/next-web/
      dockerfile: Dockerfile
      args:
        NEXT_PUBLIC_FIREBASE_API_KEY: AIzaSyAVqhwbdB8I56HAMVVlgJKZcfrBkKI2AhQ
        NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN: assistly-kubernetes.firebaseapp.com
        NEXT_PUBLIC_FIREBASE_PROJECT_ID: assistly-kubernetes
        NEXT_PUBLIC_FIREBASE_APP_ID: 1:806733379891:web:48bf124c0d9b90298e6646
        REACT_APP_BUILD_NUMBER: 0.0.1
        NEXT_PUBLIC_API_HOST: http://backend:8000
        API_HOST: http://backend:8000
    ports:
      - "3000:3000"
    links:
      - backend
    depends_on:
      backend:
        condition: service_healthy
      db:
        condition: service_healthy
    environment:
      - NEXT_PUBLIC_FIREBASE_API_KEY=AIzaSyAVqhwbdB8I56HAMVVlgJKZcfrBkKI2AhQ
      - NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN=assistly-kubernetes.firebaseapp.com
      - NEXT_PUBLIC_FIREBASE_PROJECT_ID=assistly-kubernetes
      - NEXT_PUBLIC_FIREBASE_APP_ID=1:806733379891:web:48bf124c0d9b90298e6646
      - REACT_APP_BUILD_NUMBER=0.0.1
      - NEXT_PUBLIC_API_HOST=http://backend:8000
      - API_HOST=http://backend:8000

networks:
  default:
    driver: bridge

volumes:
  db:
    driver: local

My browser error:

It looks very strange.
Could you tell me how you deploy the project to connect websocket please? @pycui @y1guo @Shaunwei  What is the correct ws address?
Any info is helpful! Thanks for any rely. Sincerely.

I have chosen other docker network to deploy it. I could show my method here:
As mentioned above, my frontend docker container can't connect websocket with the backend docker container but the http connection does well.
Instead, I used host network on the frontend and backend container. And then I used nginx to proxy the websocket and the frontend page. The nginx config is following:

# 非强制重定向https
server {
    listen 443 ssl; #侦听443端口，用于SSL
    server_name YOUR_SERVERNAME;  # 自己的域名
    ssl_certificate /etc/nginx/cert/XXX.pem;
    ssl_certificate_key /etc/nginx/cert/XXX.key;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:HIGH:!aNULL:!MD5:!RC4:!DHE;

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_pass http://<IP_ADDRESS>:3000;
    }
    location /ws/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_pass http://<IP_ADDRESS>:8000;
    }
}

BUT I still think it is not elegant. I don't know why I could not access the bakend by websocket but I could access the backend by http when using the same host: http://backend:8000.
If someone could offer more info about deployment, I would pretty appreciate.


Licence text without horizontal scrolling

Gitea version (or commit ref): 1.1.0
Git version: not relevant
Operating system: not relevant
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[x] Not relevant

Log gist:

Description
Would it be possible to break line in the licence text view instead of scrolling horizontal? I know, its a small thing but i think without scrolling it is better readable.
Screenshots

It's just a matter how it gets formatted, that's just my opinion

Sure. I think word-wrap/word-break can do the trick

We could have a JS-powered checkbox addind a wordwrap-class to the text-box 🤔

I would prefer this as a global setting not per repo base

@xoxys Obviously 🙂

Fixed by #5558


Eggplant Runner Plugin

Repository URL
https://github.com/TestPlant/eggplant-runner-jenkins
New Repository Name
eggplant-runner-plugin
Description
This plugin provides the capability for a user to run Eggplant DAI test configurations from a Jenkins pipeline.
GitHub users to have commit permission
pohkheng2
Jenkins project users to have release permission
pohkheng2
Issue tracker
GitHub issues

/hosting re-check

/hosting re-check

/hosting re-check

/hosting re-check

/hosting re-check

/hosting re-check

cc @Wadeck

@timja superficial audit provided no interesting stuff to dig deeper, so 👍

Non-blocking suggestions for improvement:

You README has yaml format for the pipeline but it should be groovy
You should use the plugin bom so you don't need to manage versions for each dependency: http://github.com/jenkinsci/bom
You are missing a <url> it appears to be commented out here: https://github.com/TestPlant/eggplant-runner-jenkins/blob/main/pom.xml#L107-L114

/hosting host


Lines

Hello author, how to make the lines detected by the pose thicker

Hi! You can increase line thickness from 2 to 4 here.

Hope, it is helped


Improve handling of Self

Based on https://github.com/dtolnay/async-trait/blob/0.1.30/src/receiver.rs

bors r+


mon: create crush rule from osd class

osd class can distinguwish ssd from hdd now so that we can leverage the features.
Resolves: #4400
Signed-off-by: Ning Yao<EMAIL_ADDRESS>

I think the change could break backward compatibility since users couldn't have define the class key in the crush rule dict.
@guits thoughts ?

I think the change could break backward compatibility since users couldn't have define the class key in the crush rule dict.
@guits thoughts ?

so we can rely on the number of args.  4 args means previous without class.  5 args means current with class. In this way, we can solve compatibility issues?

@guits,  this will compatible with the previous no class configurations.

@alfredodeza , can you help to take a look, thx.

Could you rebase your PR, add a commit message (not just title) and add a 'Resolves: #4400' (because you opened an issue related to that PR) ?

@guits wondering if it's worth it to have two tasks instead of using a single one with condition statement
something like:
   command: "{{ container_exec_cmd }} ceph --cluster {{ cluster }} osd crush rule {{ 'create-replicated' if item.class is defined else 'create-simple' }} {{ item.name }} {{ item.root }} {{ item.type }} {{ item.class | default('') }}"

condition

good suggestion,

command: "{{ container_exec_cmd }} ceph --cluster {{ cluster }} osd crush rule {{ 'create-replicated' if item.class is defined else 'create-simple' }} {{ item.name }} {{ item.root }} {{ item.type }} {{ item.class | default('') }}"

create-simple cannnot pass more args, execute command bellow would be failed:
ceph osd crush rule create-simple SSD default rack ''
So I think we would directly use create-replicated to replace the create-simple. There would be the same if create-replicated does not pass item.class args. like below:
command: "{{ container_exec_cmd }} ceph --cluster {{ cluster }} osd crush rule create-replicated {{ item.name }} {{ item.root }} {{ item.type }} {{ item.class | default('') }}"

create-simple cannnot pass more args, execute command bellow would be failed:
ceph osd crush rule create-simple SSD default rack ''

Are you sure ? using default('') will create an empty string but the actual quotes won't be present so we shouldn't have the extra argument.

create-simple cannnot pass more args, execute command bellow would be failed:
ceph osd crush rule create-simple SSD default rack ''

Are you sure ? using default('') will create an empty string but the actual quotes won't be present so we shouldn't have the extra argument.

yeah, I have tried in my test environment

yeah, I have tried in my test environment

It works for me
TASK [ceph-mon : create configured crush rules] **********************************************************************************************************************************************************************************************
task path: crush/roles/ceph-mon/tasks/crush_rules.yml:14
Wednesday 04 September 2019  09:49:55 -0400 (0:00:00.030)       0:01:11.988 ***
ok: [crush] => (item={'name': 'HDD', 'root': 'default', 'type': 'host', 'default': False}) => changed=false
  ansible_loop_var: item
  cmd:
  - docker
  - exec
  - ceph-mon-crush
  - ceph
  - --cluster
  - ceph
  - osd
  - crush
  - rule
  - create-simple
  - HDD
  - default
  - host
  delta: '0:00:00.595676'
  end: '2019-09-04 13:49:56.084427'
  item:
    default: false
    name: HDD
    root: default
    type: host
  rc: 0
  start: '2019-09-04 13:49:55.488751'
  stderr: ''
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

Can you share logs about your failure ?

Can we reopen and merge this? It's a useful feature.

@mihaiplasoianu this has already be handled by https://github.com/ceph/ceph-ansible/pull/4703


Fix build for 4.3

For some reason, the test build is failing on Linux/release. This seems
to fix it. I'm not sure why.

which unit tests fail?

@ylow No tests fail; the build fails (for most C++ tests, seemingly) inside release/test. They all have the same linker errors, about missing symbols that seem unrelated to visualization (like cppipc), and the linker command appears to provide .a files that actually do contain those symbols, so it's a mystery to me why it's erroring.

I'll merge this now, and post-4.3 we can figure out the root cause and hopefully back out this hacky fix.


Create an npm leaderboard project with webpack

Create an npm leaderboard project with webpack

Hi Robert Loterh,
I installed eslint properly and tested it in my project. please check your review again.
you can see this picture to make sure:
https://www.dropbox.com/s/h79berac0ch6xp3/Screenshot 2023-01-31 at 08.29.38.png?dl=0

Hi Robert Loterh, I installed eslint properly and tested it in my project. please check your review again. you can see this picture to make sure: https://www.dropbox.com/s/h79berac0ch6xp3/Screenshot 2023-01-31 at 08.29.38.png?dl=0

In the process of responding I notice you've solved this issue. Congrats!
Cheers! 😃

hello @youmari ,
I have called you many times to explain where to add mit licence. I added it already in the readme.md file. please check this:
https://www.dropbox.com/s/b77ifj6eabty710/Screenshot 2023-01-31 at 09.51.09.png?dl=0
https://www.dropbox.com/s/cjfhnjeolvklnp1/Screenshot 2023-01-31 at 09.51.33.png?dl=0

Hi @MMGGYY66, what @youmari is explaining is that you should edit your MIT.md file and fill in your information. In the MIT.md file, replace [YOUR NAME] caption with your actual name and replace all captions of [APP TYPE] with the name of this repository. 👍

hello @youmari , I have called you many times to explain where to add mit licence. I added it already in the readme.md file. please check this: https://www.dropbox.com/s/b77ifj6eabty710/Screenshot 2023-01-31 at 09.51.09.png?dl=0
https://www.dropbox.com/s/cjfhnjeolvklnp1/Screenshot 2023-01-31 at 09.51.33.png?dl=0

@Gambit142 Thanks for explaining to @MMGGYY66  I was already off sorry for that @MMGGYY66


Error in the" Start DreamBooth" in Colab

Hi, first, thanks for your work. :-)
I am having the next error when reaching the "Start DreamBooth" step.
No code was changed, and it's the latest version.
/bin/bash: -c: line 0: syntax error near unexpected token `('
/bin/bash: -c: line 0: `accelerate launch /content/diffusers/examples/dreambooth/train_dreambooth.py    --pretrained_model_name_or_path=/content/gdrive/MyDrive/stable-diffusion-v1-4    --instance_data_dir=/content/data/ewtest04    --output_dir=/content/models/ewtest04    --instance_prompt="photo of ewtest04 building"   --seed=    --resolution=512    --mixed_precision="fp16"    --train_batch_size=1    --gradient_accumulation_steps=1    --use_8bit_adam    --learning_rate=5e-6    --lr_scheduler="constant"    --lr_warmup_steps=0    --max_train_steps=800  print("Almost done ...")'
cp: cannot stat '/content/models/': No such file or directory
Done !

I hope it helps.

was able to fix the 'unexpected token' by removing the trailing '' from
--max_train_steps=$Training_Steps \

hi
It was fixed, but you got the colab right before it updated.

fixed

fixed

Many thanks! I really appreciate your effort. 😄

happy to help


How to reinstall XVim

After making XVim and opening Xcode, I accidentally clicked the "Do not install the plugin" or whatever the alert said when Xcode opened. I tried running make uninstall on XVim, closing Xcode, making XVim, setting "Reset 'Do Not Ask Me Again'" warnings in Xcode, and reopening Xcode, but XVim does not install.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/27155589-how-to-reinstall-xvim?utm_campaign=plugin&utm_content=tracker%2F252770&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F252770&utm_medium=issues&utm_source=github).

You can fix this by editing Xcode's plist. You can see what you need to edit with:
defaults read com.apple.dt.Xcode | less

In here there are keys formatted as:
DVTPlugInManagerNonApplePlugIns-Xcode-[XCODE-VERSION]

You can just delete the key corresponding to your Xcode version and Xcode will ask you about all bundles again. For Xcode 7:
defaults delete com.apple.dt.Xcode DVTPlugInManagerNonApplePlugIns-Xcode-7.0

Awesome, thanks for the fast response!!


[package: @scalar/api-reference] In my example query parameter "locale_id" (Enum with integer values) not shown on the page "Test Request".

What happens?
[package<EMAIL_ADDRESS>In my example query parameter "locale_id" (Enum with integer values) not shown on the page "Test Request".
openapi.json
Visible in docs:

Invisible on the page "Test Request"

What did you expect to happen?
I expected to see in the "Test Request" query parameter "locale_id" like it was in<EMAIL_ADDRESS>
OpenAPI Document
https://sandbox.scalar.com/e/8V5Hx

It looks like I missed something in #3666, so I will create a PR to parse the parameter enum for integer and boolean types as well.

It still doesn't work for me in the current version<EMAIL_ADDRESS>And here too: https://sandbox.scalar.com/e/8V5Hx

It was released in<EMAIL_ADDRESS>(#3692). Could you recheck it? @alexxbar

It was released in<EMAIL_ADDRESS>(#3692). Could you recheck it? @alexxbar

Yes. It works on 1.25.55: https://cdn.jsdelivr.net/npm/@scalar/api-reference
But sometimes for some reason the old version (1.25.54) is downloaded.
Thank you!


Contents not rendered on android

For some reason, if you add a element with a border to a popup, the popup wont render the controls on android/ios. It renders fine on windows.
I will try and do a bit of digging tomorrow…
ie
<Label Text=“blah”/>
<Border><Entry/><Border>

I believe this is an issue in .net maui itself.
It will impact anyone who uses border based controls (i.e. some material style controls, or frameworks like uraniumui)
The solution is to not use frames and use a border instead for the outer container. I will verify later today.
https://github.com/dotnet/maui/issues/18526

So in the old xamarin version of mopups, if you were creating a custom popup you used a frame, which then contained your controls. however there is a bug in the frame control where if you use a border based control, all the controls disappear. The fix is to change the frame to a border, ie
<Border StrokeShape="RoundRectangle 5,5,5,5">
<PopupPage>
<StackLayout>
<**Frame**>
<StackLayout>
<Border>
<Entry/>
</Border>
</StackLayout>
</Frame>
</StackaLayout>
</PopupPage>

Hopefully this will help any other users with this issue, its an issue in maui, not mopups...

even after MAUI Fix it still has an issue in some cases the Android content not appear i don't have a clear scenario but it happening

@DeliriumCode  Frame is deprecated and should not  be used. It is removed in net9.


Does not work in some scenarios

Running on windows similar to:
warp-packer --exec E:\path\to\folder\file.exe --input_dir E:\path\to\folder --output G:\some\test.exe --arch windows-x64

Then running test.exe. Working locally, but copying on another machine getting:
Error: Os { code: 2, kind: NotFound, message: "The system cannot find the file specified." }

If I copy E:\path\to\folder to that same machine and run file.exe everything is working as expected.
The application being wrapped in a ,net core app. Note that I went through the steps with myapp.exe from readme and that simple application also worked fine when copied to the other machine. So it looks there are some more complex interactions at work here...
Is it possible to get some debug output somehow?

It prints the following:
2018-12-11 16:14:07 TRACE [warp_runner] self_path="C:\\Users\\asavinykh\\test.exe"
2018-12-11 16:14:07 TRACE [warp_runner] self_file_name="test.exe"
2018-12-11 16:14:07 TRACE [warp_runner] cache_path="C:\\Users\\asavinykh\\AppData\\Local\\warp\\packages\\test.exe"
2018-12-11 16:14:07 TRACE [warp_runner] target_exec="E:\\Sources\\DevOps\\HelperApps\\OctoVars\\bin\\Debug\\netcoreapp2.1\\win10-x64\\publish\\OctoVars.exe"
2018-12-11 16:14:07 TRACE [warp_runner] target_path="E:\\Sources\\DevOps\\HelperApps\\OctoVars\\bin\\Debug\\netcoreapp2.1\\win10-x64\\publish\\OctoVars.exe"
2018-12-11 16:14:07 TRACE [warp_runner] cache is up-to-date
2018-12-11 16:14:07 TRACE [warp_runner::executor] target="E:\\Sources\\DevOps\\HelperApps\\OctoVars\\bin\\Debug\\netcoreapp2.1\\win10-x64\\publish\\OctoVars.exe"
2018-12-11 16:14:07 TRACE [warp_runner::executor] args=[]
Error: Os { code: 2, kind: NotFound, message: "The system cannot find the file specified." }

Note that this box does not have disk E. Is it a factor?

Seems like it's getting confused with the drive letters. Can you try going to drive E:\ and running warp-packer from there using relative directories like the example?
E:\> cd \path\to\folder
E:\path\to\folder> warp-packer --arch windows-x64 --input_dir . --exec file.exe --output G:\some\test.exe

@dgiagio yep, that worked, thanks. Do you think it's worth fixing in code?

Unrelated to the issue question, am I guessing right, that if a program is looking for it's config file in the executable folder, it's unlikely to find it because the executable is in the cache folder, and the config in the wrapped program folder?


[Snyk] Fix for 2 vulnerabilities

Snyk has created this PR to fix 2 vulnerabilities in the yarn dependencies of this project.
Snyk changed the following file(s):

package.json

Note for zero-installs users
If you are using the Yarn feature zero-installs that was introduced in Yarn V2, note that this PR does not update the .yarn/cache/ directory meaning this code cannot be pulled and immediately developed on as one would expect for a zero-install project - you will need to run yarn to update the contents of the ./yarn/cache directory.
If you are not using zero-install you can ignore this as your flow should likely be unchanged.

⚠ Warning
Failed to update the yarn.lock, please update manually before merging.

Vulnerabilities that will be fixed with an upgrade:

Issue
Score

Improper Neutralization of Special Elements in Data Query Logic SNYK-JS-MONGOOSE-8446504
  721

Regular Expression Denial of Service (ReDoS) SNYK-JS-PATHTOREGEXP-8482416
  721

[!IMPORTANT]

Check the changes in this PR to ensure they won't cause issues with your project.
Max score is 1000. Note that the real score may have changed since the PR was raised.
This PR was automatically created by Snyk using the credentials of a real user.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Improper Neutralization of Special Elements in Data Query Logic
🦉 Regular Expression Denial of Service (ReDoS)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


About features/feature folder rules

Thanks for the great resources : )
I am designing an application based on this document, but I have a question.
What I want to ask you
project-structure.md

A feature folder could also contain other features (if used only within the parent feature) or be kept separated, it's a matter of preference.

For the definition of the bolded part

Specifics
Although it is defined to be used only within the parent feature,
features/comments is referenced in features/discussions, lib/authorization.tsx.
Also, features/users is referenced in features/comments, lib/authorization.tsx, and route/protected.tsx.
Aren't features that are imported as new features supposed to be used only by the parent feature that imported them? I wondered.
However, when I took a closer look at the source code, I found that the part I was wondering about had one thing in common.
It was that I had imported the Typescript types used in the functional folder.

Based on the above, I would like to ask you three questions.

Can I import the types defined in TypeScript, since they are not features themselves?
features/comment is used in features/discussions, but
Can a feature other than features/discussions use features/comment?
featureA <- featureB
featureC <- featureA
Is this kind of dependency possible?
I can't provide a specific situation where this is needed, but I would like to know if features can be used as grandchildren in features.

Hi there,
Q: "Can I import the types defined in TypeScript, since they are not features themselves?"
A: Yes you can, just make sure you are exporting the types from the index.ts of a given feature.
Q: "features/comment is used in features/discussions, but
Can a feature other than features/discussions use features/comment?"
A: It can if it needs to, however, since the application is simple, we don't need to do that.
Q: "featureA <- featureB
featureC <- featureA
Is this kind of dependency possible?
I can't provide a specific situation where this is needed, but I would like to know if features can be used as grandchildren in features."
A: It's difficult to tell without a concrete example, but it looks like this might cause the circular dependency issue. But that would also depend on what exactly you are importing from each feature.

Thank you for taking the time to reply!😄

A: Yes you can, just make sure you are exporting the types from the index.ts of a given feature.

I understood that the type is not an entity of the feature, so if it is exported in index.ts, it is not included in the constraints of the feature and can be imported and used from any module.

A: It can if it needs to, however, since the application is simple, we don't need to do that.

So featureA and featureB with different contexts can be used as a common feature and featureC as a child feature?
I'm sorry.
I wasn't asking the right questions to get what I wanted to hear out of you.
What I wanted to ask was.
A feature folder can contain other features (if they are only used within the parent feature).
Here's a bad example that applies here

It's difficult to tell without a concrete example, but it looks like this might cause the circular dependency issue. But that would also depend on what exactly you are importing from each feature.

I should have thought a little more before asking my question.
After asking the question, I thought about it for a while, but I couldn't find a suitable case.
Besides, if it is a simple feature, it may not be a big problem, but if the impact of the feature is spread too widely,
it may ruin the clean architecture.

And I have one more question, but it shouldn't be addressed in this issue, so I've created a separate issue
-> issue#64


Auto import of React components not working if no previous import of React module...

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.59.0
OS Version: Windows 10

Steps to Reproduce:

Start to write a new component (.tsx file)
Try to import a component inside your component, from another file

Details:
When using auto import of components, vscode do not display the import line unless I have imported React before (import React from "react";);
But, since React 17, importing react is no longer needed and so I don't see why we still need to have this import to make the auto import work.
Please refer to this article about the new JSX syntax: https://reactjs.org/blog/2020/09/22/introducing-the-new-jsx-transform.html
Note that I am using TypeScript, that's why my files are .tsx files instead of .jsx.

Does this reproduce in the latest VS Code insiders build too?

Yes, same behavior.


Fix link to Homebrew installation script

With the migration of the script to the omero-install repository, the
link on the documentation page needs to be updated

--rebased-to #1092

See https://github.com/ome/omero-install/pull/11

@manics: pushed. I will open a PR against omero-install to add a README.

Links is fine now, thanks.


bump package version

Also: fix format typo in contributors field.

Ah, sorry to forget to bump package version 🙇

No worries!


docs(cli): update command return message

I built some locally projects using this amazing collection of components and for each project I wanted to add all available components. I thought CLI didn't have this option. So I checked docs and didn't find something about it.
I believe cli page is not updated (specially add section). So this PR may help! :)

@adaremitchandra I wonder if a --all argument might be useful.

@adaremitchandra I wonder if a --all argument might be useful.

There should be an issue that requests the --all argument. I suppose it would be useful,  also for auto installation scripts.


Fix for error missing translation strings when building for android

Thanks @dtaalbers, can this still be merged?

Yes this can be merged


[WandbLogger] Call wandb.finish() to ensure all artifacts are uploaded before training ends.

Bug description
Currently WandbLogger doesn't explicitly call wandb.finish(), so it's possible that not all artifacts are synced to the server at the time train.fit() returns, especially when you have a large amount of artifacts to upload.
We know that wandb.init() spawns a new background process to log data to a run, and it asynchronously upload the logs and artifacts to the wandb server. The uploading will finish after 1-2 mins or more depending on your artifact size. It's necessary to explicitly call this barrier method to ensure all artifacts are uploaded before trainer.fit() returns.
https://github.com/Lightning-AI/lightning/blob/420eb6f248560c9c721601df791bf8b2f2214fe6/src/lightning/pytorch/loggers/wandb.py#LL566C1-L573C1
@rank_zero_only
def finalize(self, status: str) -> None:
    if status != "success":
        # Currently, checkpoints only get logged on success
        return
    # log checkpoints as artifacts
    if self._checkpoint_callback and self._experiment is not None:
        self._scan_and_log_checkpoints(self._checkpoint_callback)

    # Ensure that all artifacts get uploaded before trainer.fit() returns
    wandb.finish()

What version are you seeing the problem on?
master
How to reproduce the bug
Increase save_top_k, train a large model, and set WandbLogger(log_model=True).
Error messages and logs
# Error messages and logs here please

Environment

Current environment
#- Lightning Component (e.g. Trainer, LightningModule, LightningApp, LightningWork, LightningFlow):
#- PyTorch Lightning Version (e.g., 1.5.0):
#- Lightning App Version (e.g., 0.5.2):
#- PyTorch Version (e.g., 2.0):
#- Python version (e.g., 3.9):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):
#- Running environment of LightningApp (e.g. local, cloud):

More info
No response

A kind reminder:) @awaelchli

@woshiyyya I'm not sure we can make this decision for the user. finalize() gets called after every trainer stage, e.g.,
trainer.fit()  # finalize gets called
trainer.test()  # finalize gets called again

Lightning can't know in advance when the right time to call it is. And if I'm not mistaken, wandb.finish() should only be called once.

I'm not sure the common use-case, but generally speaking, "finalize" reminds me "it is called when object died".
I think it finalize() in general meaning should be called in the end of lifetime of the object.
If the current finalize() method is used for just saving checkpoint, the method name might have to be changed.
If the lifetime of the logger is same as Trainer's lifetime, wandb.finish() should be called when the Trainer ends?


Add dimod 0.11 to test matrix

Codecov Report

Merging #270 (a4d7efa) into master (6cbc54e) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #270   +/-   ##
=======================================
  Coverage   92.46%   92.46%
=======================================
  Files          17       17
  Lines        2071     2071
=======================================
  Hits         1915     1915
  Misses        156      156

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6cbc54e...a4d7efa. Read the comment docs.


Parameter is Undefined in the next Handler

Sample code:
route.get(
  "/:id",
  openApi({
    tags: ["meeting"],
    summary: "Get one meeting",
    request: {
      param: z.object({
        id: z.string(),
      }),
    },
    responses: {
      200: meetingZod,
    },
  }),
  async (c) => {
    let param = c.req.valid("param");
    console.log(param); // empty object {}
    let id = param.id;
    let res = await service.getOne(id);
    return c.json(res);
  }
);

param is undefined and the request cannot go through the zValidator too.

I think I need more details. Assuming route is new Hono(), this test passes just fine:
  it('test', async () => {
    const route = new Hono();
    route.get(
      '/:id',
      openApi({
        tags: ['meeting'],
        summary: 'Get one meeting',
        request: {
          param: z.object({
            id: z.string(),
          }),
        },
        responses: {
          200: z.object({ id: z.string() }),
        },
      }),
      async (c) => {
        const param = c.req.valid('param');
        const id = param.id;
        return c.json({ id }, 200);
      },
    );

    const res = await route.request('/123');
    const data = await res.json();

    expect(data).toEqual({ id: '123' });
  });

How are you calling your API? What is route in your example?

Yes, route = new Hono(). I found the issue now. it happens when I use other middleware before using openApi (cors, logger).
However, it won't happen when I use it with zodValidator.  You can check the code below.
import { Hono } from "hono";
import { openApi } from "hono-zod-openapi";
import { assertEquals } from "jsr:@std/assert";
import { z } from "./deps.ts";
import { cors } from "hono/cors";

Deno.test({
  name: "test openApi",
  fn: async () => {
    const route = new Hono();
    route.use("*", cors()); // not working for openApi when use other middleware before openApi
    route.get(
      "/:id",
      openApi({
        tags: ["meeting"],
        summary: "Get one meeting",
        request: {
          param: z.object({
            id: z.string(),
          }),
        },
        responses: {
          200: z.object({ id: z.string() }),
        },
      }),
      (c) => {
        const param = c.req.valid("param");
        console.log(param);
        const id = param.id;
        return c.json({ id }, 200);
      }
    );
    route.use("*", cors()); // Works fine for openApi
    const res = await route.request("/123");
    const data = await res.json();
    assertEquals(data, { id: "123" });
  },
});

Okay, I see. I will look into it, this certainly should not happen.

This problem is related to every middleware that openApi is using internally. I created an issue in hono's repo here: https://github.com/honojs/hono/issues/3657 as I can't see an easy way to fix it at the moment.

I see. Thank you for your great work.

Please upgrade the library to 0.5.0 and upgrade Hono to v4.6.10. The fix is included there.


Remove anime from recommendations

Hello,
It's it possible to get recommendations excluding Anime ? I have a lot of them in the results but I don't watch them so I don't want them in it 😀
Thanks

We don't currently have a way to exclude genres. I'll add this as a feature request.

Hello @rectifyer ,
Any update on this one :) ?


Add minimum lines for ::std::error::Error implementation

To compile below with current (1.47) rustc:
fn f<E: ::std::error::Error>(e: E) {}
fn main() {
    f(errnor::Errno(0));
}

Sorry, for typo in commit message too. Fixed by rebase: https://github.com/chamaken/rust-errno/commit/fbff9e4652bf2d95968fab71cf1f385b1856a457

Thanks!


BackupDatabase() should yield

Currently we call sqlite3_backup_step(backup, -1), but this will lock the database for the duration of the copy.
One of the main purposes of this API is to allow users to continue uninterrupted while a backup of the database is made.
To allow the backup to be made on a background thread while giving other threads opportunities to read, we should update it to step one page at a time and yield in between.
Unfortunately, this opens up the possibility of getting SQLITE_BUSY when stepping. This means we'll need to retry. The backup can even be automatically restarted. Which means it could take a very long time to complete. We'll need to think about timeouts and cancellation when we do this.

What we currently have is useful when you want the copy done as quickly as possible. This is useful on UWP to export the database, or to save/load an in-memory database.

Triage: We decided to keep the existing method as is (no yielding), but to add additional methods/overloads for timeout, cancellation, and progress that yield.


AMBARI-23974: Add hidden attribute to mpack module, stack services and cluster services

Change-Id: Ifb9a67ebaa1f4b9be82a8883f10ecc099f1b4875
What changes were proposed in this pull request?
Some client modules in an mpack are not gateway clients and hence should be hidden in the UI.
(Please fill in changes proposed in this fix)
How was this patch tested?
Verified hidden attribute is added in

[ ] /api/v1/mpacks/<mpack-id>

GET /api/v1/mpacks/1
{   "MpackInfo" : {     "id" : 1,     "modules" : [       {         "id" : "hadoop_clients",         "displayName" : "Hadoop Clients",         "description" : "Clients for HDFS, YARN and MAPREDUCE services",         "category" : "CLIENT",         "name" : "HADOOP_CLIENTS",         "version" : "<IP_ADDRESS>-b163",         "hidden" : true,...

[ ] /api/v1/stacks/<stack-name>/versions/<stack-version>/services/<service-name>

GET /api/v1/stacks/HDPCORE/versions/1.0.0-b395/services/HADOOP_CLIENTS
{   "StackServices" : {     "comments" : "Clients for HDFS, YARN and MAPREDUCE services",     "credential_store_enabled" : false,     "credential_store_required" : false,     "credential_store_supported" : false,     "custom_commands" : [ ],     "display_name" : "Hadoop Clients",     "hidden" : false,...

[ ] /api/v1/clusters/<cluster-name>/servicegroups/<servicegroup-name>/services/<service-name>

GET /api/v1/clusters/cl1/servicegroups/HDPCORE/services/HADOOP_CLIENTS
{   "ServiceInfo" : {     "cluster_id" : 2,     "cluster_name" : "cl1",     "credential_store_enabled" : "false",     "credential_store_supported" : "false",     "hidden" : "true",...
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2499/
Test FAILed.
Test FAILured.

Also, a number of unit tests seem to be broken by this change (in addition to MpackTest):
Tests run: 4787, Failures: 31, Errors: 217, Skipped: 91
->
Tests run: 4787, Failures: 32, Errors: 235, Skipped: 91

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2516/
Test FAILed.
Test FAILured.

Closing PR as it is over 5 months old.


Save results

Close #82
This PR envolved some changes into Statistics method such as update
to avoid to serialize unecessary stuff.
Statistics can be easily retrieved using:
from qstrader.statistics import load
stats = load("output_directory/statistics_yyyy-mm-dd_HHMMSS.pkl")

Bokeh TimeSeries plot may be improved according
https://github.com/bokeh/bokeh/issues/4747
Other interactive plotting libraries can also if we can't fix this.

Coverage decreased (-0.3%) to 88.498% when pulling a30c4e597243f2ce9121d0205a34623f3a8b05fc on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

We can also use mpld3 to plot an interactive equity time series
http://mpld3.github.io/
import mpld3
mpld3.enable_notebook()
ts.plot()

Coverage decreased (-0.6%) to 88.263% when pulling e55c94cce019069f643cc403b99c6d52f7eda47b on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Coverage decreased (-0.3%) to 88.498% when pulling c72544cffc57a418be88bbac458b757e68697a39 on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Coverage decreased (-0.2%) to 88.615% when pulling c72544cffc57a418be88bbac458b757e68697a39 on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Coverage decreased (-0.5%) to 88.38% when pulling 5db6068cb6df3c8993b2518e4a449bda937e9006 on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Coverage decreased (-0.2%) to 88.615% when pulling 5db6068cb6df3c8993b2518e4a449bda937e9006 on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Nice zoom feature!

I think we should agree in the future as to whether to merge PRs if test coverage has decreased. Given the slight decrease I'm not particularly concerned here, but if we introduce major new changes and test coverage decreases significantly, I don't believe we should merge the PRs.
Thoughts?

I agree! Testing new features is as much important as adding new features.

Bokeh 0.12 fixes x-axis issue with time series so we can have this kind of interactive plot in Jupyter notebook.

Coverage decreased (-0.2%) to 88.615% when pulling f44bd075c5d2b9582c613159eb4e69491bb11d4d on femtotrader:save_results into fc95eebb1085ef3b8107cb068bc581d5ab061686 on mhallsmoore:master.

Just to check before merging, what dependencies are we adding here? Bokeh and mpld3?

Bokeh and mpld3 is only used in Jupyter Notebook. It's not dependencies that's just optional for plotting.

Cool - just merged.

Thanks


Ship logomenu by default

Kyle had this great idea. I'd like to add logomenu to our panel, like this: https://github.com/Aryan20/Logomenu

However ... what I really want is for the icon to show the distro logo for the container userspace I am using. So by default in Bluefin that would be Ubuntu, but if you swapped it out for Alpine or Fedora then those logos would show up. And if you don't ever create a distrobox just leave the logo as Fedora or something. And they can also set one manually.
I think we'd need an agreed upon name for the default container to make it centralized. Any other ideas around this?
cc @KyleGospo @EyeCantCU

How would you determine the last selected box? Trying to figure out how best to implement this

yea this would be really hard to implement specially if you have bunch of boxes and use them at same time

How about making the default box on the system be named default and then just set it according to that?

Or let the custom image set the default, in my case it would be ubuntu-toolbox and for bazzite it would be bazzite-arch, maybe an option like in this change defaults section:

Or ... we could just select "Ubuntu", "Fedora", and "Wolfi" in yafti entries that just set the logo via a dconf key as part of when they're setting up their machine.

Looks like this will be F39 only since Kyle's added some goodies but it's going to require GNOME 45+.

Some ideas from chat:
[11:56 PM]j0rge: @KyleGospo I got it
[11:57 PM]j0rge: I totally got it
[11:57 PM]j0rge: the logo dropdown
[11:57 PM]j0rge: when you choose the "distrobox" you're going to switch into, we dynamically generate that terminal screen.
[11:57 PM]j0rge: "Welcome to your Arch userspace" and then pipe out a hyfetch on first launch or whatever
[11:58 PM]j0rge: it's like, no one is making the terminal with the nice WSL GUI, so what about moving the OS "chooser" thing to the desktop level.
[11:58 PM]j0rge: then we wouldn't need to map shortcuts to terminals and stuff
[11:58 PM]j0rge: like, the terminal would just open to what you have set as the OS
[11:59 PM]KyleGospo: Sounds great to me, we could make that a simple bash script that even calls the just command for you
[11:59 PM]j0rge: and dude we could even have an icon for "Host" when you want to just have a host terminal
[11:59 PM]KyleGospo: And then have logo menu pipe to it```


[5.3][CodeCompletion] Assume non-'isSimpleDidSet' in code completion

Cherry-pick #32528 into release/5.3

Explanation: As a side effect of SE-0268 implementation, Code completion stopped suggesting implicit oldValue in didSet blocks. This patch fixes the regression by retuning false from SimpleDidSetRequest in code completion mode so that the oldValue is  correctly populated.
Scope: Code completion inside didSet block
Risk: Low
Testing: Added regression test cases
Issue: rdar://problem/64699286
Reviewer: Ben Langmuir (@benlangmuir), Suyash Srijan (@theblixguy)

@swift-ci Please test

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 752fe6c9a9c7f079deddea382443b2864f49096f

Build failed
Swift Test OS X Platform
Git Sha - 752fe6c9a9c7f079deddea382443b2864f49096f

@swift-ci Please nominate


add search field handler

adds auto suggest to pop up.

only issue is it only shows one suggestion


Extract and test DevOmni calculation

This pull request adds tests for the calculation of DevOmnis, which was taken from #338.

Hmm... looks like the tests are failing on the 32 bit builds:
https://travis-ci.org/OmniLayer/omnicore/jobs/251046360#L2455
https://travis-ci.org/OmniLayer/omnicore/jobs/251046361#L2311


gas limit

can you add option to change gas limit?

Currently the gas limit is hard coded at 200,000. See  FarmClient.h#L238
Is that not enough for your needs?  What are you mining?
I might add an option, or I might use eth_estimateGas to get an estimate of the gas required.
For now, though,  you could just change that line of code and build from source.

I'm working on a new project that needs to have a lot more gas.
Some produced blocks need about 600.000 gas.
I found the gas limit in your files and adapted according.
Building isn't working (windows) perhaps you could make a new build with a higher limit?
or an eth_estimateGas like you suggest yourself.

thanks!


feat: confetti on homepage by default

Closes https://github.com/0xBubki/donate/issues/105.

cc @GarrettJMU

🚢


support for fractional electron

Hello,
My inquiry is related to the support for fractional electrons within PySCF.
It appears to me that PySCF does not explicitly support fractional electrons at the moment.
I am interested to know if there are plans to include this feature in the future?
Or perhaps, is there a way to achieve this with the current code?
Specifically, I would like to reproduce Figure 1 of below the paper. (sorry for the link, copyright)
It is implemented naturally in Qchem, but I want it in pyscf
https://doi.org/10.1021/acs.jpclett.8b02417
Best Regards, M

The DFT code in PySCF does not have a strict requirement on the number of electrons being an integer. You can modify the get_occ method to assign fractional occupation on certain orbitals, may be similar to this  https://github.com/pyscf/pyscf/blob/master/pyscf/scf/addons.py#L41

Thank you for your response! I will definitely check your link


BMR and Calories Expended

Hi I want to know if theres any method to differenctiate BMR calories with Calories Expended in the API. If I change any personal data (in google Fit) I can get It using TYPE_BASAL_METABOLIC_RATE. But If i don't do any thing this doesnt work

Same here, the documentation says it BMR can be retrieved with that TYPE, but that's not true


No color?

Testing #21861
I honestly don't really know what I am testing/doing here but I have set the --color=yes argument but I don't see any color in the test results pane nor in the peek view. So, I guess that's unwanted
{
    "python.experiments.optInto": ["pythonTestAdapter"],
    "python.testing.pytestArgs": ["--color=yes","-o", "log_cli=true"]
}

Will investigate, the screenshot you provided below shows that the color isn't working so this looks to be a legitimate bug. Thanks!

Hi! Any chance you could set your log level to "trace" and send over the associated "python" output channel logs? I tried the same args and test and got full color. The logs can help me determine if that argument is even making it to pytest or not. Thanks!

Hello! @cwebster-99 ran into the same problem and realized that setting your pytestArgs in your user settings doesn't work if you have workspace settings defined for pytestArgs. Since we define the pytestArgs on configuration for you in your workspace you very likely have them there are that is why it isn't working in your user settings. Going to close this since Courtney had the same problem then you don't have to worry about repro-ing


Use updated Sierra item status code and rules for requesting to handle safeguarded material

We have updates to Sierra to correctly handle records for safeguarded material:

I've created a new item status code in Sierra: "Safeguarded" (g).  Requested by Toni Hardy for CALM item level records harvested and loaded into Sierra.  This status is not requestable and has opacmsg = ‘by approval’ (p).

Source: Slack.
Sierra's rules for requesting are also updated to work with the new status code. The new rule is on line 20:
rules-for-requesting.txt
We needed to be using these to present this material correctly on /works.
TO DO
Update to use:

[ ] item status code Safeguarded (g)
[ ] rules for requesting (attached)
[ ] check if the opacmsg by approval (p) should also be used/updated

Collections Information they plan to start using it from around 27/28 Feb. (Slack)


Fix removal of swarm hash from bytecode

Since Solidity 0.5.2, the swarm hash may not be the last item in the contract's bytecode (it still is on the deployedBytecode, though), as constants used by the constructor may be added after it. This means that the tryRemoveSwarmHash function from bytecode.ts will fail, since it assumes it is the last bytes sequence (note the ending dollar sign in the regex).
bytecode.replace(/a165627a7a72305820[a-fA-F0-9]{64}0029$/, '')

This has the effect that the check of whether a contract changed or not will yield a false positive whenever the contract is compiled from a different machine or directory, since the swarm hash includes the source file paths.
We need to drop the $ from the regex, and run that replacement only on the last occurrence, in case the contract's data contains a sequence that looks like the swarm hash.
We should also test what happens in the following scenarios:

If a contract A creates another contract B, then B's code should be included in A. Does this mean that B's swarm hash is also included? If so, should we also try and remove it?
If a contract A creates another contract B from A's constructor, where is B's contract placed? Will B's swarm hash be listed after A's?

Alternatively, we can also evaluate relying on the deployedBytecode instead of bytecode property for checking for changes. The difference between the two should be that the latter includes the constructor code, but that should not affect much since logic contracts should not have constructors.


Memory error when encoding syllables/utterances

I'm getting the following error
neobolt.exceptions.TransientError: There is not enough memory to perform the current task.
when trying to encode encode utterance for Switchboard, and for syllables on SOTC. This is distinct from the previous error happening during corpus import.

@james-tanner I think this is all resolved now, right?  Both Switchboard and SOTC have been enriched with syllables/utterances successfully?


THRIFT-5138: Swift - properly escape keywords

Compiler: Swift, Python

Fix one place where escaping was forgotten (Swift)

Allow keywords to be used as identifiers in Swift

Fix keyword mechanism (calling virtual method from base class constructor calls base class method, not the overridden one (used in Python and Swift)

[x] Did you create an Apache Jira ticket?  (not required for trivial changes)

[x] If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?

[ ] Did you squash your changes to a single commit?  (not required, but preferred)

[x] Did you do your best to avoid breaking changes?  If one was needed, did you label the Jira ticket with "Breaking-Change"?

[ ] If your change does not involve any code, add  [skip ci] at the end of your pull request to free up build resources.

Note: Travis failure was caused by network problem during getting rust dependencies. Maybe Travis cache should be setup so that network problems are less frequent (although Travis docs state that cache is not network local).

Not sure if the build error is related (probably not) but may I ask you to rebase or force-push again nevertheless?

Failed jobs:

UBSan: https://travis-ci.org/github/apache/thrift/jobs/665499739#L19046

  - Installing squizlabs/php_codesniffer (3.5.4): Downloading (connecting...)
Could not fetch https://api.github.com/repos/squizlabs/PHP_CodeSniffer/zipball/dceec07328401de6211037abbb18bda423677e26, please create a GitHub OAuth token to go over the API rate limit

Head to https://github.com/settings/tokens/new?scopes=repo&description=Composer+on+travis-job-51f671a1-4db4-419f-8627-a7ffe95f5d6a+2020-03-22+1515

to retrieve a token. It will be stored in "/root/.composer/auth.json" for future use by Composer.

2x  JOB="pod lib lint"

An error occurred while generating the build script.

I wonder why we need to place this in the CTORs at all.
Wouldn't init_generator() be the more natural (and less error prone) way ?


Ensure that save_and_open_page_path is re-set if save_page raises

For the past couple of months, when the testrun in my app fails particularly egregiously I would find that it left a bunch of screenshot HTML files in the project root instead of tmp/capybara, like so:
traviss-air:bridge_troll tjgrathwell$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   app/models/event.rb

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	screenshot_2015-05-03-15-52-33.521.html
	screenshot_2015-05-03-15-52-36.030.html
	screenshot_2015-05-03-15-52-36.351.html

I tracked down this problem today and found that if any individual test raises an error performing capybara.save_page, the Capybara.save_and_open_page_path remains set to nil, which causes files to start appearing in the project root.
There may be more stuff worth investigating here (like how to make save_page not crash under certain conditions) but this commit fixes the immediate problem.

FYI, the bug causing capybara.save_page to crash in my case was https://github.com/jnicklas/capybara/pull/1516, but I still think the fix here is valuable too.

Nice, looks good, thanks @tjgrathwell


Named entity recognition granularity is not reflected correctly in exported data

Describe the bug
Option granularity="word" in named entity recognition is not reflected in exported data.
To Reproduce
Steps to reproduce the behavior:

create new project, select NER template and check "Select text by words"
import 1 task from a json file listed below
click on the task to annotate it
annotate mple tex as ORG. Label studio automatically expands annotation to whole words, so simple text is annotated.
export data as json (the relevant part is listed below)

json input data:
{
  "data": {
    "text": "Some simple text with explanations"
  }
}

json output data:
{
  "value": {
      "start": 5,
      "end": 16,
      "text": "ple tex",
      "labels": [ "ORG" ]
  }
}

Expected behavior
The field text should contain simple text instead of ple tex
Notice however that start and end fields correspond to simple text, so at least those are correct.
Environment (please complete the following information):

OS version: Ubuntu 18.04.4 LTS
Label Studio Version: 1.4

Thank you for your report, we've created a ticket about this bug. We will write you when it's done.

Sorry, this issue was missed, just fixed it in LSF, will be released soon. Or you can use custom build right now.

when is the next release?

@dhirajsuvarna on the next week.


Update the firebase project to ng-be-app instead of ngbe

[x] find new project credentials
[x] deploy using new credentials
[x] check functionality

done


Add first version of x-kuberneets impl with unit tests

Description
Changes proposed in this pull request:

add unit test to validate correctness of x-kubernetes-validations
add validation for labels

Related issue(s)
https://github.com/kyma-project/kyma/issues/17562

/test all

/test all

/retest

/retest


torchao.float8 + torch.compile does not work on HuggingFace's Mixtral model

🐛 Describe the bug
Specifically, if we try to compile a float8 version of a FFN expert (MixtralBlockSparseTop2MLP), we see shape errors
Script (requires torchao and transformers):
from transformers.models.mixtral.modeling_mixtral import (
    MixtralConfig,
    MixtralSparseMoeBlock,
    MixtralBlockSparseTop2MLP,
)

import torch

from torchao.float8 import convert_to_float8_training

config = MixtralConfig()
m = MixtralSparseMoeBlock(config).cuda()

def module_filter_fn(mod, fqn):
    # gate has out_channels == 8 which is not compatible with float8 gemms,
    # filter it out from float8 for now
    return "gate" not in fqn

convert_to_float8_training(m, module_filter_fn=module_filter_fn)

# if we compile the whole module with float8, see this:
# https://gist.github.com/vkuzo/bc001fc1609bac361e6e13995c23d808
# m = torch.compile(m)

# try compiling individual experts
for name, child in m.experts.named_children():
    if isinstance(child, MixtralBlockSparseTop2MLP):
        new_child = torch.compile(child)
        setattr(m.experts, name, new_child)

# we should see each individual expert being compiled here
print(m)

# if torch.compile is on, this will fail with
# https://gist.github.com/vkuzo/b5136f21302cd2a259cbb37cda1aa717
for _ in range(10):
    x = torch.randn(1, 4096, 4096, device='cuda').requires_grad_()
    final_hidden_states, router_logits = m(x)
    # res = final_hidden_states.sum() + router_logits.sum()
    # res.backward()

print('done')

Output:
...
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 1254, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 1796, in dispatch
    return self._cached_dispatch_impl(func, types, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 1355, in _cached_dispatch_impl
    output = self._dispatch_impl(func, types, args, kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_subclasses/fake_tensor.py", line 2155, in _dispatch_impl
    r = func(*args, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_ops.py", line 723, in __call__
    return self._op(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/_meta_registrations.py", line 5558, in meta_scaled_mm
    torch._check(
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/__init__.py", line 1573, in _check
    _check_with(RuntimeError, cond, message)
  File "/home/vasiliy/.conda/envs/pt_nightly_20241006/lib/python3.11/site-packages/torch/__init__.py", line 1555, in _check_with
    raise error_type(message_evaluated)
torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
RuntimeError: Expected self.size(1) to be divisible by 16, but got self.size(1)=1007

Full output: https://gist.github.com/vkuzo/b5136f21302cd2a259cbb37cda1aa717
Versions
Collecting environment information...
PyTorch version: 2.6.0.dev20241023+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: CentOS Stream 9 (x86_64)
GCC version: (GCC) 11.5.0 20240719 (Red Hat 11.5.0-2)
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.34

Python version: 3.11.0 (main, Mar  1 2023, 18:26:19) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-6.4.3-0_fbk14_zion_2601_gcd42476b84e9-x86_64-with-glibc2.34
Is CUDA available: True
CUDA runtime version: 12.2.140
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA H100
GPU 1: NVIDIA H100
GPU 2: NVIDIA H100
GPU 3: NVIDIA H100
GPU 4: NVIDIA H100
GPU 5: NVIDIA H100
GPU 6: NVIDIA H100
GPU 7: NVIDIA H100

Nvidia driver version: 535.154.05
cuDNN version: Probably one of the following:
/usr/lib64/libcudnn.so.8.8.0
/usr/lib64/libcudnn_adv_infer.so.8.8.0
/usr/lib64/libcudnn_adv_train.so.8.8.0
/usr/lib64/libcudnn_cnn_infer.so.8.8.0
/usr/lib64/libcudnn_cnn_train.so.8.8.0
/usr/lib64/libcudnn_ops_infer.so.8.8.0
/usr/lib64/libcudnn_ops_train.so.8.8.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      52 bits physical, 57 bits virtual
Byte Order:                         Little Endian
CPU(s):                             384
On-line CPU(s) list:                0-383
Vendor ID:                          AuthenticAMD
Model name:                         AMD EPYC 9654 96-Core Processor
CPU family:                         25
Model:                              17
Thread(s) per core:                 2
Core(s) per socket:                 96
Socket(s):                          2
Stepping:                           1
Frequency boost:                    enabled
CPU(s) scaling MHz:                 79%
CPU max MHz:                        3707.8120
CPU min MHz:                        1500.0000
BogoMIPS:                           4792.82
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp ibrs_enhanced vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid overflow_recov succor smca fsrm flush_l1d
Virtualization:                     AMD-V
L1d cache:                          6 MiB (192 instances)
L1i cache:                          6 MiB (192 instances)
L2 cache:                           192 MiB (192 instances)
L3 cache:                           768 MiB (24 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-95,192-287
NUMA node1 CPU(s):                  96-191,288-383
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Vulnerable: eIBRS with unprivileged eBPF
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==<IP_ADDRESS>
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==<IP_ADDRESS>
[pip3] nvidia-cufft-cu12==<IP_ADDRESS>
[pip3] nvidia-curand-cu12==<IP_ADDRESS>
[pip3] nvidia-cusolver-cu12==<IP_ADDRESS>
[pip3] nvidia-cusparse-cu12==<IP_ADDRESS>
[pip3] nvidia-nccl-cu12==2.21.5
[pip3] nvidia-nvjitlink-cu12==12.4.127
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] pytorch-lightning==2.4.0
[pip3] pytorch-triton==3.1.0+cf34004b8a
[pip3] torch==2.6.0.dev20241023+cu121
[pip3] torchao==0.7.0+gitd2526126
[pip3] torchdata==0.8.0
[pip3] torchmetrics==1.5.0
[pip3] torchvision==0.20.0.dev20241023+cu121
[pip3] triton==3.1.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-cublas-cu12        <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
[conda] nvidia-cudnn-cu12         <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cufft-cu12         <IP_ADDRESS>                pypi_0    pypi
[conda] nvidia-curand-cu12        <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusolver-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusparse-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-nccl-cu12          2.21.5                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.4.127                 pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
[conda] pytorch-lightning         2.4.0                    pypi_0    pypi
[conda] pytorch-triton            3.1.0+cf34004b8a          pypi_0    pypi
[conda] torch                     2.6.0.dev20241023+cu121          pypi_0    pypi
[conda] torchao                   0.7.0+gitd2526126           dev_0    <develop>
[conda] torchdata                 0.8.0                    pypi_0    pypi
[conda] torchmetrics              1.5.0                    pypi_0    pypi
[conda] torchvision               0.20.0.dev20241023+cu121          pypi_0    pypi
[conda] triton                    3.1.0                    pypi_0    pypi

cc @ezyang @chauhang @penguinwu

Update:

MoE model sends tensors with varying dims to the individual experts, so the failue is expected in the absence of padding

We can enable padding with the following update to the float8 config:

config = Float8LinearConfig(pad_inner_dim=True)
convert_to_float8_training(m, module_filter_fn=module_filter_fn, config=config)

Full example: https://gist.github.com/vkuzo/f5cd488ab635ea3dfe1205aa68eca473

Unfortunately, the above script does not work in triton 3.1.0 due a triton compilation error when padding f8e5m3.  This is fixed in https://github.com/triton-lang/triton/pull/4222, which is not included in triton 3.1.0.  We can install the triton nightly to get around this error:

pip install -U --index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/Triton-Nightly/pypi/simple/ triton-nightly

After installing the triton nightly and enabling float8 padding, we can e2e compile this example.
Keeping this issue open to track documenting this in our README.md file.

We should also add performance benchmarks on float8 + compile on MoE, as padding will have an additional overhead

Thank you very much! @vkuzo Would be awesome to have benchmarking comparison on this. And just to confirm, is this for FSDP1 or FSDP2 version?

I wrote this issue to be independent from choice of FSDP version and the localized repro does not use FSDP at all, I really hope that it's orthogonal :)  If there are any issues when composing this with FSDP1 or 2, happy to help figure those out.

an additional thing: in my original repro for this issue, things worked in eager mode but not in compile, for forward pass only.  For fwd+bwd, both eager and compile were broken.  I suspect that torch._scaled_mm's meta function might have shape constraints which are too stringent compared to the actual implementation of the scaled gemm, which would explain eager working but compile not working for the forward.  Noting it here so I don't forget, but we should ensure eager matches compile here.  TODO file an issue / fix this in pytorch/pytorch.

oh @vkuzo this errors in eager mode for me too actually, if I just uncomment the lines that run the backward too:
res = final_hidden_states.sum() + router_logits.sum()
res.backward()

What's going on here is that:
(1) in eager mode, you won't actually see the shape error until your execute your backward
(2) under compile, we generate a backward graph ahead of time when we see that your forward outputs require gradients
(3) that means that instead of getting a delayed error at the time your run the backward, you'll get the error earlier (at compile time) when we trace out the backward graph
If you really want to just run the forward and not the backward under compile, you can run under no_grad (which will be more efficient anyway as we don't need to save activations in the generated compile graph.
So it seems like padding might be required here for both eager and compile?

I see, thanks for the explanation @bdhirsh !


[Urgent]: Hotfix for Automatic1111 Web UI 1.8

Closes #35
With the release of version 1.8 of a1111, the extra networks pages were changed.
This PR aims to fix some of the issues this caused. I tried my best to keep SD.Next compatible, but there may be unintended side effects.
Changelog:

Network dictionary values changed:

"search_term" -> "search_terms"
"hash" -> "shorthash"

search_terms is now represented as an array on a1111
Updated staged javascript file

Results in slight loss of functionality. NSFW filtering no longer works on a1111.

@corbin-hayden13 Haven't heard from you in a while, so I'm gonna merge this PR for now as it's rather urgent.


No prometheus metric to find all unique label names, unique label counts or unique label value pairs

Proposal
I was trying to create a high cardinality management dashboard as Grafana cloud has. See URL below:
https://grafana.com/docs/grafana-cloud/billing-and-usage/control-prometheus-metrics-usage/cardinality-management/
But I have found that there is no metric which can give me unique label names or count of unique labels or unique values for each label.
Is there any feasibility if prometheus can export metrics to publish all these label details?

To get the count of unique labels / values for each label technically you can use this metric (Prometheus  creates a different time-series for every metric-label-value combination):
# HELP prometheus_tsdb_head_series_created_total Total number of series created in the head
# TYPE prometheus_tsdb_head_series_created_total counter
prometheus_tsdb_head_series_created_total 60566

For other metrics like those on the TSDB Status page on the prometheus interface, you can check the api:
GET http://localhost:9090/api/v1/status/tsdb

The response will include informations about (in json format):

Head Stats
Top 10 label names with value count
Top 10 series count by metric names
Top 10 label names with high memory usage
Top 10 series count by label value pairs

Also check this guide on how to feed the response into prometheus:
https://grafana.com/docs/grafana-cloud/billing-and-usage/control-prometheus-metrics-usage/usage-analysis-api/
or this plugin:
https://grafana.com/grafana/plugins/grafana-simple-json-datasource/

Thanks a lot for your reply.
It is good to have these cardinality related data in the form of metrics so that we can take advantage of PromQL but anyway after your suggestion we have found below two useful plugins which helped us to access Prometheus TSDB REST endpoint and showing data on dashboard.
https://grafana.com/grafana/plugins/marcusolsson-json-datasource/
https://grafana.com/grafana/plugins/yesoreyeram-infinity-datasource/
Grafana cloud is having this cardinality related data as a paid functionality. But I believe that understanding cardinality better will help everyone. If possible you can try to add these native metrics in future releases. But this is resolved for me, for now.

Hi, are you willing to share how you got the api/v1/status/tsdb data in a grafana dashboard?

Hello @k24dizzle ,
As I mentioned above, I am using Grafana plugins which can call prometheus REST endpoint and convert response data to Grafana visualization. As we were having multiple prometheus urls to hit to fetch and collect data, yesoreyeram-infinity-datasource plugin was most suitable for us. It helped to add different different endpoints in panel only without changing anything in datasource configuration.

Hey @jagyas can you please close this issue as it is resolved?

Hello from the bug scrub.
Closing as this issue looks to be resolved.