Don't throw exception when compilation unit has multiple files with the same digest. [phab:D1930]

Created by nbeloglazovgoog at 2017-09-11 13:39:33:
See https://groups.google.com/forum/#!topic/kythe/UQ_YAhjxENY

shahms commented inline (context lost during migration) at 2017-09-11 15:49:42:
Just digest?  Also, why keep a copy of the keys rather than just catching the exception here?

schroederc commented inline (context lost during migration) at 2017-09-11 16:05:26:
The exception happens on line #55, if I remember correctly.

nbeloglazovgoog wrote at 2017-09-11 16:14:03:
Use digest variable.

nbeloglazovgoog commented inline (context lost during migration) at 2017-09-11 16:17:08:
Not using exception for "expected" flow feels cleaner. Also, it won't swallow other unrelated exceptions that can be an actual bug. Do you still prefer try/catch?

shahms commented inline (context lost during migration) at 2017-09-11 16:26:56:
No preference as such, just asking. It seems like ImmutableMap.Builder should also provide get access, but maybe that's just me ;-)

shahms accepted this revision at 2017-09-11 16:26:56

nbeloglazovgoog commented inline (context lost during migration) at 2017-09-11 16:30:18:
Agree, I expected the builder to have get method. Or a way to handle duplicated keys in some other ways.

nbeloglazovgoog wrote at 2017-09-11 16:49:29:
Shahms, could you land this change? I don't have write access which means I can't submit it, right?

shahms closed via commit 4a5fba0d102c389fecc1553fd2f1822b1906ec09 at 2017-09-11 18:40:49


NTR: Four additional subclasses of 'highest level of education data item'

This NTR is for four subclasses of 'highest level of education data item.'  The first is for cases in which it is only known that the person's highest level of education is something below high school diploma/equivalent, but the level is not further specified. The second and third are for when the data reflect that a person started but did not complete college or high school, respectively. The fourth is for when the person's highest level of education is high school completion or the equivalent.
Term 1/4:
Preferred term label
highest level of education is below high school completion or equivalent data item
Textual definition
A highest level of education data item that represents the fact that an individual in the United States is has not achieved a level of education that is equivalent to or higher than a complete high school education.
Suggested parent term
highest level of education data item
Attribution
ORCID: 0000-0003-1909-9269
Term 2/4:
Preferred term label
highest level of education is some college data item
Textual definition
A highest level of education data item that represents the fact that the highest level of education achieved by an individual in the United States is completion of some undergraduate coursework, but without attaining an undergraduate degree.
Suggested parent term
highest level of education data item
Attribution
ORCID: 0000-0003-1909-9269
Term 3/4:
Preferred term label
highest level of education is some but not all high school data item
Textual definition
A highest level of education data item that represents the fact that the highest level of education achieved by an individual in the United States is completion of some high school coursework, but without attaining a diploma or equivalent.
Suggested parent term
highest level of education data item
Attribution
ORCID: 0000-0003-1909-9269
Term 4/4:
Preferred term label
highest level of education is completion of high school or equivalent data item
Textual definition
A highest level of education data item that represents the fact that the highest level of education achieved by an individual in the United States is completion of high school or equivalent.
Suggested parent term
highest level of education data item
Attribution
ORCID: 0000-0003-1909-9269

We discussed at OMRSE meeting on August 7, and agreed to create these classes as defined.


tBTC end-to-end test script

Depends on #2
Here we improve the local environment setup scripts and introduce a new script to perform an end-to-end test of tBTC deposit and redemption. Apart from that, a script for installation and start-up of a local relay maintainer was added.

I executed the e2e test script successfully.
✗ ./run-e2e-test.sh
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
npm WARN<EMAIL_ADDRESS>No description
npm WARN<EMAIL_ADDRESS>No repository field.
npm WARN<EMAIL_ADDRESS>No license field.

audited 2984 packages in 96.299s

45 packages are looking for funding
  run `npm fund` for details

found 23 vulnerabilities (22 low, 1 high)
  run `npm audit fix` to fix them, or `npm audit` for details
(node:45730) ExperimentalWarning: Importing JSON modules is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)

Starting deposit number [1]...

Creating new deposit contract with lot size 100000 satoshis...
Looking up new deposit with address 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 backed by keep at address 0x2dB696d61B2583D3b95165422f9e3551758cA166...
Monitoring deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 for transition to ACTIVE.
Waiting for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 keep public key...
Waiting for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 to retrieve public key...
Found public key for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
        Got deposit address: tb1qu8rhru0nhdjd6rn6kufkfv2apae3r4gqmvdypg ; fund with: 100000 satoshis please.
Now monitoring for deposit transaction...
Monitoring Bitcoin for transaction to address tb1qu8rhru0nhdjd6rn6kufkfv2apae3r4gqmvdypg...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Received notification for script hash: [e75df4da483fc704b3fa4f4d8e552c063b777a6807f8878441bf6ea4b87169a5] with status: [bdff42bdc939ef10a9ee56ffc59d454146380c0ece4f65c34ac7b54547e296d1]
Closing connection to electrum server...
Waiting for 1 confirmations for Bitcoin transaction 54c211596525bca8efb82af7fa863220ae34392772cbfe47e649f1785b754ace...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Registered listener for blockchain.headers.subscribe event
Received notification of a new block at height: [1747693]
Closing connection to electrum server...
Submitting funding proof to deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 for Bitcoin transaction 54c211596525bca8efb82af7fa863220ae34392772cbfe47e649f1785b754ace...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Closing connection to electrum server...
Deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 transitioned to ACTIVE.
Deposit is active, minting...
Approving transfer of deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 TDT to Vending Machine...
Minting TBTC...
Found Transfer event for<PHONE_NUMBER>00000 TBTC.

Deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 has been created successfully.

Starting deposit number [2]...

Creating new deposit contract with lot size 100000 satoshis...
Looking up new deposit with address 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F backed by keep at address 0xC8be1842e30230A2FE1B4eCeA3E288CFd71654D0...
Monitoring deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F for transition to ACTIVE.
Waiting for deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F keep public key...
Waiting for deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F to retrieve public key...
Found public key for deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F...
        Got deposit address: tb1q977u0phzuy3lktljjpvzdsqzrr5xhw6azdr48n ; fund with: 100000 satoshis please.
Now monitoring for deposit transaction...
Monitoring Bitcoin for transaction to address tb1q977u0phzuy3lktljjpvzdsqzrr5xhw6azdr48n...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]

Received notification for script hash: [7399dea7a9fc6341dde0a60756f8f5ae986e875775aa458fbc30a645a5549de9] with status: [0482a7a9a4953861f2a7bf290c643141016217cb08b953573173c72b941da4e1]
Closing connection to electrum server...
Waiting for 1 confirmations for Bitcoin transaction 7aa63c9d40b1428af4e607f21340c64bd69b8a02a91855f0ee578f7ad6b539b1...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Registered listener for blockchain.headers.subscribe event

Received notification of a new block at height: [1747695]
Closing connection to electrum server...
Submitting funding proof to deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F for Bitcoin transaction 7aa63c9d40b1428af4e607f21340c64bd69b8a02a91855f0ee578f7ad6b539b1...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Closing connection to electrum server...

Deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F transitioned to ACTIVE.
Deposit is active, minting...
Approving transfer of deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F TDT to Vending Machine...
Minting TBTC...

Found Transfer event for<PHONE_NUMBER>00000 TBTC.

Deposit 0xFB7Fb3e8DcC8CeB24176f4737a5B9e4bA6d46C6F has been created successfully.

Starting redemption of the first deposit...

Looking up Deposit contract at address 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Looking up Created event for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Found keep address 0x2dB696d61B2583D3b95165422f9e3551758cA166.
Found existing Bitcoin address for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Looking up UTXO size and transaction fee for redemption transaction...
Approving transfer of<PHONE_NUMBER>000000 to the vending machine....
Initiating redemption of deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 from vending machine...
Looking up latest redemption details for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Finding or waiting for transaction signature for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Looking for existing signed redemption transaction on Bitcoin chain for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Closing connection to electrum server...
Broadcasting signed redemption transaction to Bitcoin chain for deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Closing connection to electrum server...
Waiting for 1 confirmations for Bitcoin transaction 998515ea4303b6786ce9cd362b1c5ab64a11c31a0f8bd9391b55e3dac00a8316...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Registered listener for blockchain.headers.subscribe event
^[[CReceived notification of a new block at height: [1747696]
Closing connection to electrum server...
Transaction is sufficiently confirmed; submitting redemption proof to deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346...
Connecting to electrum server...
You are connected to an ElectrumX 1.14.0 server.
Negotiated version: [ElectrumX 1.14.0,1.4.2]
Closing connection to electrum server...

Redemption outcome: Redeemed deposit 0x6Bffd428b2534816ABD2d45e66f9A976Bfc7e346 with Bitcoin transaction 998515ea4303b6786ce9cd362b1c5ab64a11c31a0f8bd9391b55e3dac00a8316.

Test completed successfully


add independency js lib

fix #84

Codecov Report

Merging #85 (72ee059) into main (a813f74) will not change coverage.
The diff coverage is 100.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@           Coverage Diff           @@
##             main      #85   +/-   ##
=======================================
  Coverage   98.86%   98.86%
=======================================
  Files          79       79
  Lines        1232     1232
  Branches      187      187
=======================================
  Hits         1218     1218
  Misses         13       13
  Partials        1        1

Files
Coverage Δ

src/astro/FunctionalAstrolabe.ts
98.82% <100.00%> (ø)

:mega: Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!


Py 3.4 support

Thanks for sigal!
What is needed for Python 3.4 support? Would love to see that happening!

Sigal is already compatible with python 3.
There is an encoding bug with non-ascii directory names, which is fixed on the git version, but otherwise it should be ok.

Assuming you had the same issue as #137, this is now fixed and release in version 0.9.2

Thanks, yes that was it!


Call IsActive from Introspection

@brockallen Actually - the introspection validator calls the token validator, which in turn calls the custom token validator - which calls IsActive ;)
Now it is debatable if the IsActive check should be part of the custom token validator - or the standard one.


call this.$refs.paginate.selected from other component

#HTML
<template id="A">
  <paginate
     :page-count="10"
     :container-class="'pagination'"
     :click-handler="pageEvent">
  </paginate>
</template>

i'd wish to select page back to a number after getting callback, however the issue is my main method is calling from another component, the following way is not working when i called function abc().
var bus = new Vue();

Vue.component('A', {
  template: '#A',
  created{
    bus.$on('test', function () {
      this.$refs.paginate.selected = 5;
    });
  }
}

var vm = new Vue({
  methods: {
    abc(){
      axios.get('url').then((response) => {
         bus.$emit('test')
      });
    }
  }
}

I tried to call it from the component itself and it works
Vue.component('A', {
  template: '#A',
  mounted(){
    this.$refs.paginate.selected = 2;
  }
}

Any helps?

Hello @hamutaro89 . So busy these days. Sorry for can't reply you in time. I think that's because you scope of your callback function.
Could you please try this
bus.$on('test', () => {
  this.$refs.paginate.selected = 5;
});

Hello @hamutaro89 . Any problems just reopen this issue. Thanks.


Perlmutter configuration

Update
See the latest more verbose steps here :https://github.com/nv-legate/quickstart/pull/7#issuecomment-1610066967

Goal
This PR is an extension of #3 attempting to develop a basic quickstart-based workflow that can be used used to run cunumeric code on perlmutter.
Setup
Assumes the following repos and branches checked out in the same directory, e.g. ~/work:

legion / control_replication
legate.core / branch-23.05
cunumeric / branch-32.05

Assume conda env files generated using e.g.
python legate.core/scripts/generate-conda-envs.py --python 3.10 --ctk 11.8

(NB: need to apply patches first, script has an issue on perlmutter)
Build
# from a perlmutter login node
module load python
module load cudatoolkit
module load craype-accel-nvidia80
module load cray-pmi

# prevent BLAS issue
module del cray-libsci

# apply patches (ONCE)
cd ../legion
git apply ../quickstart/0001-legion-ofi.patch

cd ../legate.core
git apply ../quickstart/0002-legate-ofi.patch

# create conda env files (see above)

# create conda env at a specific prefix (ONCE)
PREFIX="/global/common/software/dasrepo/$USER/legate"
conda env create -p $PREFIX -f environment-test-linux-py3.10-cuda11.8.yaml

conda activate $PREFIX
export LEGATE_DIR=$PREFIX
export PLATFORM=perlmutter
export CC=cc
export CXX=CC
export FC=ftn

# build legate with embedded gasnetex
cd ../legate.core
LEGION_DIR=../legion/ CONDUIT=ofi-slingshot11 NETWORK=gasnetex ../quickstart/build.sh 2>&1 | tee ~/lg-build.txt

# build cunumeric
cd ../cunumeric
CONDUIT=ofi-slingshot11 NETWORK=gasnetex ../quickstart/build.sh 2>&1 | tee ~/cn-build.txt

Runtime
# from a perlmutter login node
module load python
module load cudatoolkit

# prevent BLAS issue
module del cray-libsci

PREFIX="/global/common/software/dasrepo/$USER/legate"

conda activate $PREFIX

export LEGATE_DIR=$PREFIX
export PLATFORM=perlmutter
export ACCOUNT=dasrepo

# from cunumeric repo, set LD path for lib64 install issue
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PREFIX/lib64/ USE_CUDA=1 ../quickstart/run.sh 2 examples/gemm.py -n 20000 --gpus 4

Result
Currently, running that cunumeric example results in a gasnet error and abort:
*** WARNING: ofi-conduit failed to configure FI_CXI_* environment variables due to prior conflicting settings. This may lead to unstable behavior and/or degraded performance.  If you did not intentionally set FI_CXI_RX_MATCH_MODE, then this condition may have resulted from initializing MPI prior to initialization of GASNet. For more information on that scenario, please see "Limits to MPI interoperability" in the ofi-conduit README.
MPICH ERROR [Rank 0] [job id 8707026.0] [Fri May 12 14:58:02 2023] [nid003701] - Abort(1616271) (rank 0 in comm 0): Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(170).......:
MPID_Init(501)..............:
MPIDI_OFI_mpi_init_hook(766):
create_endpoint(1311).......: OFI EP enable failed (ofi_init.c:1311:create_endpoint:Address already in use)

aborting job:
Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(170).......:
MPID_Init(501)..............:
MPIDI_OFI_mpi_init_hook(766):
create_endpoint(1311).......: OFI EP enable failed (ofi_init.c:1311:create_endpoint:Address already in use)
gasnet_exit(): ERROR: signal 11 received during exit... goodbye.  [in gasnetc_ofi_exit()]
aborting job:
GASNet abnormal exit

This happens with or without these set:
export FI_MR_CACHE_MONITOR=memhooks
export FI_CXI_RX_MATCH_MODE=software

@manopapad here is the PR, I do not have permissions to add a reviewer:
https://gitlab.com/StanfordLegion/legion/-/merge_requests/778

@bryevdv Regarding the lib/ vs lib64/ issue, can you check if these workarounds actually work? https://github.com/nv-legate/legate.core/pull/766, https://github.com/nv-legate/cunumeric/pull/971

Login (every time)
# from a perlmutter login node
module load python
module load cudatoolkit
module load craype-accel-nvidia80
module load cray-pmi

# prevent BLAS issue
module del cray-libsci

export PREFIX="/global/common/software/dasrepo/$USER/legate"

Repo Setup (once)
mkdir work
cd work

git clone git@github.com:nv-legate/quickstart.git
git clone git@github.com:nv-legate/legate.core.git
git clone git@github.com:nv-legate/cunumeric.git

# need control replication branch present
git clone https://gitlab.com/StanfordLegion/legion.git
cd legion
git checkout control_replication

Env Setup (once)
# start in work dir

mkdir envs
cd envs

python ../legate.core/scripts/generate-conda-envs.py --python 3.10 --ctk 11.8
cd  ..

conda env create -p $PREFIX -f envs/environment-test-linux-py3.10-cuda11.8.yaml

Build
# start in work dir

conda activate $PREFIX

export LEGATE_DIR=$PREFIX
export PLATFORM=perlmutter
export CC=cc
export CXX=CC
export FC=ftn

# build legate with embedded gasnetex
cd legate.core
LEGION_DIR=../legion/ NETWORK=gasnetex ../quickstart/build.sh --gasnet-system=slingshot11

cd ..

# build cunumeric
cd cunumeric
../quickstart/build.sh

Running
# start in work/cunumeric dir

LEGATE_DISABLE_MPI=1 USE_CUDA=1 ../quickstart/run.sh 2 examples/gemm.py -n 20000 --gpus 4

@manopapad pending https://github.com/nv-legate/legate.core/pull/776 for a small fix in the env generation, the above steps work for me with this PR (we could also delete the patch files now)

@manopapad in terms of any other small simplications:

is LEGATE_DIR actually used by anything? I'm not seeing it if so
do we want to default NETWORK=gasnetex for perlmutter in common.sh?
do we want to try to handle --gasnet-system=slingshot11 automatically somehow on perlmutter?
could we assume ../legion for LEGION_DIR if not provided?

Would love to get the instructions down to:
cd legate.core
../quickstart/build.sh

cd ..

cd cunumeric
../quickstart/build.sh

@manopapad I've merged your fix on branch-23.07 and changed the base branch as well. I think this could be merged, with the improvement ideas above left for separate work?

is LEGATE_DIR actually used by anything? I'm not seeing it if so

Not anymore

do we want to default NETWORK=gasnetex for perlmutter in common.sh?

Yes

do we want to try to handle --gasnet-system=slingshot11 automatically somehow on perlmutter?

Yes, I would suggest having a $GASNET_SYSTEM variable, which operates similar to $CONDUIT. Maybe only set it on Perlmutter, and in the legate.core part of build.sh we do something like
    if [[ -n "${GASNET_SYSTEM+x}" ]]; then
        set -- --gasnet-system "$GASNET_SYSTEM" "$@"
    fi

could we assume ../legion for LEGION_DIR if not provided?

I've updated the scripts to let legate.core do an internal build of Legion if $LEGION_DIR isn't provided, so we don't need to set that anymore. A user can still provide it if they want more control over which Legion version to use.

do we want to set LEGATE_DISABLE_MPI by default for perlmutter in run.sh ?

Yes, and document the issue that it's working around: https://gasnet-bugs.lbl.gov/bugzilla/show_bug.cgi?id=4638.

@manopapad I've merged your fix on branch-23.07 and changed the base branch as well. I think this could be merged, with the improvement ideas above left for separate work?

Yes, happy to go either way (merge this PR and start a new one, or include in this PR). Might be actually quicker to roll them all in now, and make the documentation effort smaller.

Edit: well, is there a place besides this issue to document the steps above? IIRC there is some internal wiki but I don't see how that helps general users

I've been putting such info in the README.md of this repo. For most clusters it has been sufficient to document just the "Login" steps, and fall back to the generic build instructions for the rest.
Reading through your recipe, it appears the unique things on Perlmutter (besides "Login") are:

the special cases that you identified above, which can be absorbed into the scripts
the use of a $PREFIX rather than a name for conda envs; is this necessary on Perlmutter?
export PLATFORM=perlmutter: this should be handled automatically by detect_platform
export CC=cc, export CXX=CC, export FC=ftn: are these still necessary? we could move them to set_build_vars if they're always necessary
USE_CUDA=1, --gpus 4: these should be getting set by default in run.sh.

the use of a $PREFIX rather than a name for conda envs; is this necessary on Perlmutter?

AFAIK this is in order to get the env installed on to /global/common for visibility on all the nodes

export PLATFORM=perlmutter: this should be handled automatically by detect_platform
export CC=cc, export CXX=CC, export FC=ftn: are these still necessary? we could move them to set_build_vars if

Yup, none of these seem necessary. With latest commit I believe it is down to this:
Login (every time)
# from a perlmutter login node
module load python
module load cudatoolkit
module load craype-accel-nvidia80
module load cray-pmi

# prevent BLAS issue
module del cray-libsci

export PREFIX="/global/common/software/dasrepo/$USER/legate"

Repo Setup (once)
mkdir work
cd work

git clone git@github.com:nv-legate/quickstart.git
git clone git@github.com:nv-legate/legate.core.git
git clone git@github.com:nv-legate/cunumeric.git

Env Setup (once)
# start in work dir

mkdir envs
cd envs

python ../legate.core/scripts/generate-conda-envs.py --python 3.10 --ctk 11.8
cd  ..

conda env create -p $PREFIX -f envs/environment-test-linux-py3.10-cuda11.8.yaml

Build
# start in work dir

conda activate $PREFIX

cd legate.core
../quickstart/build.sh

cd ..

cd cunumeric
../quickstart/build.sh

Running
# start in work/cunumeric dir

conda activate $PREFIX

export ACCOUNT=dasrepo

../quickstart/run.sh 2 examples/gemm.py -n 20000


Reconsider default append only behavior

From a discussion on slack:
Yuval Kalugny [10:50 AM]
Another question:
When I commit the same file a few times, the results get appended.
Is there a way for this to not happen? That is, that new files replace old files with the same name?

Daniel Whitenack [10:51 AM]
@kalugny  in 1.4 or 1.3?

Yuval Kalugny [10:53 AM]
In 1.3, but looking forward, also in 1.4

[10:54]
I would love to understand why that is the default behaviour - I think that generally files are not an assortment of independent lines

...

Joey Zwicker [11:36 AM]
@dwhitena @kalugny  I'm pretty sure in v1.4 you still need to do delete, then putfile to not append. The default behavior for putfile is still appending

[11:37]
A lot of that comes from the defaults of Hadoop-land I believe but perhaps it's something we can reconsider. Especially in v1.4 where appending to a file isn't as clean of a use case since files are the lowest level datum

This seems to make sense to me, I think we definitely still want the "append" behavior for files that are put with --split. I.e. if I put files foo/1 and foo/2 with one put-file the next one should put foo/3 and foo/4 not overwrite foo/1 and foo/2 but appending to a single file doesn't seem to make a ton of sense in the new model.

I don't see how append could ever have been useful? There are versions available if needed why would appending to the file make sense?
Perhaps I am too stuck in the git way but it seems that replacing the file in a new commit is the way to go?
Issuing a delete first also requires finding the latest commit and then do a delete right?
Seems a little counter intuitive and a bit cumbersome to code it. Do I have to traverse all the history and delete the file in every commit it has ever taken place?

@dahankzter I sounds like most of these questions were already answered in our Slack channel, but I'll comment on a few of them here just to keep the issue complete.

Issuing a delete first also requires finding the latest commit and then do a delete right?

This is what branches are for in Pachyderm. Branch names are essentially tags or aliases for a particular HEAD commit. So start-commit <repo> master will start a new commit with it's parent as the HEAD of master and will then tag this new commit as the new HEAD of master. Finding the latest commit is easy.

Seems a little counter intuitive and a bit cumbersome to code it. Do I have to traverse all the history and delete the file in every commit it has ever taken place?

Commits are immutable so you never go back and delete files from old commits. To delete a file you simply start a new commit on whatever branch you want and delete the file. Let's talk through an example:

Commit1 adds file foo
Commit2 adds files bar and buz
Commit3 deletes file foo
Commit 4 adds baz

If we list files for each commit, here's what is returned:
$ pachctl list-file <repo> Commit1
foo
$ pachctl list-file <repo> Commit2
foo
bar
buz
$ pachctl list-file <repo> Commit3
bar
buz
$ pachctl list-file <repo> Commit4
bar
buz
baz

Notice that in Commit3 we deleted foo. foo is no longer there in any future commit we read from, but we can always still go back and get foo by reading from Commit1 or Commit2. This works the same way for overwrites. If instead of just deleting foo in Commit3, I instead deleted it and added in a new file foo with different content (let's call it foo' to differentiate), then foo' would be available in Commit3 and Commit4 but 'foo` would be Commit1 and Commit2.

Thanks @JoeyZwicker ! It was a bit of a confusion on my part and it is much clearer now.
I still think the append by default is weird though. Other than a legacy behaviour is there any reason to maintain it? Having putfile behave as a "replace" would be conveniten but perhaps I have not considered enough ramifications.

There are a few edge cases where it's still useful, but we opened this issue specifically because we're thinking about changing the default to replace instead of append. In pachyderm v1.3, appending was a very meaningful use case, but in v1.4 and going forward, pachyderm manages/prefers lots of small/separate files instead of appending to one big log file -- which was the common use case.
We'll almost certainly change this behavior, it just hasnt been slated into our dev timeline yet.

It might be something we would want to keep when you are manipulating files using the pachctl but it seems it is also how it is currently working when you are manipulating files inside a pipeline, which was something really weird at first for me.

I'm tentatively in favor of changing the default to replace as opposed to append.  However, I do think that we need an --append (or -a) flag for put-file that allows for appending.  Otherwise appending to a file, which we've agreed that is still a reasonable use case, would have to consist of 1) reading the file, 2) manually append to it, then 3) put it back.
I'm also not sure if having --split default to append makes sense.  It just doesn't feel very consistent to have two defaults.  I think it's fine if you have to do --split -a to append to a split file.

Chatted with @JoeyZwicker offline.  One open question that occurred to us was what the behavior should be when you are putting the same file twice in an open commit.  Should one put overwrite another?  Or should they append?  The former seems dangerous since you are losing writes.  The latter feels inconsistent with what we are proposing which is to have the default behavior of put-file being replace.

It would be good if we could supply a mode to control if overwrite or
append should be used even in an open commit.
On Tue, May 9, 2017, 04:55 Derek Chiang (Enchi Jiang) <
<EMAIL_ADDRESS>wrote:

Chatted with @JoeyZwicker https://github.com/JoeyZwicker offline. One
open question that occurred to us was what the behavior should be when you
are putting the same file twice in an open commit. Should one put overwrite
another? Or should they append? The former seems dangerous since you are
losing writes. The latter feels inconsistent with what we are proposing
which is to have the default behavior of put-file being replace.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pachyderm/pachyderm/issues/1531#issuecomment-300047499,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKcU-P-0ifttiAVy6dQEYO1i-etHxRIks5r39WcgaJpZM4Mupjj
.

Talked with @jdoliner last week. I think one of the simplest options that would help users would simply to add a --overwrite (-o) to put-file so users can have some sugar instead of having to make two calls for every file they want to add.
This also puts the control into the users hands for most the above issues without having to actually change the default behavior.
Just to double check, delete-file shouldn't fail if the file doesn't exist (pretty sure it doesn't).

i think that would be a great idea and would simplify a lot of things.

This is somewhat related to #1885.
The current file mutation behavior is not documented anywhere leading to confusion of users who haven't participated in our Slack channel or read some of the issues in this repository.
While we resolve this issue, can we please document what happens when writing to a file?


docs: document that pmbr shouldn't be used unless nedded

While testing a new build of our image, we found that it didn't boot in UEFI mode on a Lenovo M700 and tracked it down to this. I think it makes sense to call out this caveat in the rustdoc.

Oh sorry I forgot that PR


stream-progress: _percentage NaN

If I run stream-progress with simplehttp2server v3.1.2, it fails with
dial.js:47 Uncaught (in promise) Error: Percentage must be a number between 0 and 1.
    at HTMLElement.set percentage [as percentage] (dial.js:47)
    at fetch.then (stream.js:94)
    at <anonymous>

A closer look tells me that Number.isNaN(_percentage) is true.
However if I run stream-progress with python -m SimpleHTTPServer it works as it did in the video.
I'm running on a MacBook with MacOS 10.11.6 and Chrome Canary 60.0.3105.0.

I can't check it right now, but that's probably because SH2S has GZIP-on-the-fly enabled and therefore doesn't send a Content-Length header.
One solution is to craft a HEAD request to get the uncompressed file size and use that as an estimation, or use a server that pre-compresses and sends the header.


Add support for negotiating IMAP, SMTP & HTTP on 443

This PR adds support for negotiating IMAP, SMTP & HTTP on the configured HTTPS port using TLS ALPN. This is intended to be useful for deploying Mox as a chatmail server. The upstream implementation of chatmail servers uses imap and smtp as the “next protocol” values for IMAP and SMTP, respectively: https://github.com/deltachat/chatmail/blob/main/cmdeploy/src/cmdeploy/nginx/nginx.conf.j2#L16-L17
To test, configure Mox as is standard for the mox localserve mode, but add this block under Listeners.local:
		AutoconfigHTTPS:
			Enabled: true
			Port: 1443

Then run Mox and use OpenSSL’s s_client mode to connect to the local instance:
> openssl s_client -quiet -connect localhost:1443 -alpn smtp
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
220 localhost ESMTP mox 7f5e1087d42d5d7bf57649c5c6a2155c3706ce39+modifications
HELO test
550 5.5.0 your ehlo domain does not resolve to an IP address (htqp11_GJOmHEhSs_Y03eg)
QUIT
221 2.0.0 okay thanks bye
^C
> openssl s_client -quiet -crlf -connect localhost:1443 -alpn imap
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
* OK [CAPABILITY IMAP4rev2 IMAP4rev1 ENABLE LITERAL+ IDLE SASL-IR BINARY UNSELECT UIDPLUS ESEARCH SEARCHRES MOVE UTF8=ACCEPT LIST-EXTENDED SPECIAL-USE LIST-STATUS AUTH=SCRAM-SHA-256-PLUS AUTH=SCRAM-SHA-256 AUTH=SCRAM-SHA-1-PLUS AUTH=SCRAM-SHA-1 AUTH=CRAM-MD5 ID APPENDLIMIT=9223372036854775807 CONDSTORE QRESYNC STATUS=SIZE QUOTA QUOTA=RES-STORAGE AUTH=PLAIN] mox imap
c1 STARTTLS
c1 BAD STARTTLS unrecognized syntax/command: tls already active
^C
> openssl s_client -quiet -crlf -connect localhost:1443
depth=0 O = mox localserve, CN = localhost
verify error:num=18:self signed certificate
verify return:1
depth=0 O = mox localserve, CN = localhost
verify return:1
GET / HTTP/1.1
Host: localhost

HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Mon, 25 Nov 2024 07:28:00 GMT
Content-Length: 19

404 page not found
^C

As part of this change, I had to go get golang.org/x/net/http2. (The default http.Server supports HTTP2 if you leave it alone, but that built-in support is deactivated if I set TLSNextProto to a non-nil value. To ensure that Mox continues to support HTTP2, the Go documentation directs folks with "…more complex configurations…" to import x/net/http2.)  This has caused the large number of dependency updates. I can revert these and attempt a more surgical addition of the http2 library, if you’d like.
There are also two major deficiencies in this code that I’d like advice on correcting:

Right now, this ALPN feature is enabled when Mox is configured to provide Autoconfigure services to mail clients. I chose to do this because it was relatively straightforward to implement. However, the resulting behavior is extremely non-obvious. How would you recommend exposing the ALPN feature in the configuration?
I’m not sure what the best way to expose the private serve() functions in imapserver and smtpserver are. The current implementation creates a public function called ServeConn() in each module that just calls the private serve() function with all the same arguments, but this feels redundant. Would you recommend making the serve() functions public, using public wrappers but with a more limited set of parameters, or something else entirely?

Thanks!

Nice, that's quick.
About enabling the ALPN feature: What about a new field in the configs for "submissions" and "imaps", to enable it on the https port. https://github.com/mjl-/mox/blob/v0.0.13/config/config.go#L168 and IMAPS just below. "EnableOnHTTPS bool" or something, with comment explaining ALPN.
It would enable this on any https on port 443 for that listener. And should probably give an error if there is no https on that listener. The http/web.go code is collecting servers on different ports, with tls settings. I think there could be a step about adding to NextProtos at the end of gathering all the servers.
About the serve() functions. I think we can make them public for now. We can change it later, we don't have to guarantee that those functions are completely stable. We should pass in the parameters from the listener. That's why it would be good to enable this ALPN feature in each submissions/imaps section of the listener config. The parameters should come from that listener.
Adding golang.org/x/net/http2 and the dependency updates are fine. I thought we could perhaps just use a regular tls listeners, and accept connections ourselves and look at the nextproto to call the right function. But that's probably not the best approach. The net/http Server has methods for shutting down etc. I don't think we're using them at the moment, but may in the future. And there may be additional http/2 state management in the http2 package that we would miss if handling connections ourselves.
I saw the TODO where NextProtos is set. We should probably clone the TLS config before modifying, and only append to nextprotos, not overwrite it. The config may come from the autotls/ package, with adds the acme alpn nextproto for verification.

What do you think about this plan

Yes, that plan makes sense.

Actually, rather than returning a struct, the IMAP and SMTP listeners could return a function pointer. Then that function needs only to ask for the TLS configuration and the network connection, and it could contain the other arguments configured by the listener as part of its closure. Then the serve() functions wouldn't need to be public. Does that fit into Mox's established coding style and architectural conventions well, or should I avoid that kind of functional-programming-inspired abstraction and make the two functions public?

At first reading, just exporting the serve functions seems more mox-like. But if that has complications, the other option could be better.

I chose to go with the Listen()-returns-a-function approach because it keeps related code close together. Should something need to change about the serve() function, the other caller is in the same file a few dozen lines up. If instead I were to have the Listen()ers return structs, the structs themselves and also the call site over in http/web.go need to change whenever serve() changes, which is more tedious and spaghetti-like.
How does this look?

Are the variable names sufficiently clear?
Does the documentation text explain the feature well?
Is the new warning message too verbose?

Approach looks good. Documentation looks sufficient, and warnings aren't too
verbose (I do usually try to keep them a single sentence, so I can keep it
all lower case, but sometimes that's just not enough). Some styling nits
later on.
Rebasing will cause some trouble (sorry) with the recent addition of TLS
client cert authentication. The imapserver & smtpserver serve() functions now
always expect a plain tcp connection, on which they're doing tls explicitly
(so they can set up a tls.Config that is bound to the "conn" type, for
setting the "account" and "username" fields during authentication in the tls
"verify client" callback). The "tls bool" parameter to the serve() functions
now means "start a tls server". I think we may need another flag "this is
already a tls conn, don't start another tls server".
TLS client cert auth won't work with the https-alpn connections. I don't
think such support has a use case at the moment. The smtpserver/imapserver
and https tls config code could be reworked in the future to also make tls
client cert auth work for https-alpn connections.
Some nits:

The slog attributes are usually all lowercase and a bit shorter. I would
rename "listenerName" to just "listener", and "SubmissionsEnableOnHTTPS" to
"submissionhttps".
log := mlog.New("smtp-alpn", nil), the log name is usually the package
name, so it would be "smtpserver".

It could also be useful to track in logging & metrics that a connection came
in over https. Perhaps an attribute on https-alpn connections. And protocols
like "imaphttps" and "submissionhttps" in the metrics.
It would be useful to add some tests. Maybe in the integration test, enabling
it on one listener, testing the https alpn connection works on that listener
and not on the other. You're welcome to look into that, but I'm OK doing it
too.

Approach looks good. Documentation looks sufficient, and warnings aren't too verbose.

Great!

Rebasing will cause some trouble (sorry) with the recent addition of TLS client cert authentication. I think we may need another flag "this is already a tls conn, don't start another tls server".

I implemented this as a "viaHTTPS" flag, because then that also makes the protocol metrics & logging easier.

The slog attributes are usually all lowercase and a bit shorter. I would
rename "listenerName" to just "listener", and "SubmissionsEnableOnHTTPS" to
"submissionhttps".
log := mlog.New("smtp-alpn", nil), the log name is usually the package
name, so it would be "smtpserver".

I think both of these are fixed!

It could also be useful to track in logging & metrics that a connection came in over https. Perhaps an attribute on https-alpn connections. And protocols like "imaphttps" and "submissionhttps" in the metrics.

Yep! I originally fixed the merge conflict by using reflection to avoid setting up TLS when the connection is already TLS, but the "viaHTTPS" method enabled the logging/metrics to be easy to implement, and also meant I didn't need reflection.

It would be useful to add some tests. Maybe in the integration test, enabling it on one listener, testing the https alpn connection works on that listener and not on the other. You're welcome to look into that, but I'm OK doing it too.

I'm taking a look at this now—I've found the integration tests, but I'm not yet sure I understand how they work. I'll keep poking at it, and let you know if I'm not able to figure it out soon.

I've now figured out why the example integration test didn't make any sense at first: the integration tests are meant to be run in Docker via the makefile, and the code under test is in a separate Docker container from the integration_test executable. That explains a lot, and now I think I have enough pieces to be able to write an integration test for this feature.

Alright, how does the test I added look?

Tests looks good, especially with also testing that the alpn access doesn't work if not enabled.
The request for the mta-sts endpoint isn't entirely working though. http.Get() only returns a non-nil err if writing the request or reading the response fails, not if the status is not 2xx. The response code will be 404 (as I noticed when adding that check to the test), because the url isn't correct, it should be of the form https://mta-sts.<recipientdomain>/.... But the recipient domain is mox1.example, but that domain is also in use with another container.
Anyway, as this looked quite ready to merge, I took another good look, and realized the alpn imap/smtp helper is used for all listener, regardless of in which listener it was configured. I fear this will cause trouble future development, where we would have to change that code and behaviour. So I decided to try to change the https/alpn setup to make it per-listener. Attached is a diff with those changes compare to this latest MR.
It has a few more changes:

In config, rename EnableOnHTTPS to EnabledOnHTTPS (add "d"), to match the current "Enabled" bool field.
Instead of warning that EnabledOnHTTPS is set but there is no https server on port 443, just cause EnabledOnHTTPS to start listening for https on port 443.
In smtpserver, reverted the metric tracking. I pointed to it before, but I think the added logging for new connections is enough after all.
Removed snippet from moxacmepebblealpn.sh for integration test that wrote DNS records that would conflict with the non-alpn container, or aren't needed.

Could you have a look at the diff and check if it looks reasonable?
Are there any more changes we would need before getting this in?
https-alpn-mail.diff.txt

Tests looks good, especially with also testing that the alpn access doesn't work if not enabled. The request for the mta-sts endpoint isn't entirely working though. http.Get() only returns a non-nil err if writing the request or reading the response fails, not if the status is not 2xx. The response code will be 404 (as I noticed when adding that check to the test), because the url isn't correct, it should be of the form https://mta-sts.<recipientdomain>/.... But the recipient domain is mox1.example, but that domain is also in use with another container.

Ah, oops! I think it shouldn't matter for the purpose of the test, since all it really needs to do is get any valid HTTP response (instead of IMAP or SMTP), but I agree that it would definitely be better to have a 200 response from the correct domain.

Anyway, as this looked quite ready to merge, I took another good look, and realized the alpn imap/smtp helper is used for all listener, regardless of in which listener it was configured. I fear this will cause trouble future development, where we would have to change that code and behaviour. So I decided to try to change the https/alpn setup to make it per-listener. Attached is a diff with those changes compare to this latest MR.

I looked through the patch and I think I see what you mean. Thank you for catching that!

It has a few more changes:

In config, rename EnableOnHTTPS to EnabledOnHTTPS (add "d"), to match the current "Enabled" bool field.
Instead of warning that EnabledOnHTTPS is set but there is no https server on port 443, just cause EnabledOnHTTPS to start listening for https on port 443.
In smtpserver, reverted the metric tracking. I pointed to it before, but I think the added logging for new connections is enough after all.
Removed snippet from moxacmepebblealpn.sh for integration test that wrote DNS records that would conflict with the non-alpn container, or aren't needed.

Could you have a look at the diff and check if it looks reasonable? Are there any more changes we would need before getting this in?

It looks reasonable to me! The tests pass, but I haven't managed to successfully test it with a Delta Chat client yet. Part of this is that I forgot I needed to implement autoconfigure, but the main issue is that I can't get Delta Chat to talk to a Mox server which presents a certificate signed by a private certificate authority. After spending the second half of today digging into it, I realized it's because the Delta Chat core library uses the Mozilla CA certificate list exclusively, with no override options. This means I can't test with a private CA without recompiling the Delta Chat client.
Identifying this issue has exhausted me for the day, so I'm going to put it aside and decide how to approach client testing again when I revisit it with fresh eyes tomorrow. In the meantime, I'll push your patch and also the code I wrote to make autoconfigure output the port 443 information.
Also, merry Christmas/happy holidays!

After sleeping on it, I realized that I don't need the test Mox instance to be publicly accessible in order to obtain a certificate for the domain! I've now set up a test environment where I can have Delta Chat Desktop talk to Mox. I have not yet been able to convince Delta Chat Desktop to use the ALPN connection, so I'm working with the Delta Chat folks to figure that part out.
However, I've also discovered that the https-alpn-mail patch you linked above breaks the ability to view Mox webpages in Safari and Firefox. I don't yet understand why this happens, but any attempt results in this error in the Mox logs:
l=info m="https error" err="http: TLS handshake error from <IP_ADDRESS>:57881: tls: client
 requested unsupported application protocols ([h2 http/1.1])" pkg=http pkg=net/http

I've added more logging in http/web.go to verify that all of the HTTP listeners are being configured with h2 in the ALPN list, and I confirmed with Wireshark that the requests from Safari/Firefox are sending two separate protocol names, rather than one long protocol name (h2 and http/1.1 vs. h2 http/1.1).
I'm currently working on testing with Delta Chat in a branch from before I applied your patch (but with cherry-picked autoconfig changes). If you have time and motivation to investigate the h2 ALPN thing, I'd appreciate it! If not, I can figure it out after I work out how to get Delta Chat to use this feature (finally).

If you have time and motivation to investigate the h2 ALPN thing, I'd appreciate it!

I found the problem. I think it still worked when ACME was enable, but not
with manual TLS certs. With ACME enabled, the TLS config got alpn h2 &
http/1.1 early from the ACME TLS config. Without ACME, the call to
http2.ConfigureServer would add h2 & http/1.1 to the NextProtos of the
tlsconfig, but on a clone, while the call to tls.NewListener was already done
on the tlsconfig from before the clone. So the config used by tls.NewListener
didn't get h2 & http/1.1 in NextProtos.
I fixed it by cloning all TLS configs at an earlier moment. Then we don't
have to clone it between tls.NewListener and http2.ConfigureServer.
There is also a small change for setting "kind" "acme-tls-alpn-01", I can't
see how the previous code would do anything. I'm going to commit that snippet
separately.
fix-non-acme-alpn.diff.txt

Okay, I've gotten Delta Chat Desktop to talk to Mox over 443 with ALPN, and it all works!
Thank you for investigating the HTTP2 thing—I'll test that patch now and push everything once I've tested.

Yep, testing shows that it all works now! Thank you for your help!
If you're alright with the code I've added to the autoconfig system, then this is ready to merge :)


Settings Font Issues

Any idea why I'm seeing these weird font (or icon) issues under Settings > Updates, and also under Themes and Packages?

I'm running Atom on Windows 10 and have the correct font already installed.

This issue is also present on Windows 8.1 and Ubuntu 14.04.
Ubuntu 14.04:

Windows 8.1:

So I discovered after opening up developer tools that the problem is not a screwy font, but it is actually the button labels: "Settings", "Uninstall" and "Disable" are not hidden and are overflowing onto the next buttons.
I took a look in the Metro UI package "settings.less" file and found the problem...
.panels {     & > div:not(.package-detail) {       .package-card .btn:not(.install-button) {         background-color: transparent;         max-width: @component-icon-size; // shrink to hide text         padding: 0;         overflow: hidden; // THIS LINE NEEDS TO BE ADDED

I've submitted a pull request with the fix - https://github.com/Cirex/atom-metro-ui/pull/25

Just ran into the same problem and solved it in a similar way.
This will hide the button labels and leave only the icons visible, so that max-width can stay unchanged.
.settings-view .btn-toolbar .btn.icon {
  font-size: 0;
}


Default recipe with arguments that take a URL give the wrong error

Create a Justfile with this as the first recipe:
fetch URL PATH="/tmp":
    curl --output-dir "{{PATH}}" -O {{URL}}

Then run this on the command line:
just https://apple.com/

This is a default recipe with an argument, so it errors:
error: I/O error reading directory /Users/nick/Projects/fetcher/https:/apple.com/: No such file or directory (os error 2)

Instead, @casey says we should get a different (and more helpful) error:
Recipe `fetch` cannot be used as default recipe since it requires at least 1 argument.

Ah, okay, I was super confused in Discord but figured this out. When an argument contains a / or ., just will treat it as a directory that it should start the search for a justfile from. https://apple.com/ contains a /, so just tries to find a justfile starting at https://apple.com/, but that isn't a directory, so there's an I/O error.

This is confusing, but working as intended.

That seems like odd behavior given that the default recipe has a required argument not satisfied by the call.
Also, couldn't Just be updated to recognize URIs. That could be useful in the future and would fix this issue unless the URI is something like file:///path/to/resource of course which could still look for the path but might be better to just leave as a string argument so path arguments could be sent to recipes and not change the Just context.  😃


Source code error

Describe your problem here.
I tried to compile a source of 0.7.1h and got error (exatcly, type not found: SCall) and someone told me to install the experimental branch. I tried that and got this error now:
source/psychlua/CallbackHandler.hx:55: characters 18-33 : Class<llua.Lua_helper> has no field sendErrorsToLua
export/release/windows/haxe/ApplicationMain.hx:153: characters 5-36 : ... Called from macro here
source/psychlua/CallbackHandler.hx:55: characters 41-46 : Class<llua.LuaL> has no field error
export/release/windows/haxe/ApplicationMain.hx:153: characters 5-36 : ... Called from macro here
C:/HaxeToolkit/haxe/lib/flixel-ui/2,5,0/flixel/addons/ui/FlxUICursor.hx:564: characters 9-11 : Warning : Potential typo detected (expected similar values are flixel.addons.ui.SortMethod.ID)
source/psychlua/FunkinLua.hx:280: lines 280-283 : Int should be Bool
source/psychlua/FunkinLua.hx:280: lines 280-283 : ... For function argument 'f'
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No, im using clean experimental version.

You should install linc_lua from git.
Use:
haxelib git linc_luajit https://github.com/superpowers04/linc_luajit.git


Include logback config from Spring Boot.

Include defaults provided by Spring Boot otherwise the logging configuration won't log anything as it has no appenders.

Thanks for the PR! This is now merged into master


Onboard godog test framework with a basic test

What type of PR is this?
/kind enhancement
What does this PR do / why we need it:
On boarding godog test framework for writing e2e tests
Have you updated the necessary documentation?

[ ] Documentation update is required by this PR.
[ ] Documentation has been updated.

Which issue(s) this PR fixes:
Fixes part of https://issues.redhat.com/browse/GITOPS-333
How to test changes / Special notes to the reviewer:
make e2e

Codecov Report

Merging #83 (9e67d63) into master (89c02dc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #83   +/-   ##
=======================================
  Coverage   60.76%   60.76%
=======================================
  Files          52       52
  Lines        2546     2546
=======================================
  Hits         1547     1547
  Misses        850      850
  Partials      149      149

Flag
Coverage Δ

unittests
60.76% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89c02dc...9e67d63. Read the comment docs.

+ make e2e
Log successfully started, logging into: /go/src/github.com/redhat-developer/kam/test/e2e/out/test-results/integration_2020-11-16_17-50-50.log
Running integration test in: /go/src/github.com/redhat-developer/kam/test/e2e/out/test-run
Working directory set to: /go/src/github.com/redhat-developer/kam/test/e2e/out/test-run
Before suite
The bash instance has been started and will be used for testing.
Before feature
Feature: Basic test
  Checks whether KAM top-level commands behave correctly.

  Scenario: KAM version                     # features/basic.feature:4
    When executing "kam version" succeeds   # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails
    Then stderr should be empty             # shell.go:306 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldBeEmpty
    And stdout should contain "kam version" # shell.go:290 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain
After feature
After suite

1 scenarios (1 passed)
3 steps (3 passed)
21.422205ms
ok  	github.com/redhat-developer/kam/test/e2e	0.035s

Details log - https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/redhat-developer_kam/83/pull-ci-redhat-developer-kam-master-v4.5-integration-e2e/1328358083193212928/artifacts/integration-e2e/integration-e2e-steps/container-logs/test.log

Platform issue - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/redhat-developer_kam/83/pull-ci-redhat-developer-kam-master-v4.5-integration-e2e/1329371706321539072
/retest

/lgtm
/approve


dvdtyp:1.0.0

I am submitting

[x] a new package
[ ] an update for a package

Description: A simple colorful template for notes and handouts.

I have read and followed the submission guidelines and, in particular, I

[x] selected a name that isn't the most obvious or canonical name for what the package does
[x] added a typst.toml file with all required keys
[x] added a README.md with documentation for my package
[x] have chosen a license and added a LICENSE file or linked one in my README.md
[x] tested my package locally on my system and it worked
[x] excluded PDFs or README images, if any, but not the LICENSE

[x] ensured that my package is licensed such that users can use and distribute the contents of its template directory without restriction, after modifying them through normal use.

Thank you :)


MSL120DR Not Supported

Smart Bulb Not Supported
Setup: Homebridge running on Raspbian
Homebridge v.1.3.4
HB-Meross v 6.5.1 beta (Issue apparent in older versions)
Model: MSL120DR Refoss Smart Bulb
Logs:
[Meross] [Smart Bulb] is currently not supported, do create a github issue https://git.io/JtfVC with info:
{"uuid":"2103109297840990845448e1e961f16a","onlineStatus":1,"devName":"Smart Bulb","devIconId":"device036","bindTime":1628301054,"deviceType":"msl120dr","subType":"us","channels":[{}],"region":"us","fmwareVersion":"4.2.5","hdwareVersion":"4.0.0","userDevIcon":"","iconType":1,"skillNumber":"","cluster":1,"domain":"mqtt-us.meross.com","reservedDomain":"mqtt-us.meross.com","firmware":"4.2.5","hbDeviceId":"2103109297840990845448e1e961f16a","model":"MSL120DR"}

Hi,
This device should now be supported in  6.5.1-beta.1!
Thanks


info block display fixes

Q
A

Is bugfix?
✔

New feature?
✔/❌

Breaks BC?
✔/❌

Tests pass?
✔/❌

Fixed issues
https://github.com/yiisoft/yii2/issues/18715

moved the content a little so that the block is displayed correctly

updated content, checked everything in this file.
I checked the language in google translator

👍🏻


Can I export an audio file from AKSequencer?

Sorry to ask this question here. I did post it in the stackoverflow, but I don't get some solution until now.
I'm using AKSequencer to play notes in sequence and I'm also applying some filters on the sound, like reverb.
I found an example code here that exports an audio from another audio file, but it is not what I need. Actually, I need to render the notes sequence, including the filters, and export it to the user.
My code:
class Sequencer {

    let oscBank = AKOscillatorBank(waveform: AKTable(AKTableType.sawtooth))
    let sequencer = AKSequencer()

    init() {
        setup()
    }

    func setup() {

        // I instantiate a MIDI node with the oscillator bank to put it in the track output.
        // It makes the oscillator's filters are recognized in the track output.
        let midiNode = AKMIDINode(node: oscBank)
        _ = sequencer.newTrack()
        sequencer.tracks[0].setMIDIOutput(midiNode.midiIn)

        generateSequence()

        // Here I'm applying some filter, a reverb in this case.
        let reverb = AKReverb(oscBank)
        reverb.loadFactoryPreset(.plate)

        // The Audiokit's output may be the last filter applyed. It is working to play.
        AudioKit.output = AKMixer(reverb)
        try?  I want to get the song and export it with the .wav extension exactly as it was played.
    func saveFile() {
        // What should I do here?
    }
}

Can someone please help?

Sorry, I was away on travels for a week, I've responded on StackOverflow.

@aure, I saw your answer in StackOverflow, but what do you suggest we do? This feature is essential in our project.

@aure, Thanks for replying in StackOverflow, but what do you suggest we do? This feature is essential in our project.

??


ECB displays Gadget Replaced even when it creates new.

Gadgets can be imported multiple times for different roles. Thatswhy by default we are displaying Gadget  Replaced. @mitulpanch


ZenMode doesn't change the width of the article when "showTableOfContents" is set to false

Describe the bug
ZenMode doesn't change the width of the article when "showTableOfContents" is set to false.
To Reproduce
Steps to reproduce the behavior:

Set showTabeleOfContents to false in front matter
Click on ZenMode
Author info disappears, but the width remains unchanged

Expected behavior
Article width should expand the same way as it does when showTableOfContents is set to true
And more of a feature request: would also be possible to have a param for ZenMode in the front matter to have it as the default. The new width is really great and for some articles it would make sense to have it as the default.
Finally, can ZenMode right now only be activated/deactivated at the top of the article? I was wondering whether it would be possible to have it activate/deactivate anywhere (like scroll to top) as it would allow to use it without interrupting the reading flow.

@gibru yes please :)
Also tag @v20100v in these discussions since he was the original creator of the feature

Hello @gibru,

So I've played some more with ZenMode and I guess the way it works right now is intentional

That's right ! That was my first intention, but maybe you have a better idea ? If you want to open a new discussion, it's with pleasure ^^.


add multiple device should cache login info in memory

Device type and model (or SN)
none
The description of new feature
in this chanages:
https://github.com/wuwentao/midea_ac_lan/commit/1f56e69b7ebcabdf634a8eb14c44f44dec4470e7
we have removed write username/password to local json file, and not save username/password in local disk.
it caused a small issue:
if user have more than 5-10+ device to be add, they should add each device and input login username/password every time.
we can cache it in memory and not write to file, can be done in future

I investigate with the core Team and this is not allowed at the time writing.

I investigate with the core Team and this is not allowed at the time writing.

understand, username/password is sensitive data, HA core may not allow to cache it.
we can delay it, not a urgent requirement.


Update sql-database-metrics-diag-logging.md

One line change to the sample code for better clarity.

#sign-off

@danimir : Thanks for your contribution to the Azure documentation! The author, @veljko-msft, has been notified to review your proposed change.

#sign-off


Added sleep support to presets

Added simple sleep support to presets.

Useful addition!
Could you please add documentation in the readme and the example preset as well?

I'd be happy to.  I dont really see a presets.json in the http-api and the README at this level is pretty sparse.  Do you want me to make a PR against http-api?   I have one planned there anyway to return the list of presets like favorites has.

Yeah, sorry, confused it with the http-api. It's better to stick the documentation in there since most people use that library.


Cannot access API of Foxx Service due to Auth Problem

ArangoDb 3.4 RC 1 - Windows
After uploading Foxx Service, the API/Swagger is not accessible.

@vinaypyati @lservini Can you check this in latest RC?

Hi @lanluu,
Did you have a chance to test if this issue also occurs in 3.4.0 RC2 ? I tried reproducing this issue on 3.4.0 RC1 and unfortunately, it is not reproducible. API of Foxx services are accessible.  Can you please confirm the following so that we can reproduce on our side too?

Are you using Single Server?
How are you creating a service? Is it via the Store, Upload or New UI?

Thanks!

Hi @lanluu,
We haven't received feedback from you for a while, I am closing this issue for now.
Please consider testing this issue on 3.4.0 RC3. You can download it from the download section. If it does not resolve your problem, please feel free to reopen the issue by adding a new comment.
Thanks!


Flicking when rendering with Painter

Hey I'm using a Painter for a very simple game. It updates the window with a refresh loop:
spawn(move || loop {
  handle.submit_command(Selector::NOOP, (), Target::Auto);
  std::thread::sleep(Duration::from_millis(1000 / FPS));
});

But it flickers when I run it. I've tried setting FPS to 30 and 60.

The painting code is the following (abridged). Is there a setting I need to enable? Using Druid 0.7.0 on MacOS Monterey 12.6
    let my_painter = move || {
        Painter::new(move |ctx, data: &MyData, _| {
            let bounds = ctx.size().to_rect();
            let (x0, y0, x1, y1) = (bounds.x0, bounds.y0, bounds.x1, bounds.y1);

            // ... other stuff

            for cell in canvas_state.cells.iter() {
                let radius = (cell.size as f32 / PI).sqrt();
                let ratio = (x1 - x0) / WIDTH as f64;

                let radius_scaled = radius as f64 * ratio;
                let (cell_x, cell_y) = cell.position;
                let x_scaled = cell_x as f64 * ratio;
                let y_scaled = cell_y as f64 * ratio;
                // first scale down the size
                // then find the position in the bounds
                let circle = Circle::new(
                    Point {
                        x: x_scaled,
                        y: y_scaled,
                    },
                    radius_scaled,
                );
                let background = Rect::new(x0, y0, x1, y1);
                ctx.fill(background, &Color::Rgba32(0xFFFFFF));
                ctx.fill(circle, &Color::Rgba32(0xABCDEF));
            }

Just tried it out by changing my Cargo.toml:
-druid = "0.7.0"
+druid = { git = "https://github.com/linebender/druid" }

Still flickering. I wonder if I should change the way I do it and define a Widget like the Game of Life example.

Certainly- let me create the repo and you'll be able to run it.

@xStrom https://github.com/synlestidae/cello
Changes frozen in branch bug/flickering

It is supposed to be animated. I am hacking together a clone of agar.io for fun


Not working

downloading instagram data from google drive... http status: 404 Not Found
exit status 1
Error: Process completed with exit code 1.

This indicates there is something wrong with your fork setup.
Please, check your fork has a secret called FILE_ID with a value similar to dHqYnLKcM7P18cBhgmWVfUFdjnMXoKOYg and try again.


Can't programatically add multiple subdomains in powershell

Having issues adding multiple custom domains to webapps via code. We are using powershell to automate adding customers to a multitenant platform so need to be able to add mutliple custom domains.
When I run
Set-AzureRmWebApp -Name $azwebapp -ResourceGroupName $RGName -HostNames @("custom1.domain.com","custom2.domain.com")
it works.
However, as the process will be automated I need to be able to inspect what is there first, and simply add to it. I do this by collecting the custom domains first using:
$webhosts = (Get-AzureRmWebApp -ResourceGroupName $RGName -Name $azwebapp).Hostnames
I then add my new domain to it
$webhosts.Add("custom3.domain.com")
But when I pass that back to the Set-AzureRmWebApp command it fails:
Set-AzureRmWebApp -Name $azwebapp -ResourceGroupName $RGName -HostNames $webhosts
I've also tried passing it as an array, also does not work.
Set-AzureRmWebApp -Name $azwebapp -ResourceGroupName $RGName -HostNames @($addweb)
Looking at $webhosts I notice that it does not list the quotes, so I manipulate the string to add them:
$addweb = New-Object System.Collections.Generic.List[string]
$webhosts = (Get-AzureRmWebApp -ResourceGroupName $RGName -Name $azwebapp).Hostnames
$w = $null
$wo = $null
foreach ($w in $webhosts){
$w = $w.insert(0,'"')
$w += '"'
$addweb.add($w)
$wo += "$w `n"
}
I then add my new domain
$webapp = '"custom1.domain.com"'
$webapp2 = '"custom2.domain.com"'
$addweb.Add($webapp)
Then clean up to remove extra commas and line breaks
$weburls = ($addweb -join ",").TrimEnd(",")
Then pass this to the function
Set-AzureRmWebApp -Name $azwebapp -ResourceGroupName $RGName -HostNames $weburls
But it fails. I've tried many different approaches with none able to succeed (unless you manually specify -Hostnames @("hostname1","hostname2"). Why is this? is there a way around this?
WARNING: 15:21:58 - Could not set custom hostname '"webapptest344354.azurewebsites.net","custom1.domain.com","custom2.domain.com"'. Details: Microsoft.Rest.Azure.CloudException: The host name "webapptest344354.azurewebsites.net","custom1.domain.com","custom2.domain.com" is invalid.
at Microsoft.Azure.Management.WebSites.WebAppsOperations.<CreateOrUpdateHostNameBindingWithHttpMessagesAsync>d__73.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.WebSites.WebAppsOperationsExtensions.<CreateOrUpdateHostNameBindingAsync>d__137.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Commands.WebApps.Utilities.WebsitesClient.AddCustomHostNames(String resourceGroupName, String location, String webAppName, String[] hostNames)
WARNING: 15:21:59 - Could not remove custom hostname 'webapptest344354.azurewebsites.net'. Details: Microsoft.Rest.Azure.CloudException: The host names for the site must include the default hostname with dns suffix 'azurewebsi
tes.net'. Currently specified hostnames are: 'None'.
at Microsoft.Azure.Management.WebSites.WebAppsOperations.<DeleteHostNameBindingWithHttpMessagesAsync>d__74.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.WebSites.WebAppsOperationsExtensions.<DeleteHostNameBindingAsync>d__139.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Commands.WebApps.Utilities.WebsitesClient.AddCustomHostNames(String resourceGroupName, String location, String webAppName, String[] hostNames)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 875a7c86-1590-59b9-5aa0-55cff7c0f1e0
Version Independent ID: 60f04963-f7ad-f4b7-f779-f97ee98afc98
Content: Azure PowerShell Script Sample - Assign a custom domain to a web app
Content Source: articles/app-service/scripts/app-service-powershell-configure-custom-domain.md
Service: app-service-web
GitHub Login: @cephalin
Microsoft Alias: cephalin

btw, domain names above is sanitised for obvious reasons, and CNAMEs exist for domains I am trying to add as well.

@CyanMass45 Thanks for the feedback! We are currently investigating and will update you shortly.

i've opened a ms case as I am getting The host names for the site must include the default hostname with dns suffix 'azurewebsites.net'. Currently specified hostnames are: 'None' for restoring a webapp backup. I swear this was working last week. If it is anything relevant to this issue, i'll post the notes from the case

@CyanMass45 I would suggest you consider using CLI, which has a cmdlet to list all hostnames by resource group, web app, or other parameters.
az webapp config hostname list --resource-group                                --webapp-name                                [--slot]
Source: https://docs.microsoft.com/en-us/cli/azure/webapp/config/hostname?view=azure-cli-latest#az-webapp-config-hostname-list

@BryanTrach-MSFT  But I do not want to list the host names. I want to add additional hostnames, without it removing existing hostnames. I do not see how your answer is even remotely related to my issue, let alone helpful.

@CyanMass45 Your question goes beyond the intent of the article. We kindly request that you share your question on MSDN Forums or Stack Overflow.

I am having this exact problem. I cannot ADD custom domains to an existing Azure Web App without it wiping out all the existing hostnames. I have tried the suggestions above and still receive errors.

i think this is addressed in this link: https://stackoverflow.com/questions/45509231/remove-custom-domain-host-from-app-service-via-powershell/45513832#45513832
You just need to use the same variable to add additional host name rather than inputting through different variable which may have data type issue.


Parent filename could be null for resources in jar

This should fix #1170
to reproduce use datafaker-gen, especially this PR this https://github.com/datafaker-net/datafaker-gen/pull/10
to check that it is fixed use same PR and run it with (assuming this fixed is built and installed in local maven repo)
./mvnw clean install -Ddatafaker.version=2.2.2-SNAPSHOT

Hi @snuyanzin , what's the root cause for this? Any chance we could have a unit test for this?

yep, that's the idea of this draft PR https://github.com/datafaker-net/datafaker/pull/1171
Unfortunately haven't succeed yet...
The idea is first to build datafaker and then build datafker-gen as a downstream project with datafaker E2E tests
the root cause: when resource files are in jar  it might happen that the parent for some of them is just / and that means the parentname is null... at the same time it is not the case for datafaker code itself, only for downstream projects


feat: replace Insets expressions with equivalent PaddingValues.Absolute expressions

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#65  👈 (View in Graphite)
#64
#63
#62
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


G_38T61EHXo plays in browser, HTTP error 503 on youtube-dl with no special options

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.08.28. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2018.08.28

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones
[x] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'G_38T61EHXo', u'-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.08.28
[debug] Python version 2.7.15 (CPython) - Darwin-17.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.4.1, ffprobe 3.4.1
[debug] Proxy map: {}
[youtube] G_38T61EHXo: Downloading webpage
[youtube] G_38T61EHXo: Downloading video info webpage
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on u'https://r5---sn-n4v7knlz.googlevideo.com/videoplayback?requiressl=yes&keepalive=yes&sparams=aitags%2Cclen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&ipbits=0&source=youtube&mv=m&ms=au%2Crdu&fvip=5&key=yt6&ip=<IP_ADDRESS>&lmt=1378570731690611&mm=31%2C29&mn=sn-n4v7knlz%2Csn-n4v7sn7l&dur=78.845&id=o-ACR32F-NkXKd_ZKn3WKv4EB3-_vqvAen2fREKeuIAlpz&gir=yes&mt=1535759431&clen=25218720&expire=1535781133&ei=rdSJW6PuAZrOkga3wLeICQ&itag=137&pl=21&c=WEB&initcwndbps=1185000&signature=3F3C8946EF734793D156E1BF5962BA3DFDC0D933.A3E89E3328677F7E56368EBA60DD78EC2E3433CA&mime=video%2Fmp4&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278&ratebypass=yes'
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 1 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 2 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 3 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 4 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 5 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 6 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 7 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 8 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 9 of 10)...
[download] Got server HTTP error: HTTP Error 503: Service Unavailable. Retrying (attempt 10 of 10)...
ERROR: giving up after 10 retries
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 182, in establish_connection
    raise RetryDownload(err)
RetryDownload

Description of your issue, suggested solution and other information
It doesn't download. Every other video from his channels downloads fine, except this one.

bestvideo format is not available youtube side.

I'm sorry, what does that mean? Is there no way to download this video? @dstftw


Added figure to cvpr-21

👷 Deploy Preview for gladia processing.

Name
Link

🔨 Latest commit
1077286c4fad3e52312650ccaea165b8585ab9b0

🔍 Latest deploy log
https://app.netlify.com/sites/gladia/deploys/642ebc522c9d040008c489b4


code format option for ngx_http_upstream_check_module.c

code format option for ngx_http_upstream_check_module.c

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Clientset generation broken after 0.0.7

I am trying to generate a clientset for my CRD so that I can use it outside the operator. Is there a way to do this with the current version of the SDK?
I used this hack previously on 0.0.7:
vendor/k8s.io/code-generator/generate-groups.sh all github.com/user/memcached-operator/pkg/generated github.com/user/memcached-operator/pkg/apis memcached:v1alpha1

This however seems to have broken on master (ab09a3d). The problem seems to be that the generated client expects to have the AddToScheme function in the api imported from: github.com/user/memcached-operator/pkg/apis/memcached/v1alpha1
However, in the latest version of the operator-sdk, it's one package up in apis.go:
└── memcached
    ├── addtoscheme_memcached_v1alpha1.go
    ├── apis.go
    └── v1alpha1
        ├── doc.go
        ├── register.go
        ├── memcached_types.go
        └── zz_generated.deepcopy.go

Am I missing something here? Or should the function be somehow moved back into v1alpha1?
Thanks!
Environment

operator-sdk version: ab09a3d

@benceszikora Thanks for opening this. The operator-sdk has gone through API changes, have you seen the migration-guide? Hopefully following that it should make things more clear, if not feel free to come back to the issue and we can answer any further questions!

Worth mentioning that in order for the client to be generated at all, the  // +genclient tag has to be added here (directly above the type wrapped in the API).

@benceszikora Since the SDK only supports deepcopy-gen all the other generators probably won't work on pkg/apis for 0.1.0. We could add support for those if it makes sense.
In the meanwhile you can still hack around it by adding the // +genclient tag in your memcached_types.go.
// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Memcached is the Schema for the memcacheds API
// +k8s:openapi-gen=true
type Memcached struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   MemcachedSpec   `json:"spec,omitempty"`
	Status MemcachedStatus `json:"status,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// MemcachedList contains a list of Memcached
type MemcachedList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []Memcached `json:"items"`
}

And as you pointed out we're missing AddToScheme in register.go. So you'll need to add that in:
AddToScheme = SchemeBuilder.AddToScheme

And then try:
vendor/k8s.io/code-generator/generate-groups.sh all github.com/user/memcached-operator/pkg/generated github.com/user/memcached-operator/pkg/apis memcached:v1alpha1

Other generators like lister-gen, defaulter-gen might need other tags and methods to be present. We'll look into supporting all generator types by default.

@hasbro17 I was running into the same issue and tried your fix. I was able to generate the client set, but some bits of it still looks incorrect to me. For eg: https://github.com/indradhanush/debug-operator/blob/7ad260664fcb9626dc7a764e9974d7abab8b248e/pkg/client/clientset/versioned/typed/foo/v1alpha1/debuglist.go#L43
I created the above repo to easily reproduce the issue. Am I missing something?

My apologies. It appears that I had the tags placed above the wrong struct. This commit seems to have fixed it for me: https://github.com/indradhanush/debug-operator/commit/0cccb2a426ffdf336815f308d6ce237a55f5eda3

I think this can be closed as it's outdated.

@LiliC What is the best practice for generating a golang client for CRDs produced with the operatorSDK?? It appears no client is automatically generated at this point.

@wreed4 The SDK will most likely not support generating typed clients(nor listers,informers) for your CRD since the expectation is to use the controller-runtime's dynamic client to interact with your CRD objects.
https://github.com/operator-framework/operator-sdk/blob/master/doc/user/client.md
If you do have a use case that can't be fulfilled by the dynamic client then please feel free to follow up at #1646.
In the meanwhile if you do need to generate a client-set for some reason then you can use the upstream k8s code-generator. There's a generate-groups.sh helper script that you can run against your pkg/apis/... to generate client/lister/informer for your CRD.

@hasbro17 I should have been more explicit I think, our use-case is to interact with these CRDs from various other repositories/microservices that are not themselves operators.  Is requiring the controller-runtime as a dependency still the best-practice you suggest?
Put slightly differently, we have developed an operator and now we are developing several microservices that are themselves clients of that operator interacting with the CRs it manages (CRUD operations).


Removed the dependency of CP4D Operator

This modules creates the 2 version of common services which needs to be removed
This modules needs to be cleaned up for the dependency in the modules.yaml
This modules needs to be updated for the notifications module in the notify.yaml

updated all the modules and verified the build is successful


ncdump can generate nul characters &#0 ; when dumping ncml using -x.

Re e-support TSE-683024
If a char valued attribute contains embedded nul characters,
then illegal xml will be generated when using the -x (ncml) flag.

Merged, closing out.

Solution:
There is not good solution since we have a char value '\0' that
is legal in a netcdf-c char valued attribute, but is completely
illegal in ncml (i.e. xml).
So, implemented hack is to go ahead and generate '&#0 ;' entities
and then complain that we are generating illegal ncml.


Sortable Table

it would be interesting to be able to sort the tables in the different views (projects, components, vulnerabilities ...)

The majority of tables can now be sorted. Will be included in 3.1.


test by tanbo

hahahaha

Please don't test on public repos!  Not helpful to anyone I'm afraid.


Compiling Issue 0.6.3

Describe your bug here.
Just got back from a 6 month hiatus, resetted my PC, and now I dont even know whats going on. I downloaded everything im pretty sure and I get this:

Would love help.
Command Prompt/Terminal logs (if existing)
No response
Are you modding a build from source or with Lua?
Source
What is your build target?
Windows
Did you edit anything in this build? If so, mention or summarize your changes.
Nothing

use flixel addons 2.11.0


HmIP-MOD-TM "closing" and "opening" states inverted.

Garage door controller HmIP-MOD-TM is showing state "opening" while closing and "closing" while opening.
Fully closed and fully open correctly reported as "closed" and "open".
Everything else is working correctly.

DON'T DELETE THIS

The issue template seems to be removed, please restore our default issue template in the issue description and fill it out.

DONT'T DELETE THIS.
This is for issues only
If you have a feature request, add it as a new discussion, or add your point of view to an existing discussion.
Please answer all questions as good as possible.
Language: as you like -> english
Make sure you have read the documentation at https://github.com/danielperna84/custom_homematic#custom_homematic.
Make sure you have read the release notes at https://github.com/danielperna84/custom_homematic/releases.
Make sure you are running the latest version of the custom_component (and Home Assistant) before reporting an issue.
ENSURE that your Backend (CCU/Homegear/...) is working as expected and connected devices are controllable.
custom_component version (if applicable):
1.27.0
known last working custom_component version (if applicable):
N/A
Home Assistant:

Version: 2023.1.6
Installation method: HAOS

CCU:

Model Raspi3
OS RaspberryMatic
Version:  <IP_ADDRESS>30114

Problem-relevant configuration:
Do you use tls? yes
Do you use callback? no
Do you use username and password? yes
Which interfaces do you use (hmip/bidcos/wired)? hmip
Describe the bug
Garage door controller HmIP-MOD-TM is showing state "opening" while closing and "closing" while opening.
To Reproduce

Go to the entity in the developer tools of ha.
Click on the entity
Click on open
State is "closing" while the door is opening.
State turns to "open" when fully opened.

Screenshots (if applicable)

CCU firewall in CCU
device config in CCU
integrations view in HA
device view in HA
enity view in HA

Relevant HA log entries (press full log) VERY IMPORTANT
I had set logging to debug for the integration, individual messages for the state changes not shown in logs.
2023-01-22 14:53:38.617 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 Duty Cycle
2023-01-22 14:53:38.642 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503
2023-01-22 14:53:38.643 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 Rssi Device
2023-01-22 14:53:38.643 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 Operating Voltage
2023-01-22 14:53:38.643 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 Rssi Peer
2023-01-22 14:53:38.643 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 Door State
2023-01-22 14:53:38.644 DEBUG (MainThread) [custom_components.homematicip_local.generic_entity] init: Setting up HmIP-MOD-TM 001E5E12FAF503 State

Diagnostic Information (available on every device)
{
  "home_assistant": {
    "installation_type": "Home Assistant OS",
    "version": "2023.1.6",
    "dev": false,
    "hassio": true,
    "virtualenv": false,
    "python_version": "3.10.7",
    "docker": true,
    "arch": "x86_64",
    "timezone": "Europe/Madrid",
    "os_name": "Linux",
    "os_version": "5.15.80",
    "supervisor": "2022.12.1",
    "host_os": "Home Assistant OS 9.4",
    "docker_version": "20.10.19",
    "chassis": "vm",
    "run_as_root": true
  },
  "custom_components": {
    "dahua": {
      "version": "0.5.0",
      "requirements": []
    },
    "shopping_list_with_grocy": {
      "version": "0.0.1",
      "requirements": []
    },
    "tesla_custom": {
      "version": "3.9.1",
      "requirements": [
        "teslajsonpy==3.7.1"
      ]
    },
    "meross_cloud": {
      "version": "1.2.8",
      "requirements": [
        "meross_iot==<IP_ADDRESS>"
      ]
    },
    "grocy": {
      "version": "v4.5.2",
      "requirements": [
        "pygrocy==1.4.1"
      ]
    },
    "ics_calendar": {
      "version": "3.1.6",
      "requirements": [
        "ics>=0.7.2",
        "recurring_ical_events>=1.0.2b0",
        "icalendar>=4.1.0"
      ]
    },
    "shopping_list": {
      "version": "0.7",
      "requirements": []
    },
    "homematicip_local": {
      "version": "1.27.0",
      "requirements": [
        "hahomematic==2023.1.5"
      ]
    },
    "electrolux_status": {
      "version": "1.2.0",
      "requirements": [
        "pyelectroluxconnect==0.3.12"
      ]
    },
    "localtuya": {
      "version": "5.0.0",
      "requirements": []
    },
    "dash_cast": {
      "version": "1.0.0",
      "requirements": []
    },
    "smart_thermostat": {
      "version": "2021.11.1",
      "requirements": []
    },
    "hacs": {
      "version": "1.29.1",
      "requirements": [
        "aiogithubapi>=22.10.1"
      ]
    },
    "browser_mod": {
      "version": "2.2.0",
      "requirements": []
    },
    "dwains_dashboard": {
      "version": "3.3.0",
      "requirements": []
    },
    "thermal_comfort": {
      "version": "2.0.1",
      "requirements": []
    },
    "smartir": {
      "version": "1.17.6",
      "requirements": [
        "aiofiles==0.6.0"
      ]
    },
    "frigate": {
      "version": "3.0.0",
      "requirements": []
    },
    "bodymiscale": {
      "version": "3.0.6",
      "requirements": [
        "cachetools==5.1.0"
      ]
    },
    "platerecognizer": {
      "version": "1.0.0",
      "requirements": [
        "pillow",
        "requests"
      ]
    }
  },
  "integration_manifest": {
    "domain": "homematicip_local",
    "name": "Homematic(IP) Local",
    "config_flow": true,
    "documentation": "https://github.com/danielperna84/custom_homematic",
    "issue_tracker": "https://github.com/danielperna84/hahomematic/issues",
    "requirements": [
      "hahomematic==2023.1.5"
    ],
    "ssdp": [
      {
        "manufacturer": "EQ3",
        "manufacturerURL": "http://www.homematic.com"
      }
    ],
    "zeroconf": [],
    "homekit": {},
    "dependencies": [],
    "codeowners": [
      "@danielperna84",
      "@SukramJ"
    ],
    "iot_class": "local_push",
    "loggers": [
      "hahomematic"
    ],
    "version": "1.27.0",
    "integration_type": "hub",
    "is_built_in": false
  },
  "data": {
    "config": {
      "entry_id": "19932e8eccc15b06176f332eaaa4e0ed",
      "version": 1,
      "domain": "homematicip_local",
      "title": "RaspberryMatic",
      "data": {
        "instance_name": "RaspberryMatic",
        "host": "<IP_ADDRESS>",
        "username": "**REDACTED**",
        "password": "**REDACTED**",
        "tls": true,
        "verify_tls": false,
        "callback_host": null,
        "callback_port": null,
        "json_port": null,
        "interface": {
          "HmIP-RF": {
            "port": 42010
          },
          "BidCos-RF": {
            "port": 42001
          }
        }
      },
      "options": {},
      "pref_disable_new_entities": false,
      "pref_disable_polling": false,
      "source": "user",
      "unique_id": "DBE997517F",
      "disabled_by": null
    },
    "platform_stats": {
      "cover": 1,
      "switch": 1
    },
    "devices": [
      "HmIP-MOD-TM"
    ]
  }
}

Expected behavior
State to be "opening" while opening and "closing" while closing.
Additional context
Add any other context about the problem here.

We have no device definitions for HmIP-MOD-TM in pydevccu.
Please execute the export service and attach the files as zip here.

HmIP-MOD-TM.zip


[Feature request] Support for git worktree

Git repository can have multiple working trees, allowing you to check out more than one branch at a time.
https://git-scm.com/docs/git-worktree
Currently, gtm can't be used in linked working trees.
(ins) /c/work/feature gtm init
Unable to intialize Git Time Metric, Git repository not found in

gtm record just don't do anything.

I think the issue may be the version of libgit2 gtm is currently using.  Looks like they have recently added support for work trees.  I'll see if I can upgrade to the latest libgit2.  https://github.com/libgit2/libgit2/pull/3436

Currently waiting for git2go to update to a newer version of libgit2.  There's some updates in libgit2 for the support of git worktree.


Crash on start

I'm sending you the crash report to help you find the source of the problem, but I don't think I can help you any further.
https://mclo.gs/nJuDYjS

Please update EssentialClient to the latest version

I'm on the latest version

I'm playing in 1.21 and version 1.5.2 is the latest version available

All 1.21.1 mods will work on 1.21
I forgot to mark it as being compatible
https://modrinth.com/mod/essentialclient/version/2.0.0-beta.2+1.21.1

I tried with beta versions 1.21.1 and 1.21.3 of the mods but it doesn't change anything. I still have the same crash as the release version 1.5.2.

Can you please provide the crash log for the 1.21.1 version.
And can you confirm this crash happens without EssentialClient installed?

I can state with certainty that the crash does not occur without EssentialClient installed
https://mclo.gs/hQH14Up

You have the wrong version installed, please install the 1.21.1 version and send the crash log

I wasn't paying attention but I was installing it via my modpack made on the modrinth application and it was installing the latest version of the project despite the fact that I selected version 1.21.1 but I've bypassed this problem and I no longer have the crash, thank you.