added molecule entity type and updated references

added molecule entity in entity-type and folder-item-type enums
added mappings to convert entity type to api entity type, and vice versa
added mixins class, copied common columns from every other class, and added canonical_smiles column
updated generate_files in entity schemas to reflect these updates
removed #type:ignore comment in base_model to pass pre_commit checks

In that specific file, It looks like i added the molecule entity to those parts. I can't seem to find any other line where I didn't connect it


Dockerized Openspiel

I've been working on putting OpenSpiel in Docker container (since it always greatly boosts framework adoption). So far I can get the tests to run, but there are still some tests prone to failure. Is there anyone else with some Docker experience that spots the problem in the Dockerfile.
Dockerfile Reference: https://github.com/yarncraft/DockerizedOpenSpiel
Thanks in advance!

Hi @yarncraft , this is really cool!
Looks like all the python tests are failing, so it's probably the case that the Python API (pybind11) is not building properly.
We've seen this a few times. It's usually been one of these:

python3-dev is not installed.
python2 and python3 are being mixed
The python environment variables passed to CMake are not right
There are several installations of python (i.e. one via Conda and another system) and the wrong one is being found running CMake (or when building / linking)

I've never used Docker. Can you tell me what OS you are using and its version?

I've never used Docker. Can you tell me what OS you are using and its version?

Haha, clicked on literally the only link in your message and saw Ubuntu 18.04. Perfect. It's been fairly straight-forward to install Ubuntu 18.04. My guess is that you don't have python3-dev installed. Did you run install.sh to get all the dependencies?

LOL I just saw now the second line of your Dockerfile installs python3-dev. Looking at the file, seems fine.
Hmm... do you know if you can import pyspiel from python3 in the Docker instance? Before doing build_and_run_tests.sh, can you try:
python3
Python 3.7.6 (default, Feb  4 2020, 17:04:58)
[Clang 11.0.0 (clang-1<IP_ADDRESS>)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyspiel

Does this cause an error?
Before doing this , you'll need to set the PYTHONPATH environment variables, see Step 4 of https://github.com/deepmind/open_spiel/blob/master/docs/install.md

According to step 4 in the docs I read the following:
To be able to import the Python code (both the C++ binding pyspiel and the rest) from any location, you will need to add to your PYTHONPATH the root directory and the open_spiel directory.
I think this is not needed when you put the framework in the container as you would use Docker containers exactly to avoid undertaking such steps. (Since you would just run your python scripts in the containerized environment instead).
However there is a note in step 2 stating the following
Install pip deps as your user. Do not use the system's pip.
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --user
pip3 install --upgrade pip --user
pip3 install --upgrade setuptools testresources --user

So maybe I can try to install pip in this way and check if this resolves the problem. Anyway it seems quite unlikely :)

I think this is not needed when you put the framework in the container as you would use Docker containers exactly to avoid undertaking such steps. (Since you would just run your python scripts in the containerized environment instead).

Right, but currently the python tests are failing within the environment and when you're using build_and_run_tests.sh which sets up the Python paths for you as necessary. I was asking to try doing the import pyspiel from within the container to see if it was built properly. If the tests are failing, something is going wrong in the container, so we have to diagnose it. I mainly want to know if it's how we're running the Python tests from CMake or if you can't load pyspiel at all (from within the container).
I've never used Docker so I have no idea how to debug these things.

However there is a note in step 2 stating the following
Install pip deps as your user. Do not use the system's pip.
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py --user
pip3 install --upgrade pip --user
pip3 install --upgrade setuptools testresources --user

This is mostly a recommendation so that users don't mess up their system pip configuration. I doubt this will be it, but one more thing to try.

Ok I'll make sure to check out if I can run a python script whereby I import pyspiel in my containerized environment!
Note that not all tests fail (in fact 57% passes) however the tests that do fail seem to be quite random, therefore it is indeed hard to debug what exactly goes wrong. I'll do a full report on what could be the cause somewhere in the next 24 hours.

Note that not all tests fail (in fact 57% passes) however the tests that do fail seem to be quite random, therefore it is indeed hard to debug what exactly goes wrong. I'll do a full report on what could be the cause somewhere in the next 24 hours.

Right, looks like the C++ tests are passing, so OpenSpiel is building and running properly. From your screenshot it seems to be only the python tests, and we've seen this before so I think it's just that the Python API is not getting build or linked properly (or the CMake tests are not building the python tests properly).

Thanks for taking the initiative to do this!
I needed something similar a few months ago. I am no docker expert, so this
is certainly not optimal, but the following worked for me.
This was part of a larger effort; I've removed some things I know to be
irrelevant, but other unneeded items may still be there.
FROM ubuntu:20.04
RUN apt update
RUN dpkg --add-architecture i386 && apt update
RUN apt-get -y install
clang
curl
cmake
git
python3
python3-dev
python3-pip
python3-setuptools
python3-wheel
sudo
RUN git clone -b 'master' --single-branch --depth 15
https://github.com/deepmind/open_spiel.git open_spiel
WORKDIR open_spiel
RUN ./install.sh
RUN mkdir -p build &&
cd build &&
cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=which clang++ ../open_spiel &&
make -j4
RUN pip3 install absl-py scipy
COPY . build
CMD /open_spiel/build/run.sh

@elkhrt Thanks for sharing, to make your Dockerfile future proof you might consider adding a python upgrade step. I do observe that you don't make use of a virtualenv, this might not be needed when you containerize indeed.
I will give this Dockerfile a try in a minute!

@elkhrt It seems like the Dockerfile is working correctly for the TicTacToe example. It does not run tests explicitly however, does it check these internally when executing the last CMD or is it something that needs to be added as well?

Yours is far better / future proof / general. And actually runs the tests,
unlike mine. I was just sharing it in case it helped you find what was
missing in yours.
On Tue, 3 Mar 2020, 10:27 Lucas Engels<EMAIL_ADDRESS>wrote:

@elkhrt https://github.com/elkhrt It seems like the Dockerfile is
working correctly for the TicTacToe example. It does not run tests
explicitly however, does it check these internally when executing the last
CMD or is it something that needs to be added as well?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/deepmind/open_spiel/issues/156?email_source=notifications&email_token=AHAF7TG66Z55FYWBF6TMTLDRFTLPNA5CNFSM4K7X3KOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENS5RWY#issuecomment-593877211,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHAF7TE63SV33LTKELG4EE3RFTLPNANCNFSM4K7X3KOA
.

Ok I will make sure to enhance the Dockerfile with some extra obligatory steps. I think we can close this issue, I will come back with an update once I get everything up and running with the enhanced version!
Thanks @lanctot @elkhrt  for the help!

56% tests passed, 58 tests failed out of 133

Total Test time (real) = 269.80 sec

The following tests FAILED:
	 66 - python_api_test (Failed)
	 67 - python_playthrough_test (Failed)
	 68 - python_action_value_test (Failed)
	 69 - python_action_value_vs_best_response_test (Failed)
	 70 - python_best_response_test (Failed)
	 71 - python_cfr_br_test (Failed)
	 72 - python_cfr_test (Failed)
	 73 - python_deep_cfr_test (Failed)
	 74 - python_discounted_cfr_test (Failed)
	 75 - python_dqn_test (Failed)
	 76 - python_eva_test (Failed)
	 77 - python_evaluate_bots_test (Failed)
	 78 - python_expected_game_score_test (Failed)
	 79 - python_exploitability_descent_test (Failed)
	 80 - python_exploitability_test (Failed)
	 81 - python_fictitious_play_test (Failed)
	 82 - python_generate_playthrough_test (Failed)
	 83 - python_get_all_states_test (Failed)
	 84 - python_rl_losses_test (Failed)
	 85 - python_lp_solver_test (Failed)
	 86 - python_masked_softmax_test (Failed)
	 87 - python_mcts_test (Failed)
	 88 - python_minimax_test (Failed)
	 89 - python_neurd_test (Failed)
	 90 - python_nfsp_test (Failed)
	 91 - python_outcome_sampling_mccfr_test (Failed)
	 92 - python_policy_aggregator_joint_test (Failed)
	 93 - python_policy_aggregator_test (Failed)
	 94 - python_policy_gradient_test (Failed)
	 95 - python_projected_replicator_dynamics_test (Failed)
	 96 - python_generalized_psro_test (Failed)
	 97 - python_rectified_nash_response_test (Failed)
	 98 - python_random_agent_test (Failed)
	 99 - python_rcfr_test (Failed)
	100 - python_sequence_form_lp_test (Failed)
	101 - python_value_iteration_test (Failed)
	102 - python_bluechip_bridge_uncontested_bidding_test (Failed)
	103 - python_uniform_random_test (Failed)
	104 - python_alpharank_test (Failed)
	105 - python_alpharank_visualizer_test (Failed)
	106 - python_dynamics_test (Failed)
	107 - python_heuristic_payoff_table_test (Failed)
	108 - python_utils_test (Failed)
	109 - python_visualization_test (Failed)
	110 - python_catch_test (Failed)
	111 - python_cliff_walking_test (Failed)
	112 - python_data_test (Failed)
	113 - python_tic_tac_toe_test (Failed)
	114 - python_bot_test (Failed)
	115 - python_games_sim_test (Failed)
	116 - python_matrix_game_utils_test (Failed)
	117 - python_policy_test (Failed)
	118 - python_pyspiel_test (Failed)
	119 - python_rl_environment_test (Failed)
	120 - python_tensor_game_utils_test (Failed)
	121 - python_file_logger_test (Failed)
	122 - python_lru_cache_test (Failed)
	123 - python_examples_bridge_supervised_learning (Failed)
Errors while running CTest
The command '/bin/sh -c mkdir -p build &&     cd build &&     cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=`which clang++` ../open_spiel &&     make -j4 &&     ctest -j4'

@elkhrt, I added tests by running the ctest -j4 command after the build as explained in the installation docs. It seems like your Dockerfile experiences the same troubles as well! So the python build fails both through pip as through a manual compilation it seems.

So the dockerfile I'm using now is
FROM ubuntu:20.04
RUN apt update
RUN dpkg --add-architecture i386 && apt update
RUN apt-get -y install \
    clang \
    curl \
    cmake \
    git \
    python3 \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    sudo

# clone repository and install
RUN git clone -b 'master' --single-branch --depth 15 https://github.com/deepmind/open_spiel.git open_spiel
WORKDIR open_spiel
RUN ./install.sh

# build and test
RUN mkdir -p build && \
    cd build && \
    cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=`which clang++` ../open_spiel && \
    make -j4 && \
    ctest -j4
COPY . build

WORKDIR /open_spiel/build
CMD run.sh

When skipping the test command, you can indeed run the examples. However, as explained above, when running the tests this Dockerfile approach also experiences the same trouble as did mine when it comes down to the Python tests.

PYVERSION is something our script defines here: https://github.com/deepmind/open_spiel/blob/b19852be38e65de2db20dc9be6659e522a72e83d/open_spiel/scripts/build_and_run_tests.sh#L89, it's not defined by default.
So you either need to run the same command on that line before running the test or hard-code it. (Ubuntu 18.04 comes with Python 3.6 but I see you do an upgrade so you might have 3.7-- you can find out which version you have using dpkg --list | grep python or from within the Python interpreter using import sys; print(sys.version))

Also looks like your new file is missing the installing the required pip packages..?

If it helps, my go-to minimal manual install is the one on page 6 of the paper: https://arxiv.org/abs/1908.09453

I am now working with the Dockerfile provided by Lockhart, the Dockerfile uses:
Python 3.8.2
pip 18.1

I am now working with the Dockerfile provided by Lockhart, the Dockerfile uses:
Python 3.8.2
pip 18.1

Cool. But you still need to install OpenSpiel's python dependencies via pip3 install --upgrade -r requirements.txt

FROM ubuntu:20.04
RUN apt update
RUN dpkg --add-architecture i386 && apt update
RUN apt-get -y install \
    clang \
    curl \
    cmake \
    git \
    python3 \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    sudo

# clone repository and install
RUN git clone -b 'master' --single-branch --depth 15 https://github.com/deepmind/open_spiel.git open_spiel
WORKDIR open_spiel
RUN ./install.sh

RUN pip3 install --upgrade pip
RUN pip3 install --upgrade setuptools testresources
RUN pip3 install --upgrade -r requirements.txt

# build and test
RUN mkdir -p build && \
    cd build && \
    cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=`which clang++` ../open_spiel && \
    make -j4 && \
    ctest -j4
COPY . build

WORKDIR /open_spiel/build
CMD run.sh

results in
Step 10/14 : RUN pip3 install --no-cache -r requirements.txt
 ---> Running in 512b2a97f247
Requirement already satisfied: pip>=20.0.2 in /usr/local/lib/python3.8/dist-packages (from -r requirements.txt (line 2)) (20.0.2)
Collecting absl-py==0.9.0
  Downloading absl-py-0.9.0.tar.gz (104 kB)
ERROR: Could not find a version that satisfies the requirement tensorflow<2.0,>=1.15.1 (from -r requirements.txt (line 4)) (from versions: none)
ERROR: No matching distribution found for tensorflow<2.0,>=1.15.1 (from -r requirements.txt (line 4))
The command '/bin/sh -c sudo pip3 install --no-cache -r requirements.txt' returned a non-zero code: 1

Ah man, not this again. This error was a pain to fix a few months back when we upgraded to TF 1.15. Maybe it is back to haunt us in Ubuntu 20.04.
Might not work by the system's pip. I wonder if the user-based pip is necessary now.
I will test an install independently on a Ubuntu 20.04 machine and report back. Might be a few days.

FROM ubuntu:18.04
RUN apt update
RUN dpkg --add-architecture i386 && apt update
RUN apt-get -y install \
    clang \
    curl \
    git \
    python3 \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    sudo

RUN sudo pip3 install --upgrade pip
RUN sudo pip3 install matplotlib

# clone repository and install
RUN git clone -b 'master' --single-branch --depth 15 https://github.com/deepmind/open_spiel.git open_spiel
WORKDIR open_spiel
RUN ./install.sh

RUN pip3 install --upgrade setuptools testresources
RUN pip3 install --upgrade -r requirements.txt
RUN pip3 install --upgrade cmake

# build and test
RUN mkdir -p build && \
    cd build && \
    cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=`which clang++` ../open_spiel && \
    make -j4 && \
    ctest -j4
COPY . build

RUN python3 ./open_spiel/python/examples/matrix_game_example.py
WORKDIR /open_spiel/build
CMD run.sh

I managed to resolve the dependency issue by switching back to 18.04 and installing make through pip (since that downloads the latest version in contrary to apt-get). I am now running the tests again and I'm checking if I can import pyspiel in a Python script

Ok I expect that python3 command to fail unless you set the PYTHONPATH environment variables. Tests should be ok because they are set within the CMakeLists.txt IIRC.

100% tests passed, 0 tests failed out of 133

Total Test time (real) = 1053.32 sec

OK so the tests are ok, the script indeed still fails, so I'm adding the PYTHONPATH and everything should work properly in that case

I haven't spent too much time going through the conversation above but we've successfully dockerized OpenSpiel and hence I'm pasting the relevant docker files here in case they are of any help.

We use this file to build a container which is used to build the OpenSpiel project (within CI), image size is ~800MB
We use this file for building a container that actually runs the python scripts, image size ~120MB

You can see how the container from 2. is built within a container from 1. in the circleci config. Two docker files are used to obtain a much smaller final container. Also, containers are based on debian:buster-slim so not sure whether this will be completely relevant for your problems; feel free to ignore in that case 🙂. It's safe to ignore non-open-spiel related things within the project.

Ok thanks for sharing, I just got it working with the following setup:
FROM ubuntu:18.04
RUN apt update
RUN dpkg --add-architecture i386 && apt update
RUN apt-get -y install \
    clang \
    curl \
    git \
    python3 \
    python3-dev \
    python3-pip \
    python3-setuptools \
    python3-wheel \
    sudo

RUN sudo pip3 install --upgrade pip
RUN sudo pip3 install matplotlib

# clone repository and install
RUN git clone -b 'master' --single-branch --depth 15 https://github.com/deepmind/open_spiel.git open_spiel
WORKDIR open_spiel
RUN ./install.sh

# install Python dependencies
RUN pip3 install --upgrade setuptools testresources
RUN pip3 install --upgrade -r requirements.txt
RUN pip3 install --upgrade cmake

# build and test
RUN mkdir -p build && \
    cd build && \
    cmake -DPython_TARGET_VERSION=${PYVERSION} -DCMAKE_CXX_COMPILER=`which clang++` ../open_spiel && \
    make -j4 && \
    ctest -j4
COPY . build

ENV PYTHONPATH=${PYTHONPATH}:/open_spiel/
ENV PYTHONPATH=${PYTHONPATH}:/open_spiel/build/python

WORKDIR /open_spiel/build

Amazing! Thanks for working on this.
Please submit a PR. Many people would benefit from this, so we should have it somewhere!

Indeed! I am working on it! 👍

PR submitted 💯Being able to easily run Reinforcement Learning projects in the cloud will most certainly be beneficial!

@lanctot The PR still needs to be merged on the master, is there a problem still?

No problem, I just got busy with work yesterday so couldn't do it yet.
We have a weekly update cycle: we import the PR internally and then it gets merged in our weekly push back to github (on Mondays).
So I will import it in the next few days and it will be merged on March 9th.

I am wondering if you could try something for me though.
My server provider doesnt have Ubuntu 20.04 yet but I would like if we coukd fix the problem you faced before its release. I suspect if we bump the TF requirement to 2.0 I believe it will work (in requirements.txt). Is it easy for you to just test out for me?

Thanks for the update! 👍
I will check if the system still works on Ubuntu 20.04 with TF 2.0 as soon as possible

I tried internally and the upgrade to TF 2.0 breaks ~10 tests so if that's the solution we'll have to update some code first. When I can get an Ubuntu 20.04 machine then I'll look for a shorter-term solution (unless we've fixed those tests by then).

Ok as soon as you've updated the code I will try to run it in the Ubuntu 20.04. You should give Docker a try as well, it's very easy to install on whatever OS you're using! In that case you can just use the Dockerfile I provided and switch from 18.04 to 20.04. (if you can write shell scripts, you can write Dockerfiles)

@lanctot, I'm glad I could contribute, it seems like we're about to have a presentation from you for our Machine Learning project at the University of Leuven.

Indeed, I am looking forward to it. I was wondering if you were in that class. Is that how you found out about OpenSpiel?

Yes that is how I found out about OpenSpiel indeed, many people of the course struggled with getting it up and running on various operating systems so that's why I built a Dockerfile asap 👍I already mentioned the new way of installation on our forum

Just an FYI. I tried installing openspiel using the docker container from the version 0.1.0 package. It came up with this error. It seems like this one failed test causes the docker container to not be created. Since it passes 99% of tests, I just deleted the testing part of the dockerfile and have successfully created the docker container.
Assuming that since docker is supposed to take care of dependencies and stuff (though it's my first time using it) that this is not an issue on my end (using the latest version of docker).
99% tests passed, 1 tests failed out of 151

Total Test time (real) = 1504.72 sec

The following tests FAILED:
	140 - python_examples_bridge_supervised_learning (Failed)
Errors while running CTest
The command '/bin/sh -c ctest -j12' returned a non-zero code: 8

Thanks, we will look into it. Re-opening as a reminder. Tagging @elkhrt @yarncraft just so they know.
We have not had any issues with that test recently, so I am not sure what could be happening.

Just an FYI. I tried installing openspiel using the docker container from the version 0.1.0 package. It came up with this error. It seems like this one failed test causes the docker container to not be created. Since it passes 99% of tests, I just deleted the testing part of the dockerfile and have successfully created the docker container.
Assuming that since docker is supposed to take care of dependencies and stuff (though it's my first time using it) that this is not an issue on my end (using the latest version of docker).
99% tests passed, 1 tests failed out of 151

Total Test time (real) = 1504.72 sec

The following tests FAILED:
	140 - python_examples_bridge_supervised_learning (Failed)
Errors while running CTest
The command '/bin/sh -c ctest -j12' returned a non-zero code: 8

This is indeed normal behavior and not an error on your side. The testing part can be commented out for faster builds, most of the framework will behave correctly given the 99% passing grade. @lanctot, is the Jax dependency not part of the Openspiel installation? If this error is due to a missing dependency I can quickly fix it by adding it to the container spec.


Global state store

@maxogden @freeman-lab can you elaborate on what you had in mind? I'd like to get this in place before going too much further - it will get harder to do the more we build without it.

@blahah agreed! I've been really enjoying choo and I sorta think we should restructure our UI/UX to use it. It basically just combines a few really light-weight modules for state management (send-action) and view updating (yo-yo) in a simple and elegant way. Also very compatible with the composable front-end work @maxogden and co have been doing over in dat land.
As I sort of created our front-end mess in the first place, I can take a stab at doing this refactoring this weekend. Agreed it'd be good to get in place before adding much more!

I took your initial mess and ran it through a pipeline involving a mincer, a mangle and several illegal surgical procedures. Sorry about that... If you can start converting and create some examples of best practise, I can help with the rest of the conversion.

We've moved to choo now 🙌, this will land in v1


Refactor write_to function

There are some shenanigans in there that should be addressed. Namely, the manipulation of self._fd is a cause for concern:
https://github.com/spacetelescope/asdf/blob/32abe89ce024ec93b838bdf684a74662a83ee0f0/asdf/asdf.py#L1088-L1095

Bottom line is I think we should strive for writing to be an idempotent operation.

It also might make sense to deprecate the uri argument to AsdfFile (I'm not sure how much this is used anyway) and instead make it an optional argument to AsdfFile.write_to. Then we should attempt to explicitly pass the URI to all places where it is required instead of making it an attribute of the AsdfFile object.

This is related to #119.

I believe this was addressed in the block management rewrite https://github.com/asdf-format/asdf/pull/1537
Please feel free to reopen this issue if the asdf 3.0.1+ still has this issue.


Updates to accessions.tsv (57911 new, 88 removed) seqs downloaded 2021-08-20

Many of the changed mappings were caused by newly duplicated sequences in GISAID, from multiple submitting organizations.  Curiously, the sequences already in GISAID had names that were slightly different from their names in GenBank (e.g. CA-CHLA-PLM43021777 vs CA-CHLA-P-PLM43021777, PR-HF101 vs PR-FIOCRUZ-HF101, IN-GD-071321-C21194875 vs IN-GD-SID-C21194875), while the newly duplicated sequences in GISAID have names that exactly match the GenBank names.  I will try to contact the submitters to inquire about the duplicates.
Hope you have a great weekend.  :)

Thank you for digging into this @AngieHinrichs ! Really disappointing to hear that duplications are such an issue. Is there a way to keep track of these? Ideally we would want to exclude duplicates. But this might be getting into a whole nest of thorns.... 😬

Hi Emma,
I have been excluding duplicates in order to keep the mapping one-to-one.  So far, duplicate submissions have used the same names, so when my pipeline complains about a GenBank sequence matching two GISAID sequences with the same name but different IDs, I usually just exclude the newer IDs.  I've been accumulating a file of those newer IDs (currently 2169 of those, really a small number compared to 3 million), and could share that.  The best thing would be to contact all the submitters to ask them to ask GISAID to remove the duplicated sequences, but... too many things to do.
This new category is a little more annoying because the names are changing in GISAID, so they're not as easy to detect and work around.  Earlier submissions have names that are similar to GenBank names but not exactly the same.  The new duplicate submissions, which don't seem to be coming from the submitters, use the GenBank names.  Now we have a new GISAID sequence whose name exactly matches the GenBank sequence, so it is favored over the original submission with a name that doesn't exactly match, and not detected as a plain old duplicate.
Before I send an update of the mapping, I look for previous mappings that have disappeared or changed, just in case there's a bug (or a need to re-download a chunk of GISAID sequences for a name change in both GenBank and GISAID, etc).  That's how I end up finding duplicates.  The numbers are small in the grand scheme of things, but they take time to investigate.
Thanks,
Angie

Hi Angie,
Sorry for the delayed reply, but thank you so much for the detailed information and breakdown! Good to know that in the grand scheme it's a small number, but agree with you that it's good to keep a finger on, and we appreciate it!


TransferUtilityUploadDirectoryRequest.UploadFilesConcurrently is internal in net core

Not able to access UploadFilesConcurrently of TransferUtilityUploadDirectoryRequest.cs
I am using AWSSDK.S3 <IP_ADDRESS>.
Is there any plan to expose this property for concurrent processing of files?

Thanks for reporting this.  It's being treated as a bug for now.  It should go into a release soon.

Hi @ArchanaNeog,
This appears to be fixed as per comment https://github.com/aws/aws-sdk-net/issues/1227#issuecomment-657180129.
Thanks,
Ashish


[TF 2.0] Checkpoint breaking change for RNN.

Type of breakage: Breakage with changing code.
APIs that are affected:

tf.nn.dynamic_rnn
tf.nn.static_rnn
tf.nn.bidirectional_dynamic_rnn
tf.nn.static_bidirectional_rnn
All cells within the tf.nn.rnn_cell.* module.

Description of change: The current TensorFlow RNN code is going to be replaced by tf.keras RNNs in TensorFlow 2.0. User will need to update their code to use the new Keras API. The weights of the Keras RNN cell are different from the existing TensorFlow RNN cell.
Variable name change map: The RNN cell weights are in a different format between existing TensorFlow RNN cells and Keras cells. There is currently no direct mapping for that.
Target time window: Undecided since the update requires a non-trivial user side change.

@dynamicwebpaige In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thanks!

Unassigning myself, as this is more of an FYI than a solvable problem.

@dynamicwebpaige
Please let us know if this issue can be moved to closed status.

AttributeError: module 'tensorflow.compat.v1' has no attribute 'contrib'
I am facing this error while using rnn model in my code. Although I used
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
help me how can I solve it


link to learner track is broken.

Describe the bug
A clear and concise description of what the bug is.
the link to the kernel learner track is broken.
https://kernel.community/getting-started/
To Reproduce
Steps to reproduce the behavior:

Go to 'https://www.notion.so/What-is-KERNEL-b9520eb29d954f5fb709150cae3975b1
Scroll down to 'KERNEL TRACK'
Click on Kernel Learn Track.
See error

Expected behavior
A clear and concise description of what you expected to happen.
correct the link
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

Browser [e.g. chrome, safari]

Additional context
Add any other context about the problem here.

Fixed in #81 - thanks for noting this :heart:


useApi ?

just like Query Mutation in react-apollo:
<Query query={gql_document} variables={variables}>
  {({ data, error, loading }) =>

}
</Query>

<Mutation mutation={gql_document}>
  {(mutation_api, mutation_result) => {
    const { data, error, loading } = mutation_result;
    return

  }}
</Mutation>

And now we have useAsync, but it can only act as Query.
We should have an useApi to act as Mutation.
// A very very naive implementation
const useApi = (promisified_api, variables) => {
   const [state, set_state] = useState({loading: !!variables});
   const api = async (...variables) => {
    set_state({loading:true});
    const data = await promisified_api(...variables);
    set_state({data});
  }
  // we should expose `set_state`, it can act as Query's updteQuery, and use can set_state({error:null}) manually when close the error dialog.
  return [api, state, set_state];
}

const [api, state, set_state] = useApi(fn_return_promise, maybe_variables);

Hey guys, I used useAsync as a starting point, and grappled with the problem that I didn't necessarily want to run it on first load...
So I came up with the following re-implementation: https://gist.github.com/dfee/0686746c4b37ad45ba88b20a66a60432
and here's some example usage:
  const [state, utils] = useAsync<WrappedFormUtils>(auth.signInWithEmail, {
    onFinally: ({ context }) => {
      if (context !== undefined) {
        context.resetFields(["password"]);
      }
    },
  });

  const onSubmit: SignInFormProps["onSubmit"] = (e, form) => {
    utils.setArgs([form.getFieldsValue()]);
    utils.setContext(form);
  };

I'm sharing this here because I think my solution is effective, but might be improved by the community, and perhaps merged in to replace the current useAsync implementation.

So I have tackled this problem. I am not sure whether it is feasible to extend the useAsync hook or provide another one that has an API to [state, setState].
I had an array of data I had to fetch from an API and then I had to remove add or update items locally in the React app instead of just calling the API again. This is appeared to be a problem with the original implementation of the useAsync hook since it does not provide an api to the modify the state of the data return from the async fn.
I have made some modifications to the useAsyncFn hook as it was easier. (useAsync hook just calls useAsyncFn hook).
Below is the difference in the API that useAsync hook exposes using a trivial todo example:

Original version

const {
        loading,
        error,
        value
    } = useAsync(async () => {
           const result = await apiClient.get('/todos');
           return result;
    }, []);
// todos cannot be modified locally

Modified version

const {
        loading,
        error,
        data: [todos, setTodos]
    } = useAsync(async () => {
        const result = await apiClient.get('/todos');
        return result;
    }, []);

// This is will work with the local state of todos
setTodos(todos.filter(todo => todo.type !== 'done'))

I can do a pull request but I haven't really read through the contributing guidelines and Typescript can be a problem since I have not used it before. Let me know what you think.


[OPMONDEV-166]: fix typo and remute exceptions while granting accesses to PG

Previously that was quite strange except clause... Just be careful since it might have been responsible for other purpose than the comment

PG won't raise any errors if access already exists. We have upper try..except block to handle other exceptions, so we should be fine :)


Fourth assignment completion

@zkotti my final assignment is ready to be reviewed!

Excellent submission! I really liked your additional design decisions, particularly the MetricLoader; although not requested, you took it a step further, providing useful extra functionality. Congrats!

That's great! Thank you @zkotti, really appreciate it!

It would be great if you made your repo public so that I can share it with your colleagues.

I made it and I would be happy to share it!


Rebrand "Open Images" search to "Open Ledger" search

Change the current UI on the "Open Images" search page to be:

The primary page at / (move the partner API search to a subpage)
Branded as the "Open Ledger" search

Maybe include a total number of records there and a list of current datasources.

This was removed, now just the partner API page is there.


Upgrade Gliderlabs SSH to 0.3.3 and add FailedConnectionCallback

Following the merging of https://github.com/gliderlabs/ssh/pull/143 we
can now report connections to the ssh server that have failed before
public key exchange has completed using the standard fail2ban message.
This PR updates Gliderlabs SSH and adds a callback that will provide this
logging.
Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

May need to double check that this doesn't cause too much failed logging.


[br] Use correct bit position 6 for RA stub router flag (SNAC router flag)

Describe the bug A clear and concise description of what the bug is.
Position of stub router flag was initially chosen in the RA flags extension option. Now IANA has allocated it as bit 6 in the regular flags byte, see https://www.iana.org/assignments/icmpv6-parameters/icmpv6-parameters.xhtml#icmpv6-parameters-11
To Reproduce Information to reproduce the behavior, including:
See latest OT code
Expected behavior A clear and concise description of what you expected to happen.
"Stub router" flag bit set to flag position 6 and mention "SNAC router flag" as a name also.
Reference to spec in code can be updated to : https://datatracker.ietf.org/doc/html/draft-hui-stub-router-ra-flag-02
which has the bit position that is now chosen.
Console/log output If applicable, add console/log output to help explain your problem.
Additional context Add any other context about the problem here.
See SPEC-1283

FYI @abtink @jwhui

Submitted:

https://github.com/openthread/openthread/pull/10695

Address by #10695


Open Extension Page is just broken now.

Open Extension Page is just broken now.
Click it, does nothing. Started randomly happening.
Nothing shows in the console when I inspect the pop up but I can't click on the open extension page fake link (actually a button for some reason) and see the console because the pop goes away and the inspector dies.
Safari Version 17.0 (186<IP_ADDRESS>.22, 18616)
macOS 13.6 (22G120)
Userscripts Version 4.5.0 (81) - You need an edit menu in the Userscripts.app, otherwise you can't copy this information as easily.
Feels like I have been filing bugs all day so I am just really tired of everything right now. Don't close if you can't reproduce, I'll eventually come back. But the whole extension is useless now unless I go manually edit the scripts.

Hi @MikeRich88!
Sorry, which userscript are you having issues with? I apologize for not being responsive & fixing bugs ASAP. Life, and a 5 year old, has kept me busy.

I am not having an issue with any userscript
I can't even edit any of them. "Open Extension Page" is broken

A lot of websites have increased security and may be preventing script injection. I don't know is an extension or built-into Safari specifically, but I'd recommend using Tampermonkey for Safari instead.

Again, that's not the issue.
If I go to my own website, which has no such increased security, and click on "Open Extension Page", it doesn't open.
I don't know how to make this any clearer. Do you need a QuickTime video?

The problem you're describing is not related to any of the userscripts within this repo; and I don't know where to find the "Open Extension Page" link. I'm guessing it's build into Safari, or a browser extension I'm not familiar with. Either way, I don't think I can solve your issue.

Please accept my apologies. Obviously I have the wrong repo.


Update flatbuffer_conversions.cc

comment typos

Still contains a typo. It is understand and not unterstand

fixed the typo. Pls re-evaluate.

Hi @Cassini-chris  Please don't use "add file"/"update file"/"fix file"/etc. commit messages. These are hard to reason about when looking at the history of the file/repository. Instead, please write explanatory git commit messages.
The commit message is also the title of the PR if the PR has only one commit. It is thus twice important to have commit messages that are relevant, as PRs would be easier to understand and easier to analyze in search results.
For how to write good quality git commit messages, please consult https://cbea.ms/git-commit/
Thank you for your contribution!


MINOR:cleanup code

What is the purpose of the change
clean up code
Brief changelog
Since it's a small change, I don't think it's necessary to add an issue

Codecov Report

Merging #4013 (d48b054) into develop (6ff00ed) will decrease coverage by 0.09%.
The diff coverage is 64.00%.

@@              Coverage Diff              @@
##             develop    #4013      +/-   ##
=============================================
- Coverage      47.89%   47.80%   -0.10%
+ Complexity      4995     4983      -12
=============================================
  Files            633      633
  Lines          42526    42520       -6
  Branches        5573     5572       -1
=============================================
- Hits           20369    20325      -44
- Misses         19658    19686      +28
- Partials        2499     2509      +10

Impacted Files
Coverage Δ

...che/rocketmq/remoting/netty/NettyServerConfig.java
48.27% <ø> (ø)

...apache/rocketmq/acl/common/SessionCredentials.java
57.97% <33.33%> (+1.02%)
:arrow_up:

...n/java/org/apache/rocketmq/store/ConsumeQueue.java
64.55% <63.15%> (-0.29%)
:arrow_down:

...cketmq/acl/plain/RemoteAddressStrategyFactory.java
89.51% <100.00%> (-0.25%)
:arrow_down:

...org/apache/rocketmq/common/stats/StatsItemSet.java
41.79% <0.00%> (-8.96%)
:arrow_down:

...in/java/org/apache/rocketmq/test/util/MQAdmin.java
38.88% <0.00%> (-5.56%)
:arrow_down:

...org/apache/rocketmq/store/ha/WaitNotifyObject.java
66.07% <0.00%> (-5.36%)
:arrow_down:

...ketmq/common/protocol/body/ConsumerConnection.java
95.83% <0.00%> (-4.17%)
:arrow_down:

...ava/org/apache/rocketmq/filter/util/BitsArray.java
59.82% <0.00%> (-2.57%)
:arrow_down:

.../java/org/apache/rocketmq/logging/inner/Level.java
36.00% <0.00%> (-2.00%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ff00ed...d48b054. Read the comment docs.

Coverage increased (+0.04%) to 51.907% when pulling d48b05427bba53ddfc062c674c14834302d79e90 on Kvicii:MINOR_clean_up_code into 6ff00ed0530bbd94e6ed9cc444b0a2fdb57c7ba3 on apache:develop.

Coverage increased (+0.04%) to 51.907% when pulling d48b05427bba53ddfc062c674c14834302d79e90 on Kvicii:MINOR_clean_up_code into 6ff00ed0530bbd94e6ed9cc444b0a2fdb57c7ba3 on apache:develop.

Please add related issue to this pr.


revisit OnlineBenchmarkTrainer

Describe the solution you'd like
OnlineBenchmarkTrainer is overriding methods that aren't designed to be overridden, which can lead to issues with upstream divergences.
OnlineBenchmarkTrainer has two methods:

[x] _maybe_log_save_evaluate
[ ] _get_train_sampler

From #114 we can just about get rid of _get_train_sampler (it depends on transformers master for now though)
But there's more work to be done to get rid of _maybe_log_save_evaluate. I think these should be able to be replaced with callbacks, but it needs research.
Additional context
Related to #121 , #114

ok, i think we still need this class, though we've removed most of the cruft-y bits


Fix invalid meta

Otherwise we get the following error:
error: Package ‘osu-stable’ in /nix/store/nwffh179bzszwry8syj74l4qw6m13aq2-source/pkgs/osu-stable/default.nix:80 has an invalid meta attrset:
       	- key 'meta.maintainer' is unrecognized; expected one of:
       	     ['available', 'badPlatforms', 'branch', 'broken', 'changelog', 'description', 'downloadPage', 'executables', 'homepage', 'hydraPlatforms', 'insecure', 'isBuildPythonPackage', 'isFcitxEngine', 'isGutenprint', 'isHydraChannel', 'isIbusEngine', 'knownVulnerabilities', 'license', 'longDescription', 'mainProgram', 'maintainers', 'maxSilent', 'name', 'outputsToInstall', 'platforms', 'position', 'priority', 'schedulingPriority', 'sourceProvenance', 'tag', 'tests', 'timeout', 'unfree', 'unsupported', 'version'], refusing to evaluate.

Good catch, thanks!
Can I ask you to also fix that in pkgs/rocket-league while you're at it?

Good catch, thanks!
Can I ask you to also fix that in pkgs/rocket-league while you're at it?

Sure, I am just going to fix in another PR since I am doing the lazy way to fix it (using GH web interface).

Ah, nevermind then. I'll push the fix to this PR and merge it in one go.

Looks like you were faster. Thanks!


*** MAINTAIN THIS PROJECT ***

What is the problem?
There are currently 194 pull requests open and 607 issues. If the original contributors of this project are not willing to continue, please find someone who does. A major pull requester is a good option.
I think this library is too popular to just be neglected.
How can it be fixed?
If you are reading this and want to contribute to this project, please suggest yourself in the comments below.

Side Note
I'm using this library and not a contributor in any way.

There was a discussion happening about this in #884 but nothing seems to have come of it. The maintainers seem to think ngMaterial is going to replace this so it's not worth maintaining anymore.

Thanks for linking to that discussion @AdamAxtmann, that explains a lot.  It's disappointing to see the repository owner openly stating he believes the project is deprecated and rarely checks in on it anymore.  If that's the case, I'd hope he either transfers the repository or opens the floodgates to new maintainers and pull requests.
I'm a bit baffled at the assumption we're all going to jump on the ngMaterial bandwagon, to be honest.  Not all of us are chasing the newest and shiniest.  I don't think I'm the only one who appreciates the well established Bootstrap ecosystem (ngMaterial is nowhere close yet), along with it's simple, familiar UI components.

+1

+1

:+1:

md-select from mg-material has like 0 options. Why would I use that instead of this, other than the fact it's been completely abandoned.

@jcluts as the originator of this and variations of this library, I have not really felt ngMaterial serves as a good alternative, as it's kind of a widget suite. But yes, we don't have any people who are really capable of taking ownership of this repo.

@ProLoser Please make me a owner/contributor so that I can help maintain this project. Otherwise, I'll be directing others to make PRs on my fork to be published to npm.
I've already accepted several PRs into my fork and have published<EMAIL_ADDRESS>Folks, If my contributor/owner request is denied and you would like your PRs accepted and published to npm, please send your PR requests to my fork https://github.com/aaronroberson/ui-select and they will be published under the -ar namespace. If you would like to submit your PR to angular-ui and to my fork you are encouraged to do so. If you would like to submit someone else's PR which has already been created against angular-ui, you can follow the instructions below:
Navigate to the fork which has initiated the pull request (pr-fork) and would like to get published to npm under the -ar namespace.
Go to the pull requests page of the initiating fork (pr-fork)
Click the New Pull Request button.
The pending pull request(s) should be offered. Select on the left side as the base fork (aaronroberson/ui-select) (IMPORTANT)
Now the View Pull Request button should change to Create Pull Request. Click the Create Pull Request button.
Now there should be a pending pull request in aaronroberson/ui-select which I will accept if the following are true:

All tests pass.
Feature requests retain the nature of the ui-select, selectize and Select2 projects.
Merge conflict instructions are provided for merging the current base (master branch) and remote (pull request).
Your code is not ambiguous or obscure.

@aaronroberson added

Thank you! I updated my post above to reflect having been added.

Good luck @aaronroberson! :100:

I vote for @aaronroberson !

@aaronroberson Glad to see a new maintainer on this project. Do you plan to sift through older PRs?

@aaronroberson hopefully you've been following along discussions and points I've been making about modularity, simplicity and composeability. I realize that it is somewhat of a pipe dream considering attribute-directives are on the way out but if this project continues expand towards supporting a billion obscure configuration options vs just giving users the bare-bone tools they need to add obscure features themselves it will turn into ui-select2 and all the projects that came before it.

Closing 600 issues would be a huge task. All the very best @aaronroberson

@ProLoser I'm a strong believer in passing in an options config object as opposed to adding a million attributes to a directive. How do you feel about that?
As for modularity, singularity and composability I 100% agree.

I have explored the ability to do both. Simply iterating for any values on the attributes with a common prefix, letting attributes override properties in the options object, etc.


'ascii' codec can't decode byte error when using collate utf8_bin in mysql

When I am using mysql collate utf8_bin on tag name field and have tag names with unicode characters such as á, django template throws DjangoUnicodeDecodeError in force_unicode(value):
'ascii' codec can't decode byte 0xc3 in position 7: ordinal not in range(128). You passed in <Tag: [Bad Unicode data]> (<class 'taggit.models.Tag'>)
Replacing tag name column collate utf8_bin with utf8_general_ci fixes this issue but raises a different issue with IntegrityError when trying to save tags with different casing.
I have added a fix in my fork which solves the issue in my scenario by making sure the str function always returns unicode by casting to utf8 if mysqldb returns byte string: https://github.com/cimani/django-taggit/commit/bf26627f0f3f9885dfe64186a6dcb90ad34e746b
I would love for this fix to go into the master but I do realize that this changes forces utf8 which might not apply to all applications of this awesome plugin. I am guessing making encoding as an optional parameter passed through TaggableManager init function and applied to the underlying tag class to be used by this function would be one way around this. But I am not exactly sure how to implement this in a nice and clean way yet. If anyone has any good ideas on this, please point me to right direction :)

Check your MySQL-python version. It should be 1.2.5. I had similar problem and this fixed my problem.

I strongly recommend people using mysql-python to use https://pypi.python.org/pypi/mysqlclient instead. It's a fork that's well maintained and has python 3 support. Django recommends it now.

ditto mysqlclient

closing as this seems to be a mysql thing


Can we have an option to remove max-width from body?

I love everything about this framework except for the fact that it forces max-width on my body.
I am proud to wear my body fat.

We try to avoid configuration where possible. We're having to do it in 2.0.0 for browser compatibility, but I'm not sure that we're ready to do it for cosmetic items like this.
I don't want to <body>-shame though. For your use case, is this something you'd be able to override with your own custom stylesheet after water.css? For example:
html {
    max-width: initial;
}

That's what I currently resort to. Thanks.


Output not having correct css

I am trying to output my website to html so I can upload to my web host. When I run hugo -t etch the css does not show up in the outputted website.
There is the correct css when I am running the live server but not when I output to html.

This is probably because the path to the stylesheet is an absolute path and you are viewing the generated public/index.html locally where it is not in the root / folder. If you inspect the source code of index.html after opening it in the browser, you'll see <link rel="stylesheet" href="/css/style.min.css">. Usually, everything in the public directory is uploaded to a server where a web server treats that folder as the root folder, meaning /css/style.min.css will resolve correctly, but this won't be true if viewing the page on your computer.
If you want all the paths to load correctly when viewing public/index.html you can add relativeURLs = true to config.toml and it should fix the problem.

Worked just fine once I added relativeURLs = true to the config.
I did want to put it under /blog and have my main website something different so this is a big help.
Thanks for the help.


perf: avoid bytes cloning while generating digest and use faster_hex for encoding/decoding

Type of Change

[ ] Bugfix
[ ] New feature
[x] Enhancement
[ ] Refactoring
[x] Dependency updates
[ ] Documentation
[ ] CI/CD
[x] Performance

Description

see linked issue: #4403
Additional Changes

[x] This PR modifies the API contract
[x] This PR modifies the database schema
[x] This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Checklist

[ ] I formatted the code cargo +nightly fmt --all
[ ] I addressed lints thrown by cargo clippy
[ ] I reviewed the submitted code
[ ] I added unit tests for my changes where possible
[ ] I added a CHANGELOG entry if applicable

Hey @AayushyaVajpayee, thanks for the PR!
Tho, at the moment, we're not actively pursuing perf improvements. Plus, we prefer being a bit conservative when it comes to switching out crates for alternatives, especially when the alternative is less popular (as per crates.io downloads) and seemingly less well documented. As such, I'll mark this as a draft PR for now, and we'll revisit this when we start targeting perf improvements, or when the dependency switch is actively required.
Thanks for the contribution tho. I really liked your perf analysis in the issue.


RFM - adwcleaner

https://chocolatey.org/packages/adwcleaner

I've been debating with myself about this one for months as it's one of those programs that sometimes gets updated frequently, but than sits a month or two.
I'll step up.

https://github.com/bcurran3/ChocolateyPackages/tree/master/adwcleaner
Done. Will push when made a maintainer.

https://chocolatey.org/packages/adwcleaner
AdwCleaner (Portable) <IP_ADDRESS> | 1330 | Friday, October 26, 2018 | approved
This issue should be closed.


[iOS] Test Trail Address

Scenario: Receive Trail Address

Precondition: Need the trail address
Steps to reproduce:

Generate a new ID
Call TikiSdk.config().initialize
TikiSdk.instance.trail.address

Expected result:

The method return a string with the address

done in #307


Passing string as v-tooltip directive value causes TypeError in some browsers.

When v-tooltip directive is used like suggested in Getting Started/Usage: <button v-tooltip="'You have ' + count + ' new messages.'">, a TypeError will be thrown in some browsers, for example in IE11.
First, after creating element with v-tooltip directive, bind function is called. Then, createTooltip is called with string as value argument. The function extracts content from value argument and assigns it to a constant. After that, content is assigned to title parameter in opts object. But.. original value argument is used once again (because when value is an object, it should contain tooltip options) in the same object (opts) with spread operator. Using spread operator on a string is not allowed and will simply throw a SyntaxError: Invalid or unexpected token, although no one reported any issues, until now.
The issue exists, but it has been masked.
Module is bundled using rollup.js and transpiled with Babel. Babel creates _defineProperty, _objectSpread functions definitions, and replaces all spread operators with _objectSpread function calls. _objectSpread, when called, assumes every argument which is not null is an object, iterates over arguments and gets arguments own property names relying on Object.keys() method.
So, when directive is created with string as value, the Object.keys() is executed on a string (without any need). Most modern browsers can execute Object.keys() on a string, number, without any problems.
IE (11) can't. For IE, an argument really must be an Object.
Chrome/Firefox/Chromium based browsers behavior:
> Object.keys('hello world')
< ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10"]
IE behavior:
> Object.keys('hello world')
< [TypeError]: Object.keys: argument is not an Object
Even tough, passing value into the options on modern browsers is completely unnecessary, since it will only pass:
> _objectSpread({}, 'hello world', { placement: 'top' })
< {0: "1", 1: "2", 2: "3", placement: "top"} object into getOptions method, which will ignore all unknown parameters.

Duplicate of #323


Add option to dump profiler stats

Add a config option DEBUG_TB_PROFILER_DUMP_FILENAME to dump profile stats to a file, so they can be further evaluated, e.g. with flameprof.

Can you rebase / squash to a single commit on top of master?

Done.

I was able to resolve the merge conflict locally, but am running into permission errors when trying to push updates to the PR or to push updates that merge the PR into master. But if we can get past that, I think this one is ready to merge.

@Dosenpfand can you rebase? And also address my last question?

@jeffwidman Done. Sorry for my late reaction.

It looks like it's all good except for one style check that is failing. Once that's fixed I think it is ready to merge.

@macnewbold
fixed.


Add CLI flag configuration

See meow or arg for basic argument parsing

This was removed in 2.0.0, but will probably be brought back in a minor version update.

--prefix/-p adds "v" prefix in 2.1.0

[x] prefix
[ ] Changelog path
[ ] WordPress theme path
[ ] WordPress plugin path
[ ] Skip WordPress
[ ] Skip Changelog


support Node 18 for functions

Is this feature request related to a new or existing Amplify category?
function
Is this related to another service?
No response
Describe the feature you'd like to request
https://aws.amazon.com/blogs/compute/node-js-18-x-runtime-now-available-in-aws-lambda/
Notable callout on sdk migration from v2 to v3

AWS SDK for JavaScript upgrade to v3
If your existing Lambda functions are using the included SDK v2, then you must update your function code to use the SDK v3 when upgrading to the Node.js 18 runtime. This is the recommended approach when upgrading existing functions to Node.js 18. Alternatively, you can use the Node.js 18 runtime without updating your existing code if you deploy the SDK v2 together with your function code.

Describe the solution you'd like
see above
Describe alternatives you've considered
n/a
Additional context
No response
Is this something that you'd be interested in working on?

[ ] 👋 I may be able to implement this feature request

Would this feature include a breaking change?

[ ] ⚠ This feature might incur a breaking change

as a workaround:
I changed "{function-name}-cloudformation-template.json" file manually to "nodejs18.x" and it's working
(tried to do the same for lambda layer but it's not working for me)


Open and close card component, integrate content in card

Value proposition
As a user
I want to see a new window when click on a card
so that i can see the additional content of the card
Description

Acceptance criteria

[ ] When clicking on a card, a new window is opened.
[ ] When clicking the closing button of the detailed card, the detailed view is closed.
[ ] functional video player with start, pause and stop function
[ ] display video, title, description, views and ratings

Tasks

[ ] Create modal component with useState
[ ] styling the window-mock
[ ] integrate close button in the upper right corner
[ ] integrate player and video url from JSON
[ ] integrate title, description views, rating from JSON
[ ] testing

Hi Gustl,
a really good start for your user story! 🎉
💬 Let's discuss some issues which came to my mind while reading:
Acceptance Criteria

"onClick" is something special for developers, but not for the customer ;) Let's rephrase: When clicking on a card, a new window is opened.
When clicking the closing button of the detailed card, the detailed view is closed.
You can start, pause, and stop the video.

Tasks

If you haven't done so far, you might search for the React useRef() hook in order to implement a video.

Just a short question: "Beschreibung" looks like a button or a form input field. Is this just my misperception or planned? Plus one typo in line 2: the word "I" is missing before "click". Best

Just a short question: "Beschreibung" looks like a button or a form input field. Is this just my misperception or planned? Plus one typo in line 2: the word "I" is missing before "click". Best

The grey field visualizes the place for the the description. an input wont make sense here


Locking dependency versions

Thanks for this module - it's very helpful!
Problem I'm finding is that JSDom is a little unreliable.  The latest version  (0.2.10) is blowing up on Ubuntu when NPM is updated.  0.2.9 is not.
I was wondering if you could lock your dependency on this library to a fixed version, that works, please?  I'm not referencing JSDOM directly, but my deployment is breaking because JQuery is referencing it - and I can't do anything about that.
Thanks!

I'm also noticing the same issue using OS X.


java.io.FileNotFoundException - wrong jar names

Describe the bug
jars not found when starting Spark session:
...
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final.jar does not exist, skipping.
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-kqueue-4.1.68.Final.jar does not exist, skipping.
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-resolver-dns-native-macos-4.1.68.Final.jar does not exist, skipping.
...
java.io.FileNotFoundException: File file:/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final.jar does not exist

To Reproduce
For me:
emrconfig = SparkConf()\
    .setMaster("yarn")\
    .set('spark.emr.maximizeResourceAllocation','true')\
    .set('spark.dynamicAllocation.enabled', 'false')\
    .set("spark.jars.packages", "com.microsoft.azure:synapseml_2.12:0.9.5") \
    .set("spark.jars.repositories", "https://mmlspark.azureedge.net/maven")
spark = SparkSession \
    .builder \
    .config(conf=emrconfig)\
    .getOrCreate()

This is so simple so I assume 'm missing something
Expected behavior
The three files to be downloaded with correct names.
If I rename the files as follows, the application gets submitted and started successfully:
/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final-linux-x86_64.jar > io.netty_netty-transport-native-epoll-4.1.68.Final.jar
/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-kqueue-4.1.68.Final-osx-x86_64.jar > io.netty_netty-transport-native-kqueue-4.1.68.Final.jar
/home/ec2-user/.ivy2/jars/io.netty_netty-resolver-dns-native-macos-4.1.68.Final-osx-x86_64.jar > io.netty_netty-resolver-dns-native-macos-4.1.68.Final.jar

Info (please complete the following information):

SynapseML Version: 2.12:0.9.5
Spark Version 2.4.7
Spark Platform EMR

Stacktrace
[Please post the stacktrace here if applicable](https://mmlspark.azureedge.net/maven added as a remote repository with the name: repo-1
Ivy Default Cache set to: /home/ec2-user/.ivy2/cache
The jars for the packages stored in: /home/ec2-user/.ivy2/jars
:: loading settings :: url = jar:file:/usr/lib/spark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
com.microsoft.azure#synapseml_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-c0c8a1b1-ebd8-41ae-9b2a-625051f08cc3;1.0
	confs: [default]
	found com.microsoft.azure#synapseml_2.12;0.9.5 in central
	found com.microsoft.azure#synapseml-core_2.12;0.9.5 in central
	found org.scalactic#scalactic_2.12;3.0.5 in central
	found org.scala-lang#scala-reflect;2.12.4 in central
	found io.spray#spray-json_2.12;1.3.2 in central
	found com.jcraft#jsch;0.1.54 in central
	found org.apache.httpcomponents#httpclient;4.5.6 in central
	found org.apache.httpcomponents#httpcore;4.4.10 in central
	found commons-logging#commons-logging;1.2 in central
	found commons-codec#commons-codec;1.10 in central
	found org.apache.httpcomponents#httpmime;4.5.6 in central
	found com.linkedin.isolation-forest#isolation-forest_3.2.0_2.12;2.0.8 in central
	found com.chuusai#shapeless_2.12;2.3.2 in central
	found org.typelevel#macro-compat_2.12;1.1.1 in central
	found org.apache.spark#spark-avro_2.12;3.2.0 in central
	found org.tukaani#xz;1.8 in central
	found org.spark-project.spark#unused;1.0.0 in central
	found org.testng#testng;6.8.8 in central
	found org.beanshell#bsh;2.0b4 in central
	found com.beust#jcommander;1.27 in central
	found com.microsoft.azure#synapseml-deep-learning_2.12;0.9.5 in central
	found com.microsoft.azure#synapseml-opencv_2.12;0.9.5 in central
	found org.openpnp#opencv;3.2.0-1 in central
	found com.microsoft.cntk#cntk;2.4 in central
	found com.microsoft.onnxruntime#onnxruntime_gpu;1.8.1 in central
	found com.microsoft.azure#synapseml-cognitive_2.12;0.9.5 in central
	found com.microsoft.cognitiveservices.speech#client-jar-sdk;1.14.0 in central
	found com.azure#azure-storage-blob;12.14.2 in central
	found com.azure#azure-core;1.22.0 in central
	found com.fasterxml.jackson.core#jackson-annotations;2.12.5 in central
	found com.fasterxml.jackson.core#jackson-core;2.12.5 in central
	found com.fasterxml.jackson.core#jackson-databind;2.12.5 in central
	found com.fasterxml.jackson.datatype#jackson-datatype-jsr310;2.12.5 in central
	found com.fasterxml.jackson.dataformat#jackson-dataformat-xml;2.12.5 in central
	found com.fasterxml.jackson.module#jackson-module-jaxb-annotations;2.12.5 in central
	found jakarta.xml.bind#jakarta.xml.bind-api;2.3.2 in central
	found jakarta.activation#jakarta.activation-api;1.2.1 in central
	found org.codehaus.woodstox#stax2-api;4.2.1 in central
	found com.fasterxml.woodstox#woodstox-core;6.2.4 in central
	found org.slf4j#slf4j-api;1.7.32 in central
	found io.projectreactor#reactor-core;3.4.10 in central
	found org.reactivestreams#reactive-streams;1.0.3 in central
	found io.netty#netty-tcnative-boringssl-static;2.0.43.Final in central
	found com.azure#azure-core-http-netty;1.11.2 in central
	found io.netty#netty-handler;4.1.68.Final in central
	found io.netty#netty-common;4.1.68.Final in central
	found io.netty#netty-resolver;4.1.68.Final in central
	found io.netty#netty-buffer;4.1.68.Final in central
	found io.netty#netty-transport;4.1.68.Final in central
	found io.netty#netty-codec;4.1.68.Final in central
	found io.netty#netty-handler-proxy;4.1.68.Final in central
	found io.netty#netty-codec-socks;4.1.68.Final in central
	found io.netty#netty-codec-http;4.1.68.Final in central
	found io.netty#netty-codec-http2;4.1.68.Final in central
	found io.netty#netty-transport-native-unix-common;4.1.68.Final in central
	found io.netty#netty-transport-native-epoll;4.1.68.Final in central
	found io.netty#netty-transport-native-kqueue;4.1.68.Final in central
	found io.projectreactor.netty#reactor-netty-http;1.0.11 in central
	found io.netty#netty-resolver-dns;4.1.68.Final in central
	found io.netty#netty-codec-dns;4.1.68.Final in central
	found io.netty#netty-resolver-dns-native-macos;4.1.68.Final in central
	found io.projectreactor.netty#reactor-netty-core;1.0.11 in central
	found com.azure#azure-storage-common;12.14.1 in central
	found com.azure#azure-storage-internal-avro;12.1.2 in central
	found com.azure#azure-ai-textanalytics;5.1.4 in central
	found com.microsoft.azure#synapseml-vw_2.12;0.9.5 in central
	found com.github.vowpalwabbit#vw-jni;8.9.1 in central
	found com.microsoft.azure#synapseml-lightgbm_2.12;0.9.5 in central
	found com.microsoft.ml.lightgbm#lightgbmlib;3.2.110 in central
:: resolution report :: resolve 828ms :: artifacts dl 15ms
	:: modules in use:
	com.azure#azure-ai-textanalytics;5.1.4 from central in [default]
	com.azure#azure-core;1.22.0 from central in [default]
	com.azure#azure-core-http-netty;1.11.2 from central in [default]
	com.azure#azure-storage-blob;12.14.2 from central in [default]
	com.azure#azure-storage-common;12.14.1 from central in [default]
	com.azure#azure-storage-internal-avro;12.1.2 from central in [default]
	com.beust#jcommander;1.27 from central in [default]
	com.chuusai#shapeless_2.12;2.3.2 from central in [default]
	com.fasterxml.jackson.core#jackson-annotations;2.12.5 from central in [default]
	com.fasterxml.jackson.core#jackson-core;2.12.5 from central in [default]
	com.fasterxml.jackson.core#jackson-databind;2.12.5 from central in [default]
	com.fasterxml.jackson.dataformat#jackson-dataformat-xml;2.12.5 from central in [default]
	com.fasterxml.jackson.datatype#jackson-datatype-jsr310;2.12.5 from central in [default]
	com.fasterxml.jackson.module#jackson-module-jaxb-annotations;2.12.5 from central in [default]
	com.fasterxml.woodstox#woodstox-core;6.2.4 from central in [default]
	com.github.vowpalwabbit#vw-jni;8.9.1 from central in [default]
	com.jcraft#jsch;0.1.54 from central in [default]
	com.linkedin.isolation-forest#isolation-forest_3.2.0_2.12;2.0.8 from central in [default]
	com.microsoft.azure#synapseml-cognitive_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml-core_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml-deep-learning_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml-lightgbm_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml-opencv_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml-vw_2.12;0.9.5 from central in [default]
	com.microsoft.azure#synapseml_2.12;0.9.5 from central in [default]
	com.microsoft.cntk#cntk;2.4 from central in [default]
	com.microsoft.cognitiveservices.speech#client-jar-sdk;1.14.0 from central in [default]
	com.microsoft.ml.lightgbm#lightgbmlib;3.2.110 from central in [default]
	com.microsoft.onnxruntime#onnxruntime_gpu;1.8.1 from central in [default]
	commons-codec#commons-codec;1.10 from central in [default]
	commons-logging#commons-logging;1.2 from central in [default]
	io.netty#netty-buffer;4.1.68.Final from central in [default]
	io.netty#netty-codec;4.1.68.Final from central in [default]
	io.netty#netty-codec-dns;4.1.68.Final from central in [default]
	io.netty#netty-codec-http;4.1.68.Final from central in [default]
	io.netty#netty-codec-http2;4.1.68.Final from central in [default]
	io.netty#netty-codec-socks;4.1.68.Final from central in [default]
	io.netty#netty-common;4.1.68.Final from central in [default]
	io.netty#netty-handler;4.1.68.Final from central in [default]
	io.netty#netty-handler-proxy;4.1.68.Final from central in [default]
	io.netty#netty-resolver;4.1.68.Final from central in [default]
	io.netty#netty-resolver-dns;4.1.68.Final from central in [default]
	io.netty#netty-resolver-dns-native-macos;4.1.68.Final from central in [default]
	io.netty#netty-tcnative-boringssl-static;2.0.43.Final from central in [default]
	io.netty#netty-transport;4.1.68.Final from central in [default]
	io.netty#netty-transport-native-epoll;4.1.68.Final from central in [default]
	io.netty#netty-transport-native-kqueue;4.1.68.Final from central in [default]
	io.netty#netty-transport-native-unix-common;4.1.68.Final from central in [default]
	io.projectreactor#reactor-core;3.4.10 from central in [default]
	io.projectreactor.netty#reactor-netty-core;1.0.11 from central in [default]
	io.projectreactor.netty#reactor-netty-http;1.0.11 from central in [default]
	io.spray#spray-json_2.12;1.3.2 from central in [default]
	jakarta.activation#jakarta.activation-api;1.2.1 from central in [default]
	jakarta.xml.bind#jakarta.xml.bind-api;2.3.2 from central in [default]
	org.apache.httpcomponents#httpclient;4.5.6 from central in [default]
	org.apache.httpcomponents#httpcore;4.4.10 from central in [default]
	org.apache.httpcomponents#httpmime;4.5.6 from central in [default]
	org.apache.spark#spark-avro_2.12;3.2.0 from central in [default]
	org.beanshell#bsh;2.0b4 from central in [default]
	org.codehaus.woodstox#stax2-api;4.2.1 from central in [default]
	org.openpnp#opencv;3.2.0-1 from central in [default]
	org.reactivestreams#reactive-streams;1.0.3 from central in [default]
	org.scala-lang#scala-reflect;2.12.4 from central in [default]
	org.scalactic#scalactic_2.12;3.0.5 from central in [default]
	org.slf4j#slf4j-api;1.7.32 from central in [default]
	org.spark-project.spark#unused;1.0.0 from central in [default]
	org.testng#testng;6.8.8 from central in [default]
	org.tukaani#xz;1.8 from central in [default]
	org.typelevel#macro-compat_2.12;1.1.1 from central in [default]
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   69  |   0   |   0   |   0   ||   69  |   0   |
	---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-c0c8a1b1-ebd8-41ae-9b2a-625051f08cc3
	confs: [default]
	0 artifacts copied, 69 already retrieved (0kB/13ms)
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final.jar does not exist, skipping.
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-kqueue-4.1.68.Final.jar does not exist, skipping.
22/02/01 13:31:14 WARN DependencyUtils: Local jar /home/ec2-user/.ivy2/jars/io.netty_netty-resolver-dns-native-macos-4.1.68.Final.jar does not exist, skipping.
22/02/01 13:31:14 INFO SparkContext: Running Spark version 2.4.7-amzn-1
22/02/01 13:31:14 INFO SparkContext: Submitted application: pyspark-shell
22/02/01 13:31:14 INFO SecurityManager: Changing view acls to: ec2-user
22/02/01 13:31:14 INFO SecurityManager: Changing modify acls to: ec2-user
22/02/01 13:31:14 INFO SecurityManager: Changing view acls groups to:
22/02/01 13:31:14 INFO SecurityManager: Changing modify acls groups to:
22/02/01 13:31:14 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(ec2-user); groups with view permissions: Set(); users  with modify permissions: Set(ec2-user); groups with modify permissions: Set()
22/02/01 13:31:14 INFO Utils: Successfully started service 'sparkDriver' on port 33873.
22/02/01 13:31:14 INFO SparkEnv: Registering MapOutputTracker
22/02/01 13:31:14 INFO SparkEnv: Registering BlockManagerMaster
22/02/01 13:31:14 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
22/02/01 13:31:14 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
22/02/01 13:31:14 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-8f6e6fa5-f663-4604-8c74-05e280c93584
22/02/01 13:31:14 INFO MemoryStore: MemoryStore started with capacity 27.0 GB
22/02/01 13:31:14 INFO SparkEnv: Registering OutputCommitCoordinator
22/02/01 13:31:14 INFO Utils: Successfully started service 'SparkUI' on port 4040.
22/02/01 13:31:14 INFO SparkUI: Bound SparkUI to <IP_ADDRESS>, and started at http://ip-1-2-3-4.eu-west-1.compute.internal:4040
22/02/01 13:31:15 INFO RMProxy: Connecting to ResourceManager at ip-10-3-32-74.eu-west-1.compute.internal/<IP_ADDRESS>:8032
22/02/01 13:31:15 INFO Client: Requesting a new application from cluster with 3 NodeManagers
22/02/01 13:31:15 INFO Configuration: resource-types.xml not found
22/02/01 13:31:15 INFO ResourceUtils: Unable to find 'resource-types.xml'.
22/02/01 13:31:15 INFO ResourceUtils: Adding resource type - name = memory-mb, units = Mi, type = COUNTABLE
22/02/01 13:31:15 INFO ResourceUtils: Adding resource type - name = vcores, units = , type = COUNTABLE
22/02/01 13:31:15 INFO Client: Verifying our application has not requested more than the maximum memory capability of the cluster (57344 MB per container)
22/02/01 13:31:15 INFO Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
22/02/01 13:31:15 INFO Client: Setting up container launch context for our AM
22/02/01 13:31:15 INFO Client: Setting up the launch environment for our AM container
22/02/01 13:31:15 INFO Client: Preparing resources for our AM container
22/02/01 13:31:15 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
22/02/01 13:31:17 INFO Client: Uploading resource file:/tmp/spark-534569b4-be5f-4337-9a66-2d1dca59fbee/__spark_libs__5179491898158020782.zip -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/__spark_libs__5179491898158020782.zip
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-core_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-core_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-deep-learning_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-deep-learning_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-cognitive_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-cognitive_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-vw_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-vw_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-lightgbm_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-lightgbm_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.azure_synapseml-opencv_2.12-0.9.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.azure_synapseml-opencv_2.12-0.9.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.scalactic_scalactic_2.12-3.0.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.scalactic_scalactic_2.12-3.0.5.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.spray_spray-json_2.12-1.3.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.spray_spray-json_2.12-1.3.2.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.jcraft_jsch-0.1.54.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.jcraft_jsch-0.1.54.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.apache.httpcomponents_httpclient-4.5.6.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.apache.httpcomponents_httpclient-4.5.6.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.apache.httpcomponents_httpmime-4.5.6.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.apache.httpcomponents_httpmime-4.5.6.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.linkedin.isolation-forest_isolation-forest_3.2.0_2.12-2.0.8.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.linkedin.isolation-forest_isolation-forest_3.2.0_2.12-2.0.8.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.scala-lang_scala-reflect-2.12.4.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.scala-lang_scala-reflect-2.12.4.jar
22/02/01 13:31:17 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.apache.httpcomponents_httpcore-4.4.10.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.apache.httpcomponents_httpcore-4.4.10.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/commons-logging_commons-logging-1.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/commons-logging_commons-logging-1.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/commons-codec_commons-codec-1.10.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/commons-codec_commons-codec-1.10.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.chuusai_shapeless_2.12-2.3.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.chuusai_shapeless_2.12-2.3.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.apache.spark_spark-avro_2.12-3.2.0.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.apache.spark_spark-avro_2.12-3.2.0.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.testng_testng-6.8.8.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.testng_testng-6.8.8.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.typelevel_macro-compat_2.12-1.1.1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.typelevel_macro-compat_2.12-1.1.1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.tukaani_xz-1.8.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.tukaani_xz-1.8.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.spark-project.spark_unused-1.0.0.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.spark-project.spark_unused-1.0.0.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.beanshell_bsh-2.0b4.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.beanshell_bsh-2.0b4.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.beust_jcommander-1.27.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.beust_jcommander-1.27.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.cntk_cntk-2.4.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.cntk_cntk-2.4.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.onnxruntime_onnxruntime_gpu-1.8.1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.onnxruntime_onnxruntime_gpu-1.8.1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.openpnp_opencv-3.2.0-1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.openpnp_opencv-3.2.0-1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.microsoft.cognitiveservices.speech_client-jar-sdk-1.14.0.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.microsoft.cognitiveservices.speech_client-jar-sdk-1.14.0.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-storage-blob-12.14.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-storage-blob-12.14.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-ai-textanalytics-5.1.4.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-ai-textanalytics-5.1.4.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-core-1.22.0.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-core-1.22.0.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-core-http-netty-1.11.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-core-http-netty-1.11.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-storage-common-12.14.1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-storage-common-12.14.1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.azure_azure-storage-internal-avro-12.1.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.azure_azure-storage-internal-avro-12.1.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.core_jackson-annotations-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.core_jackson-core-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.core_jackson-databind-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.datatype_jackson-datatype-jsr310-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.datatype_jackson-datatype-jsr310-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.dataformat_jackson-dataformat-xml-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.dataformat_jackson-dataformat-xml-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.slf4j_slf4j-api-1.7.32.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.slf4j_slf4j-api-1.7.32.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.projectreactor_reactor-core-3.4.10.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.projectreactor_reactor-core-3.4.10.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-tcnative-boringssl-static-2.0.43.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-tcnative-boringssl-static-2.0.43.Final.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.jackson.module_jackson-module-jaxb-annotations-2.12.5.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.jackson.module_jackson-module-jaxb-annotations-2.12.5.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.codehaus.woodstox_stax2-api-4.2.1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.codehaus.woodstox_stax2-api-4.2.1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/com.fasterxml.woodstox_woodstox-core-6.2.4.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/com.fasterxml.woodstox_woodstox-core-6.2.4.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/jakarta.xml.bind_jakarta.xml.bind-api-2.3.2.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/jakarta.xml.bind_jakarta.xml.bind-api-2.3.2.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/jakarta.activation_jakarta.activation-api-1.2.1.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/jakarta.activation_jakarta.activation-api-1.2.1.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/org.reactivestreams_reactive-streams-1.0.3.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/org.reactivestreams_reactive-streams-1.0.3.jar
22/02/01 13:31:18 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-handler-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-handler-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-handler-proxy-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-handler-proxy-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-buffer-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-buffer-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-codec-http-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-codec-http-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-codec-http2-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-codec-http2-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-unix-common-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-transport-native-unix-common-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Uploading resource file:/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final.jar -> hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007/io.netty_netty-transport-native-epoll-4.1.68.Final.jar
22/02/01 13:31:19 INFO Client: Deleted staging directory hdfs://ip-10-3-32-74.eu-west-1.compute.internal:8020/user/ec2-user/.sparkStaging/application_1643705154665_0007
22/02/01 13:31:19 ERROR SparkContext: Error initializing SparkContext.
java.io.FileNotFoundException: File file:/home/ec2-user/.ivy2/jars/io.netty_netty-transport-native-epoll-4.1.68.Final.jar does not exist
	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:639)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:865)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:629)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:456)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:365)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:316)
	at org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:384)
	at org.apache.spark.deploy.yarn.Client.org$apache$spark$deploy$yarn$Client$$distribute$1(Client.scala:476)
	at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10$$anonfun$apply$6.apply(Client.scala:598)
	at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10$$anonfun$apply$6.apply(Client.scala:597)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:597)
	at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:596)
	at scala.collection.immutable.List.foreach(List.scala:392)
	at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:596)
	at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:869)
	at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:178)
	at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:57)
	at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:188)
	at org.apache.spark.SparkContext.<init>(SparkContext.scala:515)
	at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:238)
	at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
	at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.lang.Thread.run(Thread.java:748)
22/02/01 13:31:19 INFO SparkUI: Stopped Spark web UI at http://ip-1-2-3-4.eu-west-1.compute.internal:4040
22/02/01 13:31:19 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
22/02/01 13:31:19 INFO YarnClientSchedulerBackend: Stopped
22/02/01 13:31:19 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
22/02/01 13:31:19 INFO MemoryStore: MemoryStore cleared
22/02/01 13:31:19 INFO BlockManager: BlockManager stopped
22/02/01 13:31:19 INFO BlockManagerMaster: BlockManagerMaster stopped
22/02/01 13:31:19 WARN MetricsSystem: Stopping a MetricsSystem that is not running
22/02/01 13:31:19 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
22/02/01 13:31:19 INFO SparkContext: Successfully stopped SparkContext)

hmm, this has happened with another unrelated library, making me think it might not have anything to do with SynapseML, no idea what it is though

Hey thanks for reaching out! I see you are still on Spark 2.4. SynapseML 0.9.5 is for spark 3.2 and there are some known dependency challenges that will arise if these versions are mismatched. I would imagine this would be a cause of some of the challenges you are facing

@jonathansp only thing that worked for me was to manually rename the jars in their directory. it seems like whatever's managing the download isn't reliably renaming them once theyre down. its a little painful but it got me past those initial errors at least :) If you do find a solution Id appreciate a heads up :)
@mhamilton723 good catch, I missed that. Will close this for sure. I saw the same errors importing other mvn packages so I dont think its actually SynapseML. Do you maybe know what the latest version of mmlspark/synapseml will work on 2.4? we're tied to EMR.
Thanks for the feedback

@philmassie thanks, indeed renaming the file worked, as something has to do with the gradle versionless build. However, it's hard to maintain.
@mhamilton723 absolutely, I downgraded SynapseML to 0.9.4 and it worked with spark 3.1.
Thank you guys.


Fix nil when there is not background color set

If there is not background color set e.g in default created buttons, a nil stops execution when the optional is unwrapped.

Hey @juanjovn
To be honest I rather have the code crash in this instance (that's why I used the IUO), the user is required to have a background color, or the effect can be jarring, so I rather have an early crash that is easier to debug. An alternative that I might consider is add a nil check with a fatalError with a more detailed explanation, but it might be overkill for such a simple use case.


docs(skiplist): update insert function document

"it will be removed before inserting the new one", it looks like there are two steps involved in the process described.
#1122

Thanks. It seems to me that “using CAS” is a description of implementation details and does not explain what specific behavior is provided. (I guess you want to document the semantics that no intermediate state is observed during the update.)


Simplify the "Ack" feature protocol and move to Accepted

I think the simple "Ack" protocol should be moved to Accepted as that message is in some use. However in looking at the RFC, there are some more complex features in the RFC that I don't think would be considered "Accepted".
I recommend that we remove the "~please-ack" decorator and the other advanced features from the RFC, leaving just the ACK message.  The "~please-ack" decorator could go into a separate RFC if there is interest, possibly with the other related advanced features. There is not really a reason that the Ack message and the ~please-ack decorator need to be in the same RFC vs. being in separate ones with the ~please-ack decorator referencing the Ack message.
Does anyone object to that approach?
Has anyone implemented the "Ack" message?  The ~please-ack decorator?

aries-framework-go implements the did-exchange protocol which itself adopts the ack message to notify the inviter upon successful completion of the did-exchange.
RFC0015 is not a protocol since it lacks core ingredients of a protocol (roles and states). Both the messages it defines are ones up for adoption by other protocols - I'm temporarily calling these "orphaned messages".
I agree with removing ~please-ack into a separate RFC because, as a driver of a protocol's state, the use of ack should be strictly defined by a protocol. I agree that ~please-ack introduces needless complexity and ambiguity.
I also agree with moving this RFC (with just ack) into ACCEPTED status. This RFC by its own can't be tested with the protocol suite.

This issue has been addressed with the merging of #316. The 0015-acks RFC now covers only the ACK message, and a new RFC has been added for ~please_ack.
Closing.