typing-extensions package is conflicted

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS 11.5.2
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: -
TensorFlow installed from (source or binary): binary
TensorFlow version: 2.6.0
Python version: 3.8.10
Installed using virtualenv? pip? conda?: virtualenv and pip
Bazel version (if compiling from source): -
GCC/Compiler version (if compiling from source): -
CUDA/cuDNN version: -
GPU model and memory: -

Describe the problem
I installed tensorflow using pip on the virtualenv. The typing-extensions package is conflicted with black package dependency. black==21.8b0 is required typing-extensions>=<IP_ADDRESS>. But, tensorflow has 'typing_extensions ~= 3.7.4 dependency. Are there any plans to upgrade the typing-extensions dependency package version?
Provide the exact sequence of commands / steps that you executed before running into the problem
$ python3 -m venv venv
$ . venv/bin/activate
$ pip install -U pip
$ pip install black
$ pip install tensorflow

Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
$ pip install tensorflow
Requirement already satisfied: tensorflow in ./venv/lib/python3.8/site-packages (2.6.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.1.2)
Requirement already satisfied: astunparse~=1.6.3 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.6.3)
Requirement already satisfied: opt-einsum~=3.3.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.3.0)
Collecting typing-extensions~=3.7.4
  Using cached typing_extensions-<IP_ADDRESS>-py3-none-any.whl (22 kB)
Requirement already satisfied: numpy~=1.19.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.19.5)
Requirement already satisfied: wrapt~=1.12.1 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.12.1)
Requirement already satisfied: tensorflow-estimator~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: six~=1.15.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.15.0)
Requirement already satisfied: wheel~=0.35 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.37.0)
Requirement already satisfied: google-pasta~=0.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.2.0)
Requirement already satisfied: flatbuffers~=1.12.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.12)
Requirement already satisfied: tensorboard~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: keras~=2.6 in ./venv/lib/python3.8/site-packages (from tensorflow) (2.6.0)
Requirement already satisfied: protobuf>=3.9.2 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.17.3)
Requirement already satisfied: gast==0.4.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.4.0)
Requirement already satisfied: termcolor~=1.1.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.1.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (1.39.0)
Requirement already satisfied: clang~=5.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (5.0)
Requirement already satisfied: absl-py~=0.10 in ./venv/lib/python3.8/site-packages (from tensorflow) (0.13.0)
Requirement already satisfied: h5py~=3.1.0 in ./venv/lib/python3.8/site-packages (from tensorflow) (3.1.0)
Requirement already satisfied: markdown>=2.6.8 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (3.3.4)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.8.0)
Requirement already satisfied: werkzeug>=0.11.15 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (1.35.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (0.6.1)
Requirement already satisfied: setuptools>=41.0.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (56.0.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (0.4.5)
Requirement already satisfied: requests<3,>=2.21.0 in ./venv/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow) (2.26.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (4.2.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in ./venv/lib/python3.8/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (4.7.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in ./venv/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow) (1.3.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in ./venv/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow) (0.4.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (1.26.5)
Requirement already satisfied: idna<4,>=2.5 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (3.2)
Requirement already satisfied: certifi>=2017.4.17 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (2021.5.30)
Requirement already satisfied: charset-normalizer~=2.0.0 in ./venv/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow) (2.0.4)
Requirement already satisfied: oauthlib>=3.0.0 in ./venv/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow) (3.1.1)
Installing collected packages: typing-extensions
  Attempting uninstall: typing-extensions
    Found existing installation: typing-extensions <IP_ADDRESS>
    Uninstalling typing-extensions-<IP_ADDRESS>:
      Successfully uninstalled typing-extensions-<IP_ADDRESS>
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
black 21.8b0 requires typing-extensions>=<IP_ADDRESS>, but you have typing-extensions <IP_ADDRESS> which is incompatible.
Successfully installed typing-extensions-<IP_ADDRESS>

Hi @sanatmpa1 ,Could you please look into this issue?

If just the two are specified then pip is able to figure out the dependency mismatch and find a version of black (v21.7.b0) that will work

Full Example in a python:3.9 Docker image:
$ docker run --rm -ti python:3.9 /bin/bash
root@12adf30cd4d4:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@12adf30cd4d4:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@12adf30cd4d4:/# python -m pip install black tensorflow
Collecting black
  Downloading black-21.8b0-py3-none-any.whl (148 kB)
     |████████████████████████████████| 148 kB 6.3 MB/s
Collecting tensorflow
  Downloading tensorflow-2.6.0-cp39-cp39-manylinux2010_x86_64.whl (458.4 MB)
     |████████████████████████████████| 458.4 MB 40.8 MB/s
Collecting pathspec<1,>=0.9.0
  Downloading pathspec-0.9.0-py2.py3-none-any.whl (31 kB)
Collecting tomli<2.0.0,>=0.2.6
  Downloading tomli-1.2.1-py3-none-any.whl (11 kB)
Collecting click>=7.1.2
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 9.5 MB/s
Collecting mypy-extensions>=0.4.3
  Downloading mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Collecting platformdirs>=2
  Downloading platformdirs-2.3.0-py3-none-any.whl (13 kB)
Collecting typing-extensions>=<IP_ADDRESS>
  Downloading typing_extensions-<IP_ADDRESS>-py3-none-any.whl (26 kB)
Collecting regex>=2020.1.8
  Downloading regex-2021.8.28-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (759 kB)
     |████████████████████████████████| 759 kB 42.1 MB/s
Collecting keras~=2.6
  Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 51.1 MB/s
Collecting clang~=5.0
  Downloading clang-5.0.tar.gz (30 kB)
Collecting h5py~=3.1.0
  Downloading h5py-3.1.0-cp39-cp39-manylinux1_x86_64.whl (4.4 MB)
     |████████████████████████████████| 4.4 MB 59.1 MB/s
Collecting keras-preprocessing~=1.1.2
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 2.4 MB/s
Collecting opt-einsum~=3.3.0
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 5.3 MB/s
Collecting grpcio<2.0,>=1.37.0
  Downloading grpcio-1.39.0-cp39-cp39-manylinux2014_x86_64.whl (4.3 MB)
     |████████████████████████████████| 4.3 MB 31.8 MB/s
Collecting flatbuffers~=1.12.0
  Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting six~=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting gast==0.4.0
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting tensorboard~=2.6
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 29.8 MB/s
Collecting tensorflow
  Downloading tensorflow-2.5.1-cp39-cp39-manylinux2010_x86_64.whl (454.5 MB)
     |████████████████████████████████| 454.5 MB 23.6 MB/s
Collecting grpcio~=1.34.0
  Downloading grpcio-1.34.1-cp39-cp39-manylinux2014_x86_64.whl (4.0 MB)
     |████████████████████████████████| 4.0 MB 38.4 MB/s
Collecting keras-nightly~=2.5.0.dev
  Downloading keras_nightly-2.5.0.dev2021032900-py2.py3-none-any.whl (1.2 MB)
     |████████████████████████████████| 1.2 MB 72.1 MB/s
Collecting tensorflow
  Downloading tensorflow-2.5.0-cp39-cp39-manylinux2010_x86_64.whl (454.4 MB)
     |████████████████████████████████| 454.4 MB 73 kB/s
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of black to determine which version is compatible with other requirements. This could take a while.
Collecting black
  Downloading black-21.7b0-py3-none-any.whl (141 kB)
     |████████████████████████████████| 141 kB 52.8 MB/s
Collecting appdirs
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting typing-extensions~=3.7.4
  Downloading typing_extensions-<IP_ADDRESS>-py3-none-any.whl (22 kB)
Collecting termcolor~=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Collecting astunparse~=1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting numpy~=1.19.2
  Downloading numpy-1.19.5-cp39-cp39-manylinux2010_x86_64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB 41.4 MB/s
Collecting absl-py~=0.10
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 28.9 MB/s
Collecting tensorflow-estimator~=2.6
  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 29.1 MB/s
Requirement already satisfied: wheel~=0.35 in /usr/local/venv/lib/python3.9/site-packages (from tensorflow) (0.37.0)
Collecting wrapt~=1.12.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting protobuf>=3.9.2
  Downloading protobuf-3.17.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB)
     |████████████████████████████████| 1.0 MB 34.8 MB/s
Collecting google-pasta~=0.2
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 8.0 MB/s
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 40.0 MB/s
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
     |████████████████████████████████| 4.9 MB 32.5 MB/s
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/venv/lib/python3.9/site-packages (from tensorboard~=2.6->tensorflow) (58.0.2)
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 32.8 MB/s
Collecting requests<3,>=2.21.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 1.2 MB/s
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 29.1 MB/s
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 10.2 MB/s
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 32.7 MB/s
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 8.6 MB/s
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 37.6 MB/s
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.4-py3-none-any.whl (36 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 10.0 MB/s
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 28.4 MB/s
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 26.5 MB/s
Building wheels for collected packages: clang, termcolor, wrapt
  Building wheel for clang (setup.py) ... done
  Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30692 sha256=26573dc9851cb201eebc9f3acd403834783398b5f7140b7fd86d1942a44c38aa
  Stored in directory: /root/.cache/pip/wheels/3a/ce/7a/27094f689461801c934296d07078773603663dfcaca63bb064
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4847 sha256=e374182ff99d7aa8fff47b8b498e27a1a475777ed7c2ecad2abc722f62b221bf
  Stored in directory: /root/.cache/pip/wheels/b6/0d/90/0d1bbd99855f99cb2f6c2e5ff96f8023fad8ec367695f7d72d
  Building wheel for wrapt (setup.py) ... done
  Created wheel for wrapt: filename=wrapt-1.12.1-cp39-cp39-linux_x86_64.whl size=73034 sha256=92b09a8a10deab562a852acbbfe5c7cda670f0e4c1c47103f2a0591854ec8fa7
  Stored in directory: /root/.cache/pip/wheels/98/23/68/efe259aaca055e93b08e74fbe512819c69a2155c11ba3c0f10
Successfully built clang termcolor wrapt
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, werkzeug, tensorboard-plugin-wit, tensorboard-data-server, protobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, typing-extensions, tomli, termcolor, tensorflow-estimator, tensorboard, regex, pathspec, opt-einsum, mypy-extensions, keras-preprocessing, keras, h5py, google-pasta, gast, flatbuffers, click, clang, astunparse, appdirs, tensorflow, black
Successfully installed absl-py-0.13.0 appdirs-1.4.4 astunparse-1.6.3 black-21.7b0 cachetools-4.2.2 certifi-2021.5.30 charset-normalizer-2.0.4 clang-5.0 click-8.0.1 flatbuffers-1.12 gast-0.4.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.39.0 h5py-3.1.0 idna-3.2 keras-2.6.0 keras-preprocessing-1.1.2 markdown-3.3.4 mypy-extensions-0.4.3 numpy-1.19.5 oauthlib-3.1.1 opt-einsum-3.3.0 pathspec-0.9.0 protobuf-3.17.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 regex-2021.8.28 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.15.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 tensorflow-2.6.0 tensorflow-estimator-2.6.0 termcolor-1.1.0 tomli-1.2.1 typing-extensions-<IP_ADDRESS> urllib3-1.26.6 werkzeug-2.0.1 wrapt-1.12.1
(venv) root@12adf30cd4d4:/# pip list | grep "black\|tensorflow"
black                   21.7b0
tensorflow              2.6.0
tensorflow-estimator    2.6.0

but that tensorflow is pinning down to the patch release here (typing-extensions~=3.7.4) is pretty restrictive (and making it so that everyone using TensorFlow needs to treat it like an application and not a library. c.f.  PR #40789 for another example of problems coming from being overly restrictive).
In more complicated environments though I can get pip to fail.

An example for a setup where the dependency resolution fails is the combination of black, tensorflow and tensorflow-probability. Using a python:3.8 Docker environment (otherwise equivalent setup to @matthewfeickert's comment above),
python -m pip install black tensorflow tensorflow-probability

installs the following (output of pip list | grep "black\|tensorflow"):
black                   21.8b0
tensorflow              2.3.4
tensorflow-estimator    2.3.0
tensorflow-probability  0.13.0

The import of tensorflow-probability after tensorflow then fails:
python -c "import tensorflow, tensorflow_probability"

results in
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.5; Detected an installation of version 2.3.4. Please upgrade TensorFlow to proceed.

but that tensorflow is pinning down to the patch release here (typing-extensions~=3.7.4)

Doesn't this mean any version at 3.7.4 or after but not after 3.8 or 4.0? That is, 3.7.5 could still be installed.

Doesn't this mean any version at 3.7.4 or after but not after 3.8 or 4.0? That is, 3.7.5 could still be installed.

Yup, and indeed the compatible release syntax does that (get the latest in the v3.7.X series for X>=4 or more concisely ">=3.7.4,<3.8", which for typing-extensions is <IP_ADDRESS>)
$ docker run --rm -ti python:3.9 /bin/bash
root@b4bbf0636f94:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@b4bbf0636f94:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@b4bbf0636f94:/# python -m pip --quiet install black tensorflow tensorflow-probability
(venv) root@b4bbf0636f94:/# python -m pip list | grep "black\|tensorflow\|typing"
black                   21.7b0
tensorflow              2.6.0
tensorflow-estimator    2.6.0
tensorflow-probability  0.13.0
typing-extensions       <IP_ADDRESS>

Python 3.8 struggles though
$ docker run --rm -ti python:3.8 /bin/bash
root@521504a6d9b2:/# python -m venv /usr/local/venv && . /usr/local/venv/bin/activate
(venv) root@521504a6d9b2:/# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@521504a6d9b2:/# python -m pip --quiet install black tensorflow tensorflow-probability
(venv) root@521504a6d9b2:/# python -m pip list | grep "black\|tensorflow\|typing"
black                   21.8b0
tensorflow              2.3.4
tensorflow-estimator    2.3.0
tensorflow-probability  0.13.0
typing-extensions       <IP_ADDRESS>
(venv) root@521504a6d9b2:/# python -c "import tensorflow as tf; import tensorflow_probability as tfp"
2021-09-07 22:21:56.457553: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2021-09-07 22:21:56.457576: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/__init__.py", line 20, in <module>
    from tensorflow_probability import substrates
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/substrates/__init__.py", line 21, in <module>
    from tensorflow_probability.python.internal import all_util
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py", line 142, in <module>
    dir(globals()[pkg_name])  # Forces loading the package from its lazy loader.
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 61, in __dir__
    module = self._load()
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/internal/lazy_loader.py", line 41, in _load
    self._on_first_access()
  File "/usr/local/venv/lib/python3.8/site-packages/tensorflow_probability/python/__init__.py", line 63, in _validate_tf_environment
    raise ImportError(
ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.5; Detected an installation of version 2.3.4. Please upgrade TensorFlow to proceed.

This issue makes pip unable to install tensorflow==2.6.0 alongside astroid==2.8.0, compare
https://github.com/PyCQA/astroid/commit/40ea1a3b8e52bbfed43deb1725cde461f4bd8a96#diff-fa602a8a75dc9dcc92261bac5f533c2a85e34fcceaff63b3a3a81d9acde2fc52

I had to temporarily pin black in my dev-requirements.in / requirements.txt to work around this issue:
# TODO: Unpin black once upstream tensorflow conflicting dependency is resolved:
#    https://github.com/tensorflow/tensorflow/issues/51743
black == 21.7b0

Hi @mihaimaruseac, I ran into this issue myself so I just submitted a very small PR that I believe should fix it (#52136). Let me know if there are any changes you would like to it! 🙂

This seems to be fixed on master by https://github.com/tensorflow/tensorflow/commit/41e66148999080da5b2b80a59740a8f2a59f17da. The install_requires is now typing_extensions >= 3.6.6.

Closing as this has been fixed while I was OOO.

For anyone wondering, this did not make it into tensorflow==2.7.0rc0.

The commit has been cherry-picked, so the next release candidate should fully resolve this

Hi. Unfortunately we cannot do a cherry pick of this into TF 2.6 given the magnitude of the change and potential for breakages. Patch releases don't go through the same extensive QA process as final releases, so we're conservative in what changes get included there.

Ok, yeah that full change is a lot more than I realised initially. Would a PR for just the specific loosening of the typing_extensions compatibility against 2.6 be something that would be entertained?

That would potentially work

@mihaimaruseac @woodyza Since there's consideration of including this change in a 2.6 release, I went ahead and rebased my changes on the r2.6 branch and opened a new PR here:
https://github.com/tensorflow/tensorflow/pull/53250
I would have just reopened the old PR, but I had deleted the fork, so that wasn't possible 😬
If there's a preference to more exactly match the changes cherry-picked into 2.7, I can change the version constraint to typing-extensions >= 3.6.6 🙂

Awesome. Will review and approve and merge a while later when we do patch releases.


[$250] Storybook stuck loading

Regression from PR: https://github.com/Expensify/App/pull/35552
If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number:
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers): n/a
Logs: https://stackoverflow.com/c/expensify/questions/4856 n/a
Expensify/Expensify Issue URL: n/a
Issue reported by: @getusha and @pecanoro
Slack conversation: https://expensify.slack.com/archives/C01GTK53T8Q/p1708428340426319
Action Performed:
Run npm run storybook
Expected Result:
Storybook to properly load in http://localhost:6006/
Actual Result:
UI skeleton is only loaded and console error returns:

Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

View all open jobs on GitHub

As discussed here, @getusha will fix it!

Raised the PR

Proposal
Please re-state the problem that we are trying to solve in this issue.
The problem we are trying to solve in this issue is that the Storybook UI is not loading properly and is only displaying a UI skeleton. The console error message is not provided, but it is likely related to an infinite loop or performance issue caused by the  useEffect  hook in the  MyComponent  component.
What is the root cause of that problem?
The root cause of the problem is that the  useEffect  hook in the  MyComponent  component does not have a dependency array. This means that the effect runs every time the component renders, which can lead to infinite loops and performance issues.
What changes do you think we should make in order to solve the problem?
To solve the problem, we should add a dependency array to the  useEffect  hook that includes the  locale  variable. This will ensure that the effect only runs when the  locale  variable changes.
Code Solution
import React, {useEffect} from 'react';
import {StyleSheet, Text} from 'react-native';
import {LocaleContext} from '@root/LocaleContextProvider';
import type {Locale} from '@types/Locale';
import CONST from '@src/CONST';

const styles = StyleSheet.create({
    text: {
        fontSize: 14,
    },
});

const MyComponent = (props) => {
    const {locale} = React.useContext(LocaleContext);

    useEffect(() => {
        // For example, if we want to fetch data based on a user's locale:
        if (locale) {
            const localeCode = CONST.SUPPORTED_LOCALES[locale].code;
            // Make network request here using localeCode
        }
    }, [locale]);

    const someFunction = (translatedText) => {
        // For example, a function that takes the user's locale into account
        // to translate text
        return translatedText;
    };

    return (

    );
};

export default MyComponent;

This is ready for payout @mallenexpensify, automation didn't trigger on this one.

Thanks @getusha , whole lotta automation issued here :/

https://github.com/Expensify/App/pull/36939#issuecomment-1970130828

@getusha , @aimane-chnaif , can you please accept the job and reply here once you have?
https://www.upwork.com/jobs/~017343d0b98ad711bc

@mallenexpensify accepted, thanks!

Contributor: @getusha paid $250 via Upwork
Contributor+: @aimane-chnaif paid $250 via Upwork
Closing, no regression test needed, not customer facing.


Implement miden::sat::tx::add_asset_to_note

We should implement a procedure that allows the user to add an asset to a created note.

Closed by #674.


Remove the script for generating synthetic test-jars

This is now done on the Quarkus side.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/camel-quarkus-pr/240/


LuisDialog returns error after BotAddedToConversation to any message

Hi after I successfully have my response from BotAddedToConversation message I have the following error displayed as a response in my simulator;
{
    "message": "An error has occurred.",
    "exceptionMessage": "invalid need: expected Wait, have Done",
    "exceptionType": "Microsoft.Bot.Builder.Internals.Fibers.InvalidNeedException",
    "stackTrace": "   at Microsoft.Bot.Builder.Internals.Fibers.Wait`1.ValidateNeed(Need need)\r\n   at Microsoft.Bot.Builder.Internals.Fibers.Wait`1.Microsoft.Bot.Builder.Internals.Fibers.IWait.Post[D](D item)\r\n   at Microsoft.Bot.Builder.Internals.Fibers.Extensions.Post[T](IFiber fiber, T item)\r\n   at Microsoft.Bot.Builder.Dialogs.Internals.DialogContext.<Microsoft-Bot-Builder-Dialogs-Internals-IUserToBot-SendAsync>d__20.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at Microsoft.Bot.Builder.Dialogs.Conversation.<SendAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\r\n   at MyApp.MessagesController.<Post>d__0.MoveNext() in C:\\dev\\projects\\other\\msusur\\jupi-bot\\Jupi.App\\Controllers\\MessagesController.cs:line 28\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__3`1.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()"
}

here is my luis dialog. What might go wrong?
    [LuisModel("my model id", "subscription key")]
    [Serializable]
    public class MyQueryDialog : LuisDialog
    {
        [LuisIntent("LoadLinks")]
        public async Task ProcessLinkRequest(IDialogContext context, LuisResult result)
        {
            var entities = new List<EntityRecommendation>(result.Entities);
            if (entities.Count == 0)
            {
                await context.PostAsync("Thank you!");
            }
            context.Wait(MessageReceived);
        }
    }

Here is how I handle the messages
public async Task<Message> Post([FromBody]Message message)
        {
            if (message.Type == "Message")
            {
                return await Conversation.SendAsync(message, () => new MyQueryDialog(BuildForm));
            }
            return HandleSystemMessage(message);
        }

   private static IForm<QueryModel> BuildForm()
        {
            var builder = new FormBuilder<QueryModel>();

            return builder.AddRemainingFields().Build();
        }

You would see that error if your ProcessLinkRequest (for example) did not wait on MessageReceived at the end of the method.  If you reset the dialog state for the conversation (for example, by assigning message.BotPerUserInConversationData = null), does it reproduce?

@willportnoy fixed with the following code. I am not sure if this is the right way though...
    public async Task<Message> Post([FromBody]Message message)
        {
            if (message.Type == "Message")
            {
                message.BotPerUserInConversationData = null;
                return await Conversation.SendAsync(message, () => new SozlukQuery(BuildForm));
            }
            return HandleSystemMessage(message);
        }

Is the Model ID the same as the LUIS app id?

It's the ID that shows up the LUIS query url - the website will provide it to you.
        queryString["id"] = this.model.ModelID;
        queryString["subscription-key"] = this.model.SubscriptionKey;
        queryString["q"] = text;

Thank you!


CosmosDB: AAD auth related fix

Add the missing auth context when creating the barrier related requests.

yes, the Session and lower consistency are not affected by this issue.
see response in the code review comment.

At this time we do not expect the token type to be changed within the same Cosmos Client.
The main reason we want to read the token type from the provider rather is to be consistent, since that is what we inherit from the CosmosClient and not some default value that might have been used unintentionally (see the create() method on the RxDocumentServiceRequest class). Later in the same function we use that provider to handle auth headers and we risk a mismatch which can result into a null pointer exception or some other error.

/azp run java - cosmos - tests


Where nox should be installed

In the second chapter of Hypermodern Python you use pip install --user --upgrade nox to install nox did you do this inside the virtualenvironment created by poetry? OR you deactivate the virtualenvironment and install globally nox?
If you did it inside poetry virtualenvironment you use pip instead of poetry so that it doesn't stay in the pyptoject.toml packages?

The pip command installs Nox globally, into a user-specific directory.
Poetry’s virtual environment for your project is not activated in a normal shell. You use the poetry commands to modify it.

I use Pychar and when i'm working in a project with a virtual environment configurated this is activated every time I open the IDE. On the other side I have some questions, I use pylint instead flake8 and  have configured it to detect the modules that are not installed in the virtual environment, i.e. nox, when installed globally is not detected by pylint. if i install nox globally pycharm does not detect it. other thing in lint session if i not install nox locally as dev or dependency the session fail due to the same problem, pylint does not detect it and it throws up an error. what could be the best solution? to install nox as a development dependency?

Without seeing your project, two things come to mind.
The first is to re-use the virtualenv by passing --reuse-existing-virtualenvs to Nox. While poetry install is still run, this should be much faster as it will only perform updates if the versions in your environment do not match those in the lock file.
The second idea is a little more subtle. Do you use the --no-dev option of poetry install, followed by a manual installation of pylint? And is pylint listed as a development dependency? If so, Poetry will uninstall pylint on each run, which will slow down the session. For this reason, I have recently switched to use only pip install, and export dependencies from Poetry to requirements.txt format on the fly. See the Hypermodern Python Cookiecutter for how to do this.

Can You explain to me a little better what you do inside your sessions. You create in each run of your session the file requirements.txt and then install with pip -r? or do you create it once with poetry and then use it every session, and update it every time you install new modules?
PD: You were right in the second assumption, I use pylint as a development dependency and I use poetry install --no-dev. I'll leave you the session I'm trying to improve on. Every time that i run my session, this installs, upgrades, and removes packages according to your feedback this does with the development dependencies. This is the output: Package operations: 1 install, 13 updates, 12 removals.
@nox.session(python="3.8")
def pylint(session: Session) -> None:
    """Lint using pylint."""

    args = session.posargs or locations
    session.run("poetry", "install", "--no-dev", external=True)
    install_with_constraints(
        session, "pylint", "pylint_django", "pylint_celery",
    )
    session.run("pylint", *args)

You create in each run of your session the file requirements.txt and then install with pip -r? or do you create it once with poetry and then use it every session, and update it every time you install new modules?

The requirements.txt file is generated every time the session is run. It is not installed with pip install -r. Dependencies are installed with pip install <package>, but the requirements.txt file is passed as a constraints file via the --constraint option. This ensures that the versions in the constraints file are used for all dependencies and subdependencies.
Note that pylint and the plugins are also installed in the Poetry environment. So if you just need to quickly lint some files, you can always poetry run pylint. Use Nox to run full checks in a repeatable way in isolated environments. Use the Poetry environment to interact with your package during development, and to quickly run one-off checks and tasks.

Piggybacking off of this to ask some questions as well.
First, thank you very much @cjolowicz  for your series of blog posts! They were super enlightening for me.
A recurring issue I'm having with those Nox sessions is that for the sessions you advised using --no-dev it will end up actually uninstalling everything in the relevant Python's virtualenv. I figure that it has to do with external=True being used, though perhaps I may have missed something in your initial instructions for starting up Pyenv and Virtualenv with .bash_profile.
Have you encountered this issue yourself, and if so how what guidance would you have to deal with it?

@wanderrful This is a known issue. Please see these issues and comments:

https://github.com/cjolowicz/hypermodern-python/issues/109#issuecomment-647106048 above
https://github.com/cjolowicz/hypermodern-python/issues/111 (with further links)


Add firebase realtime db

Adds a Firebase real time database loader which uses the firebase-admin
Adds Readme, requirements for this loader

Hi @jerryjliu , I kindly request your review of my pull request on adding support to realtime db. Your feedback would be greatly appreciated. Thank you!


⚠ Bofrost.lu has degraded performance

In 9171c19, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 941 ms

Resolved: Bofrost.lu performance has improved in 0c5dd32.


Removed objects are not returned in query

Hi! We are trying to migrate to the new Google Ads API. Our use case is that we keep a local sync of the objects. When we query the change_status resource  the REMOVED resources (for example: ad_group_criterion) are returned with the status 'REMOVED'.
The query is:
SELECT change_status.ad_group_criterion,
       change_status.resource_type,
       change_status.last_change_date_time
  FROM change_status
 WHERE change_status.last_change_date_time >= '2019-06-01 10:00:00'
   AND change_status.resource_type = 'AD_GROUP_CRITERION' ;

But when we then query filtering by the resource_name that has been deleted (returned by the first query), that same resource is not brought by the API.
"SELECT ad_group_criterion.criterion_id ,
       ad_group_criterion.status, ad_group_criterion.type,
       ad_group_criterion.webpage.criterion_name,
  FROM ad_group_criterion
WHERE ad_group_criterion.resource_name = '<the resource-name obtained> '

How can we get those complete adgroup criterions ?

Hey,
I suspect this might due to either:

the resource_name field is not supporting filter operations (which it should) in which case you could try filtering by the ID rather than resource name,

or 2) there is an implicit filter to not return removed criteria, in which case you can add a clause to the WHERE ad_group_criterion.status = 'REMOVED'.
Also in general, this tracker is for the Java client library. For product related questions, please start a thread on the Google Ads API forum.


Plotting series (columns) fails

Calling plot from matplotlib with series (e.g. columns of a DataFrame) fails, because it tries to add an axis to the series objects. A traceback can be found here: https://gist.github.com/1281034 .
One way to solve this could be, if we try to index a series to multiple dimensions, simply return a plain numpy array of the data. Or maybe it makes more sense to make changes on the matplotlib side?
Series.plot() and DataFrame.plot() are designed to plot columns against the index, but don't appear to help with plotting one column against another.

I was aware of this issue but wasn't sure what to do about it. The preferred way of doing plot(soy_w['Depth'])) is soy_w['Depth']).plot(). I honestly think matplotlib should be sneaking in an np.asarray in there somewhere.
When you say "plot one versus the other" you mean a scatter plot?

Yes, I am thinking of scatter plots - they're the commonest type of plot in my field. The plot(soy_w['Depth']) was just a minimal example to show the bug - I was actually trying to do plot(soy_w['Depth'], soy_w['OtherColumn']).
Would it make sesnse to fall back to an array if we try to index to additional axes? The key is mpl calling y = y[:,np.newaxis]. Even if this is just a temporary measure while we get matplotlib to handle pandas structures better, I think it's pretty critical to work well with matplotlib.

I definitely agree that good integration with mpl is important-- where is the right place to report these issues? I guess it's sort of my own fault for breaking the ndarray interface in getitem, but I think it's a worthwhile tradeoff. I think we can put in a hack without breaking any unit tests, will take a look later today (after I get my PyCon proposals done)

Thanks - I look forward to testing it (as you've probably guessed, I'm
having a go with pandas in my normal workflow, where I'd previously been
using tabular).

try it out and let me know if things work like they should now

I can confirm that's doing the trick. Thanks, Wes.


Add game system integration API

This adds an API of sorts to ETL, that allows modules and game systems to modify the behavior of ETLs cost function. That way, systems and modules can inject specific rules for their game system into ETLs logic.
The design of the API as follows: ETL exposes it's TerrainLayer class to the global scope, so other modules are able to override it using libwrapper. The TerrainLayer class has a new method: calculateCombinedCost. This is the method, that other modules are supposed to override. Whenever the cost of a specific location is requested via ETL's cost function, ETL will collect all the terrain that is relevant for that location. That terrain is then passed into calculateCombinedCost, which transforms the list of terrain into a singular cost value. By default, the function simply returns the maximum cost of any indicated terrain (or 1 if no terrain is passed in). Modules may override this behavior to accomplish calculation rules that match a specific game system. Altough the API is very simplistic, module authors should be able to inject all the game system rules that are possibly out their this way.
The downside of this PR is, that the old way of influencing the behavior of the cost function (which is passing parameters like caluclateFn into the cost function) will no longer work, as it would conflict with the new API. However, no module other than Drag Ruler (and through Drag Ruler's API the Drag Ruler integration modules) does this and Drag Ruler would remove support for that API endpoint in v10 in favor of this ETL API, so that wouldn't be an issue.
There are two things to consider before merging this:

This API relies on other modules overriding a function via libwrapper to inject their logic. This is different from Drag Ruler's API, where a proper API is provided in which modules register. While the Drag Ruler solution is more elegant, providing such an API has way more overhead, as it comes with the added burden of detecting conflicts between multiple API consumers and providing a UI for users to resolve them. To me, doing all of these feels a little overkill for exposing only a single API endpoint. Using the libwrapper method we can relatively elegantly leave that job to libwrapper.
Currently, the PR exposes all of TerrainLayer to override, even though only overriding calculateCombinedCost is intended.

This was manually merged as part of 60286580cdb384a30b426475396e4afe34ff11b1.


[FEATURE] Meet an undefined operator when compiling NASNet

hello, I'm using hidet to compile PNASNET-5-LARGE defined here https://github.com/Cadene/pretrained-models.pytorch#pnasnet
And I met an undefined operator 'torch.nn.ZeroPad2d':
NotImplementedError: The following modules/functions are not supported by hidet yet:
torch.nn.ZeroPad2d
Could someone please add this module into hidet? thanks.

Hi @VincentXWD,
I have added the missing operators in #399, could you check whether it works?

Thank you @yaoyaoding, I'll check it later.
Happy holidays.

@yaoyaoding  Thanks, it works. Good job!


fix: testing to find the issue

Description:
Summarize the changes you're submitting in a few sentences, including Jira ticket ATL-xxxx if applicable.
Link to any discussion, related issues and bug reports to give the context to help the reviewer understand the PR.
Alternatives Considered (optional):
Link to existing ADR (Architecture Decision Record), if any. If relevant, describe other approaches explored and the selected approach. Documenting why the methods were not selected will create a knowledge base for future reference, helping prevent others from revisiting less optimal ideas.
Checklist:

[ ] My PR follows the contribution guidelines of this project
[ ] My PR is free of third-party dependencies that don't comply with the Allowlist
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked the PR title to follow the conventional commit specification

coverage: 48.859%. remained the same
when pulling 711c585d12bdd3eb6891ad3054c7402efda7854f on test-performance
into 5ece0dd573611c2f8f72f12c76a00aebc7841faf on main.

Only for testing can be closed without merge


Reduce height of header

Task: Introspection dashboard header "Spektate" takes up a lot of real estate


powerRuleIntegrationFunc denominator

Denominator is incorrect. It should be the (power +1)

fixes
#122
#138

fixed


Removed unnecessary type conversion

Changelog Description
Replaced usage of separate tags and type conversion with ast to storing everything in single tag.json_metadata which provides typing out of the box.
Additional info
Warnings or regexes caused unnecessary slowdown each saving or publishing with large amount of tracks in a workfile.
Fixed additional issue with key value of asset.
Unrelated issue, but causing some slowdown, is default DEBUG level in Pyblish plugins. It should be tied to debug command line argument, but it is not (imho).
Testing notes:

publish or save Hiero workfile with created AYON publish instances.
console output shouldn't contain anything like:

*** WRN: >>> { ayon_hiero.api.lib }: [  invalid decimal literal (<unknown>, line 1)  ] *** WRN: >>> { ayon_hiero.api.lib }: [  malformed node or string on line 1: <ast.Attribute object at 0x0000021367352DD0>  ] *** WRN: >>> { ayon_hiero.api.lib }: [  invalid syntax (<unknown>, line 1)  ]

Could we completely revamp the Ayon data tag content, shifting from hierarchical build-in tag metadata values to a JSON dump? This might significantly speed up the type conversion process. It would also align our workflow with other editorial DCC implementations.

I extracted content of Openpype tag to new key json_metadata which contains json. This approach should be backward compatible, but it is a question if it should be cleaned up more in this PR or it should wait for transfer to Publisher (where all OP mentions should be purged, imho).

Actually I dont think this would help slow down issue too much.
There is massive slowdown in these functions, caused by too verbose logging.

That could solve Publisher even without this update and it is questionable if this should be merged altogether.

Actually I dont think this would help slow down issue too much.
That could be solved by Publisher even without this update and it is questionable, if this should be merged altogether.

Yeah, perhaps we do not need to continue on this PR if - as you have said it will be faster in New publisher implementation. There is only openion on the json data implementation and its wider flexibility. But it could be implemented in the New publisher PR by  @robin-ynput

Will be revisited after implementation of Publisher if necessary.


mj-accordion and mj-carousel use cell-spacing and cell-padding instead of cellspacing and cellpadding

Describe the bug
The mj-accordion and mj-carousel components use cell-spacing and cell-padding attributes instead of cellspacing and cellpadding. This is invalid HTML (not supported in Chrome and likely other browsers) and does not align with the attributes other components use.

Totally a typo here, even if it's just here as a reset, let's fix that


[MRESOLVER-386] Make all injected ctors public, deprecate def ctors

All injection points are public and deprecate all def ctors (where ctor injection used).

https://issues.apache.org/jira/browse/MRESOLVER-386

WagonTrasnporterFactory injected ctor is missing public modifier


Add a model to the model executor list that is derived from RagTokenForGeneration model

How do I add a model executor that is derived from the huggingface model class  RagTokenForGeneration.
I could only find this link that documents about adding a new model. Can someone elaborate for this use case.
Thanks

I'm not sure what state the linked documentation was in when you originally read it, but today it looks like you should be able to follow the steps if you are interested in adding a new model.
I'm closing this as stale for now.


Extra rollout scrambling

Let the Recurrent batches scramble the training batches to feed to the RNN in many different ways.
In actuality this didn't improve learning performance at all and decreased FPS, so I don't want to spend much more time in it. Still, now the SamplingType is in the config of old experiments, and it may be useful...

Note: some of the RL-derived tests fail here but succeed in #27 , so we should just merge these two PRs together. Unless you think I should make this one pass too @taufeeque9


Static IPv6: Back routing from WAN /64 to LAN /65 does not work

Important notices
Our forum is located at https://forum.opnsense.org , please consider joining discussions there in stead of using GitHub for these matters.
Before you ask a new question, we ask you kindly to acknowledge the following:

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[x] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

We have an OPNsense in the data center in the rack. This has a /64 prefix from the operator 2001:-----:dead:-----::/64.
The gateway from the data center is 2001:-----:dead:-----::1
WAN:
I have given the OPNsense the static 2001:-----:dead:-----::2/64.
And selected WANv6 as the gateway.
I have a manually added WANv6 gateway with the following configuration:
Name: WANv6
Interface: WANMyLOC
Address family: IPv6
Adresse: 2001:-----:dead:-----::1
Upstream gateway: [x]
Remote gateway: [ ]
Disable gateway monitoring: [ ]
Disable Host Route [ ]
Minitor-IP: 2a00:1450:400e:810::200e
Mark gateway as Inactive: [ ]
Priority: 255

This is also displayed as online.
On the LAN interface I have static IPv6 with the following address:
2001:-----:dead:-----:8000::1/65
A ping on the firewall itself to any IPv6 address works.
A ping from a client in the LAN network goes out and arrives at the destination, but the response does not come back to the client. The client has the IP 2001:-----:dead:-----:8000::5/65
ip a excerpt:
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:50:56:8e:9a:d7 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet <IP_ADDRESS>/24 brd <IP_ADDRESS> scope global ens18
       valid_lft forever preferred_lft forever
    inet6 2001:----:dead:----:8000::5/128 scope global dynamic noprefixroute
       valid_lft 4704sec preferred_lft 2004sec
    inet6 fe80::250:56ff:fe8e:9ad7/64 scope link
       valid_lft forever preferred_lft forever

ip -6 r excerpt:
2001:----:dead:----:8000::/65 dev ens18 proto ra metric 1024 expires 86091sec pref medium
2001:----:dead:----:8000::/65 via fe80::21a:8cff:fe16:da98 dev ens18 proto ra metric 1024 expires 1491sec pref medium
fe80::/64 dev ens18 proto kernel metric 256 pref medium
default proto ra metric 1024 expires 1491sec mtu 1500 pref medium
	nexthop via fe80::21a:8cff:fe16:da98 dev ens18 weight 1
	nexthop via 2001:----:dead:----:8000::1 dev ens18 weight

fe80::21a:8cff:fe16:da98 and 2001:----:dead:----:8000::1 is from the OPNsense of the LAN side.
tcp dump from ping destination:
12:50:47.542897 IP6 2001:-----:dead:-----:8000::5 > 2a01:-----:-----:-----::2: ICMP6, echo request, seq 2498, length 64
12:50:47.542966 IP6 2a01:-----:-----:-----::2 > 2001:-----:dead:-----:8000::5: ICMP6, echo reply, seq 2498, length 64

In the firewall I have allowed all ICMPv6 traffic from everywhere to everywhere. Once as a floating rule, in the WAN and also in the LAN.

I'd avoid using anything other than /64.

I can fully understand that and I wouldn't prefer it either. But it's just what I have from the provider at the moment and it should actually work, shouldn't it?

If the goal is to get a single /64 to work use /64 on your LAN and /128 on your WAN?

With /128 on the WAN and /64 on the LAN it does not work either. In addition, OPNsense says “Misconfigured Gateway IP” under Gateways for the v6 gateway.
I assume this is because the 1 no longer fits into the /128?
I have also tried it with a /126 on the WAN. Then the gateway is online again, but the client still doesn't get a reply.
Do I have to set /128 and a manual route? What would that look like?

Ok. We now have a completely new /56 from the provider. Gateway is the 2001:----:----:100::1
Network is: 2001:----:----:100::/56
First /64 on the WAN interface:
2001:----:----:100::2/64
And the next one on the LAN:
2001:----:----:101::1/64
A client has an IP address via SLAAC:
2001:----:----:101:250:56ff:fe8e:9ad7
A tcpdump (tcpdump -n -i em1 ip6) on the WAN interface results in the following messages:
13:36:53.496899 IP6 2001:----:----:100::1 > ff02::1:ff8e:9ad7: ICMP6, neighbor solicitation, who has 2001:----:----:101:250:56ff:fe8e:9ad7, length 32

Why is the OPNsense not responding to the other gateway?
I hope you or someone else can help me with this information.

Seems to be working now. The ISP must have done something else on request.


Hide extraneous GCN plumbing from returned objects

From @adam-sandor on April 13, 2017 11:44
Fetching a list of files from my bucket produces 271K of JSON output from the server. And I just have 4 files in the bucket.
The data includes things like the complete description of the bucket itself (per every file) or the description of the googlecloud storage library (authors, dependencies, etc). Some of this is probably not loaded from the server but is added by the node api library. Even if that doesn't lead to network traffic it's still unnecessary and inconvenient when I try to find the relevant data. The screenshot below show the data structure with most elements collapsed. None of those elements are relevant for listing files in the bucket. I only want to see the data that is printed by gsutil when doing ls.
https://www.dropbox.com/s/4dgpu7jh8z6lb9h/Screen%20Shot%202017-04-13%20at%2013.29.58.png?dl=0
I might be missing some option or parameter that would limit the output from getFiles.
Environment details

OS: OSX
Node.js version: 7.2.0
npm version: 3.10.9
google-cloud-node version: 1.0.0

Steps to reproduce
var storage = require('@google-cloud/storage');
var fs = require('fs');

var gcs = storage({
    projectId: 'MYPROJECT'
});

var bucket = gcs.bucket('MYBUCKET');

bucket.getFiles(function(err, files) {
    if (!err) {
        fs.writeFileSync("files.json", JSON.stringify(files))
    } else {
        console.error(err)
    }
});

Copied from original issue: GoogleCloudPlatform/google-cloud-node#2217

From @adam-sandor on April 14, 2017 15:58
Ok so do I understand correctly that only the metadata is actually downloaded from the server?

From @stephenplusplus on April 13, 2017 13:2
There is a lot of stuff on the object-- some is plumbing that enables our API to function. The only thing that's really meant for the user is the metadata property, which is the API's raw record of the resource.
In the context of getFiles(), the API response returns all of the metadata for each file. If we didn't include it on the object, it would require further API calls to download that information. To find the data you need, you should look into to the metadata object. In the case that you just want names, it's a simple map over the response:
bucket.getFiles(function(err, files) {
    if (!err) {
        var fileMetadataObjects = files.map(function(file) {
            return file.metadata
        })
        fs.writeFileSync("files.json", JSON.stringify(fileMetadataObjects))
    } else {
        console.error(err)
    }
})

And of course, if you just want the name, you can use file.metadata.name or any other properties that are relevant for your use.

From @stephenplusplus on April 13, 2017 13:12
Just to address the 271,000 lines of JSON, though, I'm not particularly thrilled about that. From a quick test, the output of one file for me was about 1,200 lines. I'll re-open so we can define those huge objects as non-enumerable, so they're hidden from the output.

From @stephenplusplus on April 14, 2017 16:1
Yes, everything else is state and internal data.

From @adam-sandor on April 14, 2017 16:21
Cool, then it was just me panicking :)
Still would be user friendlier to not have that metadata in the request I think.

This was originally reported in the context of Storage, but applies to all of the APIs. I re-opened in the core repo.


Rewriting the PPP stack

This is the work done by me in the 2018, still using it though. I wanted to implement support for IPv6 for SSTP, but never got the time...
This is basicaly a nearly total rewrite of the Proto_PPP stack, to make it more logical and not "spaghetti code". I used this code from 2018 till... Basically, now, without any problems, but hidden ones may arise, as I am only a human =)
Some notes: the PPP stack is implemented a bit like a state machine, with the global connection state tracking instead of euristically and "hardcoded" states as in the code before.
It should be noted that I also changed the "default IPv6" IP from <IP_ADDRESS> to <IP_ADDRESS>, as <IP_ADDRESS> MAY be a real used address, and <IP_ADDRESS>/24 is reserved for documentation, which is good enough I guess (a little bit same logic when <IP_ADDRESS> was changed to <IP_ADDRESS>, although I'm not sure reusing <IP_ADDRESS> would be correct in this particular case as it may break some things, I guess...)
Also the timeout defines were set a little bit at random, and it would be more logical probably to inherit them from the actual logged user timeout settings.
I'm kinda tired fighting endless conflicts each merge with master, so here we go, let's do this pull request in hopes it would get accepted... Eventually...

I've applied all requested changes from @davidebeatrici , they seem to be mostly about code style and some slipped up changes I made once to compile and debug.

I don't want to sound annoying, but... Is there any news?

I hope the pull request won't be forgotten =)

@Evengard , thank for your contribution
due to lack of automating testing, we cannot guarantee everything will work or not, so we decided to merge it, since it Development edition :)
however, we did review, your code looks good

Wow thank you! =)

Thank you very much for your contribution!

@Evengard , you might be interested
Coverity report (static analysis tool)
*** CID 355465:  Control flow issues  (DEADCODE)
/src/Cedar/Proto_PPP.c: 2332 in PPPParseLCP()
2326
2327                            Zero(&o, sizeof(o));
2328
2329                            // Type
2330                            if (len < 1)
2331                            {
>>>     CID 355465:  Control flow issues  (DEADCODE)
>>>     Execution cannot reach this statement: "goto LABEL_ERROR;".
2332                                    goto LABEL_ERROR;
2333                            }
2334                            o.Type = buf[0];
2335                            buf++;
2336                            len--;
2337

** CID 355464:  Incorrect expression  (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 1019 in PPPProcessIPCPResponsePacket()

________________________________________________________________________________________________________
*** CID 355464:  Incorrect expression  (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 1019 in PPPProcessIPCPResponsePacket()
1013                    {
1014                            p->IPv4_State = PPP_PROTO_STATUS_CONFIG_WAIT;
1015                    }
1016                    return true;
1017            }
1018
>>>     CID 355464:  Incorrect expression  (NO_EFFECT)
>>>     The boolean test in statement "p->IPv4_State == 1U;" has no effect.
1019            p->IPv4_State == PPP_PROTO_STATUS_CONFIG;
1020
1021            PPPGetIPAddressValueFromLCP(req->Lcp, PPP_IPCP_OPTION_IP, &prevAddrStruct);
1022            prevAddr = IPToUINT(&prevAddrStruct);
1023            IPToStr(prevAddrStr, MAX_SIZE, &prevAddrStruct);
1024

** CID 355463:  Control flow issues  (MISSING_RESTORE)
/src/Cedar/Proto_PPP.c: 1668 in PPPProcessIPCPRequestPacket()

________________________________________________________________________________________________________
*** CID 355463:  Control flow issues  (MISSING_RESTORE)
/src/Cedar/Proto_PPP.c: 1668 in PPPProcessIPCPRequestPacket()
1662                    return false;
1663            }
1664
1665            //Debug("PPPAckLCPOptionsEx ok=%x, processed=%x", ok, processed);
1666            if (!ok || !PPPAckLCPOptionsEx(p, pp, processed))
1667            {
>>>     CID 355463:  Control flow issues  (MISSING_RESTORE)
>>>     Value of non-local "p->DhcpIpAllocTried" that was verified to be "0U" is not restored as it was along other paths.
1668                    return false;
1669            }
1670            Debug("ACKed IPCP options ID = 0x%x\n", pp->Lcp->Id);
1671
1672            if (ok && p->IPv4_State == PPP_PROTO_STATUS_CONFIG_WAIT)
1673            {

** CID 355462:    (REVERSE_INULL)
/src/Cedar/Proto_PPP.c: 918 in PPPProcessCHAPResponsePacket()
/src/Cedar/Proto_PPP.c: 956 in PPPProcessCHAPResponsePacket()

________________________________________________________________________________________________________
*** CID 355462:    (REVERSE_INULL)
/src/Cedar/Proto_PPP.c: 918 in PPPProcessCHAPResponsePacket()
912                             WriteBuf(lcp_ret_data, ret_str, StrLen(ret_str));
913
914                             lcp = NewPPPLCP(PPP_CHAP_CODE_SUCCESS, p->MsChapV2_PacketId);
915                             lcp->Data = Clone(lcp_ret_data->Buf, lcp_ret_data->Size);
916                             lcp->DataSize = lcp_ret_data->Size;
917
>>>     CID 355462:    (REVERSE_INULL)
>>>     Null-checking "lcp_ret_data" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
918                             if (lcp_ret_data != NULL)
919                             {
920                                     FreeBuf(lcp_ret_data);
921                             }
922
923                             res->Lcp = lcp;
/src/Cedar/Proto_PPP.c: 956 in PPPProcessCHAPResponsePacket()
950                             WriteBuf(lcp_ret_data, ret_str, StrLen(ret_str));
951
952                             lcp = NewPPPLCP(PPP_CHAP_CODE_FAILURE, p->MsChapV2_PacketId);
953                             lcp->Data = Clone(lcp_ret_data->Buf, lcp_ret_data->Size);
954                             lcp->DataSize = lcp_ret_data->Size;
955
>>>     CID 355462:    (REVERSE_INULL)
>>>     Null-checking "lcp_ret_data" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
956                             if (lcp_ret_data != NULL)
957                             {
958                                     FreeBuf(lcp_ret_data);
959                             }
960
961                             res->Lcp = lcp;

** CID 355461:  Control flow issues  (DEADCODE)
/src/Cedar/Proto_PPP.c: 2278 in PPPParseLCP()

________________________________________________________________________________________________________
*** CID 355461:  Control flow issues  (DEADCODE)
/src/Cedar/Proto_PPP.c: 2278 in PPPParseLCP()
2272
2273            c->OptionList = NewListFast(NULL);
2274
2275            // Code
2276            if (size < 1)
2277            {
>>>     CID 355461:  Control flow issues  (DEADCODE)
>>>     Execution cannot reach this statement: "goto LABEL_ERROR;".
2278                    goto LABEL_ERROR;
2279            }
2280            c->Code = buf[0];
2281            buf++;
2282            size--;
2283

** CID 355460:  Null pointer dereferences  (REVERSE_INULL)
/src/Cedar/Proto_PPP.c: 640 in PPPSendEchoRequest()

________________________________________________________________________________________________________
*** CID 355460:  Null pointer dereferences  (REVERSE_INULL)
/src/Cedar/Proto_PPP.c: 640 in PPPSendEchoRequest()
634                     if (IsIPCConnected(p->Ipc))
635                     {
636                             AddInterrupt(p->Ipc->Interrupt, p->NextEchoSendTime);
637                     }
638
639                     // Validate arguments
>>>     CID 355460:  Null pointer dereferences  (REVERSE_INULL)
>>>     Null-checking "p" suggests that it may be null, but it has already been dereferenced on all paths leading to the check.
640                     if (p == NULL)
641                     {
642                             return false;
643                     }
644
645                     pp = ZeroMalloc(sizeof(PPP_PACKET));

** CID 355459:    (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 736 in PPPProcessLCPResponsePacket()
/src/Cedar/Proto_PPP.c: 732 in PPPProcessLCPResponsePacket()

________________________________________________________________________________________________________
*** CID 355459:    (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 736 in PPPProcessLCPResponsePacket()
730                             if (*protocol == PPP_PROTOCOL_IPCP || *protocol == PPP_PROTOCOL_IP)
731                             {
732                                     p->IPv4_State == PPP_PROTO_STATUS_REJECTED;
733                             }
734                             if (*protocol == PPP_PROTOCOL_IPV6CP || *protocol == PPP_PROTOCOL_IPV6)
735                             {
>>>     CID 355459:    (NO_EFFECT)
>>>     The boolean test in statement "p->IPv6_State == 256U;" has no effect.
736                                     p->IPv6_State == PPP_PROTO_STATUS_REJECTED;
737                             }
738                     }
739             }
740
741             if (!isAccepted && pp->Lcp->Code == PPP_LCP_CODE_CODE_REJECT)
/src/Cedar/Proto_PPP.c: 732 in PPPProcessLCPResponsePacket()
726                     }
727                     else
728                     {
729                             USHORT* protocol = pp->Lcp->Data;
730                             if (*protocol == PPP_PROTOCOL_IPCP || *protocol == PPP_PROTOCOL_IP)
731                             {
>>>     CID 355459:    (NO_EFFECT)
>>>     The boolean test in statement "p->IPv4_State == 256U;" has no effect.
732                                     p->IPv4_State == PPP_PROTO_STATUS_REJECTED;
733                             }
734                             if (*protocol == PPP_PROTOCOL_IPV6CP || *protocol == PPP_PROTOCOL_IPV6)
735                             {
736                                     p->IPv6_State == PPP_PROTO_STATUS_REJECTED;
737                             }

** CID 287561:  Incorrect expression  (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 3016 in PPPParseUsername()

________________________________________________________________________________________________________
*** CID 287561:  Incorrect expression  (NO_EFFECT)
/src/Cedar/Proto_PPP.c: 3016 in PPPParseUsername()
3010            UINT i, len, last_at, first_en;
3011            char token1[MAX_SIZE];  // username
3012            char token2[MAX_SIZE];  // hub_name
3013            char src[MAX_SIZE];
3014            // Validate arguments
3015            Zero(dst, sizeof(ETHERIP_ID));
>>>     CID 287561:  Incorrect expression  (NO_EFFECT)
>>>     Comparing an array to null is not useful: "src == NULL", since the test will always evaluate as true.
3016            if (cedar == NULL || src == NULL || dst == NULL)
3017            {
3018                    return false;
3019            }
3020
3021            StrCpy(src, sizeof(src), src_username);

** CID 287533:  Control flow issues  (DEADCODE)
/src/Cedar/Proto_PPP.c: 2202 in ParsePPPPacket()

________________________________________________________________________________________________________
*** CID 287533:  Control flow issues  (DEADCODE)
/src/Cedar/Proto_PPP.c: 2202 in ParsePPPPacket()
2196
2197            buf = (UCHAR *)data;
2198
2199            // Address
2200            if (size < 1)
2201            {
>>>     CID 287533:  Control flow issues  (DEADCODE)
>>>     Execution cannot reach this statement: "goto LABEL_ERROR;".
2202                    goto LABEL_ERROR;
2203            }
2204            if (buf[0] != 0xff)
2205            {
2206                    goto LABEL_ERROR;
2207            }

I will need to think about theese a bit later, but it seems like some of them are leftovers from the old code possibly a bit incorrectly "translated", some might be typing errors (eg == instead of = etc, which are actually bugs), and some might be things left "for future use". But I think I will analyze it more in-depth. Thanks!


Error shown for anything I try and use

Hi There
We just installed this extension (v<IP_ADDRESS>) but unable to get it to work.
We are using VS 2022 Version 17.4.4 and when we are at a breakpoint and right click on any variable and select Dump As  'x' then we are presented with a new document window and at the top the following error message is displayed
error CS1503: Argument 1: cannot convert from 'string' to 'char'
We tried to dump on simple numbers, strings etc - with lines of code such as
var i = 0;
var something = "Hello World";
var lstStr = new List { "hello", "there" };
but everything just returns this CS1503 error.  We tried on dumping to c#, JSON, XML etc
Are we doing something wrong?

Hi @sabasjon
I need more details:

is project language C#?
.NetFramework or .NetCore?
Visual Studio locale en-us?

Could you check this package out and let me know if issue has gone before I published it to marketplace?
ObjectDumper.zip

I just checked out - .NetFramework support was broken - so it should work now.

Hi There
Thanks so much for the quick response on this - I have just updated to <IP_ADDRESS> and can confirm all working for us now.
Cheers

Fixed in v.<IP_ADDRESS>


Fixes incorrect assertion of 2 week in advance PushDate and adds unit…

… tests to verify

Amazing catch. Thank you!


Code from Mobile QR Image is not accepting

Hi
The code from Mobile QR image is not accepting in my application. The verify code is always returns false and timeslice value is always zero. But when I use tfa.GetCode() method, the value is accepting and verify code returns true and slice value is greater than zero.
My implementation is below and please let me know what I am missing or doing wrongly.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using TwoFactorAuthNet;

namespace TFAuthentication
{
    ///  Steps to Implement QR Code implementation
    ///  Step 1: Add GenerateQRCode Image Button in User Account aspx.
    ///  Step 2: Add Image Tag in User Account aspx to dispay QR Code Image.
    ///  Step 3: Call GenerateTFSharedSecret
    ///  Step 4: Call AddTwoFactorAuthentication
    ///  Step 5: Call GenerateQrCodeImage and display QR Code Image in aspx.
    ///  Step 6: Add QR Code Validate ASPX page
    ///  Step 7: Call ValidateQRCode to validate the QR Code entered by the user.
    public static class TFAIdentity
    {
        ///
        ///  Generate Two Factor Shared Secret. Store the secret with User. This will be used only when user wants to generate/re-generate the QR Code image.
        ///
        /// <returns>User Two Factor Authentication Shared Secret </returns>
        public static string GenerateTFSharedSecret()
        {
            var tfa = new TwoFactorAuth("MyOrg",6, 30, Algorithm.SHA256);
            var secret = tfa.CreateSecret(160);
            return secret;
        }
        ///
        ///   Generate QR Code Image and it will be used as img src in aspx page
        ///
        /// <param name="label">User Name</param>
        /// <param name="secret"></param>
        /// <returns>QR Code Image Source as string </returns>
        public static string GenerateQrCodeImage(string label, string secret)
        {
            var tfa = new TwoFactorAuth("MyOrg",6, 30, Algorithm.SHA256);
            return tfa.GetQrCodeImageAsDataUri(label, secret);
        }
        ///
        /// Validate the QR Code entered by the User. This method will be used every time the user wants to login into the application.
        ///
        /// <param name="code"></param>
        /// <param name="secret"></param>
        /// <returns>Boolean</returns>
        public static bool VerifyQrCode(string code, string secret)
        {
            var verifyCode = false;
            var tfa = new TwoFactorAuth("MyOrg",6, 30, Algorithm.SHA256);
            var a1 = tfa.GetCode(secret);
           // The below Line is working
           verifyCode = tfa.VerifyCode(secret, a1,out long timeSlice1);
           // The below line is not working. Code from Mobile Authenticator
            verifyCode = tfa.VerifyCode(secret, code,out long timeSlice2);

            return true;
        }
        ///
        ///  Update the User Shared Secret in Users Table.
        ///
        /// <param name="email"></param>
        /// <param name="secret"></param>
        public static void AddTwoFactorAuthentication(string email, string secret)
        {
            var connectionString = ConfigurationManager.AppSettings["connectMyOrgAPP"];
            using (var conn = new SqlConnection(connectionString))
            {
                conn.Open();
                string sqlQuery = "UPDATE Users SET SharedSecret = @sharedSecret, TFAEnabled=@TFAEnabled  WHERE email = @email";
                using (var cmd = new SqlCommand(sqlQuery, conn))
                {
                    cmd.Parameters.AddWithValue("sharedSecret", secret);
                    cmd.Parameters.AddWithValue("TFAEnabled", 1);
                    cmd.Parameters.AddWithValue("email", email);
                    cmd.ExecuteNonQuery();
                }
            }
        }
        ///
        ///  Remove User Two Factor Authentication for the user.
        ///
        /// <param name="email"></param>
        public static void RemoveTwoFactorAuthentication(string email)
        {
            var connectionString = ConfigurationManager.AppSettings["connectMyOrgAPP"];
            using (var conn = new SqlConnection(connectionString))
            {
                conn.Open();
                string sqlQuery = "UPDATE Users SET SharedSecret = @sharedSecret, TFAEnabled=@TFAEnabled  WHERE email = @email";
                using (var cmd = new SqlCommand(sqlQuery, conn))
                {
                    cmd.Parameters.AddWithValue("sharedSecret", string.Empty);
                    cmd.Parameters.AddWithValue("TFAEnabled", 0);
                    cmd.Parameters.AddWithValue("email", email);
                    cmd.ExecuteNonQuery();
                }
            }
        }
        ///
        ///  Check user enabled the Two Factor Authentication
        ///
        /// <param name="email"></param>
        /// <returns></returns>
        public static bool IsTwoFactorEnabled(string email)
        {
            var connectionString = ConfigurationManager.AppSettings["connectMyOrgAPP"];
            var isEnabled = false;
            using (var conn = new SqlConnection(connectionString))
            {
                conn.Open();
                string sqlQuery = "SELECT TFAEnabled FROM Users WHERE email = @email";
                using (var cmd = new SqlCommand(sqlQuery, conn))
                {
                    cmd.Parameters.AddWithValue("email", email);
                    using (var reader = cmd.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            isEnabled = bool.Parse(reader["TFAEnabled"].ToString());
                            break;
                        }
                    }
                }
            }
            return isEnabled;
        }
        ///
        ///  Retrieves User Shared Secret from Customers Table based on User Id
        ///
        /// <param name="email"></param>
        /// <returns></returns>
        public static string GetUserSharedSecret(string email)
        {
            var connectionString = ConfigurationManager.AppSettings["connectMyOrgAPP"];
            var sharedSecret = string.Empty;
            using (var conn = new SqlConnection(connectionString))
            {
                conn.Open();
                string sqlQuery = "SELECT SharedSecret FROM Users WHERE email = @email";
                using (var cmd = new SqlCommand(sqlQuery, conn))
                {
                    cmd.Parameters.AddWithValue("email", email);
                    using (var reader = cmd.ExecuteReader())
                    {
                        while (reader.Read())
                        {
                            sharedSecret = reader["SharedSecret"].ToString();
                            break;
                        }
                    }
                }
            }
            return sharedSecret;
        }
    }
}

My ASPX code is below

<img src="<%=Me.QrCodeImage%>" alt="QRCode" />

                                                            <asp:Button ID="RemoveTFA" Text="Disable Two Factor Authentication" runat="server"  OnClick="RemoveTFA_Click"/>

My ASPX code behind is below
Try
            If Not IsPostBack Then
                Dim email As String = user.Email
                Dim isTFAEnabled As Boolean
                Dim sharedSecret As String = String.Empty
                // Get TFA Information from Users Table
                isTFAEnabled = TFAIdentity.IsTwoFactorEnabled(email)
                If isTFAEnabled Then
                    sharedSecret = TFAIdentity.GetUserSharedSecret(email)
                End If
                //Store the secret in Session
                If String.IsNullOrEmpty(sharedSecret) Then
                    sharedSecret = TFAIdentity.GenerateTFSharedSecret()
                    Session("sharedSecret") = sharedSecret
                End If

                'Generate QR Code Image
                QrCodeImage = TFAIdentity.GenerateQrCodeImage(email, sharedSecret)
                 // If not enabled then enable and store the secret in users table
                If Not isTFAEnabled Then
                    'Update Shared Secret to Users Table
                    TFAIdentity.AddTwoFactorAuthentication(email, sharedSecret)
                End If

            End If
        Catch ex As Exception
            Response.Redirect("../error.htm")
        End Try

Please create a minimal reproducible example. Also please leave out all irrelevant comments for clarity.

I have tested the above code and it works for SHA1 and If I initialise  like var tfa = new TwoFactorAuth("MyOrg");

Yeah, I noticed you changed the algorithm and number of bits for the secret. What authenticator app are you using and are you aware that anything but the defaults (SHA1, 6 digits, 30 seconds, 80 bits) might not be supported?

Thanks Rob. I am using Microsoft Authenticator App in IPhone. I have tested the same in Google Authenticator and OTPAuth from AppStore.

Did you try the defaults? Do those work for you?

Yes. Defaults works fine and tested in Microsoft, Google and OTPAuth

What about the other apps you mention? Do these have the same problem or do these work? Please be a bit more clear about things.
If the defaults work then the combination you're using is most likely simply not supported by the Microsoft app. That's why you need to test 😉

Apparently it's not supported. I simply googled it...

Sorry for that.
Thanks for the quick response.
Sorry I was thinking on issue in my implementation and Sorry to bother you.

You're not "bothering" me, it's just that you could've solved it much faster yourself if you just googled it 😉
No hard feelings. You've learned a few things today:

You need to provide proper Minimal, Reproducible Examples to be helped more efficiently
You need to provide as much information as you can up front in your problem description instead of requiring a back-and-forth to finally get all the details out
You need to mention anything that you changed that's out of the ordinary
You need to test with more than one app (to 'cross verify')
Try google before asking a question

So in the end you've learned a lot and this wasn't a total loss 😉 👍


Asset Dto

Hello! Nice work!
Is there is a reason you have commented out the metadata from Asset dto?
Is there any way to obtain them?
Thanks!

Hi @kozer, Initially the plan was to keep the backend api layer minimal, so only functionalities required to build transaction and post transaction are there.
So if you want to fetch additional data, you need to go through provider api directly. For example, you can use blockfrost-java api to get asset metadata.
https://github.com/blockfrost/blockfrost-java
But as currently the lib is mostly used for NFT , I think it makes sense to expose fetching metadata through  existing AssetService.  I will try to add that in future release.

Thanks @satran004 !! Hope to include this soon!


change data of the cards

if i want to change some datas of the cards or Items after they have been built, what should i do?

Hi! The recent release means that changing the data of the cards is more efficient and effective as before.
The ExampleMaterialAboutFragment has an example of "dynamic items".
final MaterialAboutActionItem item = new MaterialAboutActionItem.Builder()
            .text("Dynamic UI")
            .subText(subText)
            .icon(new IconicsDrawable(c)
                    .icon(CommunityMaterial.Icon.cmd_refresh)
                    .color(ContextCompat.getColor(c, R.color.mal_color_icon_dark_theme)
                    ).sizeDp(18))
            .build();
item.setOnClickAction(new MaterialAboutItemOnClickAction() {
    @Override
    public void onClick() {
        item.setSubText("Random number: " + ((int) (Math.random() * 10)));
        refreshMaterialAboutList();
    }
});

Note the refreshMaterialAboutList(); method that updates the view.


Any plan to allow user to create custom plugins ?

Is your feature request related to a problem? Please describe.
I would like to add a plugin for figma preview
Describe the solution you'd like
The best option would be to add the possibility to create custom plugins for ladle
Thanks you for the job you are doing on this project

I open to it.
There is already quite a lot you can achieve just with https://ladle.dev/docs/providers#ladle-context

I would like to add a plugin for figma preview
Can you document in detail what this integration would look like, including what APIs Ladle should expose/add?

We can imagine an API like in the storybook to register custom addon in the bottom bar and display custom panel component on open
addons.register(ADDON_ID, (api) => {
  addons.add(PANEL_ID, {
    type: types.PANEL,
    title: 'My Addon',
    render: ({ active, key }) => (
      <AddonPanel active={active} key={key}>
        <div> Storybook addon panel </div>
      </AddonPanel>
    ),
  });
});

For my case (figma addon) I will display inside the panel and iframe linked to my figma in order to simplify the review of the designers.

I created a package to inject buttons into the addon bar so that they appear the same as the Ladle built-in addons do. If you would like to use it, here's the repo https://github.com/hiddenist/ladle-inject-custom-addons

@hiddenist nice, using your lib + https://ladle.dev/docs/providers#ladle-context is basically what the built-in addons are today
we could maybe inline your lib into Ladle directly since it's pretty lightweight integration and opens a lot of possibilities when it comes adding more features into Ladle

@tajo Certainly! I would be glad if this could be officially incorporated into Ladle.
I've had pretty good success with this so far, though I had to chase down a strange bug on the built bundle when using MUI for the button icons after upgrading from v2 to v3/v4 🤷🏻
The add-ons that I'm using in my private project could also probably be useful as official add-ons, too. I have a theme color selector, and one that collapses/expands the story sidebar.


parsing localhost incorrect

http://localhost/signalr
http://localhost:8080/signalr

both return null

Well... it's somewhat expected since it's not in the list of known tlds. localhost is a special case.
We could solve this with @mattlubner 's PR #3, but it's probably better to just manually add it to the list since it is so common.

Well... it's somewhat expected since it's not in the list of known tlds. localhost is a special case.
We could solve this with @mattlubner 's PR #3, but it's probably better to just manually add it to the list since it is so common.

I don't believe localhost can be added without changing how that list is built, as all other TLDs expect a preceding dot, whereas localhost won't.
It could potentially be hard-coded into lib/build/buildRegex.js at line 20 though, like such:
// line 20
src = "module.exports = /localhost|\\.(" + tld + ")$/;";

I believe this change, plus the change I added in https://github.com/Vixlet/parse-domain/commit/a9b7912bb2ad7bdb3585ee8ce290b462037dddd4 to lib/parseDomain.js:72, would be sufficient for handling localhost. Didn't want to add the above change to #3 though, as it felt out-of-scope.

Thanks for picking up this issue :)
I was looking for something that could retrieve the domain from any url, and this was the first hit.
Based on the examples in the readme I couldn't make up if localhost would be supported, and when it was not, I added this issue and continued searching for a module that could do that (a more general url-parser). (was quite in a hurry)
Coming back to this, looking at what this module does (splitting a domain in 2-3 parts), parsing localhost doesn't really fit in; the result would have an empty tld.
So I don't know if supporting localhost fits the purpose of this utility?
Whatever the outcome, I think it would be a good idea to document the usage of localhost in the readme :)

Starting with v0.2.1 you are allowed to pass in custom tlds: See #3

I am not really sure how to use it with localhost:3000 for example. When putting localhost as privateTld and going to foo.localhost:3000 it thinks foo is the domain and localhost the subdomain, how can I make it work on localhost?


Static (page_all) output is breaking zsh

Thank you for your work on this crate. I just gave 4.0.0-alpha1 version a try.
Pagination works as expected with a few minor (unrelated) issues:

I expected space bar to be same as PgDn just like in less
Search does not seem to navigate to the first/next occurence
Highlighting of search result seems to break colors of the text being highlighted

But the main issue is that after I exit my program every mouse movement in the terminal window is causing some garbage output:

It's quite hard to recover the terminal session after this.
Happy to help to test potential fixes or help narrow down what's causing this.

Could you share a little bit of code what you have tried to implement?
By the thanks for putting up this issue.

The issue is reproducible for me even with this basic example:
[package]
name = "foo"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
minus = { version="4.0.0-alpha1", features=["static_output", "search"] }

// main.rs
use std::io::Write;

fn main() {
    let mut pager = minus::Pager::new().unwrap();

    writeln!(pager, "{}", "Hello, world!\n".repeat(500)).unwrap();

    minus::page_all(pager).unwrap();
}

Fixed the last one.
The search won't go to the last occurrence if it's the on the very last page or close to it.
Also it won't go to the first one if it's already scrolled down by the user. Do tell me if you don't want this behaviour.
Could you describe more on how colors are breaking.

I built from master and the problem is gone. Thanks a lot for the quick fix! :raised_hands:

I see, I misunderstood the logic of scrolling when searching - current implementation makes perfect sense :+1:

Re. colors breaking, here's an example of output before search:

and after search:

It's super minor and might not be even worth fixing.

I see the problem, it's because of the ANSi reset code that the highlight_search function enters after the search. I will look into that now.

Well. I fixed that.

I will add the spacebar binding soon. You can see these changes in the next alpha, released probably by tomorrow.


dockerized version

I've added a Dockerfile and changed the source code from the hello.go to be able to run using docker container.
Also the http port can be passed as an env variable.

The de-facto standard from Heroku is to use PORT as the envvar that the port gets passed into.


Marking the operator as fips compliant

This will enable FIPS scanner during a CVP run so we can start fixing potential issues.
Also it should resolve the warning: [INVALID] The value of the 'features.operators.openshift.io/fips-compliant' annotation from the operator CSV is: false. The value can only be a string "true".

/retest

/retest

I think this change should either go into the upstream repo or into ossm/values.yaml

Ah, yes. Fixing

Actually, I will have to close this. I realized that we can't make the FIPS scanner passing until we have a proper proxy build


make syndie bag and rigs explosion resistant

About the PR

all bags have 10% resistance since its a little bit better than contents being on the floor
syndie bags have 90% resistance to prevent china lake + minibomb suiciding, you now need to get exploded a lot for it to get damaged enough to explode, at which point youd probably be gibbed already

Why / Balance
make china lake less sosa exclusive, nerf ieding a nukie that has a minibomb to kill them
Technical details

removed entity storage-specific explosion resistance since doing it for storage too would be bloat, when actual explosion resistance component supports per-explosion type resistances
made explosion resistance support not applying when worn, so a 90% resistance bag is for the contents only and not god armour
support custom examine to make it clearer

technically this means you cant have a storage that has 90% resistance but 10% resistance as armor but that seems very niche, just have them be the same or non worn resistance
Media

narnia the urist the urist and the urist
urist a) holding and wearing syndie bag and holding mime satchel
urist b) inside mime satchel
urist c) inside syndie bag
urist a is not saved by wearing bag

urist b isnt protected well

urist c is relatively unharmed due to 90% content protection

valid

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Breaking changes
explosionDamageCoefficient removed from EntityStorage, but nothing used it anyway
Changelog
:cl:

tweak: Syndicate duffel bags heavily protect their items from explosions, and regular bags have minor protection.

give the chest rig 50% or more resistance (but not the security one)

armor :godmode:

yes its for examine rtfc (read the friendly comment)


Tests: Add more complex e2e tests

Context https://github.com/netlify/netlify-cms/issues/3332, https://github.com/netlify/netlify-cms/issues/3372 and https://github.com/netlify/netlify-cms/pull/3411
We've had several cases in the past when having out of dirty repos/out of sync forks will cause issues for editorial workflow and open authoring.
Editorial Workflow Examples

Updating a PR that has an empty commit. We'll probably need to create an empty commit via GitHub API.
Updating a PR that had been edited manually via GitHub, specifically added media files that are not mapped to fields.

Open Authoring Examples
Example scenario:

Create a test repo with open authoring enabled.
Login with a GitHub user that doesn't have access to the repo.
Fork the repo via the CMS.
Simulate an out of date fork by reverting a bunch of commits and force pushing an out of date default branch.
Create an entry via the fork and move to Review.
Verify the created PR.

Another example for https://github.com/netlify/netlify-cms/issues/3372:

Create a test repo with open authoring enabled.
Login with a GitHub user that doesn't have access to the repo.
Fork the repo via the CMS.
Simulate an out of date fork by reverting a bunch of commits and force pushing an out of date default branch.
Reload the CMS and see that it offers the user to sync the fork.
Sync the fork via the CMS.
Validate success.

We should also test when the sync should fail (e.g. unresolved conflicts).

👋 Hello I maintain an open source library that generates Playwright code. I would be happy to submit PRs setting up e2e tests for netlify-cms, if you are cool with them using Playwright?

Hi @jperl, this sounds very cool. We already have a very extensive cypress setup.
I don't think we would like to have two different types of testing frameworks so first we would need to replace all existing tests.
Most of our tests use mocked recorded data and mock the browser fetch API, so most of our challenges were around that and not how to select items from the dom.
Also, I was trying to figure out how the parallel testing works (we use Cypress parallel flag which load balances our tests and also provides a dashboard). Looks like https://github.com/qawolf/playwright-ci/blob/dd83f2d1a0bfa136c46f21a0f2c67b735439e8cb/static/github.hbs will run the tests on a single machine.
I probably missed some requirements, but please let me know what you think. As for the tests in this issue, most of them are about having "dirty" repos to test the CMS so the challenge there is not necessarily working with the dom, but finding a consistent way of generating the "dirty" status.

Thanks for the response :slightly_smiling_face:
I see you already have an extensive setup, so it probably would be too much work to port all of the existing tests. Re parallel testing: This can be done by using a GitHub Action matrix, where each job targets a subset of the tests.
I understand if you prefer to use Cypress for your tests, if you change your mind let me know!

This can be done by using a GitHub Action matrix, where each job targets a subset of the tests.

Thanks, we use a GitHub actions matrix, but Cypress takes care of splitting the tests between machines:
https://github.com/netlify/netlify-cms/blob/8b0248bdc3f55dca451121d089b3274c8e72c09d/package.json#L28.

That is a great point! I added a ticket for creating a Jest scheduler to do this automatically.


[RFE] kube-burner index tests doesn't validate indexed metrics

Is your feature request related to a problem? Please describe.
We're not validating the indexed metrics in the kube-burner index tests
https://github.com/cloud-bulldozer/kube-burner/blob/11b9cb7f3381959086af3ca02c3eb09f4ec59dca/test/test-k8s.bats#L66-L74
Describe the solution you'd like
We need to validate that at least some documents have been indexed, looking for them by UUID and metricName

duped #519


Mollify the pedants

@michaelaye Sorry, this was totally tongue-in-cheek! Didn't expect anyone to actually see it. You were right that it should be clarified and weren't the only person to mention it. Thanks for bringing it to my attention.


Updated schedule content to reflect bank holidays

The pipelines have not ran successfully in many months and it is trying to implement in changes that I have no idea what they are for, or might be purposely not implemented.


See https://github.com/outofmbufs/python-pdp11-emulator/issues/6

This proposed fix made all the condition codes True/False (== 1/0) vs "truthy/false" but truthy is a lot simpler everywhere else in the code. Bug was fixed instead by making the specific cases in ROR/etc work correctly on truthy/false.


Ignore additional exceptions by default

The following exceptions are irrelevant and harmless in production, since they are already handled by rails and will return an appropriate response. They have been causing unnecessary noise for us in Bugsnag.

ActionController::ParameterMissing
ActionController::UnknownFormat
ActionController::UnknownHttpMethod

I also sorted the exception list alphabetically to keep sanity if more ignored exceptions are added in future.
I realize these can be easily configured to be ignored in the bugsnag interface (which we have already done), but I believe they should also be included as default ignore list.
Fixes #230

Thanks @adrianmacneil ! Totally agreed. Thanks for adding them to the list and sorting it :)