Unable to install ACDC (Failed building wheel for opencv-python-headless) on intel based Mac

Description:
Hello! I'm trying to install ACDC to analyze my data, but I'm having difficulty to install it on my machine.
I followed the installation guide using Anaconda, but couldn't complete the step 7.
I tried to install several different version of the opencv and I do have the same issue installing these versions listed below:
current from Github
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
But I can successfully install <IP_ADDRESS>.
However, I'm not sure how to alter the installation process to substitute openCV to the version that could be installed. Any advice would be appreciated!
Way to reproduce:
No response
(Optional) Link to data to reproduce error:
terminal-ACDCinstall.txt
Terminal output:
No response
Version information:
3.9.18 (main, Sep 11 2023, 08:38:23)
[Clang 14.0.6 ]
macOS-10.14.6-x86_64-i386-64bit
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'cellacdc'

Hi,
I am able to complete building wheel fo opencv-python-headless and install acdc after installing nasm in my conda environment!
However, when I initialize ACDC, it keeps asking me to install a GUI library (PyQt6). My machine (late 2019 MacBook Pro running OS 10.14.6) is not compatible with PyQt6. Here is the traceback message I got:
Initialising...
Since version 1.3.1 Cell-ACDC does not install a GUI library by default.
You can let Cell-ACDC install it now (default library is PyQt6), or you can abort (press "n")
and install a compatible GUI library with one of the following commands:
* pip install PyQt6
* pip install PyQt5 (or `conda install pyqt`)
* pip install PySide2
* pip install PySide6

Note: if PyQt6 installation fails, you could try installing any of the other libraries.
Do you want to install PyQt6 now ([y]/n)? y
Collecting PyQt6
Using cached PyQt6-6.5.3-cp37-abi3-macosx_10_14_universal2.whl.metadata (2.1 kB)
Requirement already satisfied: PyQt6-sip<14,>=13.6 in ./anaconda3/envs/acdc/lib/python3.9/site-packages (from PyQt6) (13.6.0)
Requirement already satisfied: PyQt6-Qt6>=6.5.0 in ./anaconda3/envs/acdc/lib/python3.9/site-packages (from PyQt6) (6.5.3)
Using cached PyQt6-6.5.3-cp37-abi3-macosx_10_14_universal2.whl (11.7 MB)
Installing collected packages: PyQt6
Successfully installed PyQt6-6.5.3
Traceback (most recent call last):
File "/Users/jie-ning.yang/anaconda3/envs/acdc/bin/acdc", line 8, in
sys.exit(run())
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/main.py", line 49, in run
run_gui()
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/main.py", line 59, in run_gui
_setup_gui_libraries()
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/_run.py", line 162, in _setup_gui_libraries
import pyqtgraph
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/pyqtgraph/init.py", line 18, in
from .colors import palette
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/pyqtgraph/colors/palette.py", line 1, in
from ..Qt import QtGui
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/pyqtgraph/Qt/init.py", line 181, in
import PyQt6.QtCore
ImportError: dlopen(/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/PyQt6/QtCore.abi3.so, 2): Symbol not found: _objc_opt_class
Referenced from: /Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/PyQt6/Qt6/lib/QtCore.framework/Versions/A/QtCore (which was built for Mac OS X 11.0)
Expected in: /usr/lib/libobjc.A.dylib
in /Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/PyQt6/Qt6/lib/QtCore.framework/Versions/A/QtCore
I tried to install other compatible GUI libraries (PyQt5, QtPy) but when I start acdc, it always asks me to install PyQt6. If I hit "n", here are the traceback
Initialising...
Since version 1.3.1 Cell-ACDC does not install a GUI library by default.
You can let Cell-ACDC install it now (default library is PyQt6), or you can abort (press "n")
and install a compatible GUI library with one of the following commands:
* pip install PyQt6
* pip install PyQt5 (or `conda install pyqt`)
* pip install PySide2
* pip install PySide6

Note: if PyQt6 installation fails, you could try installing any of the other libraries.
Do you want to install PyQt6 now ([y]/n)? n
Traceback (most recent call last):
File "/Users/jie-ning.yang/anaconda3/envs/acdc/bin/acdc", line 8, in
sys.exit(run())
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/main.py", line 49, in run
run_gui()
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/main.py", line 59, in run_gui
_setup_gui_libraries()
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/_run.py", line 154, in _setup_gui_libraries
raise e
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/_run.py", line 122, in _setup_gui_libraries
from qtpy.QtCore import Qt
File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/qtpy/init.py", line 287, in
raise QtBindingsNotFoundError from None
qtpy.QtBindingsNotFoundError: No Qt bindings could be found
Any suggestion to resolve this issue? I really want to get ACDC to work and try it out!

Hello Francesco,
Thank you so much for your suggestions. I've tried to uninstall PyQt6 and qtpy, and then using conda to install pyqt (pip-23.3.1, build: pyhd8ed1ab_0 from conda-forge)and qtpy (qtpy-2.4.1, build: pyhd8ed1ab_0 from conda-forge) as what you instructed.
However, I still get the same error:
(acdc) jie-nings-mbp:~ jie-ning.yang$ acdc
Initialising...
------------------------------------------------------------
Since version 1.3.1 Cell-ACDC does not install a GUI library by default.

You can let Cell-ACDC install it now (default library is `PyQt6`), or you can abort (press "n")
and install a compatible GUI library with one of the following commands:

    * pip install PyQt6
    * pip install PyQt5 (or `conda install pyqt`)
    * pip install PySide2
    * pip install PySide6

Note: if `PyQt6` installation fails, you could try installing any of the other libraries.

Do you want to install PyQt6 now ([y]/n)? n
Traceback (most recent call last):
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/bin/acdc", line 8, in <module>
    sys.exit(run())
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/__main__.py", line 49, in run
    run_gui()
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/__main__.py", line 59, in run_gui
    _setup_gui_libraries()
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/_run.py", line 154, in _setup_gui_libraries
    raise e
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/cellacdc/_run.py", line 122, in _setup_gui_libraries
    from qtpy.QtCore import Qt
  File "/Users/jie-ning.yang/anaconda3/envs/acdc/lib/python3.9/site-packages/qtpy/__init__.py", line 287, in <module>
    raise QtBindingsNotFoundError from None
qtpy.QtBindingsNotFoundError: No Qt bindings could be found

Here is the current package list from my environment:
(acdc) jie-nings-mbp:~ jie-ning.yang$ conda list
# packages in environment at /Users/jie-ning.yang/anaconda3/envs/acdc:
#
# Name                    Version                   Build  Channel
acdc                      0.1.1                    pypi_0    pypi
boto3                     1.28.70                  pypi_0    pypi
botocore                  1.31.70                  pypi_0    pypi
ca-certificates           2023.7.22            h8857fd0_0    conda-forge
cellacdc                  1.4.9                    pypi_0    pypi
certifi                   2023.7.22                pypi_0    pypi
charset-normalizer        3.3.1                    pypi_0    pypi
contourpy                 1.1.1                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
fonttools                 4.43.1                   pypi_0    pypi
gettext                   0.21.1               h8a4c099_0    conda-forge
giflib                    5.2.1                hb7f2c08_3    conda-forge
glib                      2.69.1               hfff2838_2
gst-plugins-base          1.14.1               hcec6c5f_1
gstreamer                 1.14.1               h6c40b1e_1
h5py                      3.10.0                   pypi_0    pypi
icu                       58.2              h0a44026_1000    conda-forge
idna                      3.4                      pypi_0    pypi
imageio                   2.31.6                   pypi_0    pypi
importlib-resources       6.1.0                    pypi_0    pypi
jmespath                  1.0.1                    pypi_0    pypi
joblib                    1.3.2                    pypi_0    pypi
jpeg                      9e                   hb7f2c08_3    conda-forge
kiwisolver                1.4.5                    pypi_0    pypi
krb5                      1.20.1               h049b76e_0    conda-forge
lazy-loader               0.3                      pypi_0    pypi
lerc                      3.0                  he9d5cce_0
libclang                  12.0.0          default_hbc2896b_2
libcxx                    14.0.6               h9765a3e_0
libdeflate                1.17                 hb664fd8_1
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libffi                    3.4.4                hecd8cb5_0
libiconv                  1.17                 hac89ed1_0    conda-forge
libllvm12                 12.0.1               hd011deb_2    conda-forge
libpng                    1.6.39               h6c40b1e_0
libpq                     12.15                h04015c4_1
libtiff                   4.5.1                hcec6c5f_0
libwebp                   1.3.2                hf6ce154_0
libwebp-base              1.3.2                h0dc2134_0    conda-forge
libxml2                   2.10.4               h930c0e2_0
libxslt                   1.1.37               h5d22bc9_0    conda-forge
lz4-c                     1.9.4                hf0c8a7f_0    conda-forge
matplotlib                3.8.0                    pypi_0    pypi
nasm                      2.14.02              h6c40b1e_0
natsort                   8.4.0                    pypi_0    pypi
ncurses                   6.4                  hcec6c5f_0
networkx                  3.2                      pypi_0    pypi
nspr                      4.35                 hea0b92c_0    conda-forge
nss                       3.89.1               hcec6c5f_0
numpy                     1.26.1                   pypi_0    pypi
opencv-python-headless    <IP_ADDRESS>                 pypi_0    pypi
openssl                   3.1.4                hd75f5a5_0    conda-forge
packaging                 23.1             py39hecd8cb5_0
pandas                    2.1.1                    pypi_0    pypi
pcre                      8.45                 he49afe7_0    conda-forge
pillow                    10.0.1                   pypi_0    pypi
pip                       23.3.1                   pypi_0    pypi
ply                       3.11                       py_1    conda-forge
psutil                    5.9.6                    pypi_0    pypi
pyparsing                 3.1.1                    pypi_0    pypi
pyqt                      5.15.7           py39he9d5cce_0
pyqt5                     5.15.10                  pypi_0    pypi
pyqt5-qt5                 5.15.11                  pypi_0    pypi
pyqt5-sip                 12.13.0                  pypi_0    pypi
pyqt6-qt6                 6.5.3                    pypi_0    pypi
pyqt6-sip                 13.6.0                   pypi_0    pypi
pyqtgraph                 0.13.3                   pypi_0    pypi
pyside2                   <IP_ADDRESS>                 pypi_0    pypi
python                    3.9.18               h5ee71fb_0
python-dateutil           2.8.2                    pypi_0    pypi
pytz                      2023.3.post1             pypi_0    pypi
qt-main                   5.15.2               h719ae48_7
qt-webengine              5.15.9               h90a370e_7
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
qtwebkit                  5.212                hbfab81c_5
readline                  8.2                  hca72f7f_0
requests                  2.31.0                   pypi_0    pypi
s3transfer                0.7.0                    pypi_0    pypi
scikit-image              0.22.0                   pypi_0    pypi
scikit-learn              1.3.2                    pypi_0    pypi
scipy                     1.11.3                   pypi_0    pypi
seaborn                   0.13.0                   pypi_0    pypi
setuptools                68.0.0           py39hecd8cb5_0
setuptools-scm            7.1.0            py39hecd8cb5_0
shiboken2                 <IP_ADDRESS>                 pypi_0    pypi
sip                       6.6.2            py39he9d5cce_0
six                       1.16.0                   pypi_0    pypi
sqlite                    3.41.2               h6c40b1e_0
threadpoolctl             3.2.0                    pypi_0    pypi
tifffile                  2023.9.26                pypi_0    pypi
tk                        8.6.12               h5d9f67b_0
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1            py39hecd8cb5_0
tqdm                      4.66.1                   pypi_0    pypi
typing-extensions         4.7.1            py39hecd8cb5_0
typing_extensions         4.7.1            py39hecd8cb5_0
tzdata                    2023.3                   pypi_0    pypi
urllib3                   1.26.18                  pypi_0    pypi
wheel                     0.41.2           py39hecd8cb5_0
xz                        5.4.2                h6c40b1e_0
zipp                      3.17.0                   pypi_0    pypi
zlib                      1.2.13               h4dc903c_0
zstd                      1.5.5                hc035e20_0

I'm sure that I uninstalled PyQt6 and running pip uninstall PyQt6 shows "skipping PyQT6 as it is not installed". But I still can see it on the list of packages......
Should I delete my whole environment and reinstall acdc again?

After this I would try to check if you can import qtpy and pyqt without running acdc.
In the terminal type python and then try typing import qtpy.QtCore.

I'm a bit confused about this instruction...... am I doing this correctly? Should I exit python and then start acdc again?
(acdc) jie-nings-mbp:~ jie-ning.yang$ python
Python 3.9.18 (main, Sep 11 2023, 08:38:23)
[Clang 14.0.6 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import qtpy.QtCore
>>>

Thanks again for your help!!!

Ok thanks for the information. Essentially, this is the error that we need to solve

Hi Francesco!
Although installing qtpy5-tools doesn't help, setting up the new environment works!
I followed your instructions, started a new environment, and installed and tested qtpy and pyqt first. (May I ask why they're using different methods to install?)When I used Python to test importing Qt, I saw no error messages. And then I proceed to install acdc.
After this, I can see the messages below and gui popped up!
Initialising...
Launching application...
Loading configuration files...
Using Qt 5.15.2 with Python 3.9.18
**********************************************
Welcome to Cell-ACDC v1.4.9
**********************************************
----------------------------------------------
NOTE: If application is not visible, it is probably minimized
or behind some other open windows.
----------------------------------------------

My old acdc environment (which installed QtPy6 and QtPy5 after acdc was installed) still cannot correctly launch the GUI even though python test import qtpy.QtCore shows no error messages. Therefore, for folks using an older OS like me (OS 10.14.6), don't forget to install nasm, qtpy and ptqy first prior to installing acdc!
Again, thank you so much for your help! Can't wait to get my hands on ACDC!

Nice! I'm happy to hear that it was just a question of creating the env from scratch! It's actually a good strategy when we feel that the env is broken :D.
Note that if you want to delete an env it is just a question of deleting the folder. To know the location of the envs, type the command conda env list. You will get a list of all of your envs. Go the location of the env you want to delete and remove the entire folder, that's it.
Anyway, let me know how things go and do not hesitate to contact us with questions and feedback and of course if you have issues!

(May I ask why they're using different methods to install?)

That is a very good question. The problem is that for some reason, installing PyQt5 from conda is more stable. Unfortunately, there are multiple ways of installing Python and Python packages so it boils down to personal preference. Personally, I prefer to create environments with conda, but then install packages with pip. If a package does not work with pip there is a high chance it will work with conda. This is the case for PyQt5 on macOS, hence, better to install it with conda. In the future, we will also publish Cell-ACDC to conda so users will be able to choose their favourite package manager to install it. I hope it's clear.
Cheers,
Francesco

Hi Francesco,
Oops, I am having another issue with installing javabridge. I've searched some previous issues and tried to solve it first, but I still cannot get it to work.
Here is the installing error messages from acdc:
(test) jie-nings-mbp:~ jie-ning.yang$ acdc
Initialising...
Launching application...
Loading configuration files...
Using Qt 5.15.2 with Python 3.9.18
**********************************************
Welcome to Cell-ACDC v1.4.9
**********************************************
----------------------------------------------
NOTE: If application is not visible, it is probably minimized
or behind some other open windows.
----------------------------------------------
Initializing Data structure module...
Checking if Java is installed...
======================================
Traceback (most recent call last):
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/dataStruct.py", line 1018, in __init__
    import javabridge
ModuleNotFoundError: No module named 'javabridge'
Traceback (most recent call last):
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/dataStruct.py", line 1018, in __init__
    import javabridge
ModuleNotFoundError: No module named 'javabridge'
======================================
git version 2.20.1 (Apple Git-117)
javac 1.8.0_352-internal
Collecting git+https://github.com/SchmollerLab/python-javabridge-acdc
  Cloning https://github.com/SchmollerLab/python-javabridge-acdc to /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns
  Running command git clone --filter=blob:none --quiet https://github.com/SchmollerLab/python-javabridge-acdc /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns
  Resolved https://github.com/SchmollerLab/python-javabridge-acdc to commit d62fb61a429fb1efc85d4a870eee67297978a621
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in ./anaconda3/envs/test/lib/python3.9/site-packages (from javabridge==1.0.20) (1.26.1)
Requirement already satisfied: requests in ./anaconda3/envs/test/lib/python3.9/site-packages (from javabridge==1.0.20) (2.31.0)
Requirement already satisfied: tqdm in ./anaconda3/envs/test/lib/python3.9/site-packages (from javabridge==1.0.20) (4.66.1)
Requirement already satisfied: charset-normalizer<4,>=2 in ./anaconda3/envs/test/lib/python3.9/site-packages (from requests->javabridge==1.0.20) (3.3.1)
Requirement already satisfied: idna<4,>=2.5 in ./anaconda3/envs/test/lib/python3.9/site-packages (from requests->javabridge==1.0.20) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./anaconda3/envs/test/lib/python3.9/site-packages (from requests->javabridge==1.0.20) (1.26.18)
Requirement already satisfied: certifi>=2017.4.17 in ./anaconda3/envs/test/lib/python3.9/site-packages (from requests->javabridge==1.0.20) (2023.7.22)
Building wheels for collected packages: javabridge
  Building wheel for javabridge (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [280 lines of output]
      Initialising...
      /Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
      !!

              ********************************************************************************
              Requirements should be satisfied by a PEP 517 installer.
              If you are using pip, you can try `pip install --use-pep517`.
              ********************************************************************************

      !!
        dist.fetch_build_eggs(dist.setup_requires)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-x86_64-cpython-39
      creating build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      copying javabridge/jutil.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      copying javabridge/locate.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      copying javabridge/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      copying javabridge/wrappers.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      copying javabridge/noseplugin.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge
      creating build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      copying javabridge/tests/__init__.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      copying javabridge/tests/test_cpython.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      copying javabridge/tests/test_javabridge.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      copying javabridge/tests/test_jutil.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      copying javabridge/tests/test_wrappers.py -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/tests
      creating build/lib.macosx-10.9-x86_64-cpython-39/javabridge/jars
      copying javabridge/jars/rhino-1.7R4.jar -> build/lib.macosx-10.9-x86_64-cpython-39/javabridge/jars
      running build_ext
      javac /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java/org/cellprofiler/runnablequeue/RunnableQueue.java
      jar cf /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/build/lib.macosx-10.9-x86_64-cpython-39/javabridge/jars/runnablequeue.jar -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/runnablequeue/RunnableQueue.class -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/runnablequeue/RunnableQueue$1.class
      javac /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java/org/cellprofiler/javabridge/test/RealRect.java
      jar cf /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/build/lib.macosx-10.9-x86_64-cpython-39/javabridge/jars/test.jar -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/test/RealRect.class
      javac /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java/org/cellprofiler/javabridge/CPython.java /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java
      Note: /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java/org/cellprofiler/javabridge/CPythonInvocationHandler.java uses unchecked or unsafe operations.
      Note: Recompile with -Xlint:unchecked for details.
      jar cf /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/build/lib.macosx-10.9-x86_64-cpython-39/javabridge/jars/cpython.jar -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/CPythonInvocationHandler.class -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/CPython.class -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/CPython$WrappedException.class -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/CPython$StackFrame.class -C /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/java org/cellprofiler/javabridge/CPythonInvocationHandler.class
      warning: _javabridge_osspecific.pxd:12:4: 'JavaVMOption' redeclared
      warning: _javabridge_osspecific.pxd:19:4: 'JavaVMInitArgs' redeclared
      warning: _javabridge.pyx:94:4: 'jvalue' redeclared
      warning: _javabridge.pyx:112:4: 'JavaVMOption' redeclared
      warning: _javabridge.pyx:119:4: 'JavaVMInitArgs' redeclared
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge.pyx:373:0: Global name __dead_objects matched from within class scope in contradiction to to Python 'class private name' rules. This may change in a future release.
      warning: _javabridge_osspecific.pxd:12:4: 'JavaVMOption' redeclared
      warning: _javabridge_osspecific.pxd:19:4: 'JavaVMInitArgs' redeclared
      warning: _javabridge_mac.pyx:4:4: 'jint' redeclared
      warning: _javabridge_mac.pyx:5:4: 'jboolean' redeclared
      warning: _javabridge_mac.pyx:9:4: 'JavaVM' redeclared
      warning: _javabridge_mac.pyx:14:4: 'JavaVMOption' redeclared
      warning: _javabridge_mac.pyx:21:4: 'JavaVMInitArgs' redeclared

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          void MacRunLoopStop() nogil
          void MacRunLoopReset() nogil
          int MacIsMainThread() nogil
          void MacRunLoopRunInMode(double) nogil

      cdef void StopVM(JavaVM *vm):
           ^
      ------------------------------------------------------------

      _javabridge_mac.pyx:34:5: Function signature does not match previous declaration

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
           MacStopVM()

      #
      # Unused stub in Mac
      #
      cdef int CreateJavaVM(JavaVM **pvm, void **pEnv, void *args):
           ^
      ------------------------------------------------------------

      _javabridge_mac.pyx:40:5: Function signature does not match previous declaration
      warning: _javabridge_nomac.pxd:15:4: 'JavaVMInitArgs' redeclared
      warning: _javabridge_nomac.pxd:20:4: 'JavaVMOption' redeclared
      warning: _javabridge_nomac.pyx:4:4: 'jint' redeclared
      warning: _javabridge_nomac.pyx:5:4: 'jboolean' redeclared
      warning: _javabridge_nomac.pyx:9:4: 'JavaVM' redeclared

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
               jint (*DetachCurrentThread)(JavaVM *vm) nogil
               jint (*GetEnv)(JavaVM *vm, void **penv, jint version) nogil
               jint (*AttachCurrentThreadAsDaemon)(JavaVM *vm, void *penv, void *args) nogil
          jint JNI_CreateJavaVM(JavaVM **pvm, void **penv, void *args) nogil

      cdef int MacStartVM(JavaVM **pvm, JavaVMInitArgs *pVMArgs, char *class_name) nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:19:5: Function signature does not match previous declaration

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          jint JNI_CreateJavaVM(JavaVM **pvm, void **penv, void *args) nogil

      cdef int MacStartVM(JavaVM **pvm, JavaVMInitArgs *pVMArgs, char *class_name) nogil:
          return -1

      cdef void StopVM(JavaVM *vm) nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:22:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:22:5: Exception check on 'StopVM' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return -1

      cdef void StopVM(JavaVM *vm) nogil:
          vm[0].DestroyJavaVM(vm)

      cdef void MacRunLoopInit() nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:25:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:25:5: Exception check on 'MacRunLoopInit' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          vm[0].DestroyJavaVM(vm)

      cdef void MacRunLoopInit() nogil:
          pass

      cdef void MacRunLoopRun() nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:28:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:28:5: Exception check on 'MacRunLoopRun' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          pass

      cdef void MacRunLoopRun() nogil:
          pass

      cdef void MacRunLoopStop() nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:31:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:31:5: Exception check on 'MacRunLoopStop' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          pass

      cdef void MacRunLoopStop() nogil:
          pass

      cdef void MacRunLoopReset() nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:34:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:34:5: Exception check on 'MacRunLoopReset' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          pass

      cdef void MacRunLoopReset() nogil:
          pass

      cdef int MacIsMainThread() nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:37:5: Function signature does not match previous declaration

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          pass

      cdef int MacIsMainThread() nogil:
          return 0

      cdef void MacRunLoopRunInMode(double timeout) nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:40:5: Function signature does not match previous declaration
      performance hint: _javabridge_nomac.pyx:40:5: Exception check on 'MacRunLoopRunInMode' will always require the GIL to be acquired.
      Possible solutions:
          1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.
          2. Use an 'int' return type on the function to allow an error code to be returned.

      Error compiling Cython file:
      ------------------------------------------------------------
      ...
          return 0

      cdef void MacRunLoopRunInMode(double timeout) nogil:
          pass

      cdef int CreateJavaVM(JavaVM **pvm, void **pEnv, void *args) nogil:
           ^
      ------------------------------------------------------------

      _javabridge_nomac.pyx:43:5: Function signature does not match previous declaration
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/setup.py", line 385, in <module>
          setup(name="javabridge",
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
          return distutils.core.setup(**attrs)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 364, in run
          self.run_command("build")
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
          super().run_command(command)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/setup.py", line 189, in run
          result = build_cython()
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/setup.py", line 62, in build_cython
          subprocess.check_call(cmd, env=env)
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/subprocess.py", line 373, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/Users/jie-ning.yang/anaconda3/envs/test/bin/python', '-m', 'cython', '-3', '/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/_javabridge.pyx', '/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/_javabridge_mac.pyx', '/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-8xsunqns/_javabridge_nomac.pyx']' returned non-zero exit status 1.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for javabridge
  Running setup.py clean for javabridge
Failed to build javabridge
ERROR: Could not build wheels for javabridge, which is required to install pyproject.toml-based projects
Traceback (most recent call last):
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/dataStruct.py", line 1018, in __init__
    import javabridge
ModuleNotFoundError: No module named 'javabridge'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/help/welcome.py", line 907, in launchDataStruct
    self.dataStructWin = dataStruct.createDataStructWin(
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/dataStruct.py", line 1085, in __init__
    myutils.install_javabridge()
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/cellacdc/myutils.py", line 493, in install_javabridge
    subprocess.check_call(
  File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/jie-ning.yang/anaconda3/envs/test/bin/python', '-m', 'pip', 'install', '-U', 'git+https://github.com/SchmollerLab/python-javabridge-acdc']' returned non-zero exit status 1.

I've checked the issue #7  and I'm sure that I have java8 installed.
When I'm not using PEP517, it shows the same error message as using acdc's error message.
I tried to follow issue #10 to install javabridge using PEP 517 installer. However, it shows another error:
(test) jie-nings-mbp:~ jie-ning.yang$ pip install --use-pep517 git+https://github.com/SchmollerLab/python-javabridge-acdc
Collecting git+https://github.com/SchmollerLab/python-javabridge-acdc
  Cloning https://github.com/SchmollerLab/python-javabridge-acdc to /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-amsoxiqs
  Running command git clone --filter=blob:none --quiet https://github.com/SchmollerLab/python-javabridge-acdc /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-req-build-amsoxiqs
  Resolved https://github.com/SchmollerLab/python-javabridge-acdc to commit d62fb61a429fb1efc85d4a870eee67297978a621
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      /private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-build-env-1ffoo5yd/overlay/lib/python3.9/site-packages/setuptools/build_meta.py:507: SetuptoolsDeprecationWarning: Running `setup.py` directly as CLI tool is deprecated.
      !!

              ********************************************************************************
              Please avoid using `sys.exit(0)` or similar statements that don't fit in the paradigm of a configuration file.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
      No module named 'cellacdc'
      ************************************************************
      `javabridge` requires the Cell-ACDC package. Please install it with `pip install cellacdc`. Thank you for your patience!
      ------------------------------------------------------------
      Traceback (most recent call last):
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/jie-ning.yang/anaconda3/envs/test/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-build-env-1ffoo5yd/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 398, in prepare_metadata_for_build_wheel
          self._bubble_up_info_directory(metadata_directory, ".egg-info")
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-build-env-1ffoo5yd/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 367, in _bubble_up_info_directory
          info_dir = self._find_info_directory(metadata_directory, suffix)
        File "/private/var/folders/f_/nwsn_t715r75yrr3jwxxy4wm0000gn/T/pip-build-env-1ffoo5yd/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 378, in _find_info_directory
          assert len(candidates) == 1, f"Multiple {suffix} directories found"
      AssertionError: Multiple .egg-info directories found
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

It seems that PEP517 solves some installation problems, but it couldn't find the cellacdc module.....
It doesn't seem to be the same issue as #10 because I did not see ValueError.
Again, I hope that you can provide me some suggestions to solve this issue. I really appreciate your detailed instruction and help!!

Oh yeah, I forgot to mention that javabridge does not work on macOS and there is no way to make it work 😰.
We have plans to abandon it and migrate to better libraries but it will take some time. In the meantime, these are possible solutions to create the required data structure:

Use the Fiji macro that we provide here https://github.com/SchmollerLab/Cell_ACDC/tree/main/FijiMacros.
Convert to TIFF using the microscope software
Create the data structure manually

We explain the data structure (and how to create it) in the user manual that you can download from here https://github.com/SchmollerLab/Cell_ACDC/blob/main/UserManual/Cell-ACDC_User_Manual.pdf. See the sections Create data structure using Fiji Macros and Manually create data structure from microscopy file(s).
Let me know how it goes, thanks!
Cheers,
Francesco

Oh cool! That's okay. I can use FIJI to convert my data. Thanks again for your prompt reply and now I'm gonna give it a try!!

Btw I just released v1.4.10 because I found a bug in v1.4.9. To upgrade, activate the environment and then run the command pip install --upgrade cellacdc. Thanks!
Cheers,
Francesco


InvalidAuthenticityToken after Devise login/logout in another tab

I have some problems with the InvalidAuthenticityToken exception when changing user authentication state in different tabs.
My app is quite simple. In fact, just a basic learning app for my studies. https://github.com/antonkoh/railsrep/tree/master/lesson17/my_app
It uses Devise gem for authentication:
<%= link_to "Sign out", destroy_user_session_path, method: :delete %>
<%= link_to "Sign in", new_user_session_path %>
protect_from_forgery is set to :exception as it is a browser app, nothing API related yet.
I do have <%= csrf_meta_tags %> specified in my layout.
The problem is reproduced in many different ways. Just a couple general examples:
On tab A I open a form to add/edit a post as an anonymous user.
On tab B I sign in as any user.
Back on tab A I try to add the post Result: InvalidAuthenticityToken
or
On tab A I open a list of posts as a signed-in user.
On tab B I sign out.
 Result: InvalidAuthenticityToken
I have introduced my own means to make sure that certain actions are available to certain signed-in users, but I don't even get to see them in action because of this exception. Is it supposed to work this way, just throwing itself at any PATCH request from an unrefreshed page after user authentication state change? Will my application become less secure if I change forgery protection mode to null_session or reset_session?
Thank you very much

Will my application become less secure if I change forgery protection mode to null_session or reset_session?

If you do this, then when you post on as the signed out user, it will sign the other tab out. There isn't much escape from this unfortunately. You could set config.clean_up_csrf_token_on_authentication = false but that will make you vulnerable to some attacks.  For example, if this attack existed on Twitter, they could tweet things in your name, although they would never steal any information.

@josevalim I am more interested in the opposite situation (see the 1st case described).
Why do the patch-requests fail if I SIGN IN on the other tab (not sign out)? Is this expected behavior?

Yes, it is expected behaviour, otherwise you are vulnerable to CSRF fixation attacks, which I tried to explain in my previous comment.

@josevalim Thank you.


preflight check failed when deploying the project in Docker

I made some changes to the codes of Marbles Project to make users create some accounts and manage them. My project is at: https://github.com/qianlizimu/marbles/tree/v2.0. When I deployed the project in Docker in Mac, I got this error message:
……
New GET request for /apple-touch-icon.png
Error Handeler -  /apple-touch-icon.png
GET /apple-touch-icon.png 404 168.900ms - -
[ibc-js] Deploy Chaincode - Complete
[preflight check] 1: testing if chaincode is ready
[ibc-js] read - success: {jsonrpc: '2.0', result: { status:'OK' }, id:1493795194403 }
[preflight check] 1 :  failed, trying again
……
I don't know what caused this. How to solve this problem? Thanks!

the preflight part is expecting a query resp to return null.  it looks like yours is returning status: OK.  if that is part of your changes, then you should change the preflight code, or comment it out.  its in app.js line 277.
the point of this code is to test if the chaincode is alive and healthy before moving on.

closing, inactivity.


Adds additional handlers

Coverage decreased (-40.0%) to 41.356% when pulling 262545630b56b3492ca7b2c53a40635465b5821c on improve-details into 86396b17aa8410c644bf9d9c67dff6e2a5f9e503 on master.


bpz_lite imports within functions

The original implementation of BPZ used a few tricks that require initialization in order for a few aspects to work properly.  Namely, the path to the data is set as an environment variable, $BPZDATAPATH, and the prior file is read from a specific file named in the init.  Because of this, we cannot import several functions from DESC_BPZ until after the init in bpz_lite is called.  This is also true in BPZPipe, and the solution there was to stick the imports in functions that are called after the init.  This is inelegant, and a little kludgey.  At some point we should figure out a "proper" way to handle this and implement.

As discussed in the tag up this morning, I think we've decided that this is justifiable, closing


rafthttp: fix misprint in readBytesLimit value

Simple fix for miscalculated value in readBytesLimit

LGTM.
Thanks a lot. Merging in greens.

Seems like 512mb is too much too ask
runtime: out of memory: cannot allocate 536936448-byte block<PHONE_NUMBER> in use)

Right, can you do something like
originalLimit := readBytesLimit
readBytesLimit = 1000
defer func() {
   readBytesLimit = originalLimit
}()

inside test?

I'll try right away


Addition of poll question and answer alternatives

Based on the poll used for the non-Git workshop on the 18-20 May 2020, designed by Sabry.
Please feel free to modify if the style should not be like this (i.e., block of code).

Sabry is it OK if I merge or are you still looking over this?


Fix normalization for off-policy algorithms

Description

Fixes normalization for off-policy algorithms that was not done properly (the normalization changes over time but the samples in the replay buffer were not updated).
There is a lot of duplicated code, mostly due to a missing collect_rollout :/ (this issue is already solved in v3)
Motivation and Context

[x] I have raised an issue to propose this change (required for new features and bug fixes)

closes #200
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation (update in the documentation)

Checklist:

[x] I've read the CONTRIBUTION guide (required)
[x] I have updated the changelog accordingly (required).
[ ] My change requires a change to the documentation.
[x] I have updated the tests accordingly (required for a bug fix or a new feature).
[ ] I have updated the documentation accordingly.
[x] I have ensured pytest and pytype both pass (by running  make pytest and make type).

The code is starting to look wonky with half of it having all the typing and whatnot...

true, the idea is to type everything little by little (that's what I tried to do with the replay buffer)
it is mostly an additional safety when changing the code, but in my opinion, half typed is better than not typed at all.

Maybe in v3 we could consider implementing e.g. value rescaling to avoid ad-hocish reward normalization (see page 3 of this paper on value rescaling).

I will try to take a look.


Create releases

You guys should create releases here also on Github (Eg. 1.8.0). It would be easy for developers to find code for the version they use.

Thank you, that would be definitely helpful for everybody. I will add this to our open source release runbook.

We'll do it from now on: https://github.com/twitter/digits-android/releases/tag/v1.8.0
thanks!


download_gem not being retried

It appears that retry applies to fetching specs and talking to git, but not to actually retrieving the gems themselves. We have received this error multiple times in the past month, even with bundle install ... --retry 5:
Gem::RemoteFetcher::FetchError: Errno::ECONNRESET: Connection reset by peer - SSL_connect
Bundler 1.12.5, Rubygems 2.6.5, Ruby 2.1.8p440.
For one example:
Fetching gem metadata from https://rubygems.org/
Fetching version metadata from https://rubygems.org/
Fetching dependency metadata from https://rubygems.org/
Fetching git://github.com/chef/omnibus.git
Fetching git://github.com/chef/omnibus-software.git
Fetching https://github.com/ksubrama/pedump.git
Installing awesome_print 1.7.0
Installing jmespath 1.3.1
Installing fuzzyurl 0.8.0
Installing mixlib-config 2.2.1
Installing mixlib-shellout 2.2.6
Installing chef-sugar 3.4.0
Installing cleanroom 1.0.0
Installing ffi 1.9.14 with native extensions
Installing libyajl2 1.2.0 with native extensions
Installing iostruct 0.0.4
Installing ipaddress 0.8.3
Installing mixlib-cli 1.7.0
Installing mixlib-log 1.6.0
Installing mixlib-versioning 1.1.0
Installing multipart-post 1.2.0
Installing plist 3.2.0
Installing systemu 2.6.5
Installing wmi-lite 1.0.0

Gem::RemoteFetcher::FetchError: Errno::ECONNRESET: Connection reset by peer - SSL_connect (https://rubygems.org/gems/ruby-progressbar-1.8.1.gem)
Installing thor 0.19.1
Installing progressbar 0.21.0
Installing zhexdump 0.0.2
Using bundler 1.12.5
Installing aws-sdk-core 2.5.0
Installing chef-config 12.12.15
Installing ffi-yajl 2.3.0 with native extensions
An error occurred while installing ruby-progressbar (1.8.1), and Bundler cannot
continue.
Make sure that `gem install ruby-progressbar -v '1.8.1'` succeeds before
bundling.

I can't find the code that's supposed to retry ECONNRESET (or indeed any rubygems request): rubygems' Gem::RemoteFetcher doesn't seem to take a parameter for retries, nor is bundler passing one.: https://github.com/bundler/bundler/blob/28ded71b5c399fde3ecbbe785f6077312f2c54c7/lib/bundler/rubygems_integration.rb#L277-L281
Since it happens intermittently, it's a little hard to repro, but we do enough builds to find it often. #4729 seems to have hit it as well.

can you try running with DEBUG=1 so we can get a backtrace on that? Thanks!

@segiddins sure. I'll set my builds to doing that. Not sure how long it'll take to get a repro!

@segiddins if install_from_gem doesn't have any internal retry logic, we should definitely wrap it in a retry block 😅

@segiddins @indirect Here's some debug output from one of our freebsd machines. We have had a few other repros today so let @jkeiser or myself know if you need more! https://gist.github.com/scotthain/58da989cfd632f252e3d1f7c20a2919e

The relevant bit:
Gem::RemoteFetcher::FetchError: Errno::ECONNRESET: Connection reset by peer - SSL_connect (https://rubygems.org/gems/multipart-post-1.2.0.gem)
/opt/angry-omnibus-toolchain/embedded/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:308:in `rescue in fetch_path'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:281:in `fetch_path'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:324:in `cache_update_path'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/site_ruby/2.1.0/rubygems/remote_fetcher.rb:188:in `download'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.12.5/lib/bundler/rubygems_integration.rb:615:in `download_gem'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.12.5/lib/bundler/source/rubygems.rb:410:in `fetch_gem'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.12.5/lib/bundler/source/rubygems.rb:116:in `install'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.12.5/lib/bundler/installer/gem_installer.rb:57:in `install'
/opt/angry-omnibus-toolchain/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.12.5/lib/bundler/installer/gem_installer.rb:15:in `install_from_spec'


more testing/fuzzing methods

fuzzing
mutation testing; https://cosmic-ray.readthedocs.io/en/latest/theory.html
property based testing; https://hypothesis.readthedocs.io/en/latest/

We have already added some fuzzing; https://github.com/komuw/naz/pull/178


Something went wrong on Analytic

Describe the issue:
Something went wrong on accessing  Analytic.

Pakc: Build #4926
How to reproduce:

Login to console
Navigate to Analytic page

Expected behavior:
Should be able to access the page.
Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: [e.g., IS 5.10.0, IS 5.9.0]
OS: [e.g., Windows, Linux, Mac]
Database: [e.g., MySQL, H2]
Userstore: [e.g., LDAP, JDBC]

Optional Fields
Related issues:

Suggested labels:

Closing since the issue is not reproducing
Console Version 2.6.6
MyAccount Version 2.2.16
https://github.com/wso2/product-is/assets/75057725/8963d4e4-cd0a-49b3-a3c4-896bd9ef08e3


Fixes issues where Node doesn't error if port is in use.

This fixes #206

Working on a solution now. I'm going to update my issue statement as well to clarify more what is happening and why.

After a lot of research, I have reverted part of my fix and provided test for it. The issue that I was having is an OS specific issue. If you have IPv6 enabled it will not necessarily block IPv4 for listening. I've added a comment block to explain more what is happening incase other engineers have this issue with their app.
I've also added in a catch for an error condition that may return on Linux when kernel option  ipv6.disable=1 is used. I did not force listening on IPv4 under that condition and instead opted for alerting the engineer of the issue so they can make a decision based on their environment.
Sources:
ipv6only listen/bind option
https://github.com/nodejs/node/issues/17664
Why listening "::" (all IPv6) also get "<IP_ADDRESS>"(all IPv4) listened?
https://github.com/nodejs/node/issues/9390

Sorry I haven't gotten back to this. I will try to get to this before the end of the month.


Version 2.0.0Beta2 Spacebar keyhandler is captureed by setting Shift+Spacebar keyhandler

Spacebar keyhandler is captureed by setting Shift+Spacebar keyhandler
PSReadline version 1.2 doesn't do this.
Also, no key handler is defined for Shift-Spacebar
[FYI:  Love PSReadLine]
Reproduce:
Get-PSReadlineKeyhandler | Where-Object Key -match 'Space'
set-psreadlinekeyhandler -chord 'Shift+SpaceBar' -Func Complete
Attempt to type any command with a Space in it:
Recall Get-PSReadLineKeyHandler above you can't type it
Get-PSReadlineKeyhandler | Where-Object Key -match 'Space'
Fix (trivial once the problem is understood -- must be done from profile or with pre-configured script since the spaces in the following command cannot be entered):
Remove-PSReadLineKeyHandler SpaceBar -ea Ignore
Environment:
PS version: 5.1.14409.1012
PSReadline version: 2.0.0-beta2
os: 6.1.7601.17514 (win7sp1_rtm.101119-1850)
PS file version: 10.0.14409.1005 (rs1_srvoob.161208-1155)Steps to reproduce or exception report

By some luck, the fix for this was in the most recently opened PR, so you got lucky pulling from AppVeyor :).


CFSv2 NetCDF4 Index Out of Bounds Issue

I am trying to access the SubX CFSv2 temperature forecasts used in this project in order to do some slightly different preprocessing and am running into an error with the netCDF4 back-end of xarray.
import xarray as xa
x = xa.open_dataset(
  "https://iridl.ldeo.columbia.edu/SOURCES/.Models/.SubX/.NCEP/.CFSv2/.forecast/.tas/dods",
    decode_times=False,
)
x['tas'][0, :, :, :, :].data

causes the following error
File src/netCDF4/_netCDF4.pyx:4406, in netCDF4._netCDF4.Variable.__getitem__()

File src/netCDF4/_netCDF4.pyx:5348, in netCDF4._netCDF4.Variable._get()

IndexError: index exceeds dimension bounds

This also occurs when simply trying to access the data as x['tas'].data
Is this a good way to go about this? It seems the scripts here are running into the issue mentioned in #18
Thanks!

@btickell you actually got further along with downloading data from that link than I did.  I tried it on my computer and have the same issues with you. It's a very very large file and may have which may have some part in the issue. The data I downloaded came with 1 file per initialization day (but I had to figure out which files were the correct ones). It appears that you have the entire time series because with GMAO GEOS-5, there were only 1239 initialized days. Your current script has creates 6456 days (which is still unrealistic for CFSv2).


Improve flag_samples further

The current implementation of flag_samples is much faster than its original implementation but can still be improved upon. There's a lot of type coercions that don't need to be there if we just let the type be logical instead of 0/1. We also might be able to drop some of the tidyverse dependencies and perhaps implement some of the things in C++ if it's refactored enough.

Closing this, a dataset with one speaker only seems to take about 80ms for me so I don't think it's worth rewriting anything more in C++ unless i get particularly bored


Build fails as its unable to create the .repoconfig in the root

I ve been trying to follow the instructions of running build-kitkat.sh.
How ever the build fails as its unable to create the .repoconfig in the root. I presume the .repoconfig must be created in the home dir of the user and not in /
I also could not find run.sh in the docker container in /usr/local/bin/run.sh
Env : Ubuntu 14.04
I have a user group called docker and I m running the script from a user in that usergroup.
Log :
user@palm:~/docker-aosp/tests$ ./build-kitkat.sh
+ '[' '' = docker ']'
+ aosp_url=https://raw.githubusercontent.com/kylemanna/docker-aosp/master/utils/aosp
+ args='bash run.sh docker'
+++ dirname ./build-kitkat.sh
++ cd .
++ pwd -P
++ basename ./build-kitkat.sh
+ export 'AOSP_EXTRA_ARGS=-v /home/dhanasek/docker-aosp/tests/build-kitkat.sh:/usr/local/bin/run.sh:ro'
+ AOSP_EXTRA_ARGS='-v /home/dhanasek/docker-aosp/tests/build-kitkat.sh:/usr/local/bin/run.sh:ro'
+ export AOSP_IMAGE=kylemanna/aosp:4.4-kitkat
+ AOSP_IMAGE=kylemanna/aosp:4.4-kitkat
+ '[' -n '' ']'
+ '[' -x ../utils/aosp ']'
+ ../utils/aosp bash run.sh docker
+ AOSP_IMAGE=kylemanna/aosp:4.4-kitkat
+ AOSP_VOL=/home/dhanasek/docker-aosp/tests/kitkat
+ AOSP_ARGS='--rm -it'
+ AOSP_VOL_AOSP=/home/dhanasek/docker-aosp/tests/kitkat/aosp
+ AOSP_VOL_CCACHE=/home/dhanasek/docker-aosp/tests/kitkat/ccache
+ '[' '!' -d /home/dhanasek/docker-aosp/tests/kitkat/aosp -o '!' -d /home/dhanasek/docker-aosp/tests/kitkat/ccache ']'
+ '[' -n '' ']'
+ docker run --rm -it -v /home/dhanasek/docker-aosp/tests/build-kitkat.sh:/usr/local/bin/run.sh:ro -v /home/dhanasek/docker-aosp/tests/kitkat/aosp:/aosp -v /home/dhanasek/docker-aosp/tests/kitkat/ccache:/tmp/ccache kylemanna/aosp:4.4-kitkat bash run.sh docker
+ '[' docker = docker ']'
+ TEST_BRANCH=android-4.4.4_r2.0.1
+ TEST_URL=https://android.googlesource.com/platform/manifest
++ grep '^processor' /proc/cpuinfo
++ wc -l
+ cpus=4
+ repo init --depth 1 -u https://android.googlesource.com/platform/manifest -b android-4.4.4_r2.0.1
fatal: cannot make /.repoconfig directory: Permission denied

Hey!   The problem is file permissions.  I speculate that your local user's id and the user in the container differ.  Run id as your local user and see if the uid is different then the output of id from the container.
docker run --rm -it kylemanna/aosp:4.4-kitkat id
uid=1000(aosp) gid=1000(aosp) groups=1000(aosp)

Also, the user docker is what the daemon is running as the docker group is what controls access to the docker daemon socket.  The problem here is that /home/dhanasek/docker-aosp/tests/kitkat isn't writable by uid 1000 in the docker container.
To be hacky, try chmod 777 /home/dhanasek/docker-aosp/tests/kitkat and see if that helps.
A better solution might be more like this:
export AOSP_VOL=$PWD/build
mkdir $AOSP_VOL && chown 1000:1000 $AOSP_VOL
./build-kitkat.sh

Let me know if that works for ya!


MRG: Remove 2.6

Closes #3529.

Let's see if the CIs are happy with this

Current coverage is 87.29% (diff: 100%)

Merging #3530 into master will increase coverage by 0.47%

@@             master      #3530   diff @@
==========================================
  Files           341        341
  Lines         59828      59690   -138
  Methods           0          0
  Messages          0          0
  Branches       9094       9083    -11
==========================================
+ Hits          51943      52105   +162
+ Misses         5136       4862   -274
+ Partials       2749       2723    -26

Powered by Codecov. Last update 1c11647...80fa5d4

Ready for review/merge from my end. CIs are happy, including a [circle full] build in there that included all CircleCI-relevant changes.

Looks good - especially the red in line changes and the green of happy CIs. :)

LGTM
@yarikoptic any potential issue for Debian packaging?

nope! we are well beyond 2.6.. pushed to alioth pkging update:
commit 413eda34d04aebb7df520a6aa4de0e9b7cb992c9
Author: Yaroslav Halchenko<EMAIL_ADDRESS>Date:   Fri Aug 26 12:13:43 2016 -0400

    upcoming release drops compat with python 2.6

diff --git a/debian/control b/debian/control
index 2a2af01..b5984de 100644
--- a/debian/control
+++ b/debian/control
@@ -28,7 +28,7 @@ Standards-Version: 3.9.8
 Vcs-Browser: http://anonscm.debian.org/gitweb/?p=debian-med/python-mne.git
 Vcs-Git: git://anonscm.debian.org/debian-med/python-mne.git
 Homepage: http://martinos.org/mne
-X-Python-Version: >= 2.6
+X-Python-Version: >= 2.7

Great thx guys !


Include requirements.txt in sdists

sdists cannot be installed without it

Thanks! I missed that one out :+1:


ocamlPackages.ffmpeg: 1.1.0 -> 1.1.3

Description of changes
1.1.3 (2022-04-06)
=====
* Use size of memory referred by AVFrame and AVPacket
  when allocating OCaml frame custom block to make sure
  the GC is diligent when cleaning up OCaml values
  referring to AVFrame and AVPacket.

1.1.2 (2022-03-13)
=====
* Change error returned on OCaml callback exception
  to AVERROR_EXTERNAL, add log entry.
* Get rid of naked pointers.
* Adapt to most recent FFmpeg API.
* Added offset/length to conversion functions.

1.1.1 (2021-12-26)
======
* Added support for interruption callback.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 169077 run on x86_64-linux 1

8 packages built:

ocamlPackages.ffmpeg
ocamlPackages.ffmpeg-av
ocamlPackages.ffmpeg-avcodec
ocamlPackages.ffmpeg-avdevice
ocamlPackages.ffmpeg-avfilter
ocamlPackages.ffmpeg-avutil
ocamlPackages.ffmpeg-swresample
ocamlPackages.ffmpeg-swscale


Media Library: Can't migrate to Azure Blob Storage automatically

Let's consider the following scenario:

I'm developing an Orchard application locally while using the regular file storage (Orchard.Web\Media) for media files.
After deploying the application to Azure, I also want to use Blob Storage:

Enable the Azure Media Storage Feature.
Copy all files from the local file system to the Storage Account.

I would expect the Media Library to continue to work as before, but media files are not loading at all for any folder, except "Recent". That's because Orchard stores the path of the media files based on their location on the file system (at the time they are created): NTFS uses \ as the directory separator, but AFS uses /, so when querying for content items in a specific folder while using AFS won't return any results for media items created on NTFS (and vice versa).
My suggestion is that Media Library should always use one of those (I don't think it matters which one, since both of them are illegal characters for file and folder names).

@BenedekFarkas i agree with you, there should be a simple solution to migrate to Azure Blob/File System anytime one decides to go with either storage.
I came across this problem working on a project where the site involved around 6000 media items. We kept working happily with local file system and once we were ready to go live with Azure Blob Storage we got struck with this. Later we solved it by running SQL Queries to convert all those directory separators to "/".
In HTML Content Editors and wherever media urls are used, Orchard uses URL based paths so  "/" separator would be a good candidate for this.

I agree, the database should have normalized file paths, using '/' might be the simplest.

I'm using 1.10.x with Azure Media Storage module enabled and that new bit of code is failing:

It uploaded my kitty test pictures (I can see them in the blob) but now I can't view them.
Repro
This is an almost out of the box setup, I have cloned 1.10.x at 'b8e138b2d55734c9f6257d08fa944fb48b1676cb' then set up the azure settings and media to be stored in blob. Created a folder called 'cat pictures', dropped three pictures into it and tried to go back to media browser but all pages (folder page and recent page) are showing the display error.

Hmm actually it only happens after I have uploaded and clicked "close".
When I navigate away from it (for example clicking on to Widgets admin page) and then click back to Media, everything works as expected.


fix(TreeView): add hcm styles for aria-current in TreeView

This addresses the HCM feedback from: https://github.com/github/primer/issues/1114#issuecomment-1292865037
In general, TreeView should support HCM in the following scenarios:

When an item is hovered, there is an outline

When an item receives focus, it uses the SelectedItem keyword as the outline color

When an item has aria-current="true", it uses the SelectedItem keyword as the background color

@ericwbailey would love your feedback on what the best values would be for HCM here. Mainly have just been using keywords from: https://www.w3.org/TR/css-color-4/#css-system-colors or transparent outlines.


Update onnxruntime-training nightly dockerfile

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.


Accepting proposals in English only

I suggest that the bot be set up to allow comments on sentences in English only. Proposals written in different languages can be mistranslated and therefore misunderstood by participants who speak other languages.
To be able to print words in other languages in a proposal, such as a proper name, you can set the bot to print a limit of 10% characters in other languages.

Perhaps worth checking for just ~40-50% of English, if someone wants to duplicate text in second language (pretty often our people duplicate in Russian).

Done. Also added checking if the proposal actually starts with English.

Сделанный. Также добавлена проверка, действительно ли предложение начинается с английского.

Super!


Formatter Indentation and Delimiter Logic

Added some state to the formatter for the purposes of better handling of delimiters, i.e. [], (), {}, and type parameter list delimiters <>. The formatting of delimited blocks will be determined by whether there is a newline in the whitespace following the open delimiter.

If there is a newline, then the indentation is increased for the length of the block, and separators, i.e. ',' and ';', require newlines after them, with the closing delimiter having a newline in front of it.
If there are no newlines, then delimiters have no spacing after the opening and before the closing delimiters, except the brace delimiters which have a singe space instead, and separators require a single space after them.

I've run this version of the formatter against the samples and library files to test that nothing unreasonable or unexpected occurs. I don't want to add the updated samples/libraries to this PR though, as I think that would be better for a future PR where I could include running the formatter automatically in the build pipeline.


Index access returns incorrect data type

As far as I could see, the current implementation of NDArray indexers only return a single value where they should return NDARRAY of ndim - indexers.Length?

@emrahsungu Yes, your are right. return an new NDArray with ndims.

I just want to make sure before I take on the job, I also see that the implementation of Indexers are also not completed at all and will require an overhaul, right? Especially, the indexer which takes NDArray

@emrahsungu if you can make the unit test IndexingTest.NDArrayAccess pass, It will be OK.
        [TestMethod]
        public void NDArrayAccess()
        {
            var nd = np.arange(4).reshape(2, 2);

            var row1 = nd[0];
            // Assert.AreEqual(row1.GetType(), typeof(NDArray));
            // Assert.AreEqual(row1[0][0], 0);
            // Assert.AreEqual(row1[0][1], 1);
        }

@Ocenia2018, thanks for the swift response. I have seen the unit test but what I am concerned is that it looks like Indexer which takes NDArray as parameter is not fully implemented and requires a bit of an overhaul.  Are we on the same page?😀

Yes we need an overhaul. I think it will return a object, When ndim = 0 then return scalar value, when ndim == input.ndim then rerun scalar value, when ndim < input.ndim then return ndarray. And add an implicit to convert between ndarray and ndarray when the ndim is different. I haven’t find the best solution.

@Oceania2018 Yeah I had something similar in my mind. I will try the challenge. Hopefully, I can come up with a nice solution.

Considering the performance and long term goal which is becoming productivity, I want to refactor NumSharp v2.x based on ArrayFire.

We will change our storage to unmanaged memory. We can discuss on another thread if have interested on the 2.x plan.

You can still working on current branch if you want, because the high level will not going to be changed.

Sure I would love to hear more!

@emrahsungu raised new issue of #195