Conda package is broken for Linux/Python 3.8 and 3.9

On Ubuntu 20.04.2 with Python 3.8.10, after installing via conda
The following NEW packages will be INSTALLED:

  pyabel             conda-forge/linux-64::pyabel-0.8.3-py38h8790de6_0

any attempt to import the abel module leads to a core dump with error
Illegal instruction (core dumped)

It is just the conda-forge package? Does the pip install work fine?

The zeroth question would be: who releases the conda-forge packages and why is it still v0.8.3, while our v0.8.4 has been released more than 2 months ago (and became available on PyPI at the same time)?

It is just the conda-forge package? Does the pip install work fine?

Yes, the version from PyPI seems fine.

The zeroth question would be: who releases the conda-forge packages and why is it still v0.8.3, while our v0.8.4 has been released more than 2 months ago (and became available on PyPI at the same time)?

You can see who is involved in maintaining the conda-forge package here. It's not obvious why it hasn't been updated to match the most recent package release.

The README file there lists our members as “Feedstock Maintainers”, but the last commit from them (@rth) to that repository is from 2018, and mostly other people and bots took care of it. So maybe it would help to ask there what's going on?

The feedstock maintainers should all have access to update the conda packaging, so I would suggest contacting them about this.

Interesting I can reproduce,
Here is the backtrace

Thread 1 "python" received signal SIGILL, Illegal instruction.
0x00007fffd21d84b2 in __pyx_pymod_exec_direct ()
   from /home/rth/miniconda3/envs/test-env2/lib/python3.8/site-packages/abel/lib/direct.cpython-38-
x86_64-linux-gnu.so
(gdb) bt
#0  0x00007fffd21d84b2 in __pyx_pymod_exec_direct ()
   from /home/rth/miniconda3/envs/test-env2/lib/python3.8/site-packages/abel/lib/direct.cpython-38-
x86_64-linux-gnu.so
#1  0x0000555555782328 in PyModule_ExecDef (module=0x7fffd2208130, def=<optimized out>)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Objects/moduleobj
ect.c:414
#2  0x00005555557823c3 in exec_builtin_or_dynamic (mod=0x7fffd2208130)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Python/import.c:2
169
#3  _imp_exec_builtin_impl (module=<optimized out>, mod=0x7fffd2208130)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Python/import.c:2
262
#4  _imp_exec_builtin (module=<optimized out>, mod=0x7fffd2208130)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Python/clinic/imp
ort.c.h:388
#5  0x00005555556789ab in cfunction_vectorcall_O (func=func@entry=0x7ffff7668590,
    args=<optimized out>, nargsf=nargsf@entry=1, kwnames=<optimized out>)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Objects/methodobj
ect.c:482
#6  0x000055555568381e in PyVectorcall_Call (callable=0x7ffff7668590, tuple=<optimized out>,
    kwargs=<optimized out>)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Objects/call.c:20
0
#7  0x00005555557292da in do_call_core (kwdict=0x7fffd2205500, callargs=0x7fffd22045e0,
    func=0x7ffff7668590, tstate=<optimized out>)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Python/ceval.c:49
83
#8  _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>)
    at /home/conda/feedstock_root/build_artifacts/python-split_1620713755437/work/Python/ceval.c:35
59

not sure what's causing it. Will try to update the repo to latest conda-forge recommendations, maybe something was wrong with build dependencies.

who releases the conda-forge packages and why is it still v0.8.3, while our v0.8.4 has been released more than 2 months ago (and became available on PyPI at the same time)?

Yes, @MikhailRyazanov  feel free to add yourself to https://github.com/conda-forge/pyabel-feedstock/blob/56fa1ec17bde2699ef33918e5f560cd94e17d9e6/recipe/meta.yaml#L55  It's mostly automated so one only need to merge PRs that get opened after the PyPi release (in this case https://github.com/conda-forge/pyabel-feedstock/pull/16).

I released 0.8.4 on conda-forge, but it's still broken on Linux. I think issue is the -march=native flag in setup.py https://github.com/PyAbel/PyAbel/blob/39501dc371c31e2660c47a1549a1cf13d60c19d3/setup.py#L41-L42
The VM where those packages are built for conda-forge apparently has a CPU with a larger instruction set than most users. Removing that flag for the next release should fix it I think.

@rth, thanks for looking into this!

I think issue is the -march=native flag

Makes sense. :–) It would be strange to expect that build-server's “native” is native for everybody...
But can this be somehow modified locally in conda-forge recipes instead of removing from PyAbel/setup.py (because it in principle should be useful for user's local installations)?

feel free to add yourself

I'm not using conda and don't know how it works, so for now I'd prefer to leave the responsibility to those who are more experienced. :–)

modified locally in conda-forge recipes

One solution could be to patch, but that's a bit discouraged. Another one could be to define some env variable that would allow to disable that flag.

In principle, in the C/C++ world, people use CFLAGS and LDFLAGS, which can be taken from the environment and/or set in the makefile. I guess, we can do a similar thing here. Does the conda-forge building system already define something like that?

@rth
I did try and use Python 3.5 and installed the PyAbel version that was automatically installed through conda and it worked on Linux Mint 20.2
I can try other versions once I get back to my Linux box at home just to test.

Copying @oobie2142's report from #329 here for reference:

On Linux Mint 20.2 with Python 3.9.7 trying to import package
33908 illegal hardware instruction (core dumped) python -c 'import IPython; IPython.terminal.ipapp.launch_new_instance()'
Not sure if the Python 3.8 work around applies to this situation, but will I need to update some flags to get this to work?

Could you please also tell your CPU model, just in case? (Although I don't know how to find out the target CPU for a Conda package...)
Regarding workarounds, I guess that if after installing the package you manually delete the .so file from the abel/lib subdirectory, this will prevent PyAbel from using the Cython version of the "direct" method, but everything else should work fine.

My CPU is a Ryzen Threadripper 2920x.
I will give that a shot.

I tried a new environment with Python 3.8.12 and was able to install and import pyabel 0.8.4
Thought I would give an update.

Ryzen Threadripper 2920x

OK, makes sense. I've tried to disassemble the compiled library, and it shows AVX-512 instructions:
$ objdump -d direct.cpython-39-x86_64-linux-gnu.so | grep zmm
    833a:       62 f1 fe 48 6f 1a       vmovdqu64 (%rdx),%zmm3
    8340:       62 f1 fe 48 7f 9b 50    vmovdqu64 %zmm3,0x50(%rbx)
    839b:       62 d1 fe 48 6f 10       vmovdqu64 (%r8),%zmm2
    83a1:       62 f2 fd 48 7c ca       vpbroadcastq %rdx,%zmm1
    83ab:       62 f3 f5 48 1f c8 04    vpcmpneqq %zmm0,%zmm1,%k1
    83b2:       62 f1 fe 48 7f 93 10    vmovdqu64 %zmm2,0x10(%rbx)
    83bc:       62 f1 fe c9 6f 02       vmovdqu64 (%rdx),%zmm0{%k1}{z}
    83c2:       62 f2 f6 48 27 c9       vptestnmq %zmm1,%zmm1,%k1
    83c8:       62 f3 75 48 25 c9 ff    vpternlogd $0xff,%zmm1,%zmm1,%zmm1
...

They are present in relatively recent Intel CPUs (perhaps the Conda build server has one of them) but are not yet supported by AMD CPUs. By the way, @stggh, on which CPU did you try (because i7-6700 and i7-9700, from which we have your benchmarks, apparently don't support AVX-512 either)?

I tried a new environment with Python 3.8.12 and was able to install and import pyabel 0.8.4

With some additional actions, or it just worked as is?

on which CPU did you try

Works fine for both systems:   i7-9700 CPU and i7-6700 CPU

In case it is useful, here is the build log, for the i7-6700 system: gist

Works fine for both systems

Then it's really strange. Or they do have AVX-512? Or you've installed some other packages?

avx and avx2?
lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              1
Core(s) per socket:              8
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           158
Model name:                      Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz
Stepping:                        13
CPU MHz:                         2999.999
CPU max MHz:                     4700.0000
CPU min MHz:                     800.0000
BogoMIPS:                        6000.00
Virtualisation:                  VT-x
L1d cache:                       256 KiB
L1i cache:                       256 KiB
L2 cache:                        2 MiB
L3 cache:                        12 MiB
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     KVM: Mitigation: VMX disabled
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and secc
                                 omp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitiza
                                 tion
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Mitigation; TSX disabled
Vulnerability Tsx async abort:   Mitigation; TSX disabled
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pa
                                 t pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall
                                 nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_
                                 good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq d
                                 tes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pd
                                 cm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
                                  xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb
                                  invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vn
                                 mi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 sm
                                 ep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsa
                                 veopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_noti
                                 fy hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
[Gibbo:.+oodsmith/Issues]$ lscpu | grep avx
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities

Null result for:
/home/stg121/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/lib
objdump -d direct.cpython-39-x86_64-linux-gnu.so | grep zmm

No, AVX and AVX2 are older 256-bit sets, there are supported by many CPUs. AVX-512 is a recent 512-bit extension, the flags for it should be called avx512<something>.

In case it is useful, here is the build log, for the i7-6700 system

If I understand correctly, there you were building the package yourself on that machine. This should work, of course. Have you tried to installed the prebuilt Conda packages? The problem is with them.

installed the prebuilt Conda packages

Ahh!  My bad.
I will check.

Installs OK:
conda list
# packages in environment at /home/stg121/anaconda3/envs/py39:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main
       :                              :                             :
pyabel                    0.8.4            py39hce5d2b2_0    conda-forge

Fails to run:
(py39) [Gibbo:.+master/examples]$ which python
/home/stg121/anaconda3/envs/py39/bin/python
(py39) [Gibbo:.+master/examples]$ python example_O2_PES_PAD.py
Traceback (most recent call last):
  File "/home/stg121/Software/Python/PyAbel/PyAbel-master/examples/example_O2_PES_PAD.py", line 7, in <module>
    import abel
  File "/home/stg121/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/__init__.py", line 25, in <module>
    from . import basex
  File "/home/stg121/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/basex.py", line 20, in <module>
    from abel.tools.polynomial import PiecewisePolynomial
  File "/home/stg121/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/tools/__init__.py", line 6, in <module>
    from . import analytical
  File "/home/stg121/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/tools/analytical.py", line 7, in <module>
    import scipy.interpolate
  File "/home/stg121/.local/lib/python3.9/site-packages/scipy/interpolate/__init__.py", line 166, in <module>
    from .interpolate import *
  File "/home/stg121/.local/lib/python3.9/site-packages/scipy/interpolate/interpolate.py", line 21, in <module>
    from .interpnd import _ndim_coords_from_arrays
  File "interpnd.pyx", line 1, in init scipy.interpolate.interpnd
  File "/home/stg121/.local/lib/python3.9/site-packages/scipy/spatial/__init__.py", line 96, in <module>
    from .kdtree import *
  File "/home/stg121/.local/lib/python3.9/site-packages/scipy/spatial/kdtree.py", line 5, in <module>
    from .ckdtree import cKDTree, cKDTreeNode
ImportError: /home/stg121/.local/lib/python3.9/site-packages/scipy/spatial/ckdtree.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv

Using ipython and simply import abel:
py39) [Gibbo:.+master/examples]$ which ipython
/home/stg121/anaconda3/envs/py39/bin/ipython
(py39) [Gibbo:.+master/examples]$ ipython
Python 3.9.7 (default, Sep 16 2021, 13:09:58)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import abel
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-46961e2d8939> in <module>
----> 1 import abel

~/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/__init__.py in <module>
     23 filterwarnings('default', r'^abel\.', category=DeprecationWarning)
     24
---> 25 from . import basex
     26 from . import benchmark
     27 from . import dasch

~/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/basex.py in <module>
     18 from scipy.linalg import inv
     19
---> 20 from abel.tools.polynomial import PiecewisePolynomial
     21
     22 from ._version import __version__

~/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/tools/__init__.py in <module>
      4 from __future__ import unicode_literals
      5
----> 6 from . import analytical
      7 from . import center
      8 from . import circularize

~/.local/lib/python3.9/site-packages/PyAbel-0.8.4-py3.9-linux-x86_64.egg/abel/tools/analytical.py in <module>
      5 import abel
      6 import scipy.constants as const
----> 7 import scipy.interpolate
      8
      9 # This file includes functions that have a known analytical Abel transform.

~/.local/lib/python3.9/site-packages/scipy/interpolate/__init__.py in <module>
    164 (should not be used in new code).
    165 """
--> 166 from .interpolate import *
    167 from .fitpack import *
    168

~/.local/lib/python3.9/site-packages/scipy/interpolate/interpolate.py in <module>
     19 from . import _ppoly
     20 from .fitpack2 import RectBivariateSpline
---> 21 from .interpnd import _ndim_coords_from_arrays
     22 from ._bsplines import make_interp_spline, BSpline
     23

interpnd.pyx in init scipy.interpolate.interpnd()

~/.local/lib/python3.9/site-packages/scipy/spatial/__init__.py in <module>
     94 """
     95
---> 96 from .kdtree import *
     97 from .ckdtree import *
     98 from .qhull import *

~/.local/lib/python3.9/site-packages/scipy/spatial/kdtree.py in <module>
      3 import numpy as np
      4 import warnings
----> 5 from .ckdtree import cKDTree, cKDTreeNode
      6
      7 __all__ = ['minkowski_distance_p', 'minkowski_distance',

ImportError: /home/stg121/.local/lib/python3.9/site-packages/scipy/spatial/ckdtree.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZSt28__throw_bad_array_new_lengthv

Hmm, seems to be a separate problem, an overlap with the default installation.

I'm mostly sure the problem is the '-march=native' flag,
https://github.com/PyAbel/PyAbel/blob/ab03e247fe9fef22a266648f027fc14b31b45458/setup.py#L41
as it would result in a non portable binary, that will use the same CPU features as the one used for the build. We should remove it.

OK, I did already say the same thing above. I was sure I did, but couldn't find where :)

@stggh, since you have some troubles installing the Conda package but can build your own, could you please in the meantime test how much this "native" actually helps in terms of speed on your machines? Because on my laptop it surprisingly makes the code slower, while replacing -O2 by -Ofast (which should be fine for our purposes) makes it noticeably faster:
=== Forward Abel transform ====================================
Options             n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-O2 -march=native       4.2        32.7       239.5      2055.4
-O2                     3.3        24.2       164.1      1475.4
-Ofast                  2.6        17.6       109.8      1069.3

=== Inverse Abel transform ====================================
Options             n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-O2 -march=native       5.0        35.7       254.9      2166.2
-O2                     4.3        27.4       179.4      1587.5
-Ofast                  3.6        20.6       125.5      1181.4

an overlap with the default installation.

I had to resort to using the conda installation:
(base) [Gibbo:/home/stg121]$ conda install pyabel
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/stg121/anaconda3

  added / updated specs:
    - pyabel

The following NEW packages will be INSTALLED:

  pyabel             conda-forge/linux-64::pyabel-0.8.4-py39hce5d2b2_0
  python_abi         conda-forge/linux-64::python_abi-3.9-2_cp39

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

The import fails, without any helpful error messages:
(base) [Gibbo:/home/stg121]$ python
Python 3.9.7 (default, Sep 16 2021, 13:09:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import abel
Illegal instruction (core dumped)

test how much this "native" actually helps in terms of speed on your machines?

Back to the installed version, re-building each optimized version
=== Forward Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-O2 -march=native   3.7        29.6       202.3      1669.7
-O2                 3.4        26.7       199.6      1537.3
-Ofast              2.5        17.7       118.7       942.4

=== Inverse Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-O2 -march=native   4.0        30.7       211.7      1697.4
-O2                 3.8        28.1       212.7      1577.1
-Ofast              2.7        19.2       130.2       980.3

Yes, code is is lower with -native, but not by much. -Ofast makes a big difference.
i7-9700 CPU @ 3.00GHz
[Gibbo:/home/stg121]$ gcc --version
gcc (Ubuntu 11.2.0-7ubuntu2) 11.2.0

Using cpu-boost for max frequency 4.7GHz,  tends to thermally throttle the system for large jobs, okay for this test:
CPU max MHz:                     4700.0000
=== Forward Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-Ofast                1.7        11.8        76.8       570.0

=== Inverse Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
---------------------------------------------------------------
-Ofast                2.0        13.1        83.6       607.6

Thanks, @stggh! The fact that the precompiled package fails with “Illegal instruction” on your system supports @rth's idea about -march=native. And I'm somewhat surprised that -march=native slows the code down for you as well — I could guess that my mobile CPU has some trade-offs and while implementing extended instruction sets tries to do so more power-efficiently than faster, but seeing the same effect, although much weaker, even on a high-end CPU is really strange. Perhaps GCC optimization is just poorly tuned for such less common cases...
I'm wondering whether we can find somewhere a modern CPU supporting AVX-512 to test whether -march=native helps at least there.
But overall, I think that replacing current flags with -Ofast would be the best choice, since it should not only make the Conda packages usable, but also make direct C faster for most (if not all) users.

For whatever reason, there is a small variation from the previous timings. Repeated tonight:
=== Forward Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
-----------------------------------------------------------------------------
-O2 -march=native   3.5       27.6       203.7      1546.2
-O2                  3.4       27.0      199.0       1532.6
-Ofast               2.6       16.2       111.8        852.5

=== Inverse Abel transform ====================================

Method              n = 257     n = 513    n = 1025    n = 2047
-----------------------------------------------------------------------------
-O2 -march=native  3.9        28.9       221.1      1583.4
-O2                3.8        28.5       213.7      1563.0
-Ofast             2.9        17.2       123.5       884.3

As you state not much difference between -O2 -march=native, and -O2.

-Ofast would be the best choice

I agree. Do it!

Yeah, even if -0fast was slower, it would still be worth it. We should prioritize making direct-C work easily on all systems. If users really want a fast reliable transform, they will likely be looking elsewhere anyway. Thanks for getting this sorted out!

OK, I've merged #331, which hopefully should solve the problem as soon as a new version is released. Let's keep this issue open until then as a reminder to confirm that new conda packages work properly.

We have finally released a new version, v0.8.5, and the conda package for it are now available.
@bwohlberg, @oobie2142, @stggh, could you please try it and report whether the problem is solved?

The new conda package seems to resolve the original problem; I can now import the abel module without any error.
There is another problem, though, in the failure of test test_dasch_deconvolution_array_sources:
$ pytest --pyargs abel
============================= test session starts ==============================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.10.0, pluggy-1.0.0

...
tests/test_dasch_methods.py ..F..                                        [ 20%]

...
=================================== FAILURES ===================================
____________________ test_dasch_deconvolution_array_sources ____________________

    def test_dasch_deconvolution_array_sources():
        im = abel.tools.analytical.SampleImage(101).image
        q = abel.tools.symmetry.get_image_quadrants(im)[0]

        # clean up any old deconvolution array files
        fn = 'three_point_basis_{}.npy'.format(q.shape[0])
        if os.path.exists(fn):
            os.system('rm {}'.format(fn))

        gb = abel.dasch.three_point_transform(q)
>       np.testing.assert_equal(abel.dasch._source, 'generated')
E       AssertionError:
E       Items are not equal:
E        ACTUAL: 'file'
E        DESIRED: 'generated'

Thanks, @bwohlberg.
I guess that this new test error is caused by changing the default location of basis-set files. I can reproduce it locally without conda. So could you please create a separate issue about it, so that we can discuss it there and close this one about conda packages?

Done: see #335.

I assume, the problem is now solved for everyone... If not, please reopen.


setting with object notation sets erroneous date

> var date = moment({year:1998,month:7,day:29})
undefined
> date.calendar()
'08/29/1998'

I realize now that this probably means month: in an object is 0 indexed; I don't think this is very obvious in the documentation.

All of moment is js Date compatible in terms of months -- all methods work with 0 indexed months, except parsing and formatting (human readable).


Player falls through the world if they connect to the server already in spectator mode

Found this bug while fixing issue #1980. I already have a fix implemented, just waiting to make a pull request for this issue after I've fixed #1980.

Yeah, I've had this happen to me a few times, and only switching to creative mode and then /spawn'ing helped.


create a .yml for up-for-grabs (as a PR)

http://up-for-grabs.net/
https://github.com/up-for-grabs/up-for-grabs.net#add-a-project
The link value would currently be https://github.com/LightTable/LightTable/labels/beginner.
BTW Id recommend to rename the beginner to good first issue.
cf https://help.github.com/articles/helping-new-contributors-find-your-project-with-labels/

@sbauer322 could you add the beginner label?

The label has been added to this issue, and the beginner label has been renamed good first issue to better align with Github's recommendations.

#2369


[FancyZones] Remove resolution from "device-id"

Summary of the Pull Request
What is this about:
Removed monitor resolution from the device id.
Skip default layouts on applied-layouts.json saving.
Note:  in some cases wrong layout could be applied.
Since we remove a part of the id, some ids in the applied-layouts.json could become identical. This could lead to the layout replacement with the other one, since we can't identify which layout was applied last. That's the reason why I decided to skip default layouts (anyway they'll be applied if needed). But if there are different custom or template layouts, it could be tricky to decide which one should be applied.
"applied-layouts": [
        {
            "device-id": "some-monitor-id_3000_2000_{00000000-0000-0000-0000-000000000000}",
            "applied-layout": {
                "uuid": "{C6D80181-D95A-476E-9A57-C8E2A9D3713A}",
                "type": "custom",
                "show-spacing": false,
                "spacing": 0,
                "zone-count": 3,
                "sensitivity-radius": 20
            }
        },
        {
            "device-id": "some-monitor-id_1920_1080_{00000000-0000-0000-0000-000000000000}",
            "applied-layout": {
                "uuid": "{A20AE166-F6AD-4202-8271-165BE2B26537}",
                "type": "custom",
                "show-spacing": false,
                "spacing": 0,
                "zone-count": 2,
                "sensitivity-radius": 20
            }
        }
    ]

->
"applied-layouts": [
        {
            "device": {
                "monitor": "some-monitor-id",
                "virtual-desktop": "{00000000-0000-0000-0000-000000000000}"
            },
            "applied-layout": {
                "uuid": "{C6D80181-D95A-476E-9A57-C8E2A9D3713A}",
                "type": "custom",
                "show-spacing": false,
                "spacing": 0,
                "zone-count": 3,
                "sensitivity-radius": 20
            }
        }
    ]

What is included in the PR:
How does someone test / validate:

Start FancyZones verify layout is correct (reset case described above)
Change layout (with shortcut or via Editor)
Verify the right layout is applied
Restart FancyZones, verify the layout is correct

Quality Checklist

[x] Linked issue: #14261
[ ] Communication: I've discussed this with core contributors in the issue.
[x] Tests: Added/updated and all pass
[ ] Installer: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated
[ ] Binaries: Any new files are added to WXS / YML

[ ] No new binaries
[ ] YML for signing for new binaries
[ ] WXS for installer for new binaries

Contributor License Agreement (CLA)
A CLA must be signed. If not, go over here and sign the CLA.

So we never cleaned up applied-layouts? That's why now we have a problem with these potential duplicate devices ? Did I understand that correctly ?

So we never cleaned up applied-layouts? That's why now we have a problem with these potential duplicate devices ? Did I understand that correctly ?

The problem with duplicating could be if the resolution was changed for any reason (RDP window resize, scaling change, etc.). So we have different ids with the same monitor name and virtual desktop id, but different resolutions. And since we remove resolution, those become identical.


Add NotAuthenticatedComponent to MsalAuthenticationTemplate.tsx

Background:
I wanted to use MsalAuthenticationTemplate for react on a website, where I have a logout option.
Use case:
When all content is wrapped in this component, and a user uses the logout option (having set msal.logout({ onRedirectNavigate: () => false })), no content is visible.
This change will add a property which is visible when there's no error, isAuthenticated is falsy, and context.inProgress is InteractionStatus.None.
The reason why this needs to be done at this place is that it's not possible to know whether an error occurred outside this component - all other props can be checked also outside this component.

Thanks for your contribution. Unfortunately we cannot accept this as it does not make sense in the context of MsalAuthenticationTemplate. This component is intended to ensure that a user is signed in and will automatically invoke sign-in if the user is not. In other words the states are either: signed in or in the process of signing in.
If you want a component that renders content when a user is NOT signed in I recommend you use the UnauthenticatedTemplate component.

In other words the states are either: signed in or in the process of signing in.

@tnorling I disagree with this statement - there are at least four states:

User is signed in
User is in the process of signing in
User received an error during signing in
User is signed out

Here's a link to a repo where this behavior can be tested: https://codesandbox.io/p/sandbox/react-typescript-forked-659xwk?workspaceId=a3ee7606-63a0-4cb6-a78b-0a5d8099b662
Steps to reproduce:

Open src/auth.msal.ts and insert your client-id.
Log in using the popup.
Click on the logout-button.

You'll now see an empty screen. You've now reached the state of a user who signed out.


Roll engine 101b27da7a1b..b50e3491cc62 (2 commits)

https://github.com/flutter/engine/compare/101b27da7a1b...b50e3491cc62
git log 101b27da7a1b..b50e3491cc62 --no-merges --oneline
b50e3491c Roll src/third_party/skia cb6b4bde0c2e..21a7be0741ac (3 commits) (flutter/engine#7216)
826f34264 [Fuchsia] Depend on libtrace when that is what's really meant (flutter/engine#7214)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

@liyuqian : New roll has been created by https://autoroll.skia.org/r/flutter-engine-flutter-autoroll

Commit queue failed; closing this roll.


Fix for fake keypress bypass

I'm pretty sure this fixes this (https://twitter.com/securifybv/status/594189441633296385) but I still haven't figured out how to test edits for an extension.

Thanks! I should add a section in chrome/README.md on how to compile and then locally run the extension.
Is this the bypass that results in sender.tab being undefined? http://pastebin.com/Bx0RsuSu

I also have a commit that discards keyboardEvents with a charcode of 0, which should help. There's a 7-year-old bug (?) that sets the charcode to 0 for programmatically generated keyboardEvents: https://www.google.com/url?sa=D&q=https%3A%2F%2Fbugs.webkit.org%2Fshow_bug.cgi%3Fid%3D16735

Just found out how key presses are failing. charCode is always zero.

Is this the bypass that results in sender.tab being undefined?
Nope, I thought it was but sender.tab is defined.

I also have a commit that discards keyboardEvents with a charcode of 0, which should help. There's a 7-year-old bug (?) that sets the charcode to 0 for programmatically generated keyboardEvents: https://bugs.webkit.org/show_bug.cgi?id=16735
Yeap that's why. So if that ever gets fixed then this will be broken.

OK I quadruple checked this. It should work now. I'm sorry about submitting a pull request that was untested.

No worries about not testing it the first time around. I'm happy to get pull requests in any shape.
I just committed the code I wrote yesterday. I went ahead and ignored charcode == 0 values early on: https://github.com/google/password-alert/blob/master/chrome/content_script.js#L528

Yeah that's way better.


demumble omits characters after valid ms symbol

$ echo asdf?x@@3HAasdf | ./demumble
asdfint x

Should print asdfintxasdf instead.
In Itanium as far as I know there's no way to hit this since the additional characters are always (?) interpreted to be part of the symbol.

llvm::microsoftDemangle() doesn't return how many characters it has consumed, which is probably step 1 here.

Patch for step 1: https://reviews.llvm.org/D80173


Netlify deploy fails

I'm running into an issue and can't seem to find how to fix it. Anytime I try to deploy to Netlify using your plugin, it fails because jekyll-netlify-headers is incompatible with my ruby version.
3:25:46 PM: jekyll-netlify-headers-0.1.3 requires ruby version >= 2.4, which is incompatible 3:25:46 PM: with the current version, ruby 2.3.6p384
Any guidance on how to make sure these are compatible with each other?

Cancel that — I added a .ruby-version file and it works now.

@interactivematt Glad you got things working and thanks for using the gem!


Abode Camera Streaming configuration best RPi4 performance

I've installed Abode Camera Streaming on a Raspberry Pi 4, per the instructions @ https://github.com/tradel/hassio-addons/tree/main/abode2rtc. I've verified that the components are all operational, and have been able to pull up streams via links access through the http://homeassistant.local:8123/hassio/ingress/bc47d723_abode2rtc UI.
Where I'm having trouble is getting the HASS lovelace card to work reliably. At one point I had all 5 of my cameras working individually, so I built a dashboard for all of them. Here's the code I've used:
  - title: Cams
    path: cams
    icon: mdi:cctv
    badges: []
    cards:
      - type: custom:webrtc-camera
        streams:
          - type: webrtc
            url: front_porch
        title: Front Porch
      - type: custom:webrtc-camera
        streams:
          - type: webrtc
            url: back_porch
        title: Back Porch
      - type: custom:webrtc-camera
        streams:
          - type: webrtc
            url: garage
        title: Garage
      - type: custom:webrtc-camera
        streams:
          - type: webrtc
            url: north_side
        title: North Side
      - type: custom:webrtc-camera
        streams:
          - type: webrtc
            url: driveway
        title: Driveway
When I first navigate to the dashboard on a laptop with Firefox, the images show endless spinners:

If I navigate away from the dashboard and come back, some of the streams will load while others continue spinning:

Checking the add-on status, it shows that the streams are active:

One other clue, when I load that dashboard using the HA companion app on my iPad Mini, the videos all load:

Is there something I can do in the static configuration to make the dashboard more reliable?

I think it's very browser dependent. I built a similar dashboard with six cameras. On a desktop browser, I usually get four, sometimes five cameras to load. On iOS, they usually all load after a delay.
I don't think this is a problem with my code or go2rtc as much as it is with browser implementations of WebRTC. But I'd love to hear about any ways to improve reliability.


chore: Update GitHub actions to use Node 20

Current dependencies on/for this PR:

main

PR #76   👈

This stack of pull requests is managed by Graphite.


no ending of task

Hi.
Tried to create font-face from font 28kb weight, ttf format (also tried another fonts).
this is task in gulpfile.js
gulp.task(
 'fonts', ['del-fonts'], function(){
   gulp.src(path.src.fonts)
      .pipe(plumber({errorHandler: onError})) // plumber
       .pipe(gulpfontgen({
      /*collate: true,
           css_fontpath: path.src.fontface,*/
      dest: path.build.fonts
    }))
   .pipe(gulp.dest(path.build.fonts))
    .pipe(browserSync.reload({ stream:true }))
   .pipe(notify({message: 'FONTS task complete', onLast: true}));
});
task start to run and no finishing until I break it, tried to wait about 30mins.
[11:23:14] Starting 'del-fonts'...
[11:23:14] Finished 'del-fonts' after 8.47 ms
[11:23:14] Starting 'fonts'...
[11:23:14] Finished 'fonts' after 19 ms
no fonts created, no errors in console
CPU is Intel I5 so it seems to be fast enough.

Hi @alexanderkatkov, fixed in https://github.com/agentk/fontfacegen/issues/7. Pushing out npm updates now.


feat: implement Delete func of OSS ArtifactDriver. Fixes #9349

Fixes #9349
Motivation

https://github.com/argoproj/argo-workflows/blob/3b4551b6f33bd22519cc98e2d8c157a92720b5d7/workflow/artifacts/oss/oss.go#L226
OSS Artifact Driver still NOT support Artifact Garbage Collection as Delete Func has not been implemented.
Modifications

Implemented Delete Func of OSS Artifact Driver .
Verification

Plz check with workflow like
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-gc-
spec:
  entrypoint: main
  artifactGC:
    strategy: OnWorkflowDeletion # the overall strategy, which can be overridden
    podMetadata:
      annotations:
         kubernetes.io/resource-type: eci
  templates:
    - name: main
      container:
        image: argoproj/argosay:v2
        command:
          - sh
          - -c
        args:
          - |
            echo "hello world" > /tmp/on-completion.txt
            echo "hello world" > /tmp/on-deletion.txt
      outputs:
        artifacts:
          - name: on-completion
            path: /tmp/on-completion.txt
            oss:
              endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
              bucket: my-argo-workflow
              key: on-completion.txt
              accessKeySecret:
                name: my-argo-workflow-credentials
                key: accessKey
              secretKeySecret:
                name: my-argo-workflow-credentials
                key: secretKey
            artifactGC:
              strategy: OnWorkflowCompletion # overriding the default strategy for this artifact
          - name: on-deletion
            path: /tmp/on-deletion.txt
            oss:
              endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
              bucket: my-argo-workflow
              key: on-deletion.txt
              accessKeySecret:
                name: my-argo-workflow-credentials
                key: accessKey
              secretKeySecret:
                name: my-argo-workflow-credentials
                key: secretKey

I tested it in my envirment. When workflow completed  the on-completion.txt  in oss was garbage collected.  when workflow deleted  the on-deletion.txt  in oss was garbage collected.  So LGTM.


How to split the dataset into Sunny/Rainy/Day/Night scene?

I  would like to konw how to analysis of different lighting and weather conditions in your work. Thanks a lot!

For nuscenes dataset, you could get the description of every sample from nuscenes api with sample data token and just filter out day/night/rainn afterward.

I have done with this step, where should I do scene filtering next in this code

For nuscenes dataset, you could get the description of every sample from nuscenes api with sample data token and just filter out day/night/rain afterward.

Thank you, I have completed this step to get the name of the night scene. What should I do to filter the scene next in this repo?

For nuscenes dataset, you could get the description of every sample from nuscenes api with sample data token and just filter out day/night/rain afterward.

The problem has been solved, Thank you very much!


Fix : Error [test_all_metadata_is_filled ]

PR #763
https://github.com/embeddings-benchmark/mteb/pull/669#issuecomment-2117932543

Not sure what is meant by this screenshot. I get that you refer to the comment in 669 but I don't see how the PR addresses it. Or at least it addresses a variety of other things as well. I have closed it for now, but do feel free to open a new PR.

hello @KennethEnevoldsen
I was just testing the error that appeared for me when I ran make test or make pr. It was fixed when you solved the issue in  PR on https://github.com/embeddings-benchmark/mteb/pull/781.


[HIGH PRIORITY] Start Hosting the Website on something other than GitHub pages

[ ] Figure out how to get something like Google Firebase working.
[ ] Ideally, we should delete the gh-pages branch and no longer create or push to it at all. Instead the build is just sent directly to the firebase api from the GitHub Action code. That way secret API keys etc. stay secret

Hosting a Static Site on Firebase
Firebase Hosting does not require a specific build format. It supports hosting static assets such as HTML, CSS, JavaScript, images, and other files without any specific build process. This means you can deploy your static website to Firebase Hosting directly without any additional compilation or bundling steps.
If your website consists of simple HTML, CSS, and JavaScript files, you can deploy it to Firebase Hosting without any modifications. Firebase Hosting will serve these files directly to your users, making it a convenient option for hosting static websites.
Here's a general overview of how you can deploy a simple static website to Firebase Hosting:

Set up Firebase Project: Create a Firebase project in the Firebase console if you haven't already done so.

Install Firebase CLI: Install the Firebase CLI on your local machine using npm or yarn:
npm install -g firebase-tools

Authenticate with Firebase using the CLI:
firebase login

Initialize Firebase Project: Navigate to your project directory and initialize Firebase for your project:
firebase init

Follow the prompts to select Firebase Hosting and your Firebase project. You can choose to use an existing Firebase project or create a new one.

Deploy to Firebase Hosting: Once your project is initialized, you can deploy your static website to Firebase Hosting using the Firebase CLI:
firebase deploy

Firebase Hosting will upload your static assets to the Firebase servers and provide you with a hosting URL where your website will be accessible.
Since Firebase Hosting supports SSL/TLS encryption and provides global CDN distribution, your website will be served securely and efficiently to users worldwide.
In summary, you can deploy a simple static website consisting of HTML, CSS, and JavaScript files directly to Firebase Hosting without any specific build process. Firebase Hosting is a flexible and convenient option for hosting static content and provides features such as SSL/TLS encryption, CDN distribution, and custom domain support out of the box.

After above one time steps are added, should only need to add this to github actions:
# Deploy the `deploy` directory to firebase
- name: Firebase Deploy
  run: firebase deploy --token ${{ secrets.FIREBASE_TOKEN }} --only hosting:deploy

TODO:
Document in the repo in a readme my "stack":

Squarespace for my domain
Firebase to host the static website
Google Analytics for analysis
js, html, css for custom code
GitHub Actions to automate deployment upon merge into main

TODO:

Give DNS propogation some time to make sure it has updated. Then go to firebase and make sure the changes have taken effect such that shanebonkowski.com is now on firebase. Confirm that page is actually up from firebase not github pages


Allow popups warning!!

People's browsers may very well block the oauth popup for google. We need to let them know that they should allow popups!

I added this


Add pharos-cluster kubeconfig -command

Alternative to #499
Fixes #438
$ pharos-cluster kubeconfig # dumps config from master host defined in cluster.yml to stdout
$ pharos-cluster kubeconfig -n foofoo # renames the server in the config to foofoo
$ pharos-cluster kubeconfig -C foofoo # renames the context in the config to foofoo

Not sure if this is really direct alternative to #499 ... because user should be able to opt-out from automatic kubeconfig in ~/.pharos/xyz. We could stop saving that file but we still need a kubeconfig for accessing the k8s api (in-memory?).

It could always skip saving it and instruct to use pharos-cluster kubeconfig. It's already used from in-memory (cluster_context['previous_config']) after the ConfigureClient phase.

It could always skip saving it and instruct to use pharos-cluster kubeconfig. It's already used from in-memory (cluster_context['previous_config']) after the ConfigureClient phase.

Always skip saving makes sense to me. @jnummelin @nevalla @SpComb wdyt?

Also possible: some kind of
$ pharos-cluster kubectl -c cluster.yml  -- run hello-kube --image=k8s.gcr.io/echoserver:1.10 --port=8080

that would use KubeConfigCommand to create a tempfile, set KUBECONFIG, run kubectl and remove the tempfile.
Or
$ pharos-cluster kubeconfig --env
KUBECONFIG=/tmp/pharos123345423vdfvfdvfdfvd.admin.conf
$ eval $(pharos-cluster kubeconfig --env)

Or
$ pharos-cluster kubeconfig --temp-file
/tmp/pharos123345423vdfvfdvfdfvd.admin.conf
$ export KUBECONFIG=$(pharos-cluster kubeconfig --temp-file)

I don't understand this PR, it doesn't write the ~/.pharos/* kubeconfigs anymore, but the Pharos::Kube still needs those?
==> Configure kube client @ terom-pharos-master
No such file or directory @ rb_sysopen - /root/.pharos/master.terom-pharos-dev.kontena.works
/usr/local/lib/ruby/2.4.0/psych.rb:472:in `initialize'
/usr/local/lib/ruby/2.4.0/psych.rb:472:in `open'
/usr/local/lib/ruby/2.4.0/psych.rb:472:in `load_file'
/usr/local/bundle/gems/k8s-client-0.3.1/lib/k8s/config.rb:81:in `load_file'
/app/lib/pharos/kube.rb:58:in `host_config'
/app/lib/pharos/kube.rb:45:in `client'
/app/lib/pharos/phase.rb:83:in `kube_client'
/app/lib/pharos/phases/configure_client.rb:19:in `client_prefetch'
/app/lib/pharos/phases/configure_client.rb:9:in `call'
/app/lib/pharos/phase_manager.rb:92:in `block in apply'
/app/lib/pharos/phase_manager.rb:53:in `yield_phase_with_retry'
/app/lib/pharos/phase_manager.rb:44:in `block in run_serial'
/app/lib/pharos/phase_manager.rb:43:in `map'
/app/lib/pharos/phase_manager.rb:43:in `run_serial'
/app/lib/pharos/phase_manager.rb:73:in `run'
/app/lib/pharos/phase_manager.rb:89:in `apply'
/app/lib/pharos/cluster_manager.rb:119:in `apply_phase'
/app/lib/pharos/cluster_manager.rb:93:in `apply_phases'
/app/lib/pharos/up_command.rb:104:in `configure'
/app/lib/pharos/up_command.rb:45:in `block in execute'
/app/lib/pharos/up_command.rb:44:in `chdir'
/app/lib/pharos/up_command.rb:44:in `execute'
/usr/local/bundle/gems/clamp-1.2.1/lib/clamp/command.rb:63:in `run'
/usr/local/bundle/gems/clamp-1.2.1/lib/clamp/subcommand/execution.rb:11:in `execute'
/usr/local/bundle/gems/clamp-1.2.1/lib/clamp/command.rb:63:in `run'
/usr/local/bundle/gems/clamp-1.2.1/lib/clamp/command.rb:132:in `run'
/app/lib/pharos/root_command.rb:18:in `run'
bin/pharos-cluster:12:in `<main>'

Always skip saving makes sense to me.

Users have been depending that the configs are saved, so maybe not a good idea to just drop it "silently". Maybe put it behind a flag that would still default to saving for a while, or something.

Users have been depending that the configs are saved, so maybe not a good idea to just drop it "silently".

We need to document this properly in the release notes. If someone is using locally saved config in automation it will most likely break things... but it should be easy to fix by adding an additional pharos-cluster kubeconfig command.


Install MongoDB with composer build script locally & on GCP

Removes the need to build the extension locally from source since the GCP docker build script already supports MongoDB.

Pull Request Test Coverage Report for Build 488

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.343%

Totals

Change from base Build 478:
0.0%

Covered Lines:
1009

Relevant Lines:
1026

💛  - Coveralls


[$500] iOS - IOU - Empty space below IOU preview in 1:1 DM after paying elsewhere

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: v1.3.91-6
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause-Internal Team
Slack conversation: @
Action Performed:

User A requests money from User B.
User B opens 1:1 DM with User A.
User B taps Pay elsewhere on the IOU preview.

Expected Result:
There will be no empty space below IOU preview in 1:1 DM after paying elsewhere.
Actual Result:
There is empty space below IOU preview in 1:1 DM after paying elsewhere.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[x] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native
https://github.com/Expensify/App/assets/115492554/726605a4-ac48-418a-9b83-6758706ab84e

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0164ad99e31ac32a85
Upwork Job ID:<PHONE_NUMBER>237944320
Last Price Increase: 2023-10-26

Not reproducible in production
https://github.com/Expensify/App/assets/115492554/aa884a56-75ce-49a9-9204-aa198565990f

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open StagingDeployCash deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

Identify the pull request that introduced this issue and revert it.
Find someone who can quickly fix the issue.
Fix the issue yourself.

Just to add some context: the same happens when adding an attachment (the chat doesn't scroll to the end), and when removing an attachment – the space at the bottom remains.

This stuff should be fixed with that changes
As a temporary solution, we can comment out fetching from the bottom, which causes that problem.

I tested this on this branch https://github.com/Expensify/App/pull/30409 and can't reproduce the issue, so must've been caused by it.

Asked here how to proceed https://expensify.slack.com/archives/C035J5C9FAP/p1698335858869399?thread_ts=1695998111.378719&cid=C035J5C9FAP

Investigating a fix

Confirmed this will be fixed by https://github.com/Expensify/App/pull/30461

@narefyev91 does not require payment (Contractor)

all set here!


Investigate means to leverage ORT for export control checks (ECC)

Usage of cryptography might be liable to export control checks. Maybe ORT (or more specifically, its Analyzer) could be leveraged to semi-automate those checks on dependencies with the help of a tool like CogniCrypt that is able to detect the use of cryptographic APIs (at least in some programming languages).

Another tool that might come into use here is https://github.com/Microsoft/ApplicationInspector.

Also see @tsteenbe's comment from here, and https://www.openchainproject.org/news/2024/03/12/webinar-scanoss-export-control.

Closed as part of backlog grooming. Feel free to comment if you would like to contribute to this.


OOM out of memory Mask RCNN

Click to expand!
Issue Type
Support
Source
source
Tensorflow Version
tf 1.15
Custom Code
Yes
OS Platform and Distribution
Windows 10
Mobile device
No response
Python version
3.7
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
I want to do a train for mask rcnn using Tensorflow 1.15. I'm using a GTX 1650 ti. I am using cuda 10 and cudnn 7.6.0 versions. I am getting an OOM error.

Code of my config file
class DamageConfig(Config):
    # define the name of the configuration
    NAME = "damage"

    # number of classes (background + damge classes)
    NUM_CLASSES = 1 + 4

    # number of training steps per epoch
    STEPS_PER_EPOCH = 250
    # learning rate and momentum

    # regularization penalty
    BATCH_SIZE = 8

    # validation steps
    VALIDATION_STEPS = 25

    # RPN Acnhor scales and ratios to find ROI

# prepare train dataset.
train_set = DamageDataset()
# change the dataset
train_set.load_dataset("C:/Users/hasan/maskcuda/dataset", "train")
train_set.prepare()

# prepare validation/test dataset
test_set = DamageDataset()
test_set.load_dataset("C:/Users/hasan/maskcuda/dataset", "val")
test_set.prepare()

# load damage config
config = DamageConfig()

# define the model
model = MaskRCNN(mode='training', model_dir='./', config=config)

# load weights mscoco model weights
weights_path = 'C:/Users/hasan/maskcuda/Mask_RCNN/mask_rcnn_coco.h5'

config = tensorflow.ConfigProto()
config.gpu_options.allow_growth = True
sess = tensorflow.Session(config=config)

# load the model weights
model.load_weights(weights_path,
                   by_name=True,
                   exclude=["mrcnn_class_logits", "mrcnn_bbox_fc","mrcnn_bbox", "mrcnn_mask"])

# start the training of model
# you can change epochs and layers (head or all)
model.train(train_set,
            test_set,
            learning_rate=config.LEARNING_RATE,
            epochs=15,
            layers='heads')

Standalone code to reproduce the issue
2022-12-11 17:49:33.553181: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 14829824 totalling 14.14MiB
2022-12-11 17:49:33.553215: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 26214400 totalling 25.00MiB
2022-12-11 17:49:33.553249: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 15 Chunks of size 51380224 totalling 735.00MiB
2022-12-11 17:49:33.553283: I tensorflow/core/common_runtime/bfc_allocator.cc:917] 1 Chunks of size 186075648 totalling 177.46MiB
2022-12-11 17:49:33.553317: I tensorflow/core/common_runtime/bfc_allocator.cc:921] Sum Total of in-use chunks: 2.81GiB
2022-12-11 17:49:33.553348: I tensorflow/core/common_runtime/bfc_allocator.cc:923] total_region_allocated_bytes_:<PHONE_NUMBER> memory_limit_:<PHONE_NUMBER> available bytes: 103 curr_region_allocation_bytes_:<PHONE_NUMBER>
2022-12-11 17:49:33.553391: I tensorflow/core/common_runtime/bfc_allocator.cc:929] Stats:
Limit:<PHONE_NUMBER>
InUse:<PHONE_NUMBER>
MaxInUse:<PHONE_NUMBER>
NumAllocs:                   22711
MaxAllocSize:<PHONE_NUMBER>

ResourceExhaustedError: 2 root error(s) found.
  (0) Resource exhausted: OOM when allocating tensor with shape[2,3,1030,1030] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node conv1_6/convolution}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[proposal_targets_6/strided_slice_17/_28717]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

Relevant log output
No response

It looks like you are running out of memory when you are trying to allocate a tensor with shape [2,3,1030,1030] and type float. To solve this issue, try reducing the size of the tensor or using a smaller batch size. You may also want to reduce the number of layers in your model or try to use a lighter weight model - clerkie (https://clerkie.co/)

@hasanhut !
Sorry for the late reply. 1.x  versions are not supported any more.
I agree on @krrishdholakia's point on large shape tensors which is causing OOM.
For 2.x models, You can use tf.data to do different pre-processing operations like batching, slicing, random shuffling . It helps set up a pipeline for the model .
Plus, It solves memory leaks issues which happens during model training.
You can migrate to 2.x model format using migration document or need to re-save the 1.x model using saved_model  and tf.io.gflile.Gfile(For loading the old frozen graph file)
Thank you!


data.dmlc.ml is down causing the example script (example/image-classification/train_mnist.py) to fail

should be fixed now

I am able to download the zip file now but the zip file structure has changed.
I dont think they were inside a mnist directory before.
unzip -l mnist.zip
Archive:  mnist.zip
Length      Date    Time    Name

    0  09-29-2016 04:41   mnist/

7840016  07-21-2000 18:19   mnist/t10k-images-idx3-ubyte
10008  07-21-2000 18:20   mnist/t10k-labels-idx1-ubyte
47040016  07-21-2000 18:20   mnist/train-images-idx3-ubyte
60008  07-21-2000 18:20   mnist/train-labels-idx1-ubyte

54950048                     5 files
This breaks the train_mnist.py script.

thanks, i'll fix it in a minute

should be fixed now
On Thu, Sep 29, 2016 at 12:50 PM, Anirudh Subramanian <
<EMAIL_ADDRESS>wrote:

I am able to download the zip file now but the zip file structure has
changed.
I dont think they were inside a mnist directory before.
unzip -l mnist.zip
Archive: mnist.zip
Length Date Time Name
0  09-29-2016 04:41   mnist/

7840016 07-21-2000 18:19 mnist/t10k-images-idx3-ubyte
10008 07-21-2000 18:20 mnist/t10k-labels-idx1-ubyte
47040016 07-21-2000 18:20 mnist/train-images-idx3-ubyte
60008 07-21-2000 18:20 mnist/train-labels-idx1-ubyte
54950048 5 files
This breaks the train_mnist.py script.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dmlc/mxnet/issues/3398#issuecomment-250572258, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAZv4TvegL_aBOD4qD9wENZwJvYFxWXsks5qvBadgaJpZM4KJeB0
.

@mli thank you !


Poor DNS cache performance with large number of unique domains

Description:
We are using Envoy as a dynamic forward proxy and are looking to connect upstream to a large number of hosts at unique domains (>> 100k).
We ran across performance issues in the implementation of the DNS cache when scaling to >40k hosts.
The primary issue is that both the DNS cache implementation and the dynamic forward proxy implementation each create a new copy of the entire cache on update. This is a reasonable implementation given Envoy's threading model and if you know the map is small, but it runs into scalability issues as the map grows.
With default configuration, the DNS cache will hit circuit-breaker limits before hitting this performance issue, so this issue won't be hit in overload conditions where you expect only a small number of hosts. The trouble occurs when you want to configure Envoy to handle a much larger number of host names.
This performance issue is probably not surprising, since I would expect the motivating use case was for a much smaller number of hosts. Looking at the dynamic forward proxy documentation, it's fairly clear that there are some features that would not be feasible for a large number of hosts (e.g. observability in the admin output).
I'd like to understand if the intention is for the DNS cache to be used as a general-purpose cache that will be able to scale, or if it's primarily intended to only be paired with the dynamic forward proxy cluster.
Repro steps:

Configure Envoy to use a dynamic forward proxy.
Set the max_hosts and host_ttl in the dns_cache_config to something large.
Disable the DNS cache circuit breakers (envoy.reloadable_features.enable_dns_cache_circuit_breakers)
Run traffic against Envoy destined to ~50k unique domains or more

I also created a Google Benchmark test to exercise the DNS cache implementation in isolation and observed a cache insertion complexity of greater than O(N^2) (N=number of elements in the map).

In general I wouldn't assume the implementation has to be the way that it is. Definitely open to perf improvements. Do you want to suggest some concrete changes?

The primary issue is that both the DNS cache implementation and the dynamic forward proxy implementation each create a new copy of the entire cache on update. This is a reasonable implementation given Envoy's threading model and if you know the map is small, but it runs into scalability issues as the map grows.

I don't recall the history of why I implemented it like this, but I'm guessing with some thought it could be merged into a single structure and accessed via locking.

I also created a Google Benchmark test to exercise the DNS cache implementation in isolation and observed a cache insertion complexity of greater than O(N^2) (N=number of elements in the map).

This part I find surprising. I believe the implementation is using maps everywhere? Do you have more details on this?

In general I wouldn't assume the implementation has to be the way that it is. Definitely open to perf improvements. Do you want to suggest some concrete changes?

In our use case, we expect frequent cache updates, which makes maintaining a global map with locking less attractive. For us, it seems to make sense to have each worker thread maintain their own local cache and receive updates from the main thread. This adds a bit of complexity since each worker would have to manage its own cache expiry (and there would be some memory overhead for the additional maps in each thread), but I think that would provide a noticeable improvement for large maps.
Here's a more concrete flow:

Worker thread DNS cache has cache miss, sends request to worker thread dispatcher
Main thread checks main thread map (posts to worker thread with DNS result on cache hit)
On cache miss, main thread resolves using DNS resolver, updates main thread map
Main thread posts DNS resolution result to the requesting worker thread
Worker thread receives resolution result, updates local cache

There might be some other opportunities for improvement, but I think we'll need to solve the immediate copying problem before other issues become important.

I don't recall the history of why I implemented it like this, but I'm guessing with some thought it could be merged into a single structure and accessed via locking.

It might make sense to keep the dynamic forward proxy host map separate from the DNS cache, but use a lock for access to the dfp host map. The two maps currently use two different value types, so it might be easier to maintain them separately (particularly if we're okay with posting updates to worker threads). It's also useful to be able to have the DNS cache available but not use the dynamic forward proxy cluster.

I also created a Google Benchmark test to exercise the DNS cache implementation in isolation and observed a cache insertion complexity of greater than O(N^2) (N=number of elements in the map).

This part I find surprising. I believe the implementation is using maps everywhere? Do you have more details on this?

I pushed the benchmark test here.
The implementation is using maps everywhere, but because the implementation creates a new map and populates it on each update, the insertion complexity of the cache is much worse than the complexity of a hash map insertion.
The implementation does the following on insert:

Creates a new empty flat_hash_map<string, DnsHostInfoSharedPtr>
Iterates over the existing flat_hash_map<string, PrimaryHostInfoPtr> map (an O(capacity) operation)
If an entry has been resolved, inserts the DnsHostInfoSharedPtr entry into the new map (if the load factor is exceeded, this will lead to a reallocation and copy of the existing entries)

Given that, I think it makes sense that the performance starts to degrade as the number of entries rise.

In our use case, we expect frequent cache updates, which makes maintaining a global map with locking less attractive. For us, it seems to make sense to have each worker thread maintain their own local cache and receive updates from the main thread.

The downside of this is you will multiply your memory usage by the number of workers which for a large cache seems not great. (I suppose you could evict things faster from the workers but it still seems sub-optimal).
In paging back in this code, I think I was mostly trying to avoid locking, but I'm not sure that is the best design choice given your use case. I think if you had a RW lock on the map, and then more granular locks for each host entry (allowing for address updates that are independent), things would scale quite well.
I would probably advocate just using locking and removing all of the copies everywhere in both the DNS cache as well as the forward proxy host map. WDYT?

Seems like that's a good minimal change -- if we find there's a lot of lock contention in our use case, we can always introduce a thread-local cache on the filter side of things.
This is an important change for us, so I'm going to take a preliminary pass at what you suggested this week. Feel free to assign this issue to me.

This is an important change for us, so I'm going to take a preliminary pass at what you suggested this week. Feel free to assign this issue to me.

OK sounds good. I can potentially work on it also if you run into issues but will assign to you for now.


Feat onshot

Support Komga's oneshot series

It can work with this https://github.com/Snd-R/komf-userscript/pull/14

Thanks, although logic feels too komga specific, I suppose it's fine to keep it this way for now
Komga also returns error if you try to upload series cover for oneshot. Might need to add oneshot flag to series too to handle that case

Thanks, although logic feels too komga specific, I suppose it's fine to keep it this way for now Komga also returns error if you try to upload series cover for oneshot. Might need to add oneshot flag to series too to handle that case

Yes, you're right. I fixed this error with https://github.com/Snd-R/komf/pull/131/commits/8f33a5da63073bf6fda62871dd796f0bde5affaa


Updating poco and making submodule

contrib/libpoco was not deleted.

CMAKE_BUILD_TYPE was suddenly changed to Release.
CMAKE_BUILD_TYPE:STRING=Release
We have RelWithDebInfo as default.

The vendored version breaks build with the latest OS X clang:
clickhouse/libs/libcommon/src/JSON.cpp:459:52: error: call to 'format' is ambiguous
        throw JSONException("JSON: array index " + Poco::NumberFormatter::format(n) + " out of bounds.");
                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:236:21: note: candidate function
        static std::string format(bool value, BoolFormat format = FMT_TRUE_FALSE);
                           ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:413:37: note: candidate function
inline std::string NumberFormatter::format(int value)
                                    ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:453:37: note: candidate function
inline std::string NumberFormatter::format(unsigned value)
                                    ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:579:37: note: candidate function
inline std::string NumberFormatter::format(Int64 value)
                                    ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:619:37: note: candidate function
inline std::string NumberFormatter::format(UInt64 value)
                                    ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:659:37: note: candidate function
inline std::string NumberFormatter::format(float value)
                                    ^
clickhouse/contrib/poco/Foundation/include/Poco/NumberFormatter.h:683:37: note: candidate function
inline std::string NumberFormatter::format(double value)

Hope this will help: https://github.com/yandex/ClickHouse/commit/10392d54523b8eb2f8892f52220f350fb1f61096

@alexey-milovidov It affects more places where UInt64 and size_t were used interchangeably. Like:
In file included from /clickhouse/dbms/src/Parsers/IAST.cpp:4:
In file included from /clickhouse/dbms/src/IO/Operators.h:5:
/clickhouse/dbms/src/IO/ReadHelpers.h:112:5: error: no matching function for call to 'readVarUInt'
    readVarUInt(size, buf);
    ^~~~~~~~~~~
/clickhouse/dbms/src/IO/VarInt.h:78:13: note: candidate function not viable: no known conversion from 'size_t' (aka 'unsigned long') to 'UInt32 &' (aka 'unsigned int &') for 1st argument
inline void readVarUInt(UInt32 & x, ReadBuffer & istr)
            ^
/clickhouse/dbms/src/IO/VarInt.h:92:13: note: candidate function not viable: no known conversion from 'size_t' (aka 'unsigned long') to 'UInt16 &' (aka 'unsigned short &') for 1st argument
inline void readVarUInt(UInt16 & x, ReadBuffer & istr)
            ^
/clickhouse/dbms/src/IO/VarInt.h:112:13: note: candidate function not viable: no known conversion from 'size_t' (aka 'unsigned long') to 'UInt64 &' (aka 'unsigned long long &') for 1st argument
inline void readVarUInt(UInt64 & x, ReadBuffer & istr)
            ^
/clickhouse/dbms/src/IO/VarInt.h:130:13: note: candidate function not viable: no known conversion from 'size_t' (aka 'unsigned long') to 'UInt64 &' (aka 'unsigned long long &') for 1st argument
inline void readVarUInt(UInt64 & x, std::istream & istr)
            ^
/clickhouse/dbms/src/IO/VarInt.h:143:21: note: candidate function not viable: requires 3 arguments, but 2 were provided
inline const char * readVarUInt(UInt64 & x, const char * istr, size_t size)

Or conversions from Field, etc. Not sure if we should alias the type like it was before, or fix all those uses to use Poco::UInt64.

https://github.com/yandex/ClickHouse/pull/1356


Issue with macOS and OpenGL library location in python Env

Apparently, BigSur is not longer support OpenGL library or other system libraries in the standard locations
in the file system and instead uses a cache. When I run
poetry run python gui.py -p model_parameters/default.yml
I got
ImportError: ('Unable to load OpenGL library', "dlopen(OpenGL, 0x000A):
Earlier we found the way to address that uisng the suggestion from: https://github.com/PixarAnimationStudios/USD/issues/1372
The way to address that was:

Find the file in OpenGL/platform/ctypesloader.py,  in the python Env created by Poetry.

Update/Edit the line

 fullName = util.find\_library( name )
to
fullName = '/System/Library/Frameworks/OpenGL.framework/OpenGL'

From reading the issue, it was fixed properly in upstream Python in November 2020. People then made some backports from Python 3.8 and 3.9 maybe. I think if we can figure out which version of Python has the fix, we can enable that in pyproject.toml (python = "^3.8") and avoid this entirely. I'll see if I can first reproduce the problem on my macbook.

Although, I though we have already been using python >3.8. Non important at this very moment.


Add art-by-gan

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How has this been tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] local build the book
[ ] review the artifacts from local build
[ ] local build the slides if appliable
[ ] Unit test if appliable

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have applied tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

1、Our environment do not have module 'tqdm', you may add "! pip install tqdm" into ipynb file.
2、You haven't loaded the model in yet. Do you need help running it?
3、You foeget add _toc.yml file.

Refer to this one:
https://stackoverflow.com/questions/39459467/remove-a-modified-file-from-pull-request


Issue on page /notebooks/07-R.html

But why bother with randiom number seeds? Setting a particular seed can be useful when debugging programs (coming up below).
misspelling

Fixed


[v1.33.0] Update release flow parameters to use kebab-case

Proposed changes
Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request.
Fixes
https://github.com/valory-xyz/open-aea/actions/runs/4478548807/jobs/7871444735#step:11:1
Types of changes
What types of changes does your code introduce to agents-aea?
Put an x in the boxes that apply

[ ] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist
Put an x in the boxes that apply.

[ ] I have read the CONTRIBUTING doc
[ ] I am making a pull request against the develop branch (left side). Also you should start your branch off our develop.
[ ] Lint and unit tests pass locally with my changes and CI passes too
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that code coverage does not decrease.
[ ] I have added necessary documentation (if appropriate)
[ ] Any dependent changes have been merged and published in downstream modules

Codecov Report
Patch coverage has no change and project coverage change: -0.01 :warning:

Comparison is base (9f4aac0) 92.52% compared to head (77b82e8) 92.52%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #623      +/-   ##
==========================================
- Coverage   92.52%   92.52%   -0.01%
==========================================
  Files         370      370
  Lines       29512    29512
==========================================
- Hits        27307    27305       -2
- Misses       2205     2207       +2

Flag
Coverage Δ

unittests
92.52% <ø> (-0.01%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

just wondered pypa/gh-action-pypi-publish@release/v1 works with both styles of arguments names


Recover wallet from 256bit string through REST endpoint

The choice of using a 256 bit string is temporary. This will be replaced
in favour of a 24 word seed phrase which will be used to create the
seed (BIP39).
This PR was exploratory. I will merge it after #909.
will squash after approval

bors r+

Actually, can you add a changelog entry please?

bors r+


"Fix" for nInitialDriveGears

So from my experience, while I'm testing vehicles if I want to set the amount of gears of a vehicle I use SET_VEHICLE_HIGH_GEAR as a temporary way to test gears - not sure if this is 100% the same as setting nInitialDriveGears but it works well.
Just a thought.

Did you test if the GET_VEHICLE_HANDLING_INT returns the updated value after setting it using that native?

You can also use GET_VEHICLE_HIGH_GEAR, I would assume you could use both of these natives as a complete method to emulate setting gears.

Very late, but this would be very nice. As I am currently using this and need to be able to change nInitialDriveGears, will this be added or does someone else have a fix?


Add encoding and writing JPEG ops

This PR introduces new encoding and writing operators to the torchvision image frontend, as follows:

[x] encode_jpeg: Takes a CHW tensor as input and produces a byte buffer containing the JPEG file in tensor form.
[x] write_jpeg: Takes a CHW tensor and a filename and writes the image into a JPEG file in disk.

Fixes #2691

How to I access this code from C++? I mean, I could not find it inside libtorch-win-shared-with-deps-1.8.1+cpu.zip. Is it in another library?

Hi @faranaziz
We haven't exposed the IO ops in the same binaries as the other modeling ops in torchvision. If you want those ops, you'll need to compile torchvision from source


Created hostsman autopackage

I get the following when I try to test:
The packages folder 'C:\code\mine\chocolateyautomaticpackages\name' does not exist. Perhaps you meant to specify the name as something else or you want to override the packages folder path setting of 'C:\code\mine\chocolateyautomaticpackages'. See chocopkgup.exe /help for details.

Yes, I updated choco package up. The location C:\code\mine\chocolateyautomaticpackages\name does not exist, but I have not entered that path in any file. Just cloned a VM, installed choco package up and ketarin, added script:
chocopkgup /p {appname} /v {version} /u "{preupdate-url}" /u64 "{url64}" /pp "{file}" /disablepush

Is anyone with a working setup in a position to test if this works? Maybe it's just good to go.

You do need to update the chocopkup config - "C:\code\mine\chocolateyautomaticpackages" is the default location.

As for name, I can only say it's possible that you have another ketarin item in jobs and it needs updated or removed.

@ferventcoder @gep13 so I can't seem to set up a working ketarin for some reason. I can't even build packages that are in master. I would appreciate one of you with a working setup just checkout this PR and see if it builds, so it can be merged.
Or actually, the people who are requesting the updated package would appreciate this. :smile:

We could use a vagrant box to set up a testing environment. But it's hard to set up a Windows vagrant box. The modern.ie boxes don't allow automatic setup through ssh.

Would it help to pull this in and see what happens since it seems environmental?

@gep13 thoughts!

Could you checkout hostsman in your running setup and run the ketarin.xml file, but run the command with /disablepush?
If it works, we can just merge and add it to the main ketarin machine.

@Redsandro this worked fine when I pulled it onto the ketarin machine, so looks like it is all good :+1:

I'm glad. Thank you! :+1:


Country with no recorded case

Awesome. I tried to check for Sierra Leone, Libya which have no recorded case. But no result found. Is it that you tracking only countries that have reported a case thus far? If yes then it's okay but I suggest you change the message when a country with no recorded case is not found.
Thank you.

Share the commands that led to this


Images on landing page are blury

The images on the landing page that you see when scrolling are blurry (in my opinion).

maybe the compression was set to deep... @SUPERETDUPER, do you want to try to fix this?

I would love to fix the issue however I don't know how. I have forked the repo and it is running locally but I have never done react. Where should I start?

its not about the react . its about the webpack compressor. look at webpack prod and there you find the initial path to solve this issue. @SUPERETDUPER

@jrogatis Ok I'll try.


Update Data Models README

With new folks onboarding, important to take a pass at the README to ensure that the contributions steps are clear and accurate.
Within 24-5 sprint, comment here with required update or any new additions, which can be implemented in our next sprint.

Need to remove Elvira as default reviewer


Fix Clock Propagation

OpenROAD.STA*PNR

New environment variable made accessible to SDC files used during Multi-Corner STA steps, OPENLANE_SDC_IDEAL_CLOCKS, set to 1 for pre-PnR. Band-aid until the SDC situation is properly discussed and addressed (in a potentially breaking change.)
Fixed issue where the clock was always propagated after STAPrePNR regardless the information in the SDC file.
For backwards compatibility, STAPrePNR unsets all propagated clocks and the rest set all propagated clocks IF the SDC file lacks the strings set_propagated_clock or unset_propagated_clock.

Depends on https://github.com/efabless/openlane2-step-unit-tests/pull/35
Depends on https://github.com/efabless/openlane2-ci-designs/pull/32

It is not an internal variable. It is a variable exposed to SDC files.
What I can do is add it to the step documentation.

It is not an internal variable. It is a variable exposed to SDC files.
What I can do is add it to the step documentation.

Isn't it forced to specific value in some steps?

I think users may miss OPENLANE_SDC_IDEAL_CLOCKS but since it is getting removed in OpenLane 3 perhaps it is not that big of a deal.


Syntax highlighting broken in storybook

Describe the bug
Syntax highlighting in storybook is not displaying correctly.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://consensys.github.io/rimble-ui/?path=/story/components-text--documentation'
Scroll down to any code blocks
See error

Expected behavior
Opening and closing tags should be the same color.
Screenshots

Desktop (please complete the following information):

OS: MacOS
Browser: Chrome
Version: 73.0.3683.86 (Official Build) (64-bit)

PrismJS syntax for .jsx requires that the markup get wrapped in parens () in order to highlight matching tags the same color.
If we are only showing markup for the docs, we can use language-html for all the snippets. If there are any examples that are full-blown .jsx with class declaration then we should pass in a syntax prop like syntax="language-jsx".

I'm still seeing syntax highlighting showing up like this:


New-VMSwitch fails on Azure VM running WinServer 2019

New-VMSwitch -Name "InternalNAT" -SwitchType Internal
Above command fails with:
New-VMSwitch : Hyper-V encountered an error trying to access an object on computer 'xxx' because the object
was not found. The object might have been deleted. Verify that the Virtual Machine Management service on the computer
is running.
Using Azure Image Builder to create an Azure VM shared image.  During a powershell customizer:
Install-WindowsFeature -Name 'Hyper-V' -IncludeManagementTools -IncludeAllSubFeature
Then reboot and attempt to New-VMSwitch
I've attempted to ensure both VMCompute and VMMS services are running before the New-VMSwitch.
There are lots of reports of Hyper-V is not installing correctly on WinServer and this error.  So ultimately the issue may be with the Hyper-V stack itself.  But this should work out of the box as advertised.  It does not using 100% MS tech.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7ffa9c12-4fdc-118a-4e84-624cbcc3f4c8
Version Independent ID: 6c5b1d1f-78a1-2c79-b37d-0367d76269da
Content: How to enable nested virtualization in Azure Virtual Machines - Azure Windows Virtual Machines
Content Source: articles/virtual-machines/windows/nested-virtualization.md
Service: virtual-machines-windows
GitHub Login: @cynthn
Microsoft Alias: cynthn

Thanks for the question. We are currently investigating and will update you shortly.

Apologies on the delay. Were you able to resolve this issue?

The existing Hyper-V deployment on Windows Server 2019 is fragile and its robustness should be improved by the product engineering team.
Until the product is more robust, the documentation should be updated to indicate that only viable option to enabling hyper-V is to reboot twice after enabling hyper-V with some settle (sleep) time after the initial enabling and after each reboot.  While this solution is still too fragile for my liking (since it is timing dependent), it seems to work in a repeatable enough way I’m able to use Hyper-V on an Azure VM.  I did have to tweak my solution when I moved from the Feb to March Azure marketplace drop of Server 2019 datacenter which illustrates its still too fragile.
Feel to reach out to me directly for more details.  I’m not sure who’s going to see this reply.
Bill
From: Karishma Tiwari - MSFT<EMAIL_ADDRESS>Sent: Friday, April 3, 2020 1:01 PM
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: Bill Sproule<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] New-VMSwitch fails on Azure VM running WinServer 2019 (#49190)
Apologies on the delay. Were you able to resolve this issue?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fazure-docs%2Fissues%2F49190%23issuecomment-608631915&data=02|01|william.sproule%40microsoft.com|0dfc9f1c018d4b5267e208d7d809b437|72f988bf86f141af91ab2d7cd011db47|1|0|637215408504889060&sdata=ObqQp4xLQ75IvqM16iKXfrGmA%2BdedCrL8%2BLMFHhwrMo%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FALIDXIYBEUSXIAOT27I7R33RKY567ANCNFSM4K54TBMA&data=02|01|william.sproule%40microsoft.com|0dfc9f1c018d4b5267e208d7d809b437|72f988bf86f141af91ab2d7cd011db47|1|0|637215408504889060&sdata=Zu5KWNVG9NsFKq0nC%2BRByIGAchie7ncCmKlTGfKCDms%3D&reserved=0.

Thanks for sharing the update.
Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

@Karishma-Tiwari-MSFT Can you assign this to someone from engineering that works on nested virtualization? Thanks!

@BillSproule as you are an FTE, please ping or email me offline and let's find the best route to move this forward.
#please-close

#reopen


revert to old Switch-style, keeping unmountScenes.  needs review

refs https://github.com/aksonov/react-native-router-flux/issues/976
refs https://github.com/aksonov/react-native-router-flux/issues/952

Can someone verify unmountScenes works?  It may need to be moved before the Actions[selectedKey]();

I had tested in the following structure:

// --- this got infinite loop
<Scene key="switch" initial={true} component={Switch} tabs={true} unmountScenes={true} selector={switchSelector}>
    <Scene key="login" initial={true}/>
    <Scene key="home"/>
</Scene>

and
// --- this would not switch even selector returns 'home'

<Scene key="switch" initial={true} component={Switch} tabs={true} unmountScenes={true} selector={switchSelector}>
    <Scene key="login"/>
    <Scene key="home" initial={true}/>
</Scene>

@zxcpoiu thanks for checking! Just made a change to only do the unmountScenes logic when a switch will actually occur.
@ms88privat could you also verify?

@joenoon I can test it tomorrow

I still got infinite loop in the first case with new commits.
It feels like a little weird to me, just want to make sure, is the following descriptions correct or not:
at the first case:
// --- this got infinite loop
<Scene key="switch" initial={true} component={Switch} tabs={true} unmountScenes={true} selector={switchSelector}>
    <Scene key="login" initial={true}/>
    <Scene key="home"/>
</Scene>

navState's children is login and home, and initial index is '0' since login is the initial component
that's assume in selector, we return 'home', then the selected navigationState is home and it's index mostly is 0 (or the index of it's initial child.)

so the reason I got infinite loop is because this children: [navState.children[navigationState.index]],
we always set navState to it's index-0 child which is login, but the selector return 'home' which is the index-1 child of navState and has it's own child indexes cache whcih is 0.
and no matter how, eventually, we've change the navState to selected child's navigationState
( I'm not sure, is this the right flow that we replace navState with child's state?)

I have tried several ways, but with no luck so far.
If someone want to reproduce, here is test case:
// --- this got infinite loop if switch selector returns 'home'

<Scene key="modal" component={Modal} >
    <Scene key="root">
        <Scene key="switch" initial={true} component={Switch} tabs={true} unmountScenes={true} selector={switchSelector}>
            <Scene key="login" initial={true} type="replace" />
            <Scene key="home" tabs={true} type="replace">
                <Scene key="tab1" initial={true} />
                <Scene key="tab2" initial={true} />
            </Scene>
        </Scene>
    </Scene>
</Scene>

@@ -15,6 +15,7 @@ export default class Switch extends Component {
   constructor(props, context) {
     super(props, context);
     this.updateState = this.updateState.bind(this);
+    this.resetHistoryStack = this.resetHistoryStack.bind(this);
     this.state = {};
   }

@@ -26,6 +27,23 @@ export default class Switch extends Component {
     this.updateState(props);
   }

+  resetHistoryStack(child) {
+    const newChild = child;
+    newChild.index = 0;
+    for (let j = 0; j < child.children.length; j++) {
+      if (child.children[j].initial) {
+        newChild.index = j;
+        if (!child.tabs) {
+          newChild.children = [child.children[j]];
+        }
+      }
+      if (newChild.children[j].children) {
+        newChild.children[j] = this.resetHistoryStack(child.children[j]);
+      }
+    }
+    return newChild;
+  }
+
   updateState(props) {
     const navState = props.navigationState;

@@ -43,11 +61,8 @@ export default class Switch extends Component {

     if (navigationState.key !== navState.children[navState.index].key) {
       if (props.unmountScenes) {
-        navigationState = {
-          ...navigationState,
-          children: [navState.children[navigationState.index]],
-          index: 0,
-        };
+        navState.children[navState.index] = this.resetHistoryStack(navState.children[navState.index]);
+        navigationState = { ...navState };
       }
       Actions[selectedKey]();
     }

@sarovin Maybe i'm wrong, but i believe navigationState should be changed within Reducer, not within Switch. What do you think?

@aksonov you are right but i don't understand how work your Reducer  😕
I try with reducer as soon as possible

Reducer works in the same way as Redux reducers. You may introduce param for JUMP action and add appropriate logic where JUMP is processed.

#1002

Closing it because of #1002, @joenoon does it work for you?

@aksonov so far so good!  Totally missed all the action in the other PR - nice work @sarovin and everyone else!


Implemenatation of BasicSymComponent class that creates a part of a report

Class is to implement in guide.pl.py module that is a reporting component of basic version of simulations using pandas library.

Done


tools: N6000 BOM info reformatting & add test cases

Add 2 new BOM Critical Components unit test cases
Improved robustness of BOM info reformatting

Coverage decreased (-0.02%) to 61.422% when pulling 54988cfdeaba6d8f27ce638b41863678a57166b3 on aravuri/N6000_BOM into 27d18f1875ef43d473e4ea21f970da9fb3943e80 on master.


Consistent Issues Setting Up Project Repository

Summary
Students across various platforms, and especially Mac users, tend to consistently make the same mistakes when setting up the project repository. There may be improvements we can make or changes to the required process that would minimize these errors and provide a better experience overall.
At its root, the issue typically comes down to which folder should be the base folder. A lot of students create a folder call "chess", and then unzip the "chess.zip" file which creates another folder called "chess." Then when they attempt to move the contents between these two "chess" folders, they either forget to move the hidden but important files, or they open up Intelli-J and/or their Git project in the outer "chess" folder instead of the inner "chess" folder.
TA Meeting Notes

If they unzip the starter code, then copy it into their folder, they miss the hidden files (Mac specific)

Potential solution: fork the repo
Other solution: add instruction to show hidden files
“If you have a broken process, and try to fix it with documentation, then its gonna stay broken” -Professor Jensen
Look into a better way to do than the zip file problem
Copy from the files from the zip file, before opening in IntelliJ
@Truthwatcher64  will work on making documentation better

Direction
After discussing this with @leesjensen, it has been decided to update the instructions to creating clones of the softwareconstruction240/chess repo without creating forks. We'll use the following resources as a guide when creating the new instruction:

CS 324 Private Repo Setup
StackOverflow: Bare vs Mirror repositories

I just found related resource that will greatly simplify this process: GitHub Template repositories.
Template repositories create new projects from only the latest commit of the repo, and they do not appear as forks with quick links to each other!

We can also systematically verify that the repos turned in are NOT forks of the base repository.
curl -s https://api.github.com/repos/:USERNAME/:REPO_NAME | jq '.fork'
# The value needs to come back as `false`.
# A `true` value indicates the repo IS a fork (which we want to disallow)
# A `null` value indicates the repo doesn't actually exist

Sources:

StackOverflow answer
GitHub: Forking vs Duplicating repos

@jerodw and @rodham - I think James' proposal to use GitHub Template repositories has merit especially when combined with the autograder restriction on forked repos. What do you all think?


DBZ-648 InterrupedException handled by MongoDB

https://issues.jboss.org/browse/DBZ-648

LGTM, @jpechane. Did you do a test that the connector would indeed be stopped and fully removed when being deleted while waiting to connect to a (non-existent) database?

@gunnarmorling Yes, the connector is removed and the number of threads before connector start and after connector delete is the same

Cool. Applied. Thanks, @jpechane!


Conflicting description in the README

Under the Prerequisites section, it mentions Clang 3.4
However, under the Limitations section, it says "HNSW does not work with Clang".

@ijklr this was true some time ago, I am going to re-test it shortly with a newer version of HNSW. Probably, this is not an issue anymore.


Add Found Items (10 items)

Put all work for this issue in a branch called issue-41.
Add 10 items to settings.development.json.

Created the 10 items. Planning to add them after the system has recovered from failing.

Items:

Added 10 items to settings.development.json.


Ant-design UI Modal breaks when upgrading to v0.9.1

Describe the bug
I try to upgrade my @solana/wallet-adapter-ant-design version from  v0.1.0 to v0.9.1 and it breaks the modal UI
I'm quite sure that I already provide @import '~antd/dist/antd.dark.less' on top of my .less file. The version of packages I used are:
 "@solana/wallet-adapter-ant-design": "^0.9.1",
 "@solana/wallet-adapter-base": "^0.7.0",
 "@solana/wallet-adapter-react": "^0.13.1",
 "@solana/wallet-adapter-wallets": "^0.11.3",

Screenshots

This is solved by adding require('@solana/wallet-adapter-ant-design/styles.css');


restcountries.eu API is no longer free/open

This line https://github.com/vercel/next-learn/blob/master/seo/pages/index.js#L103
Causes this failure
FetchError: request to https://restcountries.eu/rest/v2/all failed, reason: Hostname/IP does not match certificate's altnames: Host: restcountries.eu. is not in the cert's altnames: DNS:shortener.secureserver.net, DNS:www.shortener.secureserver.net
    at ClientRequest.<anonymous> (/Users/aharbick/Sites/seo/nextjs-lighthouse/node_modules/node-fetch/lib/index.js:1461:11)
    at ClientRequest.emit (node:events:376:20)
    at TLSSocket.socketErrorListener (node:_http_client:474:9)
    at TLSSocket.emit (node:events:376:20)
    at emitErrorNT (node:internal/streams/destroy:188:8)
    at emitErrorCloseNT (node:internal/streams/destroy:153:3)
    at processTicksAndRejections (node:internal/process/task_queues:80:21) {
  type: 'system',
  errno: 'ERR_TLS_CERT_ALTNAME_INVALID',
  code: 'ERR_TLS_CERT_ALTNAME_INVALID'
}

Fixed https://github.com/vercel/next-learn/pull/53