Deprecation of ```np.long`` in NumPy 1.20 in .merge() method

Hi,
I'm using  pyranges-0.0.115 with NumPy-1.22.2 and pandas-1.4.0. When calling the method range I get the following warning.
.venv/lib/python3.9/site-packages/pyranges/methods/merge.py:16: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  starts, ends, number = find_clusters(cdf.Start.values, cdf.End.values, slack)

I thought you may want to be aware of this change.
Best,
Ricardo.

Thanks, it will be supported for a long time, but this is something I should get around to fixing anyways.
Sorry for the late reply, was not notified of your post.

Hi,
I use pyranges in my function. As this warning keeps showing when I run pytest, I sincerely hope you can fix it. Thank you.

Dongze

I'll try to do so tomorrow :)
On Thu, May 26, 2022 at 7:44 PM Dongze He @.***> wrote:

Hi,
I use pyranges in my function
https://github.com/COMBINE-lab/pyroe/blob/main/src/pyroe/make_splici_txome.py.
As this warning keeps showing when I run pytest, I sincerely hope you can
fix it. Thank you.

Dongze

—
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/pyranges/issues/258#issuecomment-1138844796,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUWAM5RFJ6UG65PYTSDVL6Z55ANCNFSM5NXKTT3A
.
You are receiving this because you commented.Message ID:
@.***>

Great! Thanks so much! Here are the warnings I got. Hope it will help you locate the places to change.
  /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages/pyranges/methods/merge.py:71: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    ids, starts, ends, number = merge_by(cdf.Start.values, cdf.End.values, cdf.ClusterBy.values, slack)

tests/test_make_splici.py::test_make_splici
tests/test_make_splici.py::test_make_splici
  /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages/pyranges/genomicfeatures.py:569: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    exons.End.values, exon_ids.values)

I am looking at it now. It is actually a problem in the pyranges support libraries I wrote.

I've updated sorted_nearest, ncls, and pyrle to not use np.long anymore. Can you try updating those Python libraries and see if the issue is fixed?

Hi @endrebak,
There were errors reported by gcc((Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0) when I install the packages you mentioned.  If possible, could you also update them in conda? thanks!
Collecting sorted_nearest
  Using cached sorted_nearest-0.0.35.tar.gz (1.2 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cython in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (0.29.30)
Requirement already satisfied: numpy in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (1.21.6)
Building wheels for collected packages: sorted_nearest
  Building wheel for sorted_nearest (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for sorted_nearest (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [44 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-37
      creating build/lib.linux-x86_64-cpython-37/sorted_nearest
      copying sorted_nearest/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest
      copying sorted_nearest/version.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest
      creating build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/introns.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/annotate_clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/k_nearest_ties.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/max_disjoint_intervals.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/sorted_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/tiles.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/windows.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/k_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/merge_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/cluster_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/csorted_nearest.pxd -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/utarray.h -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/cluster_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/introns.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/annotate_clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/k_nearest_ties.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/sorted_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/k_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/tiles.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/merge_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/max_disjoint_intervals.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      copying sorted_nearest/src/windows.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src
      running build_ext
      building 'sorted_nearest.src.sorted_nearest' extension
      creating build/temp.linux-x86_64-cpython-37
      creating build/temp.linux-x86_64-cpython-37/sorted_nearest
      creating build/temp.linux-x86_64-cpython-37/sorted_nearest/src
      gcc -pthread -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include -fPIC -I. -I/home/zaza/miniconda3/envs/pyroe/include/python3.7m -c sorted_nearest/src/sorted_nearest.c -o build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o
      gcc -pthread -shared -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -L/home/zaza/miniconda3/envs/pyroe/lib -Wl,-rpath=/home/zaza/miniconda3/envs/pyroe/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/home/zaza/bin/zlib/lib -L/home/linuxbrew/.linuxbrew/lib -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o -o build/lib.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.cpython-37m-x86_64-linux-gnu.so
      /home/zaza/miniconda3/envs/pyroe/compiler_compat/ld: cannot find /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a
      collect2: error: ld returned 1 exit status
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for sorted_nearest
Failed to build sorted_nearest
ERROR: Could not build wheels for sorted_nearest, which is required to install pyproject.toml-based projects```

But the packages should be automatically updated in conda by the conda-bot?
On Fri, May 27, 2022 at 9:38 PM Dongze He @.***> wrote:

Hi @endrebak https://github.com/endrebak,
There were errors reported by gcc((Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0)
when I install the packages you mentioned. If possible, could you also
update them in conda? thanks!
Collecting sorted_nearest
Using cached sorted_nearest-0.0.35.tar.gz (1.2 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cython in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (0.29.30)
Requirement already satisfied: numpy in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (1.21.6)
Building wheels for collected packages: sorted_nearest
Building wheel for sorted_nearest (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for sorted_nearest (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [44 lines of output]
  running bdist_wheel

  running build

  running build_py

  creating build

  creating build/lib.linux-x86_64-cpython-37

  creating build/lib.linux-x86_64-cpython-37/sorted_nearest

  copying sorted_nearest/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest

  copying sorted_nearest/version.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest

  creating build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/introns.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/annotate_clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest_ties.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/max_disjoint_intervals.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/sorted_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/tiles.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/windows.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/merge_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/cluster_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/csorted_nearest.pxd -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/utarray.h -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/cluster_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/introns.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/annotate_clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest_ties.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/sorted_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/tiles.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/merge_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/max_disjoint_intervals.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/windows.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  running build_ext

  building 'sorted_nearest.src.sorted_nearest' extension

  creating build/temp.linux-x86_64-cpython-37

  creating build/temp.linux-x86_64-cpython-37/sorted_nearest

  creating build/temp.linux-x86_64-cpython-37/sorted_nearest/src

  gcc -pthread -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include -fPIC -I. -I/home/zaza/miniconda3/envs/pyroe/include/python3.7m -c sorted_nearest/src/sorted_nearest.c -o build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o

  gcc -pthread -shared -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -L/home/zaza/miniconda3/envs/pyroe/lib -Wl,-rpath=/home/zaza/miniconda3/envs/pyroe/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/home/zaza/bin/zlib/lib -L/home/linuxbrew/.linuxbrew/lib -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o -o build/lib.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.cpython-37m-x86_64-linux-gnu.so

  /home/zaza/miniconda3/envs/pyroe/compiler_compat/ld: cannot find /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a

  collect2: error: ld returned 1 exit status

  error: command '/usr/bin/gcc' failed with exit code 1

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sorted_nearest
Failed to build sorted_nearest
ERROR: Could not build wheels for sorted_nearest, which is required to install pyproject.toml-based projects```
—
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/pyranges/issues/258#issuecomment-1139972712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUXVB4MHYARWBJHVLVDVMEQCTANCNFSM5NXKTT3A
.
You are receiving this because you were mentioned.Message ID:
@.***>

I think it was not updated in conda due to a bug in sorted_nearest. Fixing
On Mon, Jun 6, 2022 at 11:44 AM Endre Bakken Stovner @.***>
wrote:

But the packages should be automatically updated in conda by the conda-bot?
On Fri, May 27, 2022 at 9:38 PM Dongze He @.***>
wrote:

Hi @endrebak https://github.com/endrebak,
There were errors reported by gcc((Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0)
when I install the packages you mentioned. If possible, could you also
update them in conda? thanks!
Collecting sorted_nearest
Using cached sorted_nearest-0.0.35.tar.gz (1.2 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cython in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (0.29.30)
Requirement already satisfied: numpy in /home/zaza/miniconda3/envs/pyroe/lib/python3.7/site-packages (from sorted_nearest) (1.21.6)
Building wheels for collected packages: sorted_nearest
Building wheel for sorted_nearest (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for sorted_nearest (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [44 lines of output]
  running bdist_wheel

  running build

  running build_py

  creating build

  creating build/lib.linux-x86_64-cpython-37

  creating build/lib.linux-x86_64-cpython-37/sorted_nearest

  copying sorted_nearest/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest

  copying sorted_nearest/version.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest

  creating build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/__init__.py -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/introns.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/annotate_clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest_ties.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/max_disjoint_intervals.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/sorted_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/tiles.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/windows.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/merge_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/cluster_by.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/clusters.pyx -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/csorted_nearest.pxd -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/utarray.h -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/cluster_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/introns.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/annotate_clusters.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest_ties.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/sorted_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/k_nearest.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/tiles.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/merge_by.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/max_disjoint_intervals.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  copying sorted_nearest/src/windows.c -> build/lib.linux-x86_64-cpython-37/sorted_nearest/src

  running build_ext

  building 'sorted_nearest.src.sorted_nearest' extension

  creating build/temp.linux-x86_64-cpython-37

  creating build/temp.linux-x86_64-cpython-37/sorted_nearest

  creating build/temp.linux-x86_64-cpython-37/sorted_nearest/src

  gcc -pthread -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include -fPIC -I. -I/home/zaza/miniconda3/envs/pyroe/include/python3.7m -c sorted_nearest/src/sorted_nearest.c -o build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o

  gcc -pthread -shared -B /home/zaza/miniconda3/envs/pyroe/compiler_compat -L/home/zaza/miniconda3/envs/pyroe/lib -Wl,-rpath=/home/zaza/miniconda3/envs/pyroe/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/home/zaza/bin/zlib/lib -L/home/linuxbrew/.linuxbrew/lib -I/home/zaza/bin/zlib/include -I/home/linuxbrew/.linuxbrew/include build/temp.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.o -o build/lib.linux-x86_64-cpython-37/sorted_nearest/src/sorted_nearest.cpython-37m-x86_64-linux-gnu.so

  /home/zaza/miniconda3/envs/pyroe/compiler_compat/ld: cannot find /usr/lib/x86_64-linux-gnu/libpthread_nonshared.a

  collect2: error: ld returned 1 exit status

  error: command '/usr/bin/gcc' failed with exit code 1

  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for sorted_nearest
Failed to build sorted_nearest
ERROR: Could not build wheels for sorted_nearest, which is required to install pyproject.toml-based projects```
—
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/pyranges/issues/258#issuecomment-1139972712,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUXVB4MHYARWBJHVLVDVMEQCTANCNFSM5NXKTT3A
.
You are receiving this because you were mentioned.Message ID:
@.***>

Am I correct that this still is an issue?
As far as I can see, sorted_nearest is still using np.long here: https://github.com/pyranges/sorted_nearest/blob/master/sorted_nearest/src/clusters.pyx#L11
I get this problem when using pyranges version 0.0.120 from conda.

I haven't looked at the pyranges-supporting libraries. I'll do so tomorrow!
Thanks for reminding me.
On Tue, Dec 20, 2022 at 11:03 AM ivargr @.***> wrote:

Am I correct that this still is an issue?
As far as I can see, sorted_nearest is still using np.long here:
https://github.com/pyranges/sorted_nearest/blob/master/sorted_nearest/src/clusters.pyx#L11
I get this problem when using pyranges version 0.0.120 from conda.
—
Reply to this email directly, view it on GitHub
https://github.com/biocore-ntnu/pyranges/issues/258#issuecomment-1359110033,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEHURUSEL3IGGEFBATTGOCTWOF75PANCNFSM5NXKTT3A
.
You are receiving this because you were mentioned.Message ID:
@.***>

This seems to be causing an error for me. Confusing b/c this was working until this evening...I did update my environment, so * may have * updated pyranges. I am currently using 0.0.120
concat_ranges
+--------------+-------------------+--------------+-----------+-------+
| Chromosome   | Source            | Feature      | Start     | +6    |
| (category)   | (object)          | (category)   | (int32)   | ...   |
|--------------+-------------------+--------------+-----------+-------|
| chr1         | hg002_sqanti_fltr | transcript   | 1013503   | ...   |
| chr1         | hg002_sqanti_fltr | transcript   | 1053929   | ...   |
| chr1         | hg002_sqanti_fltr | transcript   | 1308594   | ...   |
| chr1         | hg004_sqanti_fltr | transcript   | 1013496   | ...   |
| ...          | ...               | ...          | ...       | ...   |
| chr1         | hg005_sqanti_fltr | transcript   | 1211340   | ...   |
| chr1         | hg005_sqanti_fltr | transcript   | 1216908   | ...   |
| chr1         | hg005_sqanti_fltr | transcript   | 1216914   | ...   |
| chr1         | hg005_sqanti_fltr | transcript   | 1253914   | ...   |
+--------------+-------------------+--------------+-----------+-------+

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/oguzkhan/code/isocomp/.venv/lib/python3.9/site-packages/pyranges/pyranges.py", line 1145, in cluster
    df = pyrange_apply_single(_cluster, self, **kwargs)
  File "/home/oguzkhan/code/isocomp/.venv/lib/python3.9/site-packages/pyranges/multithreaded.py", line 348, in pyrange_apply_single
    result = call_f_single(function, nparams, df, **kwargs)
  File "/home/oguzkhan/code/isocomp/.venv/lib/python3.9/site-packages/pyranges/multithreaded.py", line 31, in call_f_single
    return f.remote(df, **kwargs)
  File "/home/oguzkhan/code/isocomp/.venv/lib/python3.9/site-packages/pyranges/methods/cluster.py", line 14, in _cluster
    ids = annotate_clusters(cdf.Start.values, cdf.End.values, slack)
  File "sorted_nearest/src/annotate_clusters.pyx", line 10, in sorted_nearest.src.annotate_clusters.annotate_clusters
  File "/home/oguzkhan/code/isocomp/.venv/lib/python3.9/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'long'

the pyranges obj is attached:
concat_ranges.gtf.gz

looks like this is causing problems elsewhere, including at NASA:
https://github.com/nasa/puma/issues/9#issuecomment-1368069775
downgrading from numpy 1.24 to 1.23 resolved this error. But, this issue should stay open and get fixed.

Looking at it now, thanks!

I've updated sorted_nearest: https://github.com/pyranges/sorted_nearest/blob/master/CHANGELOG
PyPI version should be working now, bioconda in a few hours.

Fyi, bioconda seems stuck on v0.0.37

This issue is still present, as bioconda installs version 0.0.3 of sorted_nearest.
As consequence, today on a clean pyranges installation, some methods fail, e.g. max_disjoint


python37Packages.HAP-python: 2.9.2 -> 3.0.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python37Packages.HAP-python is: "HomeKit Accessory Protocol implementation in python"
meta.homepage for python37Packages.HAP-python is: "https://github.com/ikalchev/HAP-python"
meta.changelog for python37Packages.HAP-python is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 3.0.0 with grep in /nix/store/wrigyh8ilsb44zhgg2q28wyvsh7z1g5q-python3.7-HAP-python-3.0.0

directory tree listing: https://gist.github.com/06e6be568e4265301c2b84edc4ea4d96

du listing: https://gist.github.com/b101b7daa6c1c5dfb67b6088c17a19c3

Rebuild report (if merged into master) (click to expand)

12 total rebuild path(s)

3 package rebuild(s)

3 x86_64-linux rebuild(s)
3 i686-linux rebuild(s)
3 x86_64-darwin rebuild(s)
3 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
python37Packages.HAP-python
python38Packages.HAP-python
python39Packages.HAP-python

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wrigyh8ilsb44zhgg2q28wyvsh7z1g5q-python3.7-HAP-python-3.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python37Packages.HAP-python https://github.com/r-ryantm/nixpkgs/archive/61c302e76df1c7641ba9bdb70c0461eec3ea1e8c.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wrigyh8ilsb44zhgg2q28wyvsh7z1g5q-python3.7-HAP-python-3.0.0
ls -la /nix/store/wrigyh8ilsb44zhgg2q28wyvsh7z1g5q-python3.7-HAP-python-3.0.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review 1

3 packages built:

python37Packages.HAP-python
python38Packages.HAP-python
python39Packages.HAP-python

Maintainer pings
cc @oro for testing.

Result of nixpkgs-review pr 105788 run on x86_64-darwin 1

3 packages built:

python37Packages.HAP-python
python38Packages.HAP-python
python39Packages.HAP-python


Improve promtool test rules output

Proposal
Use case. Why is this important?
The promtool test rules utility produces error outputs that are sometimes hard to read. This is because the expected and actual data structures being compared are printed using a %#v dump. In particular, it is hard to read the difference between the two structures when they are large - eg. a lot of labels and/or a lot of annotations.
Moreover, it's hard to improve/collect the output from a third-party tool, because the "%#v" dump is not easily parsable.
What are the changes I'm proposing?
For promtool test rules introduce a flag --format, initially, with only a single supported value: "json". This will encode the output as JSON and allows third party tools to easily parse the output and format it as needed.
In this PR https://github.com/prometheus/prometheus/pull/7549 I'm trying to solve my initial issue, which is diffing the exp and got data structures so I can understand my tests is failing. At this point, I think that method has disadvantages: - it introduces a new dependency; - it's not flexible; - it contributes to the problem, instead of fixing it.
I'm happy to work on this if I get a green light!

I'm not sure about making all of our tools be able to additinoally output in JSON, but making the output more readable to humans sounds useful.

Hey @brian-brazil. What do you think about #7549 then?

@simonpasquier maintains that bit of the codebase, so would be the best person to review it.

I'd like to see the same for check rules
When running CI against a medium size PR, being able to parse the syntax errors and linter errors from JSON would be great.
At least, parsing the text output feels quite flaky.


How to get negative_sampler file?

Hi, I have found the link in here is unavailable. I am so confused about how to get negative_sampler file. Could you please offer more details about negative_sampler file?

Hi, I have found the link in here
is unavailable. I am so confused about how to get negative_sampler file. Could you please offer more details about negative_sampler file?

The document is here: https://easyrec.readthedocs.io/en/latest/models/dssm_neg_sampler.html#id3

Thanks. But I still not sure the means of the schema, could you give a simple demo for negative_sampler file?

https://easyrec.readthedocs.io/en/latest/models/dssm_neg_sampler.html @kinghuin maybe this will help you, you can join DingDing Group: 32260796.

Thanks for reply :)


Some messages do not have response data in multithreaded environment

We are experiencing an issue where about 20% of email does not have response data after calling the .deliver method. The messages are being sent fine, but we are unable to consistently get the message ID due to this issue.
We are using the default :ActiveSupport JSON adapter, and here is our config:
config.action_mailer.postmark_settings = {api_key: CONFIG[:postmark_api_key],
return_response: true, track_opens: true}

Our environment is as follows:

Rails <IP_ADDRESS>
JRuby <IP_ADDRESS>
Postmark 1.7.1
Postmark-rails 0.12.0

I am in process of deploying a monkey-patched API client with additional logging to hopefully determine what is going on.   Any additional advice would be appreciated.

Nevermind. This is not actually happening!! Code error in the application.


Update AudioPlayer.svelte

tried to create a loop button, since obviously i wanna rock with just one anime opening all the time. if there is any error, please do tell.

@Achintyaay thanks for your contribution.
Yes, there are couple of errors:

Because the el var is an HTML element available only after mounting the component, this throws an error. There are many ways to solve this issue, but the easiest one is to replace the event listener code with this one:

// the code after the comma will execute every time $ended changes
$ended, ($ended && looping) ? play() : null;

you put all the UI in the AudioPlayer.svelte file, but that's only used to manage the audio element playing music and logic behind, not to display something. I would suggest to add it to the icons on the left of the player playing image, which are on the PlayingPreview.svelte file, starting from here:
https://github.com/Bellisario/musicale/blob/3747191bca8f9a01413ad526aa2d0e15ec8b61c1/src/components/PlayingPreview.svelte#L96

you should also provide an icon for the "loop" button, currently, there is no one.


Embed job id in test report path for tt-github-actions

@vmilosevic the job id method fails with:
/__w/_temp/1c5acf9c-4819-41c1-89c6-0c6ad5c57343.sh: line 9: curl: command not found
/__w/_temp/1c5acf9c-4819-41c1-89c6-0c6ad5c57343.sh: line 9: jq: command not found

It's insane to me that GH doesn't already have an env/string with this information, seriously wtf
I'm not sure what the best solution is, I could hack an apt install, but this seems wasteful, I guess we should install curl and jq in our containers?

It's insane to me that GH doesn't already have an env/string with this information, seriously wtf
Yes, and no other way to link attachments to jobs that produced them :(

I'm not sure what the best solution is, I could hack an apt install, but this seems wasteful, I guess we should install curl and jq in our containers?

I'll sort this out in https://github.com/tenstorrent/tt-mlir/pull/1420

This was handled as part of #1420, closing.


Add probabilistic skill scores

A good start could be brier_score which can be adapted from https://github.com/TheClimateCorporation/properscoring
as was stated in the original conversation of starting up this repo: https://groups.google.com/forum/#!topic/xarray/z8ue0G-BLc8

https://github.com/TheClimateCorporation/properscoring/issues/6

Closed via https://github.com/raybellwaves/xskillscore/pull/10
Welcome to open another issue to suggest other metrics.


Kythe release with libmemcached11 [phab:T145]

Created by astroilov at 2016-06-21 21:25:25:
The Kythe release at https://github.com/google/kythe/releases/download/v0.0.21/kythe-v0.0.21.tar.gz is linked against libmemcached10. This breaks on Debian Jessie (and presumably some other systems), where libmemcached10 is unavailable - only libmemcached11 is present. We should have an official release that uses libmemcached11.
The older version will still be relevant for other users (e.g. those on Debian Wheezy) so this should be an addition, not a replacement.

zarko wrote at 2016-06-22 00:17:55:
Would it be better to instead link libmemcached statically?

astroilov wrote at 2016-06-22 18:22:13:
Possibly? Either approach should work for Kodiak, I don't know which is better for Kythe's goals.

fromberger wrote at 2016-06-22 19:33:49:
It looks like mainly the C++ libraries care about this particular library. If there's a straightforward way to statically link for the release build that seems simpler than forking the release. I had been thinking we might just bump our required version to 11.


spelling mistake in the word analyze

spelling mistake in the word analyze

Can one of the admins verify this patch?


Backend health provided by EDS doesn't work properly with stateful session/override health status

Title: Backend health provided by EDS doesn't work properly with stateful session/override health status
Description:
We use EDS (over ADS) to dynamically control cluster load assignments and we control the health of each member from the management server. In our use case we use the HDS (Health Discovery Service) to collect endpoint health and will propagate this health status via EDS. We also use StatefulSession using an HTTP cookie and use overrideHostStatus with HEALTHY, UNKOWN and DEGRADED.  Since upgrading to Envoy 1.26 from Envoy 1.22, we now have an issue where stateful sessions no longer work.
#24094 introduced a change that uses healthStatus() instead of coarseHealth() which introduced this issue. Setting the envoy.reloadable_features.validate_detailed_override_host_statuses: false resolves the issue for us but this isn't a permanent solution.
It seems that endpoint health delivered by EDS doesn't properly work with this change. For example, when an endpoint is marked as unhealthy via EDS coarseHealth() properly returns Unhealthy but healthStatus() returns HealthStatus.HEALTHY
Repro steps:
Running on Envoy 1.26 but change was introduced in 1.25. Use StatefulSession/Cookie and update endpoint health with EDS from healthy to unhealthy.
cc @wbpcode

I will check it in next days. Thanks for the report.


Fix travis.

https://app.travis-ci.com/github/dmlc/xgboost/jobs/537824539

Ah .. pylint is updated.  Will open a different PR for fixing that.


tools service fails to start on mac if it has not been previously downloaded on machine

This issue is observed only on a mac using the latest insiders build produced by the new yaml pipeline. When I run the insiders build produced by the old pipeline, this issue is not observed.
Repro steps:

Download the latest insiders build for mac using the URL: https://azuredatastudio-update.azurewebsites.net/latest/darwin/insider
Unzip the folder
run the 'Azure Data Studio - Insiders' package.
On startup the following message pops up:

Observed: The following error pops on the bottom right as a notification.
'Failed to start SQL Tools Service'
Source: mssql (Extension)
Expected: Sqltoolservice to start-up successfully and azure sql server connectivity to work.
Upon investigation in the Contents/Resources/app/extensions/mssql/sqltoolsservice/OSX/2.0.0-release.35 folder is not found.
This is is not observed when we run the latest insiders build produced by the old pipeline from here:
https://mssqltools.visualstudio.com/CrossPlatBuildScripts/_build/results?buildId=47760

This should be fixed for #8464 you can just produce a new manual build to test.

Yes  #8464 has fixed it.


Minor gameplay consistency

ver 3.3.6
Issue p1: multiple unshielded VSP ships do not have the Rugged Construction hullmod in line with vanilla standards, as they were last updated before the hullmod existed.
Solution p1: Add "rugged" under the builtInMods section of the .ship file for
vayra_greyhound
vayra_mining_hound
vayra_razorback
vayra_battlement
vayra_fortress
vayra_motte
vayra_redoubt
vayra_tower
Issue p2: Stormhawk Thunder retains full power ion cannon and less OP than baseline version, making it wholly superior in every way instead of a minor upgrade with cost (also due to last being updated before the vanilla Thunder was tweaked).
Solution p2: change ioncannon to ioncannon_fighter in vayra_thunder_warhawk_fighter.variant & change OP cost from 9 to 11 or 12 in wing_data.csv

vayra_greyhound has a shield though. But you know what, fine, implemented in commit  c9ad379 c04843e


Child transactions in scheduled transactions are failing in 2.9.1

Description
A user reported seeing strange behavior with scheduled transactions. We jumped on a call and during troubleshooting found that scheduled transactions (token and HBAR transfers) were not going through in v2.9.1 of the JS SDK. We tried this version with two separate examples and both failed.
When we reverted to using v2.7.1, the scheduled transactions went through as expected. See repro steps for more details.
Steps to reproduce

Copy and run this code from GitHub: https://github.com/ed-marquez/hedera-sdk-js/blob/main/examples/hts-nftP2-kyc-upate-schedule.js
See lines 228-269 (Notice that the ScheduleInfoQuery even says that the schedule executed, but the child tx still didn't go through and no error is reported)
Run in v2.9.1 and v2.7.1 and notice that the transfer of NFT with serial 2 does not go through in 2.9.1 (see screenshots for console output confirmations

)

Additional context
No response
Hedera network
testnet
Version
v2.9.1
Operating system
Windows

Just wanted to corroborate with this - scheduled transactions seem to fail in v2.9.1 of the js-sdk. I had a difficult time tracking down the TX that gets fired off within the scheduled transaction, but I repeatedly found this INSUFFICIENT_TX_FEE error for a CRYPTOTRANSFER (the tx set inside the scheduled transaction)
It can be found here:
https://testnet.mirrornode.hedera.com/api/v1/transactions?account.id=0.0.12717560
Should be the transaction with consensus timestamp:<PHONE_NUMBER>.427760590
I wasn't able to prevent those errors despite having HBARs in my test accounts, but if someone can offer more insight - I'd appreciate it.

Could be related to https://github.com/hashgraph/hedera-sdk-js/issues/957
Notably one person brought up "It seems now one needs to set maximum fee for any transaction manually", haven't had a chance to test it myself but worth trying if you have a test rigged up already.

As mentioned in #957 this might be solved in the latest beta release v2.11.0-beta.1. Ran our e2e tests suite which does not set max transaction fees, and 99% passed.

I believe https://github.com/hashgraph/hedera-sdk-js/issues/957 supersedes this issue as it went into further detail, but it too has now been closed so this should also be marked closed.


[Storage Blob] Server failed to authenticate the request after upgrade to v12.12.0

Package Name: @azure/storage-blob
Package Version: 12.12.0
Operating system: Linux and Windows
[x] nodejs

version: 16.18.0

[ ] browser

name/version:

[x] typescript

version: 4.8.4

Is the bug related to documentation in

[ ] README.md
[ ] source code documentation
[ ] SDK API docs on https://docs.microsoft.com

Describe the bug
We are using BlockBlobClient.uploadStream to upload batches composed of several files to Azure Blob Storage. After upgrading from @azure/storage-blob 12.11.0 to 12.12.0 we are receiving the following error:
"RestError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature."
The exception is thrown after around 100 files, from a specific batch, have been uploaded, We suppose that credentials are valid because the first files are uploaded correctly and also reverting to @azure/storage-blob 12.11.0 solves the issue.
Any help would be appreciated.
To Reproduce
Steps to reproduce the behavior:

Calling BlockBlobClient.uploadStream more than 100 times

Expected behavior
All the upload operations should succeed.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Following the error message and stack trace:
RestError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:266a0736-301e-0006-3c35-ec72e8000000
Time:2022-10-30T07:56:28.0808035Z

    at handleErrorResponse (/opt/app/node_modules/@azure/core-http/dist/index.js:3161:19)
    at /opt/app/node_modules/@azure/core-http/dist/index.js:3097:49
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async StorageRetryPolicy.attemptSendRequest (/opt/app/node_modules/@azure/storage-blob/dist/index.js:14673:24)
    at async StorageClientContext.sendOperationRequest (/opt/app/node_modules/@azure/core-http/dist/index.js:4934:31)
    at async BlockBlobClient.stageBlock (/opt/app/node_modules/@azure/storage-blob/dist/index.js:21038:20)
    at async BufferScheduler.outgoingHandler (/opt/app/node_modules/@azure/storage-blob/dist/index.js:21407:17)
    at async BlockBlobClient.uploadStream (/opt/app/node_modules/@azure/storage-blob/dist/index.js:21423:13)
    at async AzureBlobStore.uploadStream (/opt/app/node_modules/@textlcm/storage/lib/services/blobStore/providers/azureBlobStore.js:161:51)

@josendf-basetis thanks for reporting! Can you share which mechanism you are using for authentication?

Hi @xirzec,
We are creating the blob service client from a connection string:
const client = BlobServiceClient.fromConnectionString(connectionString);

@josendf-basetis thanks! Can I assume you are using account key authentication with the connection string rather than a SAS token?

Hi @xirzec
We are using account key authentication, connection string provided by Azure Portal.

Hi @EmmaZhu
Thank you for reviewing this issue, following details of a new request, storage account in Azure West Europe region:
2022-11-04T13:43:43.062Z error RestError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:f6898d33-801e-0013-4353-f0655b000000
Time:2022-11-04T13:43:43.3544795Z
{
  "error": "RestError",
  "message": "Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:f6898d33-801e-0013-4353-f0655b000000\nTime:2022-11-04T13:43:43.3544795Z",
  "stack": [
    "RestError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.",
    "RequestId:f6898d33-801e-0013-4353-f0655b000000",
    "Time:2022-11-04T13:43:43.3544795Z",
    "    at handleErrorResponse (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\core-http\\dist\\index.js:3161:19)",
    "    at D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\core-http\\dist\\index.js:3097:49",
    "    at runMicrotasks (<anonymous>)",
    "    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
    "    at async StorageRetryPolicy.attemptSendRequest (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\storage-blob\\dist\\index.js:14673:24)",
    "    at async TracingPolicy.sendRequest (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\core-http\\dist\\index.js:4596:30)",
    "    at async StorageClientContext.sendOperationRequest (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\core-http\\dist\\index.js:4934:31)",
    "    at async BlockBlobClient.stageBlock (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\storage-blob\\dist\\index.js:21038:20)",
    "    at async BufferScheduler.outgoingHandler (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\storage-blob\\dist\\index.js:21407:17)",
    "    at async BlockBlobClient.uploadStream (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@azure\\storage-blob\\dist\\index.js:21423:13)",
    "    at async AzureBlobStore.uploadStream (D:\\Development\\ec\\services\\ec-tasks-service\\node_modules\\@textlcm\\storage\\lib\\services\\blobStore\\providers\\azureBlobStore.js:161:51)"
  ]
}

@josendf-basetis
The request failed due to "signature not match" error, which means: 1. Key used by client side is different with the one used on service. 2. string to sign used by client side is different with the one on service side.
I compared "string to sign" on service side with one generated from the SDK, they match well.
From the account's info, I didn't see key rotation either.
I have no clue why this happened for now.
If it's OK, could you help to repro the issue again?  You mentioned that it can succeed with other files. Can you share request id for a succeeded sample and for a failed sample to me?  Maybe I can locate the root cause with comparing difference between them.
Thanks
Emma

Hi @EmmaZhu,
Below you will find a ZIP file containing two directories: v12.11.0/ and v12.12.0/, each directory  contains the requests for a batch of 406 files, same OS, client software and identical files, only replacing the @azure/storage-blob package version. The v12.11 version succeeds to upload the files, the v12.12 version fails at file number 399.
storage-blob-issue-23647.zip
Thank you for your help

Hi @josendf-basetis ,
Apologize for the late response.  I took a long vacation.
I still cannot find any clue about how the error happens. The StringToSign used in the SDK seems totally the same with the one used in Service side.
Is this error always reported on the same file? If so, could you help to do a test:

Send request to the blob with old version blob package with an invalid account key and send me the request id.
Sent request to the blob with the latest version @azure/storage-blob with a valid account key and send me the request id.

I try to compare difference between these two requests.
Thanks a lot for helping me with the investigation.

Hi @EmmaZhu ,
Finally we were able to track the cause of the issue, we are uploading the content of a ZIP file, streaming directly from the unzipper, and the files at the ZIP root directory are named as: ./some-file, having the ./ prefix.
The @azure/storage-blob package v12.11.0 url-encodes the / characters in the blob name, but the v12.12.0 doe not, this seems to produce the signature mismatch. We have solved the issue trimming the ./ prefix from the file name.
The v12.11.0 service request URL encoding:
https://examples.blob.core.windows.net/example/azure-storage-issue-23647%2F1670759117513%2F.%2Fexample.txt?comp=blocklist

The v12.12.0 service request URL encoding:
https://examples.blob.core.windows.net/example/azure-storage-issue-23647/1670759047009/./example.txt?comp=REDACTED&blockid=REDACTED

Below is a test script to reproduce the issue:
import {
    BlobServiceClient,
    BlobUploadCommonResponse,
} from "@azure/storage-blob";
import { createReadStream } from "node:fs";

interface UploadFileOptions {
    connectionString: string;
    containerName: string;
    blobName: string;
    blobContentType: string;
    blobCacheControl: string;
}

async function uploadFile(
    options: UploadFileOptions,
    sourcePath: string
): Promise<BlobUploadCommonResponse> {
    const {
        connectionString,
        containerName,
        blobName,
        blobContentType,
        blobCacheControl,
    } = options;

    const service = BlobServiceClient.fromConnectionString(connectionString);

    const container = service.getContainerClient(containerName);

    const blob = container.getBlockBlobClient(blobName);

    const sourceStream = createReadStream(sourcePath);

    const response = await blob.uploadStream(
        sourceStream,
        undefined,
        undefined,
        {
            blobHTTPHeaders: {
                blobContentType,
                blobCacheControl,
            },
        }
    );

    return response;
}

async function main(): Promise<void> {
    const connectionString = process.env["AZ_STORAGE_CONNECTION_STRING"] ?? "";

    const containerName = process.env["AZ_STORAGE_CONTAINER_NAME"] ?? "";

    const sourcePath = "test/fixture/example.txt";

    const blobPrefix = `azure-storage-issue-23647/${Date.now()}`;

    //
    // Note that the filename has a `./` prefix.
    // Works in v12.11.0 but produces a signature mismatch in v12.12.0
    //
    const blobName = `${blobPrefix}/./example.txt`;

    const blobContentType = "text/plain";

    const blobCacheControl = "public, max-age=2592000, immutable";

    const uploadOptions: UploadFileOptions = {
        connectionString,
        containerName,
        blobName,
        blobContentType,
        blobCacheControl,
    };

    await uploadFile(uploadOptions, sourcePath);
}

main().catch(function (err) {
    console.error(err);
    process.exit(1);
});

@josendf-basetis thanks for the detailed repro! For what it's worth I tried it against my changes in #24141 and it seems to be resolved there.

Good news @xirzec ,
Thank you for your help.

hello can you help me with the error
The server could not authenticate the request. Make sure the authorization header value is correctly formed, including the signature.
I am using this connection
const blobClientService = BlobServiceClient.fromConnectionString(process.env.BLOB.CONNECTION);
const customerContainer =
blobClientService.getContainerClient('name');
const blobClient = containerClient.getBlockBlobClient('NAME');

Was this resolved? I am getting the same problem in the Document translator.

The cause may not just be limited to the encoded blob names. I am able to reproduce this issue by tweaking the permissions. For example, for a blob with this name "Screenshot 2021-07-28 133744.png", the signature will work if the permission is "racwdxy" (without tags) but it won't work if the permissions is "racwdxty" (with tags).

Fixed here: https://github.com/Azure/azure-sdk-for-js/pull/24953
Latest<EMAIL_ADDRESS>has already included this fix.

Iam also facing the same issue in @azure/storage-blob npm package on version 12.17.0. Once i deleted and added the details again is working.
I tried to list the containers to check the whether azure connections is establishing or not.

@SowmiyaJeevanandham
Hello I figured out the issue on my end maybe this will help you:
In the following code I had to explicitly pass my credential when trying to GET from azure, I didnt have to pass the credentiall when uploading.
`
/**

Streams a file from Azure Blob Storage to the client in chunks using a callback function

@param {string} blobUrl - the URL of the blob in Azure Blob Storage

@param {function} callback - the callback function to handle each chunk of data

@returns {Promise} - a promise that resolves when the streaming is complete
*/
export function StreamFileFromAzure(blobUrl: string, callback: (chunk: Buffer) => void): Promise {
return new Promise(async (resolve) => {
if (!BlobServiceClient) throw new Error("Azure Blob Storage Controller not initialized");
const blobClient = new Azure.BlockBlobClient(blobUrl, BlobServiceClient.credential);
try {
const response = await blobClient.download(0);
const readableStream = response.readableStreamBody;
if (!readableStream) throw new Error('readable stream was null');
     readableStream.on("data", (chunk) => {
         callback(chunk);
     });

     readableStream.on("end", () => {
         // Stream has ended
         // You can perform any cleanup or additional operations here
         return resolve();
     });

     readableStream.on("error", (error) => {
         // Handle stream errors
         console.error("Stream error:", error);
         return resolve();
     });

 } catch (error) {
     if (error instanceof Error) {
         console.error("Error streaming file from Azure:", error);
         return resolve();
     }
 }

})

}`


Support nested properties with the 'get' expression

Fixes #15505.
This pull request implements what was discussed in #15505, and thus is a breaking change. I did not remove the type hints or replace them with type assertions. Instead, I introduced a 'type' call expression. That was quite a minimal change. Given the fact that it is not necessary to use these type hints a lot, I think this is a good solution.
Examples for the new features:
Nested properties:
{
  context: {properties: {deeply: {nested: {property: 42}}}},
  expression: ['get', 'deeply', 'nested', 'property'],
  expected: 42
}

Nested properties with array:
{
  context: {properties: {answer: {at: {index: [17, 42]}}},
  expression: ['get', 'answer', 'at', 'index', 1],
  expected: 42
}

Type hint:
{
  context: {properties: {fill: [0, 255, 0]}},
  expression: ['to-string', ['get', 'fill', ['type', 'color']]],
  expected: 'rgba(0,255,0,1)'
}

Nice!  Thanks for taking this on.
I realize this may sound stubborn, but I’d like to see a real example where type hints are needed before introducing another operator that we might just remove later.  By “a real example” I mean an expression used in a style object.

@tschaub Actually it's the ambiguity of size and scale properties, which use NumberType | NumberArrayType as type. This could be fixed by introducing a SizeType. Once we have that, we can get rid of the type hints. I'm going to create a separate pull request with that change.

@tschaub Actually it's the ambiguity of size and scale properties, which use NumberType | NumberArrayType as type. This could be fixed by introducing a SizeType. Once we have that, we can get rid of the type hints. I'm going to create a separate pull request with that change.

See #15818.

Tests will pass once #15818 is merged and this pull request is rebased.

Rebased again, without a merge commit. Thanks for any review.

Thanks for your work on this, @ahocevar.  It feels like a simpler and more functional solution now (together with #15818).
Hope my comment above makes sense.  Since we aren't currently using the parsing context properties set on the Cavnas side, it doesn't break anything to leave the code as you have it.  So feel free to merge with or without the suggested change.

Thanks for the thorough review, @tschaub. Your suggested change sounds interesting. I think it makes sense as a follow-up change at a later time, and maybe it also makes sense to solve the problems a dot delimited properties path would impose before doing that.

Great feature for flat styles, thanks for this!

I think it makes sense as a follow-up change at a later time

I'm working on support for accessing nested properties in WebGL here: https://github.com/openlayers/openlayers/compare/main...tschaub:openlayers:nested (this reworks the property handling as described above).


Benchmark +fp against mainline 5.1.1

The new post about frame pointers by Brendan Gregg  has made me curious about how the +fp variant of the compiler performs against mainline.
Notes for reviewer:

Should it be 5.1.1 or 5.1.1+stable?
I'm unsure, at this point, what the expiry should be. Feedback is very welcome.

Motivated by,
https://twitter.com/kc_srk/status/1769340923685810402

Notes for reviewer:
1. Should it be `5.1.1` or `5.1.1+stable`?

You can use 5.1.1+trunk.

2. I'm unsure, at this point, what the expiry should be. Feedback is very welcome.

You set the expiry to be a few days from now, just to capture a snapshot of the benchmarks.
We currently run benchmarks against the latest OCaml trunk every night, and configure benchmarks against specific branches for short periods, just to collect some snapshots of the data.


Exceptions in resolver are caught (no traceback)

Version:
GraphQL-core-next 1.0.1
Python 3.7.1
Current Behaviour:
Exceptions in resolvers are caught by default and the traceback of the original_error doesn't help debugging.
Traceback output of log.exception('', exc_info=result.errors[0].original_error)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/graphql/execution/execute.py", line 664, in complete_value_catching_error
    return_type, field_nodes, info, path, result
  File "/usr/local/lib/python3.6/site-packages/graphql/execution/execute.py", line 731, in complete_value
    raise result
graphql.error.graphql_error.GraphQLError: My exception

Expected Behaviour:

There is a flag raise_exception which switches if exceptions are caught or if they are raised out of the library resolver.

The traceback of the original_errors is meaningful: leads to the source line where the exception  occurred.

@ceelian - thanks for the feedback. Changeset e0bb443 should improve this.
Here is an example how you can get error tracebacks now:
from traceback import print_tb
from graphql import *

def bad_resolver(obj, info):
    raise RuntimeError("the runtime error")

schema = GraphQLSchema(query=GraphQLObjectType(
    name='RootQueryType',
    fields={'bad': GraphQLField(GraphQLString, resolve=bad_resolver)}))

error = graphql_sync(schema, '{ bad }').errors[0]
original_error = error.original_error

print(error)
print_tb(error.__traceback__)

Output:
the runtime error

GraphQL request (1:3)
1: { bad }
     ^

  File ".../execute.py", line 664, in complete_value_catching_error
    return_type, field_nodes, info, path, result
  File ".../execute.py", line 731, in complete_value
    raise result
  File "../\execute.py", line 619, in resolve_field_value_or_error
    result = resolve_fn(source, info, **args)
  File "test_graphql.py", line 7, in bad_resolver
    raise RuntimeError("the runtime error")

Will release an upate with this improvement soon.

Great, thank you for the improvement!

Thank you, I was affected by this too.

Version 1.0,2 with this fix has now been released on PyPI.


Implement view manifest pod action

Is your feature request related to a problem? Please describe.
Similar to #538.
PodTable showing pods for Deployments, DaemonSets and StatefulSets has a column called actions with a menu. When expanded, the menu currently only has a button for checking pod logs.

The scope of the issue is to implement the View Manifest functionality and add a button to the same menu. When the button is clicked, a Modal showing the pods manifest needs to pop up.
You can fetch pod manifest from the /manifest backend endpoint.
You can check for a similar implementation of the View Manifest button in the module details page.

Hello, could you kindly assign the issue to me?

Hey @knav33n, sure

@petar-cvit Should I include a copy button and add managed fields checkbox similar to those in the Deployment Manifest Modal?

@knav33n yup, you can include that as well!

@petar-cvit I've raised a pull request to address this issue.


support ngChange

i use this range selection for a table filter, ng change is really in need!

Looks like this repo no longer exists.


Fix wide grants for gitlab auth

Description
I noticed that DD requires some wide grants than it was needed. This PR is solving this problem.
I remove grants email, profile - this providing grant "openid". Grant "api" is replace to read_repository

Good idea. Adding to the docs would be a good idea too so people are not quite as surprised if their SSO no longer works. In the upgrade notes would prolly be best

@Maffooch agree with you. If developer enable gitlab_auto_import_project setting this functional will not working. I will write it in documentation.

@Maffooch  done!


Adds HTTPS server options to the internal web server.

Run make -C test/ssl/ generate to generate self-signed certificate.

/botio test

Trivial


Interactive Tutorial - Creating a Cluster

Getting Connection error using Interactive tutorial
kubectl cluster-info dump
The connection to the server localhost:8080 was refused - did you specify the right host or port?

Hi @BrettPhilip
I think this issue is about the page at https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-interactive/
Are those symptoms still occurring?
/priority awaiting-more-evidence


ADBDEV-5498: Add filter to prohibit using encoding for columns at foreign tables

Add filter to prohibit using encoding for columns at foreign tables
Setting ENCODING for columns of foreign tables is useless, but not forbidden
by grammar, so it was silently ignored. This patch adds an error message when
the user tries to explicitly specify the encoding for a foreign table.
We also make sure that GUCs like default_table_access_method and
gp_default_storage_options are correctly ignored for foreign tables.
The test is taken from the original commit, with slight modifications to
account for GPDB 7 differences.
(cherry picked from commit bb5a6b85b52161d40aa081dc2c5ec9561eabdb65)

Maybe change this

Add filter to prohibit using encoding for columns at foreign tables (#565)

to
Add filter to prohibit using encoding for columns at foreign tables (#1139)

to avoid confusion?

The changes in the code are significantly different from the original commit, but this is not stated anywhere!

Okay, I'll add a note about this


Can [Feature Request] add a tree form component

Existing Component
No
Component Name
treeGrid
Description
类似于easyUI的treeGrid

Sorry, we have no plan to support this.


Twitter::Error::NotFound: Sorry, that page does not exist when client.follow('cnn')

This is an authenticated client. Calling client.user, client.followers, client.update(msg), etc. works. But every time I call client.follow(any_account_name) I receive the error: Twitter::Error::NotFound: Sorry, that page does not exist

@greenmindPDX's tip on string -> integer was the key for me. Here's the implementation.

I just solved a similar problem with hitting the unfollow endpoint by changing my params from string to integer.


Merge meta and additional fields of logger

Closes https://github.com/api3dao/airnode/issues/1181
I also noticed that we stopped including some metadata in the logger in https://github.com/api3dao/airnode/pull/1341 so I added it back.
TODO:

[x] - I want to test this on AWS to see if the logs work as expected.

See that in the first screenshot only the coordinator ID is set, in the second one all of coordinator, chain ID and provider name are set.


.sbtopts file ignored under Windows?

Options set in .sbtopts don't seem to be used under Windows. Looking at https://github.com/sbt/sbt-launcher-package/blob/master/src/universal/bin/sbt.bat, the file isn't accessed.

I can relate that too. I'm on Windows and .sbtopts is ignored, on OSX and Linux it's ok.

Since this doesn't work on Windows out of the box, you either need to configure local sbt.bat batch files which inject SBT_OPTS or perhaps use the global SBT config (unless you have project-specific configuration).
I just ran into a problem of having to configure project specific truststores, so... any idea if this is ever going to be worked on? Much obliged!


ARROW-14958: [C++][Python][FlightRPC] Implement Flight middleware for OpenTelemetry propagation

11920

 Quality gate: Passed
3 Issues

 3 Code issues

Details

 Quality gate: Failed
2 Issues

 2 Code issues

Details

 Quality gate: Failed
 Everything looks good
Details


ipmag.zeq_magic

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
zeq_magic was intended as a quick and dirty tool to look at data quickly.  it now looks for the current data model and is very slow when the internet is slow.  There should be a way to suppress that behavior and use the cached version.  Can we have a flag offline=True to do that?  I'm not sure where that is accomplished.
Additional context
Add any other context or screenshots about the feature request here.

@ltauxe, we have a few options in that regard.  Right now I have a timeout parameter set to 3 seconds.  We could set that lower -- in my experiments, with my normal/fast internet I get the cached data model about half of the time if I set the timeout to .23 seconds, and all of the time if I set the timeout to .25 seconds.  So you could try setting the timeout to .5 or something and see if that makes the time acceptably fast.
If you are willing to do a little experimenting with this, the two places where those timeouts are set:
pmagpy/controlled_vocabularies3.py   line 79:
req = requests.get(url, timeout=3)

and
pmagpy/data_model3.py   line 87:
req = requests.get("https://earthref.org/MagIC/data-models/3.0.json", timeout=3)

And/or, I could create an "offline" constant in pmag_env/set_env.py that a user could set if they are in a perpetually bad internet situation, like on a boat.
I prefer having a constant than setting an 'offline' command-line flag for lots of different programs, but I could do that too if it seems best.
And of course, if you have the internet fully "off" everything should be fast.
Let me know which way you want to try!

i’ll try setting the timeout but i think an offline flag for this one program makes sense.  i use it a lot and it is annoying that it has gotten to be so fussy.  also the axes got changed at some point and now they look funny when the data do cross the axes = see issue in git hub.  i dont know when that happened, but if it could be fixed along with the timeout then all would be right with the world.  :)
except i’m tracking down a bad bug in the instrument software that is sometimes giving us mirror images of the direction we want (wrong y!).  very weird.
lisa

On Apr 6, 2019, at 11:20 AM, Lori J<EMAIL_ADDRESS>wrote:
@ltauxe https://github.com/ltauxe, we have a few options in that regard. Right now I have a timeout parameter set to 3 seconds. We could set that lower -- in my experiments, with my normal/fast internet I get the cached data model about half of the time if I set the timeout to .23 seconds, and all of the time if I set the timeout to .25 seconds. So you could try setting the timeout to .5 or something and see if that makes the time acceptably fast.
If you are willing to do a little experimenting with this, the two places where those timeouts are set:
pmagpy/controlled_vocabularies3.py line 79:
req = requests.get(url, timeout=3)
and
pmagpy/data_model3.py line 87:
req = requests.get("https://earthref.org/MagIC/data-models/3.0.json", timeout=3)
And/or, I could create an "offline" constant in pmag_env/set_env.py that a user could set if they are in a perpetually bad internet situation, like on a boat.
I prefer having a constant than setting an 'offline' command-line flag for lots of different programs, but I could do that too if it seems best.
And of course, if you have the internet fully "off" everything should be fast.
Let me know which way you want to try!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/556#issuecomment-480526115, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDd01saotaoauqkPprksZdK6TAS21ks5veOVYgaJpZM4cgI5y.

I ended up adopting your first suggestion with timeout = .2  and there is an acceptable delay after which it just moves on.  It is fine.
lisa

On Apr 6, 2019, at 11:20 AM, Lori J<EMAIL_ADDRESS>wrote:
@ltauxe https://github.com/ltauxe, we have a few options in that regard. Right now I have a timeout parameter set to 3 seconds. We could set that lower -- in my experiments, with my normal/fast internet I get the cached data model about half of the time if I set the timeout to .23 seconds, and all of the time if I set the timeout to .25 seconds. So you could try setting the timeout to .5 or something and see if that makes the time acceptably fast.
If you are willing to do a little experimenting with this, the two places where those timeouts are set:
pmagpy/controlled_vocabularies3.py line 79:
req = requests.get(url, timeout=3)
and
pmagpy/data_model3.py line 87:
req = requests.get("https://earthref.org/MagIC/data-models/3.0.json", timeout=3)
And/or, I could create an "offline" constant in pmag_env/set_env.py that a user could set if they are in a perpetually bad internet situation, like on a boat.
I prefer having a constant than setting an 'offline' command-line flag for lots of different programs, but I could do that too if it seems best.
And of course, if you have the internet fully "off" everything should be fast.
Let me know which way you want to try!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/556#issuecomment-480526115, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDd01saotaoauqkPprksZdK6TAS21ks5veOVYgaJpZM4cgI5y.

I added the OFFLINE flag as well, since it was easy to do.  Just go into pmag_env/set_env.py and change OFFLINE to True.  It will apply globally, not just in zeq_magic.

Seems like this issue is done, and I will have a look at https://github.com/PmagPy/PmagPy/issues/554 with the truncated axes?

I think i fixed this - but one of the axes is still too short sometimes….
lisa

On Apr 8, 2019, at 2:02 PM, Lori J<EMAIL_ADDRESS>wrote:
Seems like this issue is done, and I will have a look at #554 https://github.com/PmagPy/PmagPy/issues/554 with the truncated axes?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/556#issuecomment-481006685, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDccqj78cYuoLwNOecri9znQMl-Lxks5ve65PgaJpZM4cgI5y.

ok.  thanks
lisa

On Apr 8, 2019, at 2:01 PM, Lori J<EMAIL_ADDRESS>wrote:
I added the OFFLINE flag as well, since it was easy to do. Just go into pmag_env/set_env.py and change OFFLINE to True. It will apply globally, not just in zeq_magic.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/PmagPy/PmagPy/issues/556#issuecomment-481006389, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKrDTJtQEq9FVnCyIkNqU7Ct9m5zajkks5ve64YgaJpZM4cgI5y.


Test explorer will be broken when the test item's id is the same as the test controller

Does this issue occur when all extensions are disabled?: Yes/No

Version: 1.71.0 (user setup)
Commit: 784b0177c56c607789f9638da7b6bf3230d47a8c
Date: 2022-09-01T07:36:10.600Z
Electron: 19.0.12
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22000
Sandboxed: No
When the test item's id is the same as the test controller, the test explorer will be broken.
I was thinking that test controller id should be unique among all other test controllers and did not realize the test items are also needed to be taken into consideration. Maybe adding more description in the document?
Or is it ok to allow id duplication among test item and test controller.

Confirmed fixed in
Version: 1.73.0-insider (user setup)
Commit: fad3a77833b9249158dfd88477114a06435e46a2
Date: 2022-10-07T05:18:17.049Z
Electron: 19.0.17
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: Yes
/verified


Update understanding_metric_types.md

Page markdown 'title' to match the link on the left (title). Not all pages do this, but most do and where they don't they're usually some extension of the link one the left (e.g. Visualization/Grafana -> "GRAFANA SUPPORT FOR PROMETHEUS")
Add link to the other page that documents the same information.

The two pages perhaps could be combined and conflated, but as a less review-heavy discoverability step, just linking from the later to the former.
Chose to link from this to the other, due to the other being more recently updated.
Specifically discovered this problem given this page did not link to the "choosing Summaries vs Histograms" page
Signed-off-by: andrewa-stripe<EMAIL_ADDRESS>I believe this falls under prometheus for the "Which Maintainers", so @roidelapluie & @LeviHarrison

@roidelapluie & @LeviHarrison


Support for Custom User Model added

This is my first contribution to the open-source community. Please give me your useful feedbacks to make my code better.

In this pull request, I added some new codes in order to make the library work with other fields rather than username for authentication.

Hi @mongkok
This is my first contribution to the open-source.
Notes:
1- I couldn't run the tests. I used pytest to run unit tests, but due to not having SECRET_KEY, I faced some errors.
How can I prepare a secret key in order to fix this issue?


IQueryProviderWithIncludes.Include doesn't populate Description column in the referenced Poco

In our program we have 2 classes. One is called Order, which contains a field called "Description". The other one is called Ticket, which contains a reference to Order. They look like this:
    [TableName("Tickets"), PrimaryKey("TID", AutoIncrement = true, UseOutputClause = true)]
    public class Ticket
    {
        public Guid TID { get; set; }
        public Guid? OID { get; set; }
        [ComputedColumn]
        [Reference(ReferenceType.OneToOne, ColumnName = "OID", ReferenceMemberName = "OID")]
        public Order Order { get; set; }
    }

    [TableName("Orders"), PrimaryKey("OID", AutoIncrement = true, UseOutputClause = true)]
    public class Order
    {
        public Guid OID { get; set; }
        public string Description { get; set; }
    }

We have recently switched our code from running raw SQL to using Linq provided by Database.Query(). However, when executing the following code, y.Order.Description is null, while x.Description is properly populated:
                var x = db.Query<Order>()
                          .Where(i => i.OID == new Guid("3686a4b6-75a6-e911-8e46-5cc5d488af58"))
                          .First();
                var y = db.Query<Ticket>()
                          .Include(i => i.Order)
                          .Where(i => i.TID == new Guid("8440F7B5-F1A6-E911-8100-005056833617"))
                          .First();

Other fields in y.Order are having the correct value. We are still investigating this issue.
Edit: We are using NPoco 4.0.1 with Sql Server 2014.

NPoco generated SQL for y:
SELECT [T].[TID] as [TID], [T].[OID] as [OID], [O3].[OID] as [Order__OID], [O3].[Description] as [Order__Description] FROM [Tickets] [T]
  LEFT JOIN [Orders] [O3] ON [T].[OID] = [O3].[OID]
WHERE ([T].[TID] = @0)

I built a debug build of NPoco Nuget package, however that only get me to QueryProvider.ExecuteQuery() and I cannot step into Database.QueryImp(). If anyone can help me on this that would be great.

I'm running into the same problem. If I end up figuring it out I'll leave an update.

Can't reproduce this. Either the data is null in the description field or something else isn't quite right.
https://github.com/schotime/NPoco/commit/f5e3d745ae026e81b373f7f57a5af3cacab9a0ac#diff-98cd766e139224fa87103526d509dbdb shows your example that works.


Webpack minification problem

Hi thanks for your library, but I have a problem with minification using webpack it says:
ERROR in main.2465bbd007eb4bee4c62.js from UglifyJs
Unexpected token: name (helper) [main.2465bbd007eb4bee4c62.js:67980,12]

I tried these imports but it gives error:
require('jconfirm/jConfirm') or
import 'jconfirm/jConfirm'

Do you know how to use it with webpack. Can you help me?

I haven't used jConfirm with webpack yet, but I believe it would be
const jconfirm = require('jconfirm');
Try that and let me know!

Just tried now it says:
ERROR in main.307931e5f4784ae190b0.js from UglifyJs
Unexpected token: name (d) [main.307931e5f4784ae190b0.js:67953,288]

I just remembered I never really set this up to work correctly with webpack. It's hacktoberfest...do you want to submit a pull request modifying it? You'll just need to create a new entry point. If not, I'll get to it, but probably not today. I suggest you just include the plugin as a script tag if you can't wait.

Changing webpack version form 3 to 4 solved this problem.

Interesting, thanks for letting me know


HackAUBG Sponsors Section

This issue requires an onClick function for redirecting to the particular sponsor's main website landing page.

@DemonHashira
FYI, this issue already has a branch created. If you want to check changes and work on this issue, please checkout to the #182-Sponsors-Section branch


Execution of sequenced shared object certs may race, leading to unnecessary errors

Here's the situation:

Suppose consensus adapter has timed out for some number of certs, but those certs have actually made it into consensus.
A validator will then receive those certs from consensus, sequence them, and put them into the execution driver queue.
Execution driver wakes up, and attempts to execute all certs in its queue by sending them to NodeSyncState.
The pending queue is ordered by insert time, so it is guaranteed to preserve the ordering of certs as they arrived from consensus (assuming that none of those certs arrived via some other channel in the meantime).
However, NodeSyncState parallelizes cert execution, and it is unaware of dependencies among its inputs. Given certs A and B which are sequenced in the order A, B, even though we send A to NodeSyncState first, it will race with B, and B may fail before A is finished, even though it would have completed successfully if A had been executed first.
This leads to NodeSyncState deciding that parent certs are missing, which causes it to ask other validators to provide it with the cert's effects. Those other validators may be in the same situation.

Some additional observations:

If a cert was enqueued for execution by the consensus task, then all the shared object dependencies of that cert must have been enqueued previously. If this was the only way that certs were enqueued, then we could be assured that all shared object parents of a cert in the queue must precede it in the queue, or have been previously executed.
However, certs may also be enqueued by gossip which doesn't provide this guarantee. (Although gossip transmits digests in causal order, validators randomly shuffle their gossip peers, and so may receive a cert via gossip without first receiving that certs parents).
Additionally, a shared object cert that arrives via consensus may have missing owned object parent certs (e.g. for the gas object), which themselves do not have any shared inputs, and hence will not have arrived via consensus. While the 2f+1 signers of the cert must have executed the missing parent certs, they may not yet be able to execute the cert itself if they have not sequenced it, hence they will not be able to provide TransactionEffects, which is how we normally obtain parent information.
Because of the above, and also because parallel execution of the pending list will be required to achieve desired throughput, we should treat the execution driver queue as unordered. It's not reliable, and even if it were, it would not be useful since we can't serialize execution of the whole queue.

Proposed solution:
While the execution driver / NodeSyncState lacks explicit parent information (because it does not have TransactionEffects), we can compute a partial ordering of the certs in the queue, as follows:

Two certs with a shared input in common can be ordered based on the local shared locks. If shared locks are missing the cert is not executable anyway.
Two certs with an owned input in common can be order based on the version numbers in the inputs.

Note that this is a "partial partial ordering" - we can't detect the case where a cert depends on a newly-created output of some other cert, because that output will not be known until execution, so there may be dependency information not captured here.
Nevertheless, computing this partial order and then executing certs accordingly should greatly increase the throughput of sync, and reduce the frequency with which we call execute_cert_to_true_effects to discover missing parents. But, note that we cannot eliminate the need for that function entirely. It is always possible to receive a cert that has missing owned object parents.

Also IIUC when a validator signs an owned txn the validator must have all parent objs locally.I wonder what happens when the validator does not have the parent objs locally. This also has some parallel to receiving txn from consensus.

In the new world of owned certificate sync via Narwhal, we no longer have a need for execute_cert_to_true_effects(), right?

In the new world of owned certificate sync via Narwhal, we no longer have a need for execute_cert_to_true_effects(), right?

Can you explain a bit why?

Even before the "owned objects via narwhal" change, I did not have the context about the trade off between executing certificates on other nodes for effect, vs waiting for gossip and checkout to catch up on the missing certificates. I guess the decision comes down to latency.
@andll in the new world of propagating certificates via narwhal, if two validator nodes are running "normally", what is the bound on how long a certificate committed in one node can remaining missing on another node?

Other node will see certificate when narwhal is committed. If everything is normal then it is around ~commit latency. But of course if node have network connection problems or do not receive narwhal commits for other reasons there is no upper bound


NIFI-7842 Return Lists when multiple records are returned to the restlookupservice

Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here: Noticed when trying to query a rest service for a list of items, only the first record was attached in place and the other records were ignored. This pull request will return the same single record if only a single record is found or a list if multiple are returned. This does change the restlookupservice from implementing the recordlookupservice to a lookupservice. This works well with the LookupRecord processor since it will fall back on inferring the type in the chance it is not a record.
Description of PR
fixes bug NIFI-7842
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically main)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on JDK 8?
[x] Have you verified that the full build is successful on JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible.

+1 Looks good to me.
Commits need to be squashed. Is there concern about backwards compatibility? I'd say the previous implementation was a bug due to how it could handle an array response but would just pop the result, but this may break some people using the current functionality since it is a change to that.

I don't think we can accept this change into the codebase. It has the potential to break a lot of things. Any component that previously declared a property to reference a RecordLookupService would no longer be able to use this service, and that component would become invalid. And any component that declared a property as a LookupService that previously worked as expected could start to behave very differently. Consider that a REST service previously had a response that had 3 records, and it just returned the first one. Now it returns 3. That can result in all kinds of problems in the users' data.
However, I can understand the desire to support returning multiple Records. So we could either have a separate implementation (with shared, common code), or we could add a property that determines what to do if multiple records are returned (defaulting to return the first record in order to maintain backward compatibility). Adding the property comes with 2 downsides, though. (1) It means that the Controller Service still would no longer be a RecordLookupService but just a LookupService<List<Record>>. This is probably ok, though. I don't think we even have anywhere in the codebase where we expect a RecordLookupService explicitly to be configured. (2) The default really should be to return multiple records, not to return the first and ignore the rest. But defaulting to that would change the default behavior in a very unexpected way for some users.

I don't think we can accept this change into the codebase. It has the potential to break a lot of things. Any component that previously declared a property to reference a RecordLookupService would no longer be able to use this service, and that component would become invalid. And any component that declared a property as a LookupService that previously worked as expected could start to behave very differently. Consider that a REST service previously had a response that had 3 records, and it just returned the first one. Now it returns 3. That can result in all kinds of problems in the users' data.
However, I can understand the desire to support returning multiple Records. So we could either have a separate implementation (with shared, common code), or we could add a property that determines what to do if multiple records are returned (defaulting to return the first record in order to maintain backward compatibility). Adding the property comes with 2 downsides, though. (1) It means that the Controller Service still would no longer be a RecordLookupService but just a LookupService<List<Record>>. This is probably ok, though. I don't think we even have anywhere in the codebase where we expect a RecordLookupService explicitly to be configured. (2) The default really should be to return multiple records, not to return the first and ignore the rest. But defaulting to that would change the default behavior in a very unexpected way for some users.

Yeah, I thought about the default current behavior. It just seems "wrong" or not intuitive to return the first element in the array. The only processor that I found that used this service was LookupRecord and it fully handles the change in response and puts it more inline with the other lookup services. I do agree a simple boolean to keep the current default behavior would be a good in-between solution

Yeah, I thought about the default current behavior. It just seems "wrong" or not intuitive to return the first element in the array. The only processor that I found that used this service was LookupRecord and it fully handles the change in response and puts it more inline with the other lookup services. I do agree a simple boolean to keep the current default behavior would be a good in-between solution

Or as you said, shared code and create a RestObjectLookupService

Or as you said, shared code and create a RestObjectLookupService


Ensure we don't break when cooloff time is 0 ('permanent' lock)

Coverage increased (+0.56%) when pulling 70fa41ff0de412f0125dd0c23070fdbc5a234c29 on shin-:fix_cooloff_0 into 20d2f9652d4b30a4a17b45925389581906e60685 on kencochrane:master.

LGTM, thanks.


Update Code - Proof_Of_ContributionSnapshot5.md

Elaboration of the MRC36.
Processing of the feedback received.
Assistance in 5 cases of weights discrepancy and hack.
Implied weight value calculation and future system.

Thanks for the contributions.
Seems very high for an MRC not yet merged.
Also if the cost of doing weights checks exceeds the entire value of the weights in discussion it isn't efficient.
Let me know if you can come back with a smaller request.

Thanks for the review David.
I have adjusted the weights to a value comparable to that requested by other MRCs submitted in the past before being merged.
It is worth noting that MRC 36 is the second most commented MRC. There are still outstanding issues that I have not been able to address due to time constraints.
I have also halved the weights requested by weight revisions. Just a note, my revisions have identified about 900k recoverable weights. The application is still efficient. On the other hand, responding to requests may not always have a direct return in terms of weights but it does have a direct return in terms of community satisfaction.

Thanks for the quick reply.

Glad to see the engagement. But a long post with lots of comments, replies and opinions is not the same as value accepted or delivered to the community. Otherwise the project will find it self with many lengthy discussions of things that can't be implemented or everyone giving their opinion to receive rewards.

While the analysis proposed that high number of deprecations, further analysis found most of the contributions in question were maintained and thus only a small number were removed and even those now the authors are claiming the were, would have or meant to maintain them. Which created this further work.

This whole process should and must more toward automation as the manual version will not scale.
I'll post a entry for some weights, but most of the weights for MRC36 will need to come when its finalized and delivered.

David, I have read your comments about my PR to Snapshot 5 and I understand your point of view.
However, it is dangerous to use the ambiguity of ‘value delivered’ to the detriment of a contribution made.
Objectively, I can quote you at least 4 MRCs who in the past have asked for a very similar value of around 75,000 USD for the work done in submitting it. We can accept that each MRC is unique and has different values but to give my MRC 10% of the value of others is a comparative tort that I find hard to understand.
I agree that any opinion should not be enough to be considered a contribution but I must remind you that you encouraged me to write this MRC and I understand that my draft MRC was reviewed and published as relevant content. And the 75+ comments of interesting debate and the 2 counter-proposals I think reflect that it is a topic of interest and can bring value to Morpheus.
To consider all this work worth 6500 weights is, with all due respect, ridiculous and offensive. Even more so when in the past we have seen more banal contributions with much higher values.
If these are the expectations, I am seriously concerned. What should I expect going forward? Could you give me concrete details? This uncertainty hardly encourages anyone to contribute.
My enthusiasm and values aligned with the project encouraged me to get involved. But maybe I was wrong after all.


Configuring Log Analytics to send Azure diagnostics - Set-AzDiagnosticSetting

Is it possible to enable only metrics instead of metrics and logs with this cmdlet?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fb75e535-de4f-e730-4dfb-dfc160386f45
Version Independent ID: 6b355634-913c-65e4-62fa-a51cfeff143a
Content: Use PowerShell to Create and Configure a Log Analytics Workspace
Content Source: articles/azure-monitor/platform/powershell-workspace-configuration.md
Service: log-analytics
GitHub Login: @richrundmsft
Microsoft Alias: richrund

@darivuki Thanks for the comment! Yes, it is possible to enable only metrics. Refer to link1 and link2

Ahhh! The trick is to explicitly enable metrics without explicitly enabling logs. Thank you!

@darivuki Happy to help! We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Mention LinkGenerator as an alternative to IUrlHelper.

To generate URL's LinkGenerator' class also can be used, and some of its methods do not require any context contrary to IUrlHelper` which won't be resolved by default.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8ee6ad2f-1667-8d5c-1f36-9d0511e3720f
Version Independent ID: 7838ccec-c8e8-9baa-2b3e-220b2af5fc58
Content: Routing to controller actions in ASP.NET Core
Content Source: aspnetcore/mvc/controllers/routing.md
Product: aspnet-core
Technology: aspnetcore-mvc
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@Rick-Anderson It's covered in the Routing topic, and IUrlHelper uses the Link Generator under-the-covers. Since the context is available in a controller and the Routing topic is already cross-linked here, I don't think this is actionable.
https://docs.microsoft.com/aspnet/core/fundamentals/routing#url-generation

@guardrex  So docs contain 2 sections about URL generation. It would be nice to crosslink them at least.
Google shows me  this one first for search query asp.net core url generation.
And when I have two options with titles:
Routing to controller actions in ASP.NET Core | Microsoft Docs and Routing in ASP.NET Core | Microsoft Docs I will obviously select the former one, because that's exactly what I am interested in: I am trying to find out how to generate absolute url outside controller for hitting particular action.

They are cross-linked. First paragraph in the controllers topic's URL generation section, Routing is linked over ...

See Routing for a detailed description of URL generation.

It's also linked in the introduction (Routing is linked) ...

This document will explain the interactions between MVC and routing, and how typical MVC apps make use of routing features. See Routing for details on advanced routing.

In the Routing topic, Action Results is linked over ...

The API can be used outside of the context of an executing request. MVC's IUrlHelper and scenarios that rely on IUrlHelper, such as Tag Helpers, HTML Helpers, and Action Results, use the link generator to provide link generating capabilities.

WRT the rest, you didn't explain that when you opened the issue. Your request is clear now. At first, it just sounded like you were asking for cross-linking, which is already present.

Thanks for contacting us. We believe that the question you've raised have been answered. If you still feel a need to continue the discussion, feel free to reopen it and add your comments.