Paths as URIs

This PR closes #242 at the data model level - all paths are coerced to absolute URIs (i.e. file:///directory/test.nc or s3://bucket/test.nc) as they go into the Manifest.
As this forbids constructing manifests using relative paths, it requires minor changes to many tests (e.g. test.nc-> /test.nc). It also will require slightly more invasive changes to any tests that involve kerchunk references.

[x] Closes #242
[ ] Tests added
[ ] Tests passing
[x] Full type hint coverage
[ ] Changes are documented in docs/releases.rst
[ ] New functions/methods are listed in api.rst
[ ] New functionality has documentation

What should we do with the paths in kerchunk references? Are they are always meant as absolute? I guess we should assume they are absolute, unless they have ./ or ../ at the start? Would fsspec ever produce relative paths like that?
cc @martindurant

Are they are always meant as absolute?

They are always meant "as interpreted by the target filesystem". The nature of that filesystem might be implied by the protocol of a path alone, but commonly additional arguments are also required. This means, that relative paths do work if the target happens to be the local filesystem (file://), but I think of the other filesystems, only ssh supports this concept at all. I would not expect this to be meaningful for basically any practical case.
Note that the dir:// filesystem adds prefixes to URLs for any filesystem, if that's useful at all.

(I am happy to require absolute paths even if it makes some tests slightly more verbose)

Thanks @martindurant !

The nature of that filesystem might be implied by the protocol of a path alone, but commonly additional arguments are also required.

But is the nature of the filesystem explicitly recorded in the kerchunk references format anywhere? Obviously if the prefix is explicit (e.g. s3://) then you know but otherwise?

that relative paths do work

I guess we should assume they are absolute, unless they have ./ or ../ at the start?

Would this approach work then?

(I am happy to require absolute paths even if it makes some tests slightly more verbose)

This might be helpful if the above approach doesn't work.

is the nature of the filesystem explicitly recorded in the kerchunk references format

No. The original intention was to have these in the "templates", but in practice, the remote_protocol, remote_options and fss arguments to ReferenceFileSystem are used (and often encoded in Intake prescriptions) in cases of ambiguity.

@sharkinsspatial and I discussed this earlier and we think that we need to add some kind of optional fs_root kwarg to specify the location of the root of the fsspec filesystem that the relative paths were created on. (This is the same thing as specifying what to prefix the relative paths with to turn them into absolute paths.)
The user could then do
cwd = str(Path.cwd().as_uri())
vds = open_virtual_dataset('refs.json', reader_options={'fs_root': cwd})

which would prefix any relative paths found to be relative to the current working directly on this system.
If fs_root was not specified and a relative path found then an exception would be raised.

I am happy to require absolute paths

Changing kerchunk upstream actually wouldn't really help as we still want to be able to parse kerchunk references that were created with older versions of kerchunk.

You could try updating that and let me know if that fixes it.

I would expect that to fix it - what I'm wondering about is whether or not existing DMR++ files in the wild generally contain absolute or relative paths.

I have seen both. Sometimes the file contains s3://data.nc but more commonly if the path is s3://data.nc, the file just contains <Dataset xmlns="http://xml.opendap.org/ns/DAP/4.0#" ... name="data.nc">

the file just contains <Dataset xmlns="http://xml.opendap.org/ns/DAP/4.0#" ... name="data.nc">

And this does not tell me that the true path is an S3 path does it?

the file just contains <Dataset xmlns="http://xml.opendap.org/ns/DAP/4.0#" ... name="data.nc">

And this does not tell me that the true path is an S3 path does it?

No, the true path is not known with only the dmrpp file contents

No, the true path is not known with only the dmrpp file contents

Thanks. So I was right - this is equivalent to reading kerchunk json/parquet in that sense.

Okay important question: Are we trying to support manifests with http URLs in them? Right now we have some tests which create virtual datasets containing http:// URLs as the path, e.g:
virtualizarr/tests/test_backend.py::TestReadFromURL::test_read_from_url[netcdf3-https://github.com/pydata/xarray-data/raw/master/air_temperature.nc-HDF5VirtualBackend]

But these tests (cc @scottyhq ) doesn't actually try to read the data back as loadable xarray variables. AFAIK Icechunk could not read data from a manifest containing http URLs (cc @mpiannucci ), but fsspec presumably could?
Handling this case in the manifest validation is possible but a bit annoying as cloudpathlib doesn't support http paths (yet https://github.com/drivendataorg/cloudpathlib/pull/468), and pathlib will incorrectly conclude that the http url is a relative posix path.

AFAIK Icechunk could not read data from a manifest containing http URLs

It's planned for Icechunk to support HTTP apparently.

Handling this case in the manifest validation is possible

I added support for HTTP in https://github.com/zarr-developers/VirtualiZarr/pull/243/commits/fefab909540ad790db811295c38e69f5b8190565

Are we trying to support manifests with http URLs in them? I added support for HTTP in https://github.com/zarr-developers/VirtualiZarr/commit/fefab909540ad790db811295c38e69f5b8190565

If I'm following correctly, I'd say yes! There are lots of datasets out there that are not in cloud buckets, but are on servers that support http range requests. Agreed that it would be nice if cloudpathlib handled http:// paths

This PR now builds upon #318 FYI @ayushnag

I got the DMR++ tests passing, so everything now passes locally (🥳) but I'm not sure if the changes to the DMR++ tests are okay (https://github.com/zarr-developers/VirtualiZarr/pull/243#discussion_r1861180719)...
Also the CI fails because the new dependency on cloudpathlib wants to install S3 dependencies to instantiate S3Path objects, even though I'm not using S3Path to actually communicate with S3, I'm just using it as a convenience wrapper class. That's annoying and I'm not sure what the simplest solution is. The crux of the issue is that in VirtualiZarr a user could in theory want to manipulate S3 URLs in manifests without ever talking to S3. Some options:

Remove dependency on cloudpathlib, which would mean more parsing code needed in this PR,
See if cloudpathlib are amenable to delaying their import checks until S3Path is actually used to communicate with S3,
Change tests to not use S3 paths as examples when botocore is not installed,
Change CI to always have S3 dependencies installed, but that would require users to install S3 dependencies when they actually aren't even trying to talk to S3.

On cloudpathlib, is upath any use? Do you really need a wrapper?

On cloudpathlib, is upath any use?

I think that would have the same problem as cloudpathlib - I don't necessarily want these paths to have any connection to an actual storage layer, I just want to strictly validate and manipulate their forms.

Do you really need a wrapper?

Turns out I didn't need a wrapper, I can get away with just more if...else logic for each of the possible types of paths. It's slightly less neat but it avoids the dependency issues.


npm start doesn't work after upgrade to 0.15.0

npm start stopped working after upgrading to React-Native 0.15.0 yielding:
><EMAIL_ADDRESS>start Dev/tmp/AwesomeProject
> react-native start

Looks like you installed react-native globally, maybe you meant react-native-cli?
To fix the issue, run:
npm uninstall -g react-native
npm install -g react-native-cli

Output of npm -g list --depth=0:
/usr/local/lib
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>
Output of npm list --depth=0:
<EMAIL_ADDRESS>Dev/tmp/AwesomeProject
└──<EMAIL_ADDRESS>
To be noted, using react-native start works as expected, which make me think this is a npm bug but I'm not sure on how to get started.

Related to http://stackoverflow.com/questions/33908314/packager-wont-start

Try adding react-native-cli to your (dev) dependencies.

It works perfectly ! Thanks @radko93

Thanks, this works but is it the expected behavior? Shouldn't react-native-cli be installed globally? Why the change in the 0.15.0?

npm 3 might have broken global lookup of modules from npm run-scripts (I haven't verified... just a guess)

@ide That's what my first thought was, but under the same node/npm configuration it works with React-Native 0.14.0, while it doesn't with the 0.15.0+.
I would be interested in fixing this myself, but I don't know where to begin looking.
Thanks for your answer.

It's not npm 3 case, I've tested it on npm v2.

Ran into the issue, resolved after plenty of head scratching by cleaning node_modules (rm -rf ./node_modules) and then upgrading NPM (npm install npm -g) and install deps in the generated React Native project (npm install). Upgrade clash I guess, didn't have this issue last week.

Solved on Stackoverflow http://stackoverflow.com/questions/33908314/packager-wont-start

I got it to work by running npm install -g react-native-cli again and then closing the terminal window and opening a new one. npm start works now.

:+1:  adding cli as a dependency worked for me for android deploys

Something went fishy with today's build.   I had to:
npm un react-native-cli and npm i react-native-cli
FYI for others.

@GantMan same thing happened to me too. Reinstalling react-native-cli once you update react-native solves the problem as you mentioned.

Adding react-native-cli to dev-deps doesn't solve the issue. What is the real issue here?
"react-native": "0.19.0",
"react-native-cli": "~0.2.0"

$ node -v
v6.1.0
$ npm -v
3.8.6

After npm install,
$ ls -l node_modules/.bin/react-native
lrwxr-xr-x  1 madhu  staff  47 May 17 15:04 node_modules/.bin/react-native -> ../react-native/local-cli/wrong-react-native.js

Ah, nevermind
npm un react-native-cli -D
npm un react-native-cli -g
rm -rf node_modules
npm cache clear
npm i
npm i react-native-cli -D

and then it works. Weird !!


jetbrains.*: 2019.1.3 -> 2019.2 (mostly); jetbrainsjdk: 11.0.2b164 -> 11.0.3b360.2

Motivation for this change
New major release of the IDE suite. I had issue with the older jdk, so I upgraded that also, which resolved it.
Things done

I've tested idea-ultimate, clion, and rider, I do not use the other IDEs.

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

So, compiling jetbrains.webstorm fails, others succeed
builder for '/nix/store/b1vwv58y4x4f80b616b8yn27jx21nj5x-webstorm-2019.2.drv' failed with exit code 1; last 6 log lines:
  unpacking sources
  unpacking source archive /nix/store/hb4ljqxfps1cgwj4shwrla3w9y7agwg0-WebStorm-2019.2.tar.gz
  source root is WebStorm-192.5728.87
  setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file WebStorm-192.5728.87/product-info.json
  patching sources
  rm: cannot remove 'jre64': No such file or directory

jetbrains.webstorm now builds and starts as well

GoLand was updated to 2019.2 meanwhile too.

/cc maintainers @Mic92 @loskutov @miltador @baldo

Updates for datagrip and ruby-mine which were meanwhile released are in #65837. As a further improvement for another PR, I am thinking of removing the jbr directory from all jetbrains tools, not just webstorm. It will save ~130 MB for each package where it is now uselessly present.

Rider 2019.2 is in https://github.com/NixOS/nixpkgs/pull/66426

Obsolete, there is now newer versions in nixpkgs


Update Makefile to handle "replaces" CSV field

Add "replaces" field to CSV and update Makefile's target bundle-update
to update the field with correct value
see: ECOPROJECT-1769

/test 4.14-openshift-e2e

/lgtm

/retest


Unsupported gpu architecture 'compute_60'

Dear all,
I have a problem compiling master branch on centos7 with CUDA version 7.5. I'm using cmake-gui v 3.10 for generating build files. For my environment I'm  using Anaconda3 with python 2.7. I disabled ROCKSDB MPI MOBILE OPENGL and METAL. My CUDA ARCH NAME is 50.
First of all, if I compile old caffe with Manual CUDA ARCH NAME where I delete all names that are beyond 50 I can compile just fine.
The problem for caffe2 is as follows:
No matter what CUDA ARCH NAME I choose I always get
[  7%] Building CXX object third_party/gloo/gloo/CMakeFiles/gloo.dir/transport/tcp/pair.cc.o
Compiling src/libwrap.cu                      > /home/gregork/src/caffe2/third_party/nccl/build/obj/libwrap.o
nvcc fatal   : Unsupported gpu architecture 'compute_60'
make[3]: *** [/home/gregork/src/caffe2/third_party/nccl/build/obj/libwrap.o] Error 1
make[2]: *** [nccl_external-prefix/src/nccl_external-stamp/nccl_external-build] Error 2
make[1]: *** [CMakeFiles/nccl_external.dir/all] Error 2
[  7%] Linking CXX static library ../../../lib/libbenchmark.a
[  7%] Built target benchmark
[  7%] Linking CXX static library ../../../lib/libgloo.a
[  7%] Built target python_copy_files
[  7%] Built target gloo
[  7%] Linking CXX static library ../../../lib/libgtest.a
[  7%] Built target gtest
make: *** [all] Error 2

If I choose CUDA ARCH NAME Manual and CUDA ARCH BIN 50 and CUDA ARCH PTX 50. I still get the same error.
Bellow is output of my configuration:
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/libprotoc.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/libprotoc.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/libprotoc.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/libprotoc.so.14)

/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/./libprotobuf.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.8' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/./libprotobuf.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/./libprotobuf.so.14)
/home/gregork/anaconda3/envs/caffe2/bin/protoc: /home/gregork/anaconda3/envs/caffe2/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /home/gregork/anaconda3/envs/caffe2/bin/../lib/./libprotobuf.so.14)

CMake Warning at /home/gregork/local/share/cmake-3.10/Modules/FindProtobuf.cmake:455 (message):
  Protobuf compiler version doesn't match library version 3.4.0
Call Stack (most recent call first):
  cmake/ProtoBuf.cmake:60 (find_package)
  cmake/Dependencies.cmake:8 (include)
  CMakeLists.txt:72 (include)

Using protobuf compiler /home/gregork/anaconda3/envs/caffe2/bin/protoc
The BLAS backend of choice:Eigen
Could NOT find NNPACK (missing: NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY)
Brace yourself, we are building NNPACK
Found PythonInterp: /home/gregork/anaconda3/envs/caffe2/bin/python (found version "2.7.14")
-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/six-download

[100%] Built target six

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/enum-download

[100%] Built target enum

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/opcodes-download

[100%] Built target opcodes

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/peachpy-download

[ 11%] Performing update step for 'peachpy'

Current branch master is up to date.

[ 22%] No configure step for 'peachpy'

[ 33%] No build step for 'peachpy'

[ 44%] No install step for 'peachpy'

[ 55%] No test step for 'peachpy'

[ 66%] Completed 'peachpy'

[100%] Built target peachpy

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/fp16-download

[ 11%] Performing update step for 'fp16'

Current branch master is up to date.

[ 22%] No configure step for 'fp16'

[ 33%] No build step for 'fp16'

[ 44%] No install step for 'fp16'

[ 55%] No test step for 'fp16'

[ 66%] Completed 'fp16'

[100%] Built target fp16

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/fxdiv-download

[ 11%] Performing update step for 'fxdiv'

Current branch master is up to date.

[ 22%] No configure step for 'fxdiv'

[ 33%] No build step for 'fxdiv'

[ 44%] No install step for 'fxdiv'

[ 55%] No test step for 'fxdiv'

[ 66%] Completed 'fxdiv'

[100%] Built target fxdiv

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/psimd-download

[ 11%] Performing update step for 'psimd'

Current branch master is up to date.

[ 22%] No configure step for 'psimd'

[ 33%] No build step for 'psimd'

[ 44%] No install step for 'psimd'

[ 55%] No test step for 'psimd'

[ 66%] Completed 'psimd'

[100%] Built target psimd

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/pthreadpool-download

[ 11%] Performing update step for 'pthreadpool'

Current branch master is up to date.

[ 22%] No configure step for 'pthreadpool'

[ 33%] No build step for 'pthreadpool'

[ 44%] No install step for 'pthreadpool'

[ 55%] No test step for 'pthreadpool'

[ 66%] Completed 'pthreadpool'

[100%] Built target pthreadpool

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googletest-download

[100%] Built target googletest

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/fxdiv-download

[ 11%] Performing update step for 'fxdiv'

Current branch master is up to date.

[ 22%] No configure step for 'fxdiv'

[ 33%] No build step for 'fxdiv'

[ 44%] No install step for 'fxdiv'

[ 55%] No test step for 'fxdiv'

[ 66%] Completed 'fxdiv'

[100%] Built target fxdiv

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googletest-download

[100%] Built target googletest

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googlebenchmark-download

[100%] Built target googlebenchmark

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googletest-download

[100%] Built target googletest

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googlebenchmark-download

[100%] Built target googlebenchmark

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googletest-download

[100%] Built target googletest

-- Configuring done

-- Generating done

-- Build files have been written to: /home/gregork/src/caffe2/build/googlebenchmark-download

[100%] Built target googlebenchmark

Found gflags  (include: /home/gregork/anaconda3/envs/caffe2/include, library: /usr/lib64/libgflags.so)
Found glog    (include: /home/gregork/anaconda3/envs/caffe2/include, library: /usr/lib64/libglog.so)
-- git Version: v0.0.0
-- Version: 0.0.0
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- compiled but failed to run
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
Found lmdb    (include: /home/gregork/anaconda3/envs/caffe2/include, library: /usr/lib64/liblmdb.so)
Found LevelDB (include: /home/gregork/anaconda3/envs/caffe2/include, library: /usr/lib64/libleveldb.so)
Found Snappy  (include: /home/gregork/anaconda3/envs/caffe2/include, library: /usr/lib64/libsnappy.so)
OpenCV found (/home/gregork/anaconda3/envs/caffe2/share/OpenCV)
CMake Warning at cmake/Dependencies.cmake:241 (find_package):
  By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Eigen3", but
  CMake did not find one.

  Could not find a package configuration file provided by "Eigen3" with any
  of the following names:

    Eigen3Config.cmake
    eigen3-config.cmake

  Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
  "Eigen3_DIR" to a directory containing one of the above files.  If "Eigen3"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:72 (include)

Did not find system Eigen. Using third party subdirectory.
Found PythonInterp: /home/gregork/anaconda3/envs/caffe2/bin/python (found suitable version "2.7.14", minimum required is "2.7")
NumPy ver. 1.12.1 found (include: /home/gregork/anaconda3/envs/caffe2/lib/python2.7/site-packages/numpy/core/include)
Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
CUDA detected: 7.5
Added CUDA NVCC flags for: sm_50 compute_50
Found libcuda: /usr/lib64/libcuda.so
Found libnvrtc: /usr/lib64/libnvrtc.so
Could NOT find CUDNN (missing: CUDNN_LIBRARY)
Could NOT find NCCL (missing: NCCL_INCLUDE_DIRS NCCL_LIBRARIES)
NCCL: /home/gregork/src/caffe2/third_party/nccl/build/lib/libnccl_static.a
Could NOT find CUB (missing: CUB_INCLUDE_DIR)
Could NOT find Gloo (missing: Gloo_INCLUDE_DIR Gloo_LIBRARY)
CUDA detected: 7.5
Found libcuda: /usr/lib64/libcuda.so
Found libnvrtc: /usr/lib64/libnvrtc.so
Does not need to define long separately.
Current compiler supports avx2 extention. Will build perfkernels.
GCC 4.8.5: Adding gcc and gcc_s libs to link line
Include NCCL operators
Include NNPACK operators
Including image processing operators
Excluding video processing operators due to no opencv
Excluding mkl operators as we are not using mkl
MPI operators skipped due to no MPI support
Include Observer library
Automatically generating missing __init__.py files.

   If you don't care about NCCL you can disable it with -DUSE_NCCL=OFF. Otherwise you will have to modify its Makefile.
Another option is to update CUDA to version 8 or 9 as @laetitiaoist suggests.


RuntimeError: Failed to run command: , errors: Command 'cd /home/tianyu/.cache/paddle_extensions && /home/tianyu/miniconda3/envs/paddle3/bin/python3 ms_deform_attn_setup.py build' returned non-zero exit status 1. LAUNCH INFO 2023-08-24 11:00:59,752 Exit code 1

Hello there,
When I was running BEVFormer model, I met this error:

Does anyone know where's the problem from?
Thanks so much!!

When I check the log it gives me the info:
2023-08-24 13:13:42,775 -  WARNING - No custom op ms_deform_attn found, try JIT build
2023-08-24 13:13:42,775 -  WARNING - No custom op ms_deform_attn found, try JIT build
2023-08-24 13:13:44,339 -    ERROR - ms_deform_attn builded fail!
2023-08-24 13:13:44,339 -    ERROR - ms_deform_attn builded fail!

I also encountered this problem

The custom op ms_deform_attn is probably not built correctly. Try to clean the .so in paddle3d/model/ops or manually build the ops by running python setup.py install in this directory.


Terminal hides permanently when using "toggle terminal window"

When I use CTRL+space to hide the terminal (when active), it hides the window as expected. However, using the same key combination again doesn't bring it back, even though the process is still active (as seen in my Task Manager).
On Windows 10, using Git Bash shell.

Is there any other app on your system that might be swallowing this combo? Please also try setting a different hotkey and see whether it makes a difference

I have this issue too on Windows 10.  It seems not so much that there is a keystroke conflict, but that the application is still in the foreground when the window is hidden and the keystroke has no effect unless you bring another application into the foreground i.e. click on something else first.  Ideally on hide, the next application should be brought to the foreground; I believe this is how hyper.js works with it's summon extension.

I have a similar issue on Windows 7. I press CTRL-Space and Terminus goes away (as I'd expect), but it also is removed from the taskbar. Pressing CTRL-Space does nothing. I don't see it in the ALT-Tab application selector and it does not show up in the Applications list in Windows Task Manager. However, in the Windows process list Terminus is still active. If I select Terminus in the process list and then press CTRL-Space again, the application comes back.

I'm also having this same problem.
One weird wrinkle to add: If I hit alt-tab and cycle through the windows but land back on the app I was using (in this case Chrome) and then hit ctrl space, it opens, and I can then also close. BUT then it won't open again.
Also I had trouble finding documentation for what the global hotkey was, and I also don't see anything in the settings for the hotkeys to change this hotkey.
I'm on Win 10

Hey just wanted to add, after some experimentation I've realized that the hide and pull up hotkey works fine, as long as a window is in focus.  But if I pull it up, hide it, and then click on a window or something, putting that window in focus, it will work. No clue why that is.

This is actually a really annoying issue! I have Terminus open almost all the time when I am writting code in VS Code. Now, the shortcut for autocomplete suggestions for VS Code is ctrl space, but whenever Terminus is open, it overwrites the VS Code shortcut and shows the Terminus terminal instead.

Thanks @jcklpe - finally nailed it

Is there any solution of @jcklpe  issue. I am also facing the same issue. I don't know even how to reinstall. Can't see on Control panel.

Hello @Eugeny,
It seems your commit https://github.com/Eugeny/terminus/commit/dc864781e4b5627126e8b533c903710ea17f9a61 doesn't solve the problem. Tested on Windows 10 1809 and macOS 10.15 with Terminus 1.0.102, as described by @jcklpe, the toggle window terminal hotkey will only work if the window has been hidden with the hotkey but not if the window remains visible but in the background.
Do you think this can be addressed anytime soon? Terminus has become my favourite terminal. But this hotkey behaviour really annoys me as I use the toggle hotkey constantly to bring the window to the foreground.

Added a possible fix - please check!

That was quick! Can you make a release so that I can check changes?

It will appear on the nightly builds page soon https://ci.appveyor.com/project/Eugeny/terminus/build/artifacts

I downloaded the artifact build and the toggle functionality works always no mater the app has been already hidden with the hotkey or not. Great job! 👍👍👍


Slow Loading Speed Issue

Summary:
This ticket addresses the slow loading speed issue in the app, which requires immediate attention and resolution. The purpose of this ticket is to provide a comprehensive understanding of the problem, its background, potential use cases, and the benefits/impact of resolving this issue.
Background/Reason:
The app currently experiences significant delays in loading, negatively impacting user experience and overall satisfaction. This issue has been reported by multiple customers, indicating a widespread problem that needs to be addressed urgently. The slow loading speed hampers the app's functionality and efficiency, hindering users from accessing desired features and content promptly.
Use Cases:

User Scenario: When a user opens the app, it takes an unusually long time to load the main interface, causing frustration and impeding their ability to perform desired actions.
Content Loading: Loading specific content within the app, such as images, videos, or articles, takes an excessive amount of time, leading to a poor user experience.
Navigation Delay: Moving between different sections or screens within the app is slow, resulting in a cumbersome and frustrating user journey.

Benefits/Impact:

Enhanced User Experience: Resolving the slow loading speed issue will significantly improve the overall user experience, ensuring faster and more efficient app usage.
Increased User Engagement: With improved loading speed, users will be more likely to engage with the app, explore its features, and spend more time utilizing its functionalities.
Customer Satisfaction: Addressing this issue promptly will demonstrate the app's commitment to customer satisfaction, leading to increased loyalty and positive word-of-mouth recommendations.
Improved App Performance: By optimizing loading speed, the app's performance will be enhanced, resulting in smoother navigation, quicker content access, and reduced frustration for users.
Competitive Advantage: A fast-loading app will give the company a competitive edge over similar apps in the market, attracting more users and potentially increasing revenue.

Please prioritize this ticket to ensure a swift resolution of the slow loading speed issue, as it directly impacts the app's usability and customer satisfaction.

The app has slow loading speed, which needs to be fixed.

<EMAIL_ADDRESS>:

The app has slow loading speed, which needs to be fixed.

Summarized feedback from<EMAIL_ADDRESS>:

The loading speed of the app is slow and needs to be fixed.

Summarized feedback from<EMAIL_ADDRESS>:

The loading speed of the app is slow and needs to be fixed.


GesturesInput.cs missing?

Overview
Tried to use the InputManager prefab, but none of the gestures worked. When i checked the scripts attached to the prefab I noticed that the GesturesInput.cs script is missing. Didnt find it anywhere in the toolkit
Unity Editor Version
2017.1.0f3
Mixed Reality Toolkit Release Version
HoloToolkit-Unity-v1.2017.2.0

You should be using InteractionInputSource

Looks like GitHub didn't automatically close these issues when merged into master. I'll close these manually and look into why.


Upgrade scalatest version to 3.2.16

Closes https://github.com/NVIDIA/spark-rapids/issues/8600
A prerequisite to supporting Scala 2.13 is upgrading our scalatest version to a version that supports Scala 2.13.
Overview of changes:

Upgrade scalatest version to 3.2.16 (this is the same version used in Apache Spark master branch)
FunSuite is now AnyFunSuite
scalatest Mockito support is now in separate dependency (org.scalatestplus:mockito-4-11-2_12)
package name for Matcher changed

build

LGTM


[Event Change Request] table 130 "Incoming Document" OnCreateGenJnlLineOnBeforeShowRecord

Could you please pass the paramter IncomingDocument: Record "Incoming Document"  by reference instead of by value?
[IntegrationEvent(false, false)]
//+Change
//-local procedure OnCreateGenJnlLineOnBeforeShowRecord(IncomingDocument: Record "Incoming Document"; var IsHandled: Boolean)
local procedure OnCreateGenJnlLineOnBeforeShowRecord(var IncomingDocument: Record "Incoming Document"; var IsHandled: Boolean)
//-Change
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 11199.


Update change-the-emergency-location-for-a-user.md

#1791
Action Taken:
Added article: Set-CsOnlineVoiceUser for powershell cmdlet to assign  PSTN specific parameters
https://github.com/MicrosoftDocs/office-docs-powershell/blob/master/skype/skype-ps/skype/Set-CsOnlineVoiceUser.md
Uservoice related article
https://www.skypefeedback.com/forums/299913-generally-available/suggestions/18440797-powershell-emergency-location-telephone-number-l
https://www.skypefeedback.com/forums/299913-generally-available/suggestions/19341262-change-emergency-location

@CarolynRowe Hi Carolyn,  changes look correct. Could you please check and merge? Thanks!


added helm chart for deploying stream registry

Added a helm chart to enable easy deployment of stream registry in a kubernetes cluster.

❤


Cannot assign hotkey to TIMEOTP

Hello Again,
First, thanks for #18 !!
Another (smaller) issue: I cannot assign a HotKey to TIMEOTP. I tried CTRL+T, CTRL+Alt+T, CTRL+Shift+T, I just get an error beep and nothing happens.
Use global Hotkey is, of course, enabled.
I also found more conveniant to have to Copy TOTP Option bellow Copy Username and Copy Password Menus than in the KeeOtp2 Submenu. Would it be possible to have an option to show/hide it ?
Thanks again for this great plugin !
AVB

Hi @avbfr,
thank you for your feedback. The "Copy TOTP" thing should be an easy one, I will implement with the next patch.
Your mentioned bug may be based on the implementation of NHotkey. I will check this out. Thank you!

Hello @tiuub,
Thanks !
Just to clarify a little: I'm able to Assign the HotKey in the config menu but When I try to use it I get the error beep.and the TOTP is not copied.

Hi @avbfr,
I tried to reproduce your error, but Im not able to get it. Can you give me some more details about your environment?
Which version of Windows do you use? Which system type do you run, 32-bit or 64-bit? Do you use any other software which handles hotkeys, like AutoHotkey or anything else?

Hello @tiuub
I'm using Windows 10 Enterprise 20H2 64 Bits. I've AutoHotkey installed but it's not activated.
Other Standard KeePass Hotkeys are working fine (CTRL+B, CTRL+V...).
CTRL+T for TOTP stopped working with 1.5.4. I've also tried to set different HotKey but none are working (CTRL+T, CTRL+Alt+T, CTRL+Shift+T).
Other KeePass plugins: KeeAgent <IP_ADDRESS>, KeeChallenge <IP_ADDRESS>, KeePassQRCodeView <IP_ADDRESS>
KeePass Version: 2.48.1 (Latest)
Hope this help.
AVB

Okay, thank you. I have tweaked a little bit around. Please check out the newes pre release.
KeeOtp2 - <IP_ADDRESS>(pre).zip
Please tell me, if it worked for you.

Just tested and CTRL+T works !! Also it's Good to have Copy TOTP under Copy Password ! :-) Thanks a lot !!
On this version, I just noticed a small delay (about 1.5s) when copying TOTP using HotKey or directly by clicking on "Copy TOTP" (My KeePass window reduce automatically when I use such menus that's how I can notice the delay).
The configurable Global HotKey looks to not be used anymore but I assume that's the tweak you made. ;-)
AVB

Great! I will check the issue with the delay.
The Global Hotkey is still usable. It's similar to the built-in Ctrl + Alt + A hotkey. If configured, you can instantly paste the TOTP into the target window by hitting Ctrl + Alt + T. You can configure this at "Edit Entry" - "Auto-Type". For example you configure Auto-Type with window title "Google", then hop over to the browser, open Google-Login and hit Ctrl + Alt + T. The TOTP should be pasted right now.
But for now, I will close this issues.


Need a more versatile hiding strategy for modal dialogs

I notice that, despite being hidden, modal dialogs like the keyboard shortcuts dialog still appear in the tab order. For instance, in the development environment, tabbing from the Reply button (which I assume is the last button on-screen) moves focus into the hidden keyboard shortcuts dialog.
This does appear to have aria-hidden="true" set, but some subelements appear to still have a tabindex. Instead of some crazy scheme for toggling tabindexes based on hidden state, would it be possible to use a more robust hidden mechanism that actually removes the elements from the DOM? display: none maybe? I'm guessing these modal dialogs may be handled by a plugin, but I haven't dug deep enough in the code to figure out which plugin this might be.
But there are a few instances where hidden elements appear to be concealed in a way that doesn't remove them from the DOM, and this is going to cause all sorts of subtle bugs for screen reader users. aria-hidden is a good fix for simple circumstances, but any kind of complex tree structure is going to cause issues where tab focus lands in places screen readers can't introspect.
Thanks.

I'm kinda surprised they aren't display: none when not active already; deserves some investigation.  @shubhamdhama has done some recent work on modals and this might be a quick project for him, but I'd be happy for anyone to take a look.


rerank 3.5 notebook

This pull request adds a new notebook to the repository, which demonstrates how to deploy the Cohere-rerank-english model package from the AWS Marketplace using Amazon SageMaker. The notebook includes instructions for subscribing to the model package, creating an endpoint, performing real-time inference, and cleaning up resources.

The notebook begins with an introduction to Cohere's Large Language Models (LLMs) and their use cases, such as generation, summarization, classification, content moderation, information extraction, semantic search, and contextual entity extraction.
It then provides pre-requisites for running the notebook, including the need to open it from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio, and ensuring the IAM role has the required permissions.
The notebook guides users through subscribing to the model package on the AWS Marketplace, specifying the model package ARN, and creating an endpoint using Boto3.
It includes code for creating an input payload, performing real-time inference using the rerank method, and visualizing the output.
Finally, the notebook provides instructions for cleaning up resources, including deleting the model and unsubscribing from the listing.

@lfayoux @jpekmez
In the markdown of the notebook, change the following:

cohere-rerank-v3.5 should be replaced with the new market listing
or your AWS account has a subscription to cohere-rerank-v3.5 should be replaced with the new market listing
The model package listing page should be updated to cohere-rerank-v3.5 listing page

For the code:
From the docs: https://docs.cohere.com/docs/cohere-works-everywhere#sagemaker it looks like we do not support v2 API yet @lfayoux which one do you want to use? I also looked at the client code itself here: https://github.com/cohere-ai/cohere-python/blob/main/src/cohere/sagemaker_client.py and it's not clear to me how the model arn packages work in this case

@mahjongmen There should be a new sdk release with sagemaker v2 client. Either was we should replace the sdk with the normal one

@lfayoux when I spoke with @mkozakov yesterday it was determined that because @billytrend-cohere is gonna push the new SDK on monday morning but because @gloriacohere needs to push all of this monday morning at 6:30am EST it won't make it in time.
Issue right now is that the cohere sdk is only compatible with V1 API

We can actually use the cohere sdk with client v1 and then later swap to v2

great - can someone do that. I looked through our documents and it wasn't obvious to me whether or not we needed ARN etc... so I don't want to change the docs without testing

as per https://cohereai.slack.com/archives/C07SBRVE8G2/p1733079402682219 we should go ahead with cohere-aws sdk and then change to using the cohere sdk during monday (post release)


Sugestion: make it easier to add description to a task

In gulp v4, gulp.task does not support 3 parameter any more. So it is
gulp.task(taskName, function () {});
To add a description, I have to
var fn = function () { };
fn.description = 'a great task';
gulp.task(taskName, fn)
I would like to have gulp.task to support description like
gulp.task(taskName, fn, description);
description is an optional.

Sorry, we've already discussed this and aren't going to implement it.  You can implement this with your own function wrapper or whatever.


botframework-connector 4.11.0 requires msal==1.2.0, but you'll have msal<2.0.0,>=1.6.0 by azure-sdk-for-python

Sample information

Sample type: [\samples]
Sample language: [python]
Sample name:

Describe the bug
dependency conflict
To Reproduce
Steps to reproduce the behavior:
pip install
pip install -r requirements.txt
Expected behavior
Is there any further plan on this?
Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
https://github.com/Azure/azure-sdk-for-python/search?l=Text&q=msal
https://github.com/microsoft/botbuilder-python/blob/main/libraries/botframework-connector/requirements.txt

Please let us know what sample this occurs in so that we can pin msal to 1.2.0 in it?

Please let us know what sample this occurs in so that we can pin msal to 1.2.0 in it?
Thank you for the response❤ this is not for a specific sample. From botbuilder-python dependency, botframework-connector 4.11.0 requires msal==1.2.0

Any update mate?

Dear Jessica,
I have replied in the dialog.
botframework-connector 4.11.0 requires msal==1.2.0, but you'll have msal<2.0.0,>=1.6.0 by azure-sdk-for-python
This is conflict between bot connector and auth process dependency
BR,
Juncheng

From: Jessica Wailes<EMAIL_ADDRESS>Sent: Wednesday, December 9, 2020 3:15 AM
To: microsoft/BotBuilder-Samples<EMAIL_ADDRESS>Cc: CharingJC<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [microsoft/BotBuilder-Samples] botframework-connector 4.11.0 requires msal==1.2.0, but you'll have msal<2.0.0,>=1.6.0 by azure-sdk-for-python (#2978)
Please let us know what sample this occurs in so that we can pin msal to 1.2.0 in it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/BotBuilder-Samples/issues/2978#issuecomment-740880224, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AR3M2NTZSPMDSXLZOF27A4DSTZ3LLANCNFSM4URWBQNQ.

I'm also hitting this issue. Any recommendation on which versions to pin to to work around this?

I've submitted an issue to the botbuilder-python repo which mantains the library that introduced the dependency conflict: https://github.com/microsoft/botbuilder-python/issues/1450

Closing as it is being tracked in https://github.com/microsoft/botbuilder-python/issues/1450

Closing as it is being tracked in https://github.com/microsoft/botbuilder-python/issues/1450


Pass through current agent Consul datacenter name

Instead of only filling __meta_consul_dc when datacenter is set in consul_sd_config this change fills the label based on what the agent reports it's current data center is, if datacenter isn't manually set, otherwise it uses whatever datacenter was set to.

This seems to make sense in Consul terms.
Can you please squash your commits?
@fabxc

:+1: It's cool that the commit message is descriptive but let's keep the first line self-sufficient and the body less code-specific.

Periods have been added, this should be good to go

Merging this now, as it seems like all points have been addressed.


RFE: stream-oriented versions of strings.Unquote, strconv.ParseInt, etc

I often find myself needing variants of the basic parsing functions (strconv.ParseInt, strings.Unquote, and so on) that consume a prefix of an input string and return the unconsumed part.  I realize you can accomplish this using text/scanner, but it is complex and heavyweight.
Could we add these to the standard library?

Dup of #2632?
Returning the unconsumed part is new. fmt.Scan* exists but probably slower than you want.
FWIW, there's a fork of strconv.ParseUint that works on []byte at https://godoc.org/go4.org/strutil#ParseUintBytes that I find myself using enough to warrant existing.

Although fmt.Sscanf(_, "%q%s", _, _) can be used to scan a double-quoted or raw string literal, it doesn't work for single-quoted rune literals (whereas strconv.Unquote does).
The []byte vs string issue is orthogonal to the issue of parsing a prefix versus the entire string.

I don't believe strconv should be in the business of telling you where the data ends. If you know where it ends, break it before handing it to strconv. If you don't know where the data ends, I think you have a bad data format.


Adding dismiss button for the GalleryVC

I tried adding a dismiss button, but it did not show, is there any way to have a dismiss button in the zooming page?

Not at this point. I need to work on it. But the view does dismiss itself on tap. Isn't that good enough for you?

Please, i would really a dismiss button. We have funny users in our Country here, am sure they will complain about something like this.
THank you very much, for this wonderful library.

Do you think could do it yourself? You can fork the project and then create a pull request when you are done. I can then review it and accept it or give you advice on how to make it better. You can practice writing code in that way? Do you like the idea?

Yes, I like the idea, actually, i tried adding the button myself, but it does not show, let me share my code.
class GalleryVC: UIPageViewController {

    var closeButton: UIButton = {
        let button = UIButton()
        button.setImage(#imageLiteral(resourceName: "close"), for: .normal)
        return button
    }()

        view.backgroundColor = .black

        view.addSubview(closeButton)
        closeButton.topAnchor.constraint(equalTo: view.topAnchor, constant: 20).isActive = true
        closeButton.leftAnchor.constraint(equalTo: view.leftAnchor, constant: 20).isActive = true
        closeButton.widthAnchor.constraint(equalToConstant: 40).isActive = true
        closeButton.heightAnchor.constraint(equalToConstant: 40).isActive = true

        view.bringSubview(toFront: closeButton)

        closeButton.addTarget(self, action: #selector(dismissView), for: .touchUpInside)
    }```

```func dismissView() {
        dismiss(animated: true, completion: nil)
    }

Hey, there, what VC did you out the code into?

Hi, I did that in the GalleryVC, I also tried it in the GalleryItemVC, still the same issue.

Cause it should go in the GalleryItemVC, yes. Did you try making it work by using the storyboard?
Your code seems alright and all nice and i wonder why it doesn't work. I will give it a try and let you know what happens. Meanwhile, could you try doing the same using the storyboard?

OK, I will try with a storyboard.

I do see the issue with your code now. You need to give the button a size using CGRect, etc.
For example:
let button = UIButton(frame: CGRect(x: 100, y: 100, width: 100, height: 50))
Check out the link for a full description of how to do it:
https://stackoverflow.com/questions/24030348/how-to-create-a-button-programmatically
Afterwards, you can add the constraints programmatically and it should work fine.
It is quite a simple operation though and I would recommend using storyboards for that.

Thanks, I got it working, the issue was with the image, I had the drag the image to the Resource of SwiftImageCarousel.

Yeah, that seems another plausible reason? Do you want to create a pull request? It would be nice to have it as a feature in the library for other people to use as well.


element not interactable

I try this image simaofsilva/noip-renewer:3.9.1-slim-buster on a buster raspbian and I have an error. Maybe the login page changed ?
Thank you for your help
Opening browser
Login page
Traceback (most recent call last):
File "//renew.py", line 48, in
browser.find_element_by_name("Login").click()
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webelement.py", line 633, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
(Session info: headless chrome=88.0.4324.182)

I think the image, I used, was too old
I change for this image : simaofsilva/noip-renewer:debian
and I have another error
Opening browser
Traceback (most recent call last):
File "//renew.py", line 41, in
browser = webdriver.Chrome(options=browserOptions)
File "/root/.local/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in init
RemoteWebDriver.init(
File "/root/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init
self.start_session(capabilities, browser_profile)
File "/root/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/root/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/root/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed.
(chrome not reachable)
(The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

The image with tag 3.9.1 is a year old so I do not recommend use it, so please use tag debian and update that from time to time.
I have the image with tag debian running on a arm/v7 device without any issues. Are you running the right image for your processor's architecture? What is the output of uname --m?

The output if uname --m is
armv7l
is debian tag the right image for my raspberry ?

Yes, it seems to be the right version, what can be wrong ?
REPOSITORY                 TAG       IMAGE ID       CREATED      SIZE
simaofsilva/noip-renewer   debian    bc1c130454d9   6 days ago   462MB

That seems alright.
It can be a memory problem with the container exhausting the RAM and crashing. Have you check the Raspberry status while the container is running?

I didn't find some trouble with my raspberry. So I build my own image but I change bullseye by buster because my raspberry run with buster
And it works with this new image
My change :
FROM python:3.10.4-slim-buster@sha256:c2824bf9bdb9dbeed967df6805388e90e67439aae4934c02526d8ce5c29ae6d1
Could you upgrade your debian image to buster ?
I also use renew.py and not renew_legacy.py : I don't know why you keep a specific version for debian : maybe you can merge.
Thanks a lot

Your running in a container so it does not matter which version of Debian you are using on your Raspberry Pi. Still, going from bullseye to buster is a downgrade, not an upgrade.
I am still wondering how the current image does not work in your environment but I am glad you found a solution for your issue.


[ENG-724] Hierarchical list

Purpose
Add hierarchical-list/item, partial-registration-modal and their respective manager components for rendering a project hierarchy list for partial registration.
Summary of Changes
Two components are added.
The hierarchical-list/item related components takes a NodeModel and will invoke itself recursively to render the model's children.
The partial-registration-modal implements a HierarchicalListManager class which maintains selectedNodes list to keep track of all selected nodes.
Ticket
https://openscience.atlassian.net/browse/ENG-724
Reviewer Checklist

[ ] meets requirements
[ ] easy to understand
[ ] DRY
[ ] testable and includes test(s)
[ ] changes described in CHANGELOG.md

Pull Request Test Coverage Report for Build 3283

73 of 74 (98.65%)  changed or added relevant lines in 4 files are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.02%) to 75.907%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

lib/osf-components/addon/components/registries/hierarchical-list/item-manager/component.ts
18
19
94.74%

Files with Coverage Reduction
New Missed Lines
%

lib/osf-components/addon/helpers/get-ancestor-descriptor.ts
4
59.68%

Totals

Change from base Build 3278:
0.02%

Covered Lines:
7306

Relevant Lines:
8425

💛  - Coveralls


set returning of knex#insert to this.id

otherwise when using PostgreSQL the returned rows is [], which causes the subsequent this.get(rows[0], params) to fail.

Sweet. Thank you! Published the fix as another patch release.

ta :smile_cat:

/cc @iainkirkpatrick


Upgrade Linux Kernel for flatcar-3815 from 6.1.77 to 6.1.80

Subject says it all.
CI: http://jenkins.infra.kinvolk.io:8080/job/container/job/packages_all_arches/3584/cldsv/

Fixed build issues with CONFIG_NET_SCH_{CBQ,DSMARK} that started to disappear with Kernel 6.1.80+. See upstream commits of CBQ, DSMARK.

Merging, to avoid further CI breakage.


fix: CasparCG restart error: Timeout after 10000ms SOFIE-2588

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

fix

What is the new behavior (if this is a feature change)?

The handling of cronjob CasparCG restarts is improved to produce more better errors, and minimise logging at the error level.
Additionally, some typings for the ui have improved for executing tsr actions.
This relies on https://github.com/nrkno/sofie-timeline-state-resolver/pull/295 for accurate logging

Other information:

Status

[ ] Code documentation for the relevant parts in the code have been added/updated by the PR author
[x] The functionality has been tested by the PR author
[ ] The functionality has been tested by NRK

I think the tests still need fixing: https://github.com/nrkno/sofie-core/actions/runs/6430122872/job/17460498391?pr=1042

@jstarpl fixed


Elevations not present for some layers

I am not sure if this is caused by the gallery or by faulty data, but during testing of the gallery on the campaigns2019 server these errors keep popping up:
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/wms.py", line 308, in generate_gallery
    add_image(plot_driver.plot(), plot_object, generate_code, sphinx, url_prefix=url_prefix)
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/mss_plot_driver.py", line 538, in plot
    image = self.plot_object.plot_vsection(data, self.lats, self.lons,
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/mpl_vsec.py", line 188, in plot_vsection
    UR(self.driver.vert_units).check("[temperature]")):
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/registry.py", line 1263, in parse_expression
    return build_eval_tree(gen).evaluate(
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/pint_eval.py", line 92, in evaluate
    left = self.left.evaluate(define_op, bin_op, un_op)
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/pint_eval.py", line 102, in evaluate
    return define_op(self.left)
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/registry.py", line 1264, in <lambda>
    lambda x: self._eval_token(x, case_sensitive=case_sensitive, **values)
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/registry.py", line 1159, in _eval_token
    {self.get_name(token_text, case_sensitive=case_sensitive): 1}
  File "/home/mss-campaigns2019/miniconda3/envs/stable/lib/python3.9/site-packages/pint/registry.py", line 643, in get_name
    raise UndefinedUnitError(name_or_alias)
pint.errors.UndefinedUnitError: 'unknown' is not defined in the unit registry

and
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/wms.py", line 294, in generate_gallery
    elevations = plot_object.get_elevations()
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/mss_2D_sections.py", line 165, in get_elevations
    return [str(x) for x in self.driver.get_elevations(self._vert_type)]
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/mss_plot_driver.py", line 316, in get_elevations
    return self.data_access.get_elevations(vert_type)
  File "/home/mss-campaigns2019/src/Marilyth/MSS/mslib/mswms/dataaccess.py", line 366, in get_elevations
    return self._elevations[vert_type]["levels"]
KeyError: 'pl'

My first guess is faulty data, since self._elevations seems to not be filled up, paired with a unknown units warning during a vert_units check.
It would be nice if you could also have a look at this @joernu76

Starting up the server does not show errors.
(stable) mss-campaigns2019@icg1317:~/CAMPAIGNS2019/config$ PYTHONPATH=. mswms --debug >log 2>&1

At least not with the stable version. I'll try your code.

Mmmh, I get different errors. Can you create an environment where I can reproduce the error?

in stable environment we have the version of Mays repo builded

OK, one step further. If a variable has no "units" attribute, mss will assign the units the "unknown units" string. But such variables will be "discarded" in the dataaccess class. So you should not "encounter" them, even if present.

OK, the stable version does "something". Could use a bit more of debug messages to get an impression of what it is doing.

OK, can reproduce the problem.

see into the src folder there is the version used

There are a bunch of files without units or None units. These aren't all but should bring the point across.
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.ml.nc', '/srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.sfc.nc'] unknown units
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/transfer/2019-11-05T00:00:00.an.ml.nc', '/srv/mss/CAMPAIGNS2019/mssdata/transfer/2019-11-05T00:00:00.an.sfc.nc'] unknown units
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.sfc.nc'] None
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.bt.nc'] None
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/transfer/2019-11-05T00:00:00.an.sfc.nc'] None
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/airs/2019-11-27T00:00:00_AIRS_filt_tracks.nc'] None
INFO: Vert units of ['/srv/mss/CAMPAIGNS2019/mssdata/airs/2019-11-27T00:00:00_AIRS_filt_all.nc'] None

All variables of /srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.bt.nc have units:
ncdump -h /srv/mss/CAMPAIGNS2019/mssdata/analysis/2019-11-27.an.bt.nc
variables:
	float BTemp_ecmwf_res_15mu_high(time, lat, lon) ;
		BTemp_ecmwf_res_15mu_high:standard_name = "brightness_temperature_residual_15mu_high" ;
		BTemp_ecmwf_res_15mu_high:long_name = "ECMWF set to look like Brightness T at 15mu_high AIRS T" ;
		BTemp_ecmwf_res_15mu_high:units = "kelvin" ;
	float BTemp_ecmwf_res_15mu_low(time, lat, lon) ;
		BTemp_ecmwf_res_15mu_low:standard_name = "brightness_temperature_residual_15mu_low" ;
		BTemp_ecmwf_res_15mu_low:long_name = "ECMWF set to look like Brightness T at 15mu_low AIRS T" ;
		BTemp_ecmwf_res_15mu_low:units = "kelvin" ;
	float BTemp_ecmwf_res_4mu(time, lat, lon) ;
		BTemp_ecmwf_res_4mu:standard_name = "brightness_temperature_residual_4mu" ;
		BTemp_ecmwf_res_4mu:long_name = "ECMWF set to look like Brightness T at 4mu AIRS T" ;
		BTemp_ecmwf_res_4mu:units = "kelvin" ;
	double lat(lat) ;
		lat:standard_name = "latitude" ;
		lat:long_name = "latitude" ;
		lat:units = "degrees_north" ;
	double lon(lon) ;
		lon:standard_name = "longitude" ;
		lon:long_name = "longitude" ;
		lon:units = "degrees_east" ;
	double time(time) ;
		time:standard_name = "time" ;
		time:long_name = "time of overpass is 05:30 UTC" ;
		time:units = "hours since 2019-07-01T00:00:00" ;

// global attributes:

Don't see a problem with the others either.
Maybe it happens with plots not associated to a particular elevation level? Cloud cover comes to mind.

Ah, I see. The unknown unit error occurs for *Vertical Sections" of *model level" data.
Yes. The vertical axis has indeed no known or sensbile units in that particular case.

I added that error.

Does that also explain the absence of self._elevations for the HS layers?

Maybe. One step at a time.
I propose this fix
diff --git a/mslib/netCDF4tools.py b/mslib/netCDF4tools.py
index 248bb07a..11d7b342 100644
--- a/mslib/netCDF4tools.py
+++ b/mslib/netCDF4tools.py
@@ -111,7 +111,7 @@ def identify_vertical_axis(dataset):
         name, var = identify_variable(dataset, standard_name)
         orientation = hybrid_orientation(var)
         if var is not None:
-            units = getattr(var, "units", "unknown units")
+            units = getattr(var, "units", "dimensionless")
             result.append((name, var, orientation, units, layertype))
     if len(result) == 0:
         return None, None, None, None, "sfc"

Seems safer...?

The gallery is pretty nice for debugging plotting plugins :-)

In case that the plot supports styles, you should select one (first?). I have at least one plot bugging our because none of the three styles was selected.

style="default" is always passed as an argument. I suppose taking the first style instead would work better.

There may be no styles as well. Certainly so far, we do not enforce a "default" style being present.

Aannnd another bug. Strange that it does not trigger normally.
This should fix the remaining problems:
diff --git a/mslib/mswms/dataaccess.py b/mslib/mswms/dataaccess.py
index 20c92d82..4ab0ff6b 100644
--- a/mslib/mswms/dataaccess.py
+++ b/mslib/mswms/dataaccess.py
@@ -225,7 +225,6 @@ class DefaultDataAccess(NWPDataAccess):
                 raise ValueError(f"variable type {vartype} not available for variable {variable}")

     def _parse_file(self, filename):
-        elevations = {"levels": [], "units": None}
         with netCDF4.Dataset(os.path.join(self._root_path, filename)) as dataset:

             time_name, time_var = netCDF4tools.identify_CF_time(dataset)
@@ -260,6 +259,8 @@ class DefaultDataAccess(NWPDataAccess):
                     if elevations["units"] != self._elevations[vert_type]["units"]:
                         raise IOError(f"vertical level units do not match previous "
                                       f"file '{self._elevations[vert_type]['filename']}'")
+                else:
+                    self._elevations[vert_type] = elevations

             standard_names = []
             for ncvarname, ncvar in dataset.variables.items():

I can prepare a fix for develop if you like.

That would be great. I am glad the bugs were not caused by the plot gallery fault.

They discovered some. A nice side-effect is that one can easily test all plots of the existing server including all hacked-in plotting classes. Very useful!

I would prefer to have this in stable @joernu76

The one bug isn't even in stable and the other "simply" skips a test the first time a file is checked.


Setup with "@cloudflare/vitest-pool-workers"?

Hey there,
has someone a working setup in conjunction with "@cloudflare/vitest-pool-workers"?
I'm currently running in different issues, such as:
Failed to load url @opentelemetry/sdk-trace-base (resolved id: @opentelemetry/sdk-trace-base)

and / or:
No such module "Users/.../node_modules/.pnpm/@opentelemetry+resources@1.26.0_@opentelemetry+api@1.9.0/node_modules/@opentelemetry/resources/build/esm/detectors/platform/node/os".
  imported from "Users/.../node_modules/.pnpm/@opentelemetry+resources@1.26.0_@opentelemetry+api<EMAIL_ADDRESS>
I'd assume for now that it's due to some configurations on my end. If somebody has a working sample together with the vitest pool workers it would save me some time and I'd be grateful :)

I believe this issue is related to https://github.com/cloudflare/workers-sdk/issues/6581

Yeah.. that certainly seems like a similar issue indeed. And not sure if there is anything I could do to fix that in this library.
If the issue is only in @opentelemetry/sdk-trace-base and not in @opentelemetry/api, or if you do not do any manual instrumentation, you should be able to test the handler without the instrument calls.

After messing with this for way too long, I found a solution that worked for me. I created a package that bundles otel-cf-workers and @opentelemetry/api using esbuild to target browser, which seems to treeshake out all the Node-specific things: https://www.npmjs.com/package/@jahands/otel-cf-workers
It feels kind of hacky, but it was either this or stop using otel :(


Fix broken headings in Markdown files

GitHub changed the way Markdown headings are parsed, so this change fixes it.
See bryant1410/readmesfix for more information.
Tackles bryant1410/readmesfix#1

Thanks. But please don't send any more PRs to this repo, it's just full of junk.


Running setup.py bdist_wheel for grpcio: still running...

Iniside of a debian buster-slim image, I am using the python version that ships (3.7.3-1) and I am using the shipped pip version (18.1-5) to upgrade pip (22.2.1) and then install grpcio (1.48.0)
... and it takes 17 minutes to complete setup.py bdist_wheel for grpcio.
Why is it taking so long? What can I do to speed it up? All historic posts I read seem to imply this issue was supposed to go away with some change that pip shipped around about v19.

If I add --prefer-binary, I collect the wheel rather than the source and it solves the issue.
The question is, why is pip defaulting to the source tarball over the wheel? I guess the source tarball must be "newer" even though it's for the same version? Maybe there was some quirk with the release order specifically for the most recent 1.48.0?

Thank you. Working


github connector: chunking v2

Uses mdast + gfm (github flavour) support to parse github markdown and create a hierarchical section object, limiting prefix sizes to 128 characters and depth to 2 (to avoid reaching half the chunk size in prefixes);
Which mean we can have as succesion of prefixes:

issue title
comments author name
(sub comment author name for discussions)
level 1 headings
level 2 headings

There's still a chance that we overflow the 512/2 = 256 tokens limit for prefixes, but we'll see at scale. We can remove level 2 headings if that's the case.
Tested on real github data:
https://github.com/breach/breach_front/discussions/3
Discussion #3 [breach_front]: Test idea discussion
This is
a
Test
This is a discussion about testing testing is interesting
> spolu:
This is the real response here
>> spolu:
This is a reply
>> spolu:
Interesting
> spolu:
And this is a wrong response here with an image

https://github.com/breach/mod_strip/pull/65
Issue #65 [mod_strip]: removed devtools (to be in separate module)
DevTools should be in separate module. I have created a development module for this purpose (<https://github.com/jonathan-reisdorf/mod_development>). Feel free to comment or discuss, this is my first PR here, please double check :)
This also contains exposed add/remove methods for the context menu, so that other modules can add/remove entries. Side effect: "Close DevTools" is now only shown when the DevTools are actually opened.
>> @spolu:
Looks great! You should add the `context_menu_builder` infrastructure though to construct your context menu in devtools. No need for new `add` and `remove` function I believe.
>> @jonathan-reisdorf:
Maybe I am missing something, but how would you do that? I mean, there are already events in the strip module that will fire the context menu for example on right click; if I add the infrastructure in mod\_development these events would sum up and build two context menus, or it would at least overwrite potential other context menu entries that might get added in strip module. That way we would always have to keep these entries in sync imho, that's why I added add & remove functionality for entries. But maybe I am thinking in the wrong direction?
>> @spolu:
The code is made such that it can handle context menu builders from multiple modules: <https://github.com/breach/breach_core/blob/master/lib/core_tabs.js#L189>
>> @jonathan-reisdorf:
ok thanks, it's now changed and the inspector is now also working without mod\_strip
>> @spolu:
This is great!
>> @spolu:
You should add your module there: <https://github.com/breach/breach_core/wiki/List-of-modules>
>> @jonathan-reisdorf:
I did, but you can also add it here as a repo instead if you like
>> @spolu:
Gotcha. Didn't see it before. CooL!
>> @spolu:
Installing it right now :)

Will merge and deploy tomorrow morning :+1:


Parse error. Invalid RegExp flag 's'

I am running command:
java -jar compiler.jar --js_output_file=out.js myFile.js
and meet following error:
ERROR - Parse error. Invalid RegExp flag 's'
              e = e.replace(/;.*/s, "");

I am manipulating strings like this:
try {
          // I get first 3 most important nodes and trim titles after chars: ; - (
          title = getMostRelevantNodes( graph , 3).map(function(item){

              // decorate Query Nodes on Graph > using same function
              highlightQueryNode(item.key);

              var e = readCache( item.key).name;

              // names like 2,6-Diaminopurine: reutrn
              if ( /\d/.test(e) ) return e;

              e = e.replace(/;.*/s, "");
              e = e.replace(/-.*/s, "");
              e = ( e.indexOf('(') > -1) ? e.substring(0, e.indexOf('(')).replace(/\s+$/, '') : e;

              return e
          });

How should I fix it to make closure not return an error?

The error message here seems pretty clear: the s flag you are using isn't a valid RegExp flag.
I'm not sure what you expect the s flag to do - but it's not valid in EcmaScript regular expressions.

Actually, the '/s' flag is valid in ES_2018.
You'll need to set both language in and language out to be ES_2018 to avoid getting compile errors, though, because closure-compiler doesn't try to new regular expression features for older language outputs.


Extra json on mongo output causing provider 'mongodb': 783: unexpected token at '{

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 7.26.0
Ruby:  2.7.8p225
Distribution:  ubuntu
Module version: master

How to reproduce (e.g Puppet code you use)
we are running this on an established mongodb instance,
What are you seeing
running any /usr/bin/mongo command cli returns an extra json line, this is related to SERVER-78540 it would seem and was fixed in 4.x and 6.x but not yet in 5.x (running 5.0.21 atm)
output of command will result in something like
{"t":{"$date":"2023-09-12T06:33:42.971Z"},"s":"I",  "c":"NETWORK",  "id":5490002, "ctx":"thread4","msg":"Started a new thread for the timer service"}
{
  "databases" : [
    {
      "name" : "admin",
      "sizeOnDisk" : NumberLong(184320),
      "empty" : false
    },
    {
      "name" : "config",
      "sizeOnDisk" : NumberLong(110592),
      "empty" : false
    },
    {
      "name" : "local",
      "sizeOnDisk" : NumberLong(81920),
      "empty" : false
    }
  ],
  "totalSize" : NumberLong(1261928448),
  "totalSizeMb" : NumberLong(1203),
  "ok" : 1
}

Error will be something like
Error: Could not prefetch mongodb_database provider 'mongodb': 783: unexpected token at '{

What behaviour did you expect instead
In my opinion this extra line should be removed in lib/puppet/util/mongodb_output.rb

/usr/bin/mongo I  deprecated in favor of /usr/bin/mongosh ..  And it is removed in version 6.  Working on https://github.com/voxpupuli/puppet-mongodb/pull/677 which only supports mongosh, and version 4.4+ of mongodb.
But i need to finalize this PR.  (like cleaning commits, get extra acceptance tests ...)   Just need more hours in a day :)

This sounds like a better approach, .. I just whipped up this small fix in the mean time as it was causing the puppet run to fail. If there is anything that I could do please let me know.

I'm only testing my branch on Redhat flavours ..  So in time someone testing it in Ubunto and aother flavours could help.  And since version 7.0 is out .. thats also new ground to cover :)


Error with DoctrineBundle

I'm getting the error below. I'd appreciate any help.

Note: Using configuration file /path/to/phpstan.neon.
PHP Fatal error:  Trait "Symfony\Component\VarExporter\LazyGhostTrait" not found in /myproject/vendor/doctrine/doctrine-bundle/Repository/LazyServiceEntityRepository.php on line 31
Fatal error: Trait "Symfony\Component\VarExporter\LazyGhostTrait" not found in /myproject/vendor/doctrine/doctrine-bundle/Repository/LazyServiceEntityRepository.php on line 31
make: *** [stan] Error 255

refs #409  and refs https://github.com/doctrine/DoctrineBundle/issues/1614
I am using

Symfony 5.4.18
DoctrineBundle 2.8.2
phpstan/phpstan 1.9.7
phpstan/phpstan-doctrine 1.3.30
phpstan/extension-installer 1.2.0

# phpstan.neon.dist
parameters:
    paths:
        - src
    level: 3
    scanFiles:
        - vendor/twig/twig/src/Extension/EscaperExtension.php
    ignoreErrors:
        - '#Call to an undefined method Symfony\\Component\\Console\\Helper\\HelperInterface::ask\(\).#'

I'm not 100% certain I have phpstan/phpstan-doctrine configured correctly so the problem could be on my end. As you can see, I am using phpstan/extension-installer and the docs say "If you also install phpstan/extension-installer then you're all set!". Perhaps I should have something more?

Please create a small reproducing repository that shows the problem. I have no idea how this could happen.

No need to reproduce it, I figured it out: https://github.com/doctrine/DoctrineBundle/issues/1614#issuecomment-1374912770

Thank you so much @ondrejmirtes ! ❤

Please test phpstan-doctrine 1.3.31 :)

Please test phpstan-doctrine 1.3.31 :)

Fix confirmed! 🎉 Thank you for your quick attention!


Problem: credo reports refactoring opportunities

Solution: refactor reported code parts

Coverage remained the same at 98.333% when pulling 3e280f1336ae01f88180cd880f4dae264b915f65 on credo-feedback into 01f7cfa84067fc634bd91bb6e5951910140d36fb on master.

@maennchen feel free to merge it!


Creating Single Instance Database Clone fails

Creating Single Instance Database Clone fails with the following error:
[2023:12:05 08:44:20]: Acquiring lock .ORAW.create_lck with heartbeat 30 secs
[2023:12:05 08:44:20]: Lock acquired
[2023:12:05 08:44:20]: Starting heartbeat
[2023:12:05 08:44:20]: Lock held .ORAW.create_lck
ORACLE EDITION: ENTERPRISE
[WARNING] [DBT-11217] Unable to check available shared memory on specified node(s) ([10]).
Prepare for db operation
[FATAL] [DBT-06006] Unable to create directory: (/opt/oracle/oradata/ORAW).
   CAUSE: Proper permissions are not granted to create the directory or there is no space left in the volume.
[ 2023-12-05 08:44:32.566 UTC ] [WARNING] [DBT-11217] Unable to check available shared memory on specified node(s) ([10]).
[ 2023-12-05 08:46:43.404 UTC ] Prepare for db operation
[ 2023-12-05 08:46:43.473 UTC ] [FATAL] [DBT-06006] Unable to create directory: (/opt/oracle/oradata/ORAW).

LSNRCTL for Linux: Version <IP_ADDRESS>.0 - Production on 05-DEC-2023 08:46:43

Copyright (c) 1991, 2021, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
TNS-12541: TNS:no listener
 TNS-12560: TNS:protocol adapter error
  TNS-00511: No listener
   Linux Error: 111: Connection refused

Configuration file used was based on https://github.com/oracle/oracle-database-operator/blob/main/config/samples/sidb/singleinstancedatabase_clone.yaml, just updated name, namespace, sid and cloneFrom.
Before attempting to create the clone I had one Primary and two Physical standby instances running without issues:
kubectl -n oracle-database get singleinstancedatabase
NAME      EDITION      STATUS    ROLE               VERSION      CONNECT STR             TCPS CONNECT STR   OEM EXPRESS URL
szczyrk   Enterprise   Healthy   PHYSICAL_STANDBY   <IP_ADDRESS>.0   <IP_ADDRESS>:32533/ORAS    Unavailable        https://<IP_ADDRESS>:32428/em
ustron    Enterprise   Healthy   PHYSICAL_STANDBY   <IP_ADDRESS>.0   <IP_ADDRESS>:31519/ORAU    Unavailable        https://<IP_ADDRESS>:31105/em
zywiec    Enterprise   Healthy   PRIMARY            <IP_ADDRESS>.0   <IP_ADDRESS>:31761/ORAZ    Unavailable        https://<IP_ADDRESS>:30398/em

After executing kubectl apply on singleinstancedatabase_clone:
kubectl -n oracle-database get singleinstancedatabase
NAME      EDITION      STATUS     ROLE               VERSION       CONNECT STR             TCPS CONNECT STR   OEM EXPRESS URL
szczyrk   Enterprise   Healthy    PHYSICAL_STANDBY   <IP_ADDRESS>.0    <IP_ADDRESS>:32407/ORAS    Unavailable        https://<IP_ADDRESS>:31656/em
ustron    Enterprise   Healthy    PHYSICAL_STANDBY   <IP_ADDRESS>.0    <IP_ADDRESS>:32678/ORAU    Unavailable        https://<IP_ADDRESS>:31596/em
wisla     Enterprise   Creating   Unavailable        Unavailable   <IP_ADDRESS>:32557/ORAW   Unavailable        Unavailable
zywiec    Enterprise   Healthy    PRIMARY            <IP_ADDRESS>.0    <IP_ADDRESS>:32193/ORAZ     Unavailable        https://<IP_ADDRESS>:31245/em

kubectl -n oracle-database get pods
NAME            READY   STATUS             RESTARTS      AGE
szczyrk-wx4zj   1/1     Running            0             161m
ustron-9ifi2    1/1     Running            0             154m
wisla-mtqfs     0/1     CrashLoopBackOff   21 (4m ago)   139m
zywiec-vlf14    1/1     Running            0             3h17m

kubectl -n oracle-database get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
szczyrk   Bound    pvc-ce59ebf8-11b3-4b90-8493-3a9bcd1a8d06   10Gi       RWO            gp2            27m
ustron    Bound    pvc-55696dbb-714a-4c20-8bbb-8ba85314cf8a   10Gi       RWO            gp2            20m
wisla     Bound    pvc-283d044d-74f4-4289-b3b7-4cd769e35bfa   10Gi       RWO            gp2            5m13s
zywiec    Bound    pvc-3f4b83cb-d861-4963-8bae-5d7badf1eca6   10Gi       RWO            gp2            63m

Environment:

AWS EKS 1.25
StorageClass gp2, provisioner kubernetes.io/aws-ebs
oracle-database-operator 1.0.0

Hi,
Can you share the operator pod and clone db pod logs ?

Hi,
Yes, I cloned the primary instance. I tried again, now with only one instance in total (a Primary) prior to the clone operation.
$ kubectl --kubeconfig ~/.kube/config-sinch-op-smsf-1-andbos -n oracle-database get singleinstancedatabase
NAME     EDITION      STATUS    ROLE      VERSION      CONNECT STR             TCPS CONNECT STR   OEM EXPRESS URL
zywiec   Enterprise   Healthy   PRIMARY   <IP_ADDRESS>.0   <IP_ADDRESS>:31170/ORAZ   Unavailable        https://<IP_ADDRESS>:30994/em

$ kubectl --kubeconfig -n oracle-database get pvc
NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
zywiec   Bound    pvc-37535c5d-f6df-489f-b91b-ed3656bd6e8e   50Gi       RWO            gp2            8d

$ kubectl -n oracle-database get pods
NAME           READY   STATUS    RESTARTS   AGE
zywiec-vpxps   1/1     Running   0          8d

$ kubectl -n oracle-database apply -f ~/oracle/singleinstancedatabase_clone.yaml
singleinstancedatabase.database.oracle.com/wisla created

$ kubectl -n oracle-database get pods
NAME           READY   STATUS    RESTARTS      AGE
wisla-wnnrh    0/1     Running   1 (72s ago)   3m54s
zywiec-vpxps   1/1     Running   0             8d

$ kubectl -n oracle-database get pvc
NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
wisla    Bound    pvc-1cec290e-f461-4b66-81fc-e0069a22480b   50Gi       RWO            gp2            3m58s
zywiec   Bound    pvc-37535c5d-f6df-489f-b91b-ed3656bd6e8e   50Gi       RWO            gp2            8d

$ kubectl -n oracle-database get singleinstancedatabase
NAME     EDITION      STATUS     ROLE          VERSION       CONNECT STR             TCPS CONNECT STR   OEM EXPRESS URL
wisla    Enterprise   Creating   Unavailable   Unavailable   <IP_ADDRESS>:30200/ORAW     Unavailable        Unavailable
zywiec   Enterprise   Healthy    PRIMARY       <IP_ADDRESS>.0    <IP_ADDRESS>:31170/ORAZ   Unavailable        https://<IP_ADDRESS>:30994/em

$ kubectl -n oracle-database logs -f wisla-wnnrh
Defaulted container "wisla" out of: wisla, init-permissions (init), init-wallet (init)
[2023:12:22 12:39:46]: Acquiring lock .ORAW.create_lck with heartbeat 30 secs
[2023:12:22 12:39:46]: Lock acquired
[2023:12:22 12:39:46]: Starting heartbeat
[2023:12:22 12:39:46]: Lock held .ORAW.create_lck
ORACLE EDITION: ENTERPRISE
[WARNING] [DBT-11217] Unable to check available shared memory on specified node(s) ([10]).
Prepare for db operation
[FATAL] [DBT-06006] Unable to create directory: (/opt/oracle/oradata/ORAW).
   CAUSE: Proper permissions are not granted to create the directory or there is no space left in the volume.
[ 2023-12-22 12:39:57.762 UTC ] [WARNING] [DBT-11217] Unable to check available shared memory on specified node(s) ([10]).
[ 2023-12-22 12:42:07.441 UTC ] Prepare for db operation
[ 2023-12-22 12:42:07.506 UTC ] [FATAL] [DBT-06006] Unable to create directory: (/opt/oracle/oradata/ORAW).

LSNRCTL for Linux: Version <IP_ADDRESS>.0 - Production on 22-DEC-2023 12:42:07

Copyright (c) 1991, 2021, Oracle.  All rights reserved.

Connecting to (ADDRESS=(PROTOCOL=tcp)(HOST=)(PORT=1521))
TNS-12541: TNS:no listener
 TNS-12560: TNS:protocol adapter error
  TNS-00511: No listener
   Linux Error: 111: Connection refused
$ date
Fri Dec 22 13:42:27 CET 2023

oracle-database-operator-controller-manager_log.txt
singleinstancedatabase_clone.yaml.txt
singleinstancedatabase.yaml.txt
I have uploaded the operator-controller-manager pod logs (taken with stern so logs from all three pods are present) my YAML files.
Best regards,
Andreas

@andbos please check if the /opt/oracle/oradata which is a directory mounted with a volume have enough space, if not then try expanding the volume source or using a new volume.

Hi,
Thanks. It turns out I had assigned insufficient storage (tried to save money during testing). I doubled the storage to 100Gi and after that the creation of the clone went well. A bit strange that it was possible to create the initial primary with as little at 10Gi but not a clone. What's the minimal amount?
$ kubectl -n oracle-database apply -f ~/oracle/singleinstancedatabase_clone.yaml
singleinstancedatabase.database.oracle.com/wisla created

$ kubectl -n oracle-database get pods
NAME           READY   STATUS    RESTARTS   AGE
wisla-ly4z2    1/1     Running   0          14m
zywiec-q14pe   1/1     Running   0          26m

$ kubectl -n oracle-database get singleinstancedatabase
NAME     EDITION      STATUS    ROLE      VERSION      CONNECT STR            TCPS CONNECT STR   OEM EXPRESS URL
wisla    Enterprise   Healthy   PRIMARY   <IP_ADDRESS>.0   <IP_ADDRESS>:30531/ORAW   Unavailable        https://<IP_ADDRESS>:30474/em
zywiec   Enterprise   Healthy   PRIMARY   <IP_ADDRESS>.0   <IP_ADDRESS>:32102/ORAZ   Unavailable        https://<IP_ADDRESS>:31749/em

From above output it is impossible to know which instance is the original primary and which is the clone. Maybe that could be highlighted somehow in a future release? As far as I understand the clone is like a snapshot or a backup taken at a specific time and there is no syncing, the two primary instances are not clustered.
Best regards,
Andreas

Aha now I see that you can determine which instance is a clone by checking Clone From under Status and Spec in the describe singleinstance output.
$ kubectl --kubeconfig ~/.kube/config-sinch-op-smsf-1-andbos -n oracle-database describe singleinstancedatabase wisla
Name:         wisla
Namespace:    oracle-database
Labels:       <none>
Annotations:  <none>
API Version:  database.oracle.com/v1alpha1
Kind:         SingleInstanceDatabase
Metadata:
  Creation Timestamp:  2024-02-02T08:57:59Z
  Finalizers:
    database.oracle.com/singleinstancedatabasefinalizer
  Generation:        1
  Resource Version:  156829610
  UID:               3a3ec8cc-302e-4320-aabc-22f2feebc918
Spec:
  Admin Password:
    Keep Secret:  true
    Secret Key:   oracle_pwd
    Secret Name:  db-admin-secret
  Clone From:     zywiec
  Image:
    Pull From:     container-registry.oracle.com/database/enterprise:latest
    Pull Secrets:  oracle-container-registry-secret
  Init Params:
  Pdb Name:  ORCLPDB1
  Persistence:
    Access Mode:    ReadWriteOnce
    Size:           100Gi
    Storage Class:  gp2
  Replicas:         1
  Sid:              ORAW
Status:
  Archive Log:             false
  Clone From:              zywiec

@andbos closing this issue as I think the issue was resolved upon expanding the block volume attached to the sidb resource pod.
Feel free to reopen or reach out if problem persists


Polymer.dart+Angular.dart app fails to run when compiled to javascript

 Issue by bgourlie
Originally opened as dart-lang/sdk#16163

Attachment:
polymer_repro.rar (4.20 KB)

 Comment by jmesserly

Removed Library-Polymer label.

 Comment by sigmundch

We've done a few changes that should make it easier to make angular and polymer work together.
The error above sounds like if the polyfills were included more than once. In particular, with polymer 0.9.5 and 0.10.x we used to include the shadow_dom polyfill independently of whether you had it in the page for angular. Now the polyfill has to be added in the page explicitly, which matches what polymer.js and angular do.
If you could give polymer 0.11.0 a shot, and let us know if it doesn't work, that would be great!
If it still fails, feel free to reply here again or open a new bug. I was not able to extract the attached example in my linux box, so if it still fails please send us a .zip (which is more widely available than .rar).

Added AssumedStale label.

 Comment by jmesserly

Removed Area-Polymer label.
Added area-pkg, Pkg-Polymer labels.

 Comment by sigmundch

Added Library-Polymer label.

 Comment by kasperl

Added Area-Polymer, Triaged labels.


Account Update visualizer

Summary
Transactions can result in a series of account updates. Account updates can include token transfers, state updates, and changing delegation. These updates can be seen in the JSON file created by toPretty() function call on the transaction object.
Ex:
const txn = await Mina.transaction(senderAccount, async () => {
        await zkApp.verify(id, creditScore, signature);
      });
      await txn.prove();
      await txn.sign([senderKey]).send();
     console.log('txn', txn.toPretty());

Output:
    txn [
      {
        label: 'feePayer',
        publicKey: '..e1Co',
        fee: '0',
        nonce: '0',
        authorization: '..gUdq'
      },
      {
        label: 'OracleExample.verify()',
        publicKey: '..CVr7',
        events: '[["1"]]',
        preconditions: {
          account: '{"state":["5381126474252558361076605461681340039227154619768231462994659962968677770124","1",null,null,null,null,null,null]}'
        },
        mayUseToken: { parentsOwnToken: false, inheritFromParent: false },
        authorizationKind: {
          isSigned: false,
          isProved: true,
          verificationKeyHash: '3392518251768960475377392625298437850623664973002200885669375116181514017494'
        },
        authorization: { proof: '..KQ==' }
      }
    ]

This transacation has two account updates

The fee payer, which is signed by the fee payer
The smart contract which is authorized by proof with these preconditions: x, y, z.

The output of the tx, if it is accepted by the network, will be that an event with the value 1 is emitted by the contract.
These JSON documents are harder to read as the transactions get more complicated(see Other accounts raw.).
A tool could be built  (as a start to locally) to generate the account update JSON and have these updates visualized and easy to navigate/understand. For developers it's more useful to visualize a transaction without having to send it.  And this can be expanded to support wallets or block explorers.
Submitted by 45930
Resources

Account updates
Account update visualiser - Existing visualizer limited in interpretation/presentation.

@hgedia Please can you assign this to me.

@hgedia Proposal submitted. Please can you put this issue under review or something similar?

hey @YashBit  - your current proposal is taking a different route - so will leave this open and in existing state.


Add support for LoongArch

This PR adds basic support for LoongArch and does not include CI for now.

r? @asomers

@asomers Friendly ping.

Hello, @asomers Could you review this PR please? Thanks.

Hello, thank you for sharing your perspective. I understand your view on the LoongArch architecture. However, I believe that our work in the open-source community goes beyond satisfying personal vanity; it's about fostering technological diversity and innovation. LoongArch, as a new architecture, might present new opportunities and challenges for the computing landscape in China.
While you hold a different opinion, I think it's important for us as volunteers to maintain an open-minded approach, respecting various technological explorations, and striving to contribute to the community. If you find the complexity of supporting LoongArch to be overwhelming, I understand your reluctance to invest extensive time. Nevertheless, I hope you can consider that such work might have positive impacts on certain individuals. In open-source projects, our efforts often benefit a broader user base, even if it's not immediately evident.
Ultimately, the decision rests with you, but I encourage us to collectively work towards the betterment of the community, contributing to the advancement and innovation of technology. Thank you for your time and understanding. :heart:


Update LLOS_InputObserversGDB.lyr to include feature template

The "LLOS_InputObserversGDB.lyr" is updated to include a feature template. This fixes an issue with the LLOS and RLOS tools in Pro 2.3 not giving a feature template when the pencil button is clicked.

@csmoore - I updated this LYR file to have a feature template (in ArcMap 10.3.1).  When I tested locally, this resolved the issue with RLOS and LLOS not giving a point feature for featureset input.
Please review and merge.
@dfoll

Seeing slightly strange behavior when I copy the repo to another location and re-test. The LLOS Input Observers "pencil" icon now gives me a dropdown list of point, line, and polygon options.  When I click the point option, an input point featureset is created.
The Targets input feature set simply opens the point featureset input for the targets with no dropdown.
Also, the input location is symbolized with a "pin" symbol rather than a circle, as I had expected.

OK, with latest commit I am seeing the correct behavior.
@dfoll - please review. Thanks!

@BobBooth - unfortunately, this version didn't work for me - perhaps we may need to switch these to lyrx files when running in Pro

unassigning myself until there is an approval on this.... don't to forget about my own assignment because i left it sitting

@csmoore @dfoll  please retest using the updated LYR file from my latest PR.

This on worked for me on 2.3.15512


[#3044] Apply HostAndPort

#3044

Codecov Report

Merging #3047 into master will decrease coverage by <.01%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master    #3047      +/-   ##
==========================================
- Coverage   38.81%   38.81%   -0.01%
==========================================
  Files        1933     1933
  Lines       64917    64912       -5
  Branches     9073     9072       -1
==========================================
- Hits        25200    25197       -3
+ Misses      37143    37142       -1
+ Partials     2574     2573       -1

Impacted Files
Coverage Δ

...r/HttpRequestExecutorExecuteMethodInterceptor.java
0% <0%> (ø)
:arrow_up:

...om/navercorp/pinpoint/common/util/StringUtils.java
87.93% <100%> (ø)
:arrow_up:

...rcorp/pinpoint/common/plugin/util/HostAndPort.java
90.47% <100%> (ø)
:arrow_up:

...npoint/rpc/server/SimpleServerMessageListener.java
80% <0%> (-20%)
:arrow_down:

.../com/navercorp/pinpoint/rpc/packet/PongPacket.java
50% <0%> (-14.29%)
:arrow_down:

...om/navercorp/pinpoint/rpc/codec/PacketDecoder.java
68.67% <0%> (-7.23%)
:arrow_down:

.../profiler/sender/StandbySpanStreamDataStorage.java
72.88% <0%> (-3.39%)
:arrow_down:

...point/rpc/client/DefaultPinpointClientHandler.java
71.29% <0%> (-0.31%)
:arrow_down:

...orp/pinpoint/rpc/server/DefaultPinpointServer.java
76.51% <0%> (+1.13%)
:arrow_up:

...corp/pinpoint/rpc/stream/StreamChannelManager.java
72.41% <0%> (+1.72%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 335522d...e75eea8. Read the comment docs.


Explore VA Interface

Figure out how the VA interface works on a very close level. Think about what questions to ask Dr. Sen about it.

Blocker: VA Vista does not appear to be publicly available. Planning on asking the synthetic data generation team at VA how this database can be accessed.

At the end of the year we should be able to give a presentation on Vista so if you could work with Dr. Sen to understand how VA clinicians use Vista to access patient data, that would be fantastic.
I have some questions about it from Raja: What system to they use (ie Vista)? What type of data can he access? What data can be entered/input? Can he download anything from this interface? etc.

I am in correspondence with Dr. Sen to get preliminary information that I can use to help me access this resource. I have unsuccessfully been able to access this resource in previous attempts, so I have asked Dr. Sen for some guidance and whether this is the platform he intends to upload his patients' records from the keto and weight loss drug studies into.

The VA VISTA is actually named VISTAWeb. It is a platform designed for VA clinicians to provide necessary information pertaining to their patients. There are several VISTA-like login pages; however, none of them had a URL ending in .gov. I found a pdf document discussing VISA's use-case and it appears the website may have been redacted.
Link to document:

Hey @ubhuiyan how is research on CPRS going?

@lorikrammer It is going well. I will document what I have found within the Justification Document v2.0 if that works.


SlidingFeatureView supports sliding window with different sizes

What is the purpose of the change
This PR update the SlidingFeatureView and FlinkTableBuilder to support applying sliding window with different sizes in one SlidingFeatureView.
The implementation only stores the rows for the maximum window size, instead of rows for each sliding window, which uses less space in the Flink state backend than the approach that applies multiple sliding windows and then joins them together.
Brief change log

Introduce Java implementation of MultiSizeWindowProcessFunction.
SlidingFeatureView supports sliding window with different sizes

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

@lindong28 Could you take a look at this PR?

@lindong28 Thanks for your review! I updated the PR. Could you please take another look?

@lindong28 Thanks for your review! I updated the PR. Could you please take another look?

@lindong28 Thank you for your review! I updated the PR. Could you please take another look?


Typo: my -> by

The second paragraph in "Cause" section:

This can happen because a NULL pointer dereferenced or a random incorrect address was accessed. This in turn can be caused my memory being freed prematurely, or data structure corruption.

Here "caused my" should be "caused by".

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b19eb27e-40b0-c765-d826-8b9962e4c1e6
Version Independent ID: f9231818-2d1d-be08-7ce2-92b97edd5613
Content: Bug Check 0x3B SYSTEM_SERVICE_EXCEPTION - Windows drivers
Content Source: windows-driver-docs-pr/debugger/bug-check-0x3b--system-service-exception.md
Product: windows-hardware
Technology: debugger
GitHub Login: @DOMARS
Microsoft Alias: domars

Thanks Qingnan, this is now fixed.  FYI if you see anything you want to fix in the future, you can also submit a pull request.  To do that, start by clicking the Edit button in the upper right of the page on docs.microsoft.com.


Use Maven CI friendly versions

As described on https://maven.apache.org/maven-ci-friendly.html
This should also fix the problems with Websphere container image dependencies not being resolved (it does locally).
TODO:

Check if this does not accidentally break something in CICD/release plugin workings
Contributing for required maven version

You've now turned all parent references into a property defined in the parent. How does the system know what value to resolve the property to if it cannot find the parent?

https://medium.com/outbrain-engineering/faster-release-with-maven-ci-friendly-versions-and-a-customised-flatten-plugin-fe53f0fcc0df

Ik denk dat het zo goed is.