Add option to install libtiledb.so

This PR aims to allow an option to include/exclude libtiledb.so in the install group. The install group affects what is included in the PYPI packages and since these packages are used in Conda recipes also Conda packages. This way we can produce these configurations:

PYPI wheels should contain libtiledb.so in them, so we install with default variables pip install .
PYPI sdist should produce the same package as wheel (when finished) so the default CMake environment includes libtiledb.so in the install group
Conda recipe uses PYPI sdist so we have to modify its script to execute: TileDB_DIR=${PREFIX} INSTALL_LIB_TILEDB="OFF" pip install --no-deps . This way we will use libtiledb from Conda prefix and we also have to remove it from install group.

This PR's sole purpose is to enable Conda build to not package libtiledb.so. No other packages/functionality should be affected.
Conda feedstock PR (that requires this change):
https://github.com/TileDB-Inc/tiledb-vector-search-feedstock/pull/35

pip install .
  -- Installing: /tmp/tmpeetkpcfm/wheel/platlib/tiledb/vector_search/./_tiledbvspy.cpython-311-x86_64-linux-gnu.so
  -- Set non-toolchain portion of runtime path of "/tmp/tmpeetkpcfm/wheel/platlib/tiledb/vector_search/./_tiledbvspy.cpython-311-x86_64-linux-gnu.so" to "$ORIGIN/lib"
  -- Installing: /tmp/tmpeetkpcfm/wheel/platlib/tiledb/vector_search/lib/libtiledb.so.2.22 << HERE
  *** Installing project into wheel...
  -- Install configuration: "Release"
  *** Making wheel...
  *** Created tiledb_vector_search-0.2.3.dev41+gda176afc.d20240506-cp311-cp311-linux_x86_64.whl...
  Building wheel for tiledb-vector-search (pyproject.toml) ... done
  Created wheel for tiledb-vector-search: filename=tiledb_vector_search-0.2.3.dev41+gda176afc.d20240506-cp311-cp311-linux_x86_64.whl size=6868083 sha256=9ed54d767b3f7a380d32e39437ff8ee5ff8eb88e65894aa7dc1389b13fdae9e1
  Stored in directory: /home/dudoslav/.cache/pip/wheels/16/d2/38/a9b8e505638ba122beb55762af7fcf1d8a013d27ebdf19587f
Successfully built tiledb-vector-search
Installing collected packages: tiledb-vector-search
  Attempting uninstall: tiledb-vector-search
    Found existing installation: tiledb-vector-search 0.2.3.dev41+gda176afc.d20240506
    Uninstalling tiledb-vector-search-0.2.3.dev41+gda176afc.d20240506:
      Removing file or directory /home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb/vector_search/
      Removing file or directory /home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb_vector_search-0.2.3.dev41+gda176afc.d20240506.dist-info/
      Successfully uninstalled tiledb-vector-search-0.2.3.dev41+gda176afc.d20240506
Successfully installed tiledb-vector-search-0.2.3.dev41+gda176afc.d20240506

INSTALL_LIB_TILEDB="OFF" pip install .
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/include/experimental/__p2630_bits/submdspan_extents.hpp
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/include/experimental/__p2630_bits/submdspan_mapping.hpp
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/include/experimental/__p2642_bits
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/include/experimental/__p2642_bits/layout_padded.hpp
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/include/experimental/__p2642_bits/layout_padded_fwd.hpp
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/lib/cmake/mdspan/mdspanConfig.cmake
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/lib/cmake/mdspan/mdspanConfigVersion.cmake
  -- Installing: /tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/./_tiledbvspy.cpython-311-x86_64-linux-gnu.so
  -- Set non-toolchain portion of runtime path of "/tmp/tmp49y6peo6/wheel/platlib/tiledb/vector_search/./_tiledbvspy.cpython-311-x86_64-linux-gnu.so" to "$ORIGIN/lib"
  *** Installing project into wheel...
  -- Install configuration: "Release"
  *** Making wheel...
  *** Created tiledb_vector_search-0.2.3.dev41+gda176afc.d20240506-cp311-cp311-linux_x86_64.whl...
  Building wheel for tiledb-vector-search (pyproject.toml) ... done
  Created wheel for tiledb-vector-search: filename=tiledb_vector_search-0.2.3.dev41+gda176afc.d20240506-cp311-cp311-linux_x86_64.whl size=1271127 sha256=25fcabc87bac093aaf2d21757f75afe69dc4e66a8ec3d6d52dbb320fe23e15b0
  Stored in directory: /home/dudoslav/.cache/pip/wheels/16/d2/38/a9b8e505638ba122beb55762af7fcf1d8a013d27ebdf19587f
Successfully built tiledb-vector-search
Installing collected packages: tiledb-vector-search
  Attempting uninstall: tiledb-vector-search
    Found existing installation: tiledb-vector-search 0.1.1.dev42+gd8f58b51
    Uninstalling tiledb-vector-search-0.1.1.dev42+gd8f58b51:
      Removing file or directory /home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb/vector_search/
      Removing file or directory /home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb_vector_search-0.1.1.dev42+gd8f58b51.dist-info/
      Successfully uninstalled tiledb-vector-search-0.1.1.dev42+gd8f58b51
Successfully installed tiledb-vector-search-0.2.3.dev41+gda176afc.d20240506

No libtiledb.so in sight.

I just noticed that when you run: TILEDB_PATH=${CONDA_PREFIX} pip install --no-deps .
I get:
(zoo) taco:TileDB-Vector-Search dudoslav$ python -c 'import tiledb.vector_search'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb/vector_search/__init__.py", line 5, in <module>
    from .flat_index import FlatIndex
  File "/home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb/vector_search/flat_index.py", line 5, in <module>
    from tiledb.vector_search import index
  File "/home/dudoslav/.miniforge3/envs/zoo/lib/python3.11/site-packages/tiledb/vector_search/index.py", line 7, in <module>
    from tiledb.vector_search import _tiledbvspy as vspy
ImportError: libtiledb.so.2.22: cannot open shared object file: No such file or directory

So this needs fixing

Ok the shared object search path was modified here:
  if (APPLE)
    set_target_properties(${VSPY_TARGET_NAME} PROPERTIES INSTALL_RPATH "@loader_path/lib")
  elseif(UNIX)
    set_target_properties(${VSPY_TARGET_NAME} PROPERTIES INSTALL_RPATH "\$ORIGIN/lib")
  endif()

so this logic needs to only happen when we also package libtiledb.so

Why are we adding TILEDB_PATH here rather than just using TileDB_DIR?

@ihnorton I believe that was at my request. I would like to keep the build configuration options for vector search, soma, and vcf as consistent as possible. SOMA also uses TILEDB_PATH

I would like to keep the build configuration options for vector search, soma, and vcf as consistent as possible. SOMA also uses TILEDB_PATH

Isn't that only an environment variable, not a CMake variable?
https://github.com/search?q=repo%3Asingle-cell-data%2FTileDB-SOMA tiledb_path&type=code

Isn't that only an environment variable, not a CMake variable?

Yes, good point. I only care about the name of the env var, not the CMake variable

Thanks for working on this @dudoslav! I apologize for my delay in reviewing this PR.
I did some local testing. Within a conda environment, it works as I expected. However, outside of a conda environment, I am having trouble getting it to find libtiledb.so via TILEDB_PATH. Am I doing something wrong?
Also, is there is a small test that I can run to confirm that tiledb_vector_search is properly linked to the correct libtiledb.so? I noticed that neither RPATH or RUNPATH are set for the Python shared object (ie _tiledbvspy.cpython-XX-x86_64-linux-gnu.so).
The following demonstrates that within a conda env, the build finds libtiledb.so and does not copy it unnecessarily.
# conda

mamba create --yes -n test-vector-so \
  -c conda-forge --override-channels \
  python=3.9 tiledb
## + python                          3.9.19  h0755675_0_cpython  conda-forge     Cached
## + tiledb                          2.23.0  hfa691db_2          conda-forge        4MB
mamba activate test-vector-so
ls $CONDA_PREFIX/lib/libtiledb*
## /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so
## /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23

## current behavior
git log -n 1 --pretty=reference
## f564a8d (Share max int / float values and index lists in Python (#396), 2024-05-31)
python -m pip install -v .
## -- Found TileDB: /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23
## -- Installing: /tmp/tmp3nkssy3o/wheel/platlib/tiledb/vector_search/lib/libtiledb.so.2.23
ls $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/lib
## cmake  libdocopt.a  libtiledb.so.2.23  pkgconfig
python -m pip uninstall -v --yes tiledb_vector_search

## proposed behavior
git checkout db/sc-46695/install_group
git log -n 1 --pretty=reference
## 08e8fda (Change install directory, 2024-05-14)
python -m pip install -v .
## -- Found TileDB: /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23
ls $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/lib
## cmake  libdocopt.a  pkgconfig
ldd $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so
## linux-vdso.so.1 (0x00007ffd0c919000)
## libtiledb.so.2.23 => not found
## libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f29d6197000)
## libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f29d60b0000)
## libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f29d6090000)
## libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f29d5e67000)
## /lib64/ld-linux-x86-64.so.2 (0x00007f29d6742000)
readelf -d $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so | grep R*PATH
python -c "from tiledb.vector_search.version import version; print(version)"
## 0.3.1.dev28+g08e8fda
mamba deactivate

The below demonstrates that setting TILEDB_PATH is not sufficient for finding an external libtiledb.so (at least not how I expected to be able to use it).
# No conda

# Download pre-built libtiledb binary
wget -P /tmp/ https://github.com/TileDB-Inc/TileDB/releases/download/2.23.0/tiledb-linux-x86_64-2.23.0-152093b.tar.gz
mkdir /tmp/tmp-tiledb
tar xzf /tmp/tiledb-linux-x86_64-2.23.0-152093b.tar.gz -C /tmp/tmp-tiledb
ls /tmp/tmp-tiledb/lib/
## cmake  libtiledb.so  libtiledb.so.2.23  pkgconfig
# Create Python virtual env
python -m venv ./venv-vector
mamba deactivate
source ./venv-vector/bin/activate

## proposed behavior
export TILEDB_PATH=/tmp/tmp-tiledb
python -m pip install -v .
## -- Setting TileDB_DIR to /tmp/tmp-tiledb
## -- Could NOT find TileDB (missing: TileDB_DIR)
## -- Adding TileDB as an external project

export TILEDB_PATH=/tmp/tmp-tiledb/lib
python -m pip install -v .
## -- Setting TileDB_DIR to /tmp/tmp-tiledb/lib
## -- Could NOT find TileDB (missing: TileDB_DIR)
## -- Adding TileDB as an external project

deactivate

Thanks @jdblischak for your reply. Well, I am not sure what was the original path that we used to put there but right now a path to TileDBConfig.cmake is needed this works:
(venv-vector) taco:TileDB-Vector-Search dudoslav$ export TILEDB_PATH=/tmp/tmp-tiledb/lib/cmake/TileDB
(venv-vector) taco:TileDB-Vector-Search dudoslav$ python -m pip install -v .
...
...
  -- Found TileDB: /tmp/tmp-tiledb/lib/libtiledb.so.2.23

Of course, I can change this behavior. Another question is: Should I also set RPath even if I don't copy libtiledb.so into the wheel? So in this case set RPath to /tmp/tmp-tiledb/lib/libtiledb.so.2.23?

right now a path to TileDBConfig.cmake is needed this works

My proposal would be the following:

TILEDB_PATH is the path to the installation directory. For a conda env this would be $CONDA_PREFIX, for conda-build this would be $PREFIX, and for CMake this would be CMAKE_INSTALL_PREFIX
If TILEDB_PATH is defined, then update TileDB_DIR to $TILEDB_PATH/lib/cmake/TileDB

Another question is: Should I also set RPath even if I don't copy libtiledb.so into the wheel? So in this case set RPath to /tmp/tmp-tiledb/lib/libtiledb.so.2.23?

I think it makes sense to set RPath even in the case of using an external libtiledb.so. This will prevent runtime errors. On Linux setting LD_LIBRARY_PATH usually works, but this same trick typically doesn't work on macOS because of SIP, which blocks DYLD_LIBRARY_PATH.

@jdblischak I modified the build logic to ALWAYS set RPATH. When installing libtiledb.so (so for wheels) the RPATH is set to relative path: @loader_path/lib. When using external libtiledb.so the RPATH is set to the directory that the libtiledb.so is in. So for the second example you mentioned few comments ago the resulting readelf looks like this:
(venv-vector) taco:TileDB-Vector-Search dudoslav$ readelf -d venv-vector/lib/python3.10/site-packages/tiledb/vector_search/_tiledbvspy.cpython-310-x86_64-linux-gnu.so

Dynamic section at offset 0x358220 contains 30 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libtiledb.so.2.23]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [/tmp/tmp-tiledb/lib]
 0x000000000000000c (INIT)               0x6c000
...

Additionally, I modified the search logic to use CMAKE_PREFIX_PATH instead of TileDB_DIR so even export TILEDB_PATH=/tmp/tmp-tiledb works.
Please let me know if additional changes are required.

I re-ran my tests from above. From my perspective everything looks good. When an external libtiledb.so is used, it isn't copied unnecessarily into the installed Python package. I assume the existing CI ensures that the shared object is copied into the wheel for distribution to PyPI.
I did notice that ldd returned many more results in the conda env after modifying the RPATH, but I assume that is because there are so many shared objects installed there. I don't expect this to be a problem, but I wanted to point out the change in case that could signal a potential problem.
# conda

mamba activate test-vector-so
ls $CONDA_PREFIX/lib/libtiledb*
## /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so
## /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23

# Clean up from last time and update
python -m pip uninstall -v tiledb-vector-search
git pull upstream db/sc-46695/install_group

git log -n 1 --pretty=reference
## a7bf012 (Fix TileDB search directory and use RPATH, 2024-06-04)
python -m pip install -v .
## -- Found TileDB: /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23
ls $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/lib
## cmake  libdocopt.a  pkgconfig
ldd $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so | tail -n 3
## libicui18n.so.73 => /home/wsl/mambaforge/envs/test-vector-so/lib/./././libicui18n.so.73 (0x00007f39aa65b000)
## libicuuc.so.73 => /home/wsl/mambaforge/envs/test-vector-so/lib/./././libicuuc.so.73 (0x00007f39aa44f000)
## libicudata.so.73 => /home/wsl/mambaforge/envs/test-vector-so/lib/./././libicudata.so.73 (0x00007f39a85c0000)
ldd $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so | wc -l
## 95
ldd $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so | grep libtiledb
## libtiledb.so.2.23 => /home/wsl/mambaforge/envs/test-vector-so/lib/libtiledb.so.2.23 (0x00007f8c8d5ee000)
readelf -d $CONDA_PREFIX/lib/python3.9/site-packages/tiledb/vector_search/_tiledbvspy.cpython-39-x86_64-linux-gnu.so | grep R*PATH
##  0x000000000000001d (RUNPATH)            Library runpath: [/home/wsl/mambaforge/envs/test-vector-so/lib]
python -c "from tiledb.vector_search.version import version; print(version)"
## 0.3.1.dev29+ga7bf012
mamba deactivate

# No conda

# Download pre-built libtiledb binary
wget -P /tmp/ https://github.com/TileDB-Inc/TileDB/releases/download/2.23.0/tiledb-linux-x86_64-2.23.0-152093b.tar.gz
mkdir /tmp/tmp-tiledb
tar xzf /tmp/tiledb-linux-x86_64-2.23.0-152093b.tar.gz -C /tmp/tmp-tiledb
ls /tmp/tmp-tiledb/lib/
## cmake  libtiledb.so  libtiledb.so.2.23  pkgconfig
mamba deactivate
source ./venv-vector/bin/activate
# Clean up from last time
python -m pip uninstall -v tiledb-vector-search

export TILEDB_PATH=/tmp/tmp-tiledb
python -m pip install -v .
## -- Setting TileDB_DIR to /tmp/tmp-tiledb
## -- Could NOT find TileDB (missing: TileDB_DIR)
## -- Adding TileDB as an external project

export TILEDB_PATH=/tmp/tmp-tiledb/lib
python -m pip install -v .
## -- Adding TILEDB_PATH to CMAKE_PREFIX_PATH
## -- Found TileDB: /tmp/tmp-tiledb/lib/libtiledb.so.2.23
## -- TileDB_DIR is set to /tmp/tmp-tiledb/lib/cmake/TileDB -- find_package will search there first.
## -- Found TileDB: /tmp/tmp-tiledb/lib/libtiledb.so.2.23
## -- Setting RPATH to /tmp/tmp-tiledb/lib

ls ./venv-vector/lib/python3.10/site-packages/tiledb/vector_search/lib
## cmake  libdocopt.a  pkgconfig
ldd ./venv-vector/lib/python3.10/site-packages/tiledb/vector_search/_tiledbvspy.cpython-310-x86_64-linux-gnu.so | wc -l
## 10
ldd ./venv-vector/lib/python3.10/site-packages/tiledb/vector_search/_tiledbvspy.cpython-310-x86_64-linux-gnu.so | grep libtiledb
##         libtiledb.so.2.23 => /tmp/tmp-tiledb/lib/libtiledb.so.2.23 (0x00007f1db76b1000)
readelf -d ./venv-vector/lib/python3.10/site-packages/tiledb/vector_search/_tiledbvspy.cpython-310-x86_64-linux-gnu.so | grep R*PATH
##  0x000000000000001d (RUNPATH)            Library runpath: [/tmp/tmp-tiledb/lib]
python -c "from tiledb.vector_search.version import version; print(version)"
## 0.3.1.dev29+ga7bf012
deactivate


support custom plugin sources

In some organizations they may have developed their own custom vagrant plugins that are not available on rubygems.org. This will allow users to pass alternate source locations to look for plugins.
The new attribute will support an Array to support an admittedly limited edge case. That edge case is where the plugin is in one custom source, but depends on some other internally developed gem which lives in a different gem repository.

@jtimberman any thoughts on this?

Hey @dpetzel
This needs a rebase with master, it won't merge cleanly.

Otherwise I'm :+1: :cake:

@jtimberman done!
Thanks

side note: I noticed during my rebase, it appears the install action no longer flags as updated, so I expected notifications might not work anymore?

@jtimberman ping.

I'm behind on review for a bunch of issues and PRs this week. Maybe next week will open up.


When eureka registration is occuring the app name is empty in the payload

Describe the bug
The app name in the registration payload is empty
Steps to reproduce
Steps to reproduce the behavior:

Have application perform registration with eureka server
Observe body of request doesn't have app name set but is a url parameter

Expected behavior
A clear and concise description of what you expected to happen.
Environment (please complete the following information):

Steeltoe Version 3.2.6
OS: Windows,
.NET 8

I'm afraid this doesn't provide sufficient information to help you. Can you be more specific?

Apologies for that @bart-vmware what I have noticed is that in the payload received by my custom eureka endpoint is that the app name is not set in the payload but the other fields ie app group is.

Can you try debugging into Steeltoe to identify why?

Sure thing will do that tomorrow & report back

Interesting findings so far, is that data is correctly being included when the request is being constructed however it is not recieved when binding the body to the same class. What i do notice is the JsonProperty name differs to the property name, making them the same hasn't helped either.
Next step for me is to make the post using postman and see if it is a de-serialisation problem or a serializatioon problem.

I'm closing this issue, as it concerns a serialization issue in custom code, not a bug in Steeltoe. Feel free to open a new issue when you've identified a Steeltoe bug.

found the root cause. I was using newtonsoft for my deserialisation rather than system.text.json hence it didn't respect the JsonPropertyName attributes.


During training about unet and controlnet bin

after training 10000 steps there is not saved unet and conrtolnet bin

thanks

i am not sure the reason why must be saved  after all epoches done

Please check FAQ Q6.


WIP: Php 7 webserver

Current coverage is 47.07% (diff: 100%)

Merging #151 into 2.x will not change coverage

@@                2.x       #151   diff @@
==========================================
  Files            47         47
  Lines           820        820
  Methods         190        190
  Messages          0          0
  Branches          0          0
==========================================
  Hits            386        386
  Misses          434        434
  Partials          0          0

Powered by Codecov. Last update 7f995d0...147ad42


Ammo blacklist

The way it works right now is that when you select a round type, then the round class, you may not combine the two to create illogical stuff like smoke AGM-114s.
However, if you select the round class first, say the AGM-114, and then select smoke, you bypass the blacklist altogether.
I'm not sure if this is just on my end, or whether it's restricted to the AGM-114s, but this definitely needs investigating.

my GOD wrong ISSUE
Fixt 2ac554f52cc548949d3dc6142090014fd5dd516f
(i goofed up with the link-time checking on the racks, but funnily enough this is a common bug with acf-tool:)


Cannot checkpoint Optimizer as its not a trackable object.

Check pointing optimizer fails in TF 2.16 with Keras 3.  We have may tensorflow.org tutorials that do similar things - https://www.tensorflow.org/tutorials/generative/pix2pix/
This code snippet works in TF 2.15
import numpy as np
import tensorflow as tf

print(tf.__version__)

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1)
])
optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)
x, y = np.random.random((10, 2)), np.random.random((10, 1))
model.compile(optimizer, "mse")
model.fit(x, y)

checkpoint = tf.train.Checkpoint(model) # Works
checkpoint = tf.train.Checkpoint(model, optimizer=optimizer)  # Fails in TF 2.16 when we checkpoint optimizer

/usr/local/lib/python3.10/dist-packages/tensorflow/python/checkpoint/checkpoint.py in _assert_trackable(obj, name)
   1571   if not isinstance(
   1572       obj, (base.Trackable, def_function.Function)):
-> 1573     raise ValueError(
   1574         f"`Checkpoint` was expecting {name} to be a trackable object (an "
   1575         f"object derived from `Trackable`), got {obj}. If you believe this "

ValueError: `Checkpoint` was expecting optimizer to be a trackable object (an object derived from `Trackable`), got <keras.src.optimizers.adam.Adam object at 0x7bd306cedfc0>. If you believe this object should be trackable (i.e. it is part of the TensorFlow Python API and manages state), please open an issue.

Thanks for the report. Will take up this one.

Done.


midori|vimb: fix tarball job

Motivation for this change
this is needed after c6e51d03695efa90a8d09b41fe6441953e21d52a
This is apparently news to everyone involved in #130019, including myself, but wrapFirefox is used by browsers other than firefox. Who would've thought.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

I'm surprised this wasn't detected by OfBorg.


Not getting same results in cli sqlmap

Is there some kind of magic happening between sqlipy in burp and the sqlmapapi?  I take the exact command used in sqlipy in burp that will find a sql injection in a post request.  I take the same command in the command line and it doesn't find it...
Also where is sqlipy storing it's found sql injections?
Thanks!

Not that I am aware. The sqlmapapi script from the sqlmap project really just stands up a RESTful API server that turns requests into full sqlmap commands passed and creates instances of sqlmap using those commands.
So you are saying you are finding issues within the extension that you ARE NOT finding just using the command line? That is interesting if so. Are you using the same version on the command line (the one bundled with the extension)?
Are you using BurpSuite Pro or the free version? If using Pro, then any findings are populated in the "Issues" section of the "Target" tab for the URL. If using the free version, then findings are created as .html files with the long sqlmapapi task ID as the file name - these should get saved in either the root folder for the extension or the path where Burp resides.

Thanks for the reply,
I am using Burp Pro, and I have tried the most recent version of sqlmap as well as the sqlmap.py that is within the .Burpsuite etc etc etc folder that houses the extension.  When I take the request as shown in the log of the extension within burp suite, it does not find the sql injection, However when I save the request from within Burp and run sqlmap with -r it works fine I found out.  Just a strange thing happening.
Using --flush-session with the -r function finds it again.  However when taking the sqlmap.py command shown in the extension log (which finds the sql injection within the extension) and trying to re create it from command line it does not find it.

The version of sqlmap included with the extension is not the most current, maybe that is your issue. If not, then it is a sqlmap issue and not anything I can resolve.


Cannot get pod energy information

Hi, I'm trying to use Kepler in my k8s cluster. It was deployed on one node (node1) together with Prometheus and Grafana.
There are many pods running on this node. I was expecting all the pod energy can be displayed in Grafana dashboard, however I can only see one record of "pod_energy_stat" in Prometheus, which is about "pod_name="system_processes", pod_namespace="system"", and this pod/namespace doesn't even exist in my cluster. Do you have any clue on what the issue is about?

thank you for reporting this issue. In your issue, it looks Kepler failed to resolve the pod and attributed all energy usage to the system process (i.e. using system_process as pod name and system as pod namespace). If Kepler cannot find the pods, the root cause are likely in the container runtime and kubelet port in your cluster.
Can you check the following and share your cluster info?

what is the container runtime? We've tested cri-o. If your container runtime is different, there might be a different sysfs path for the pod, and that makes Kepler unable to find the pod name
which port does kublet uses? Kepler assumes default port 10250. If your kubelet runs on a different port, Kepler may not be able to find out. The new port has to be set in the env var
do you have the kepler log available, i.e. the output of kubectl logs -n monitoring daemonset/kepler-exporter?

Thanks for prompt response. Please see my answers inline.

do you use cgroup v1 or v2? v2 is the assumed default.

I think my kernel supports both v1 and v2
$ mount | grep '^cgroup' | awk '{print $1}' | uniq
cgroup2
cgroup

what is the container runtime?

It's containerd://1.5.9-0ubuntu3

which port does kubelet uses?

It should be the default port.

do you have the kepler log available?

The log is like the following, all about "system_processes" in namespace "system"
2022/08/16 03:20:48 energy count: core 126086.00 dram: 6008.00 time 0.000000 cycles<PHONE_NUMBER>8 instructions<PHONE_NUMBER>4 misses 33231048 node memory<PHONE_NUMBER>.000
2022/08/16 03:20:48     energy from pod: name: system_processes namespace: system
eCore: 183834(48858366737) eDram: 132(9223372036862737727) eOther: 0(0) eGPU: 0(0)
CPUTime: 0.00 (NaN)
cycles:<PHONE_NUMBER>8 (1.0000)
instructions:<PHONE_NUMBER>4 (1.0000)
DiskReadBytes: 0 (0)
DiskWriteBytes: 0 (0)
misses: 33231048 (1.0000)       ResidentMemRatio: 0.1929
avgCPUFreq: 1707.8185 MHZ
pid: 1002212 comm: containerd-shim
cgroupfs: map[]

Thanks, I'll check a setup using containerd

turns out containerd on my setup has a different path pattern:
# kubectl describe pod nginx-7cd588b686-mkpzs |grep "Container ID"
    Container ID:   containerd://286b15051ec43375190802e1f40562536980a8fd97e75bb89c7f2eec6f995f17
# find  /sys/fs/cgroup/systemd/ -iname "*286b15051ec43375190802e1f40562536980a8fd97e75bb89c7f2eec6f995f17"
/sys/fs/cgroup/systemd/system.slice/containerd.service/kubepods-burstable-poda3b200c9_db51_40b4_9d2d_53f8fdf80d7f.slice:cri-containerd:286b15051ec43375190802e1f40562536980a8fd97e75bb89c7f2eec6f995f17

while the regex used to parse container path doesn't capture this pattern
cc @marceloamaral

this was tested on ubuntu 20.04 with containerd version
# containerd -v
containerd github.com/containerd/containerd 1.5.9-0ubuntu1~20.04.4

@ruomengh would you like to provide a fix for this? We can go through the development process to help you get started.

@ruomengh would you like to provide a fix for this? We can go through the development process to help you get started.

Sounds good. I'd like to have a try but I may need some guidance.

cool!

I've email the development process, please let me know if you have any issues there, looking forward to your contribution!

@ruomengh can you try the latest kepler container image? I just hit the same issue on RHEL 8 with cri-o, but it seems #108 fixed it

Re-deploy kepler with the latest image and the issue remains.

@ruomengh
What is the detected kernel version?
Could you post head part of the kepler?
> kubectl logs kepler-exporter-6tbj9 -n monitoring | head
2022/08/18 01:50:57 InitSliceHandler: &{map[] /sys/fs/cgroup/system.slice /sys/fs/cgroup/system.slice /sys/fs/cgroup/system.slice}
use sysfs to obtain power
config EnabledEBPFCgroupID enabled:  true
config getKernelVersion:  4.18
config set EnabledEBPFCgroupID to  true

2022/08/18 01:33:03 InitSliceHandler: &{map[] /sys/fs/cgroup/cpu /sys/fs/cgroup/memory /sys/fs/cgroup/blkio}
use sysfs to obtain power
config EnabledEBPFCgroupID enabled:  true
config getKernelVersion:  5.15
config set EnabledEBPFCgroupID to  false

2022/08/18 01:33:03 InitSliceHandler: &{map[] /sys/fs/cgroup/cpu /sys/fs/cgroup/memory /sys/fs/cgroup/blkio}
use sysfs to obtain power
config EnabledEBPFCgroupID enabled: true
config getKernelVersion: 5.15
config set EnabledEBPFCgroupID to false

According to this log, the only condition that causes EnabledEBPFCgroupID disabled is cgroup.
The current implementation check the cgroup from this static path exists /sys/fs/cgroup/cgroup.controllers.
Could you confirm that this path exists on your host?
If exists on host, double check that Kepler deployment manifest is mounted to /sys path (this should be defined in the provided manifest).
If not, I think we need to fix the method to detect cgroup version to cover this.

The path doesn't exist -
ls: cannot access '/sys/fs/cgroup/cgroup.controllers': No such file or directory

I see. It seems like your system is not installed cgroupv2 by this approach.
So that the file isn't created there as expected from this issue https://github.com/sustainable-computing-io/kepler/issues/29.
I think we should change the way to detect cgroupv2 by mount point on host as you did or check /proc/filesystems file.

Seems cgroup v2 is disabled on the node to avoid another issue.
I tried this for cgroup v1 as @rootfs suggested but it doesn't work either https://github.com/sustainable-computing-io/kepler/blob/main/manifests/kubernetes/deployment.yaml#L71

Since we support cgroup v1 now, @ruomengh do we still have this issue?

@jiere I assume the issue has been gone in our environment. Please help to confirm. Thanks.

Using quay.io/sustainable_computing_io/kepler:release-0.5 on Fedora Core 33 with cGroup v1 (systemd.unified_cgroup_hierarchy=0 in grub) does now work as expected:
See bellow an extract of the logs (verbose=5) where every container has the same counters:
I0623 15:45:09.166734       1 metric_collector.go:137] energy from pod/container (0 active processes): name: csi-cinder-nodeplugin-4jx6c/cinder-csi-plugin namespace: kube-system
    cgrouppid: 0 pid: [] comm:
    Dyn ePkg (mJ): 9512 (4461128) (eCore: 9512 (4461128) eDram: 150 (70350) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    Idle ePkg (mJ): 0 (0) (eCore: 0 (0) eDram: 0 (0) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    CPUTime:  0 (0)
    NetTX IRQ: 0 (0)
    NetRX IRQ: 0 (0)
    Block IRQ: 0 (0)
    counters: map[cache_miss:0 (0) cpu_cycles:0 (0) cpu_instr:0 (0) cpu_ref_cycles:0 (0)]
    cgroupfs: map[block_devices_used:0 (0) cgroupfs_cpu_usage_us:0 (0) cgroupfs_ioread_bytes:0 (0) cgroupfs_iowrite_bytes:0 (0) cgroupfs_kernel_memory_usage_bytes:0 (0) cgroupfs_memory_usage_bytes:0 (0) cgroupfs_system_cpu_usage_us:0 (0) cgroupfs_tcp_memory_usage_bytes:0 (0) cgroupfs_user_cpu_usage_us:0 (0)]
    kubelets: map[container_cpu_usage_seconds_total:0 (1503) container_memory_working_set_bytes:0 (13774848)]

I0623 15:45:09.166748       1 metric_collector.go:137] energy from pod/container (0 active processes): name: csi-cinder-nodeplugin-4jx6c/node-driver-registrar namespace: kube-system
    cgrouppid: 0 pid: [] comm:
    Dyn ePkg (mJ): 9512 (4461128) (eCore: 9512 (4461128) eDram: 150 (70350) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    Idle ePkg (mJ): 0 (0) (eCore: 0 (0) eDram: 0 (0) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    CPUTime:  0 (0)
    NetTX IRQ: 0 (0)
    NetRX IRQ: 0 (0)
    Block IRQ: 0 (0)
    counters: map[cache_miss:0 (0) cpu_cycles:0 (0) cpu_instr:0 (0) cpu_ref_cycles:0 (0)]
    cgroupfs: map[block_devices_used:0 (0) cgroupfs_cpu_usage_us:0 (0) cgroupfs_ioread_bytes:0 (0) cgroupfs_iowrite_bytes:0 (0) cgroupfs_kernel_memory_usage_bytes:0 (0) cgroupfs_memory_usage_bytes:0 (0) cgroupfs_system_cpu_usage_us:0 (0) cgroupfs_tcp_memory_usage_bytes:0 (0) cgroupfs_user_cpu_usage_us:0 (0)]
    kubelets: map[container_cpu_usage_seconds_total:0 (13) container_memory_working_set_bytes:0 (3063808)]

Using quay.io/sustainable_computing_io/kepler:release-0.5 on Fedora Core 33 with cGroup v1 (systemd.unified_cgroup_hierarchy=0 in grub) does not work as expected:
See bellow an extract of the logs (verbose=5) where every container has the same counters:
I0623 15:45:09.166734       1 metric_collector.go:137] energy from pod/container (0 active processes): name: csi-cinder-nodeplugin-4jx6c/cinder-csi-plugin namespace: kube-system
    cgrouppid: 0 pid: [] comm:
    Dyn ePkg (mJ): 9512 (4461128) (eCore: 9512 (4461128) eDram: 150 (70350) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    Idle ePkg (mJ): 0 (0) (eCore: 0 (0) eDram: 0 (0) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    CPUTime:  0 (0)
    NetTX IRQ: 0 (0)
    NetRX IRQ: 0 (0)
    Block IRQ: 0 (0)
    counters: map[cache_miss:0 (0) cpu_cycles:0 (0) cpu_instr:0 (0) cpu_ref_cycles:0 (0)]
    cgroupfs: map[block_devices_used:0 (0) cgroupfs_cpu_usage_us:0 (0) cgroupfs_ioread_bytes:0 (0) cgroupfs_iowrite_bytes:0 (0) cgroupfs_kernel_memory_usage_bytes:0 (0) cgroupfs_memory_usage_bytes:0 (0) cgroupfs_system_cpu_usage_us:0 (0) cgroupfs_tcp_memory_usage_bytes:0 (0) cgroupfs_user_cpu_usage_us:0 (0)]
    kubelets: map[container_cpu_usage_seconds_total:0 (1503) container_memory_working_set_bytes:0 (13774848)]

I0623 15:45:09.166748       1 metric_collector.go:137] energy from pod/container (0 active processes): name: csi-cinder-nodeplugin-4jx6c/node-driver-registrar namespace: kube-system
    cgrouppid: 0 pid: [] comm:
    Dyn ePkg (mJ): 9512 (4461128) (eCore: 9512 (4461128) eDram: 150 (70350) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    Idle ePkg (mJ): 0 (0) (eCore: 0 (0) eDram: 0 (0) eUncore: 0 (0)) eGPU (mJ): 0 (0) eOther (mJ): 0 (0)
    CPUTime:  0 (0)
    NetTX IRQ: 0 (0)
    NetRX IRQ: 0 (0)
    Block IRQ: 0 (0)
    counters: map[cache_miss:0 (0) cpu_cycles:0 (0) cpu_instr:0 (0) cpu_ref_cycles:0 (0)]
    cgroupfs: map[block_devices_used:0 (0) cgroupfs_cpu_usage_us:0 (0) cgroupfs_ioread_bytes:0 (0) cgroupfs_iowrite_bytes:0 (0) cgroupfs_kernel_memory_usage_bytes:0 (0) cgroupfs_memory_usage_bytes:0 (0) cgroupfs_system_cpu_usage_us:0 (0) cgroupfs_tcp_memory_usage_bytes:0 (0) cgroupfs_user_cpu_usage_us:0 (0)]
    kubelets: map[container_cpu_usage_seconds_total:0 (13) container_memory_working_set_bytes:0 (3063808)]

@marceloamaral

@rootfs the problem is the BPF code.
With the apiserver we can now identify the containers, but the apiserver does not give the PIDs into the containers. You can see in the logs that all pid list are empty. The cgroup metrics can only be extracted if we have the pid of a process in the cgroup....
The PID information comes from the eBPF code, which is probably not working.
How is this environment? Is it bare-metal or VM?
How k8s was deployed? Mini-clusters like Kind and minikube does not expose the host /proc folder by default.
So, when we create a kind cluster we mount the host folder so that Kepler can access the pid informations.

This Kubernetes environment has been deployed using Openstack Magnum on VMs.

@fadam-csgroup
Can you check if you have visibility of the PIDs?
The command oc exec -it -n kepler service/kepler-exporter -- ls /proc should show a lot of PIDs.

@marceloamaral
Yes, the command give me lots of PIDs:
$ kubectl -n monitoring exec -it ds/kepler -- ls /proc -1 | grep '[0-9]' | wc -l
553

@fadam-csgroup can you please share the Kepler logs?
Put it in https://pastebin.com/

@marceloamaral Here are the logs: https://pastebin.com/2i7RX0Y8

I0629 12:29:27.877410       1 process_metric.go:147] cannot extract: container_cpu_usage_seconds_total
I0629 12:29:27.877413       1 process_metric.go:147] cannot extract: container_memory_working_set_bytes
I0629 12:29:27.877418       1 process_metric.go:147] cannot extract: cgroupfs_memory_usage_bytes
I0629 12:29:27.877421       1 process_metric.go:147] cannot extract: cgroupfs_kernel_memory_usage_bytes
I0629 12:29:27.877424       1 process_metric.go:147] cannot extract: cgroupfs_tcp_memory_usage_bytes
I0629 12:29:27.877427       1 process_metric.go:147] cannot extract: cgroupfs_cpu_usage_us
I0629 12:29:27.877431       1 process_metric.go:147] cannot extract: cgroupfs_system_cpu_usage_us
I0629 12:29:27.877434       1 process_metric.go:147] cannot extract: cgroupfs_user_cpu_usage_us

this is probably the same issue as https://github.com/sustainable-computing-io/kepler/discussions/750#discussioncomment-6265641
@mcalman

@fadam-csgroup on your setup, is CPU and memory accounting turned on? The following is from my setup, the cpu and memory accounting are turned on for kubelet
# sudo grep -Ri accounting /etc/systemd/ |grep -v \#
/etc/systemd/system/user-.slice.d/override.conf:CPUAccounting=yes
/etc/systemd/system/user-.slice.d/override.conf:MemoryAccounting=yes
/etc/systemd/system/user-.slice.d/override.conf:IOAccounting=yes
/etc/systemd/system/user-.slice.d/override.conf:TasksAccounting=yes
/etc/systemd/system.conf.d/kubelet-cgroups.conf:DefaultCPUAccounting=yes
/etc/systemd/system.conf.d/kubelet-cgroups.conf:DefaultMemoryAccounting=yes
/etc/systemd/system.conf.d/kubelet-cgroups.conf:DefaultBlockIOAccounting=yes

@rootfs
It seems that Memory accounting in enable but CPU is not:
# systemctl show kubelet |grep -i accounting
CPUAccounting=no
IOAccounting=no
BlockIOAccounting=no
MemoryAccounting=yes
TasksAccounting=yes
IPAccounting=no

@marceloamaral Here are the logs: https://pastebin.com/2i7RX0Y8

According to the log, it seems that you use cgroupv1 because it determines cgroup path from /proc.
I think the proc namespace of the container and the machine are different. I had a similar issue before with minikube with docker driver. I had to mount the machine /proc to the cluster node to solve the issue.
I debug that locally by printing the command together with pid number from the ct variable. I think we should add that in the log for determining the issue too.

@fadam-csgroup can you turn on kubelet cpu accounting and restart kepler?
systemctl set-property --runtime kubelet CPUAccounting=yes

If you hit the issue explained by @sunya-ch , can you turn on hostPID in Kepler pod spec?
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kepler-exporter
  namespace: system
spec:
  selector:
    matchLabels:
      app.kubernetes.io/component: exporter
      app.kubernetes.io/name: kepler-exporter
  template:
    metadata:
      labels:
        app.kubernetes.io/component: exporter
        app.kubernetes.io/name: kepler-exporter
    spec:
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccountName: kepler-sa
      hostPID: true  <---- turn on host PID so kepler sees the processes the same way as the pids on the host.

@rootfs Here are the logs with the changes: https://pastebin.com/M4K616sg
This has changed a little bit the Kepler Graph in Grafana for the system processes (1 watt more), but has not change anything for real containers.
However I did not restart kubelet after setting the CPU Accounting, and this may be required. I will to it later, when I won't impact to many running pods.

@fadam-csgroup I am not quite familiar with creating kubernetes on Fedora CoreOS. On my own setup created by kubeadm, I have this file that seems to enable CPUAccounting for kubelet
# more /etc/systemd/system.conf.d/kubelet-cgroups.conf
[Manager]
DefaultCPUAccounting=yes
DefaultMemoryAccounting=yes
DefaultBlockIOAccounting=yes


ERROR IN CASE OF UNSYNC HISTORIES IN TIME DEP BASIC STATISTICS!

Issue Description
A user reported a crash in the time-dependent basic statistics if the histories are not syncronized
What did you expect to see happen?
An error by RAVEN (as it used to happen)
What did you see instead?
An not-understandable crash!
Do you have a suggested fix for the development team?
check the sync of the histories!
Please attach the input file(s) that generate this error.  The simpler the input, the faster we can find the issue.

For Change Control Board: Issue Review
This review should occur before any development is performed as a response to this issue.

[x] 1. Is it tagged with a type: defect or task?
[x] 2. Is it tagged with a priority: critical, normal or minor?
[x] 3. If it will impact requirements or requirements tests, is it tagged with requirements?
[x] 4. If it is a defect, can it cause wrong results for users? If so an email needs to be sent to the users.
[x] 5. Is a rationale provided? (Such as explaining why the improvement is needed or why current code is wrong.)

For Change Control Board: Issue Closure
This review should occur when the issue is imminently going to be closed.

[x] 1. If the issue is a defect, is the defect fixed?
[x] 2. If the issue is a defect, is the defect tested for in the regression test system?  (If not explain why not.)
[x] 3. If the issue can impact users, has an email to the users group been written (the email should specify if the defect impacts stable or master)?
[x] 4. If the issue is a defect, does it impact the latest release branch? If yes, is there any issue tagged with release (create if needed)?
[x] 5. If the issue is being closed without a pull request, has an explanation of why it is being closed been provided?

foulty_input.inl.zip

added input file and database to replicate the error

@alfoa FYI, The reason for the crash is caused by the duplicated values in the pivot parameter.

Closure checklist passed...
Approved closure via #1012


Rubinius 3.107 includes a fix for the stalled testing bug

Fixed by: https://github.com/rubinius/rubinius/commit/387c488798d8b911f160d36278710474e526d73f

Thanks @brendon!

You're welcome :) Good to have the whole suite running :)


Violations for IaC templates lack accurate line number

terrascan version: 1.13.2
Operating System: Windows

Description
I tried scanning a copy of the Azure Quickstart Templates repository to get an idea of what types of issues would be found.  I noticed that in all violations reported, the line number is always 1.  It would be helpful if the line number reported was actually where the failure is found in the template.  For example, in the violations below, the line numbers should be a line within the declarations of the storage account and subnet that violate the rules.
More IaC file types may have this same issue, but I only tested this with ARM.
What I Did
d:\terrascan-1.13.2>terrascan.exe scan -i arm -f ..\azure-quickstart-templates\quickstarts\microsoft.compute\vm-simple-windows\azuredeploy.json

Violation Details -

        Description    :        Ensure that 'Secure transfer required' is enabled for Storage Accounts
        File           :        azuredeploy.json
        Line           :        1
        Severity       :        HIGH
        -----------------------------------------------------------------------

        Description    :        Ensure that Azure Virtual Network subnet is configured with a Network Security Group
        File           :        azuredeploy.json
        Line           :        1
        Severity       :        MEDIUM
        -----------------------------------------------------------------------

Scan Summary -

        File/Folder         :   d:\azure-quickstart-templates\quickstarts\microsoft.compute\vm-simple-windows\azuredeploy.json
        IaC Type            :   arm
        Scanned At          :   2022-04-21 00:24:29.8549459 +0000 UTC
        Policies Validated  :   4
        Violated Policies   :   2
        Low                 :   0
        Medium              :   1
        High                :   1

I've also confirmed this is a problem for Docker templates as well, scanning the TerraGoat repository.
d:\terrascan-1.13.2>terrascan.exe scan -i docker -d ..\terragoat

Violation Details -

        Description    :        Ensure platform flag with FROM command is not used for Docker file
        File           :        terraform\aws\resources\Dockerfile
        Line           :        1
        Severity       :        MEDIUM
        -----------------------------------------------------------------------

Scan Summary -

        File/Folder         :   d:\terragoat
        IaC Type            :   docker
        Scanned At          :   2022-04-21 00:32:25.3564555 +0000 UTC
        Policies Validated  :   3
        Violated Policies   :   1
        Low                 :   0
        Medium              :   1
        High                :   0


No way to contact team to join effort

A few weeks ago I discovered this project and wanted to get involved. I both attempted to message this Discord ID and email the address from the readme, but ran into the following issues:

The Discord user doesn't accept messages from people not in the server, so Clyde auto-blocked me.
The email seems to be invalid, as the mail system it's a part of autoreplied saying that the email doesn't exist in the system.

As neither of the ways of contact expressed are usable, I am creating an issue proposing that a discord server or some equivalent system of communication is implemented, as there is at present little information available about this project, what has been completed, its design goals, or anything else, really.
Personally I have experience with UI design and wish to learn Rust, and I was contacted by someone on Twitter earlier who also expressed an interest in assisting with this project after I directed them here. They found a post of mine on the SMWCentral forums and messaged me on Twitter before finding this project because there is very little footprint. I and many others would be happy to help, we just need direction, and communication is vital for that.

Hey, thanks for bringing this up. The contact details were indeed out of date, I've just updated them in the readme. Feel free to reach out using those.


I know how to AllowPublicKeys on the laptop version but how do I restrict access similarly to only specific public keys on the Android APK version?

I know how to AllowPublicKeys on the laptop version but how do I restrict access similarly to only specific public keys on the Android APK version?

For starters and at the very least please update the comment section of the config file so it is very clear that AllowedPublicKeys does not do what people think it does.

Did the other day: https://github.com/yggdrasil-network/yggdrasil-go/commit/ff0ef7ff567b772c6a45a60eb37ed9982a55aac7


arch-arm: Hook TLBIOS instructions to the TlbiShareable obj

FEAT_TLBIOS has been introduced by a recent patch 1 which was however missing to include the outer shareable case in the Msr disambiguation switch. Which meant the TLBIOS instructions were decoded as normal MSR instructions, with no effect whatsoever on the TLBs
Change-Id: I41665a4634fbe0ee8cc30dbc5d88d63103082ae9

I've assigned you to this change, Andreas. We are going to use "assignees" as the point maintainer who is responsible for merging the change. Thanks!


Thumbnail not showing and Icon on section has gone sometime

Description
Thumbnail not showing and Icon on section has gone sometime
https://imgur.com/a/HKIYogL
Steps To Reproduce
Thumbnail not showing and Icon on section has gone sometime
https://imgur.com/a/HKIYogL
Expected behavior
Thumbnail not showing and Icon on section has gone sometime
https://imgur.com/a/HKIYogL
Files Version
<IP_ADDRESS>
Windows Version
10.0.22543.1000
Relevant Assets/Logs
Thumbnail not showing and Icon on section has gone sometime
https://imgur.com/a/HKIYogL

Could you post the two app's debug logs? You can get them from settings > about


runtimes.rb:  autodetect Could not find a JavaScript runtime

execjs: 2.2.1
ruby:  2.1.0
OS:  Debian/testing on PowerPC
I am trying to use Jekyll (http://www.jekylrb.com) and I get the following error when I do
anything with it:
% jekyll --version
/var/lib/gems/2.1.0/gems/execjs-2.2.1/lib/execjs/runtimes.rb:51:in `autodetect': Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes. (ExecJS::RuntimeUnavailable)
        from /var/lib/gems/2.1.0/gems/execjs-2.2.1/lib/execjs.rb:5:in `<module:ExecJS>'
        from /var/lib/gems/2.1.0/gems/execjs-2.2.1/lib/execjs.rb:4:in `<top (required)>'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/lib/gems/2.1.0/gems/coffee-script-2.2.0/lib/coffee_script.rb:1:in `<top (required)>'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/lib/gems/2.1.0/gems/coffee-script-2.2.0/lib/coffee-script.rb:1:in `<top (required)>'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/lib/gems/2.1.0/gems/jekyll-coffeescript-1.0.0/lib/jekyll-coffeescript.rb:2:in `<top (required)>'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/lib/gems/2.1.0/gems/jekyll-2.1.0/lib/jekyll.rb:75:in `<top (required)>'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /var/lib/gems/2.1.0/gems/jekyll-2.1.0/bin/jekyll:6:in `<top (required)>'
        from /usr/local/bin/jekyll:23:in `load'
        from /usr/local/bin/jekyll:23:in `<main>'

'''

I have Rhino and Spidermonkey installed.   I have also installed therubyrhino as per the documentation.

Add "gem 'therubyracer'" on Gemfile works for me...

@welligton1993 adding that to the gemfile then using bundle install worked.

@red-nova  Exactly... Fist add "gem 'therubyracer' to Gemfile and after  execute bundle install

execjs: 2.2.1
ruby: 1.9.3-p551
OS: Ubuntu 14.04.01
Reproduced using this tutorial code https://github.com/gotealeaf/bootstrap-todo-app
Fixed adding 'therubyracer' to Gemfile as workaround worked for me

This repository has been deprecated and has moved to https://github.com/rails/execjs.

installing therubyracer gem worked for me

nano Gemfile
add in the file : 'gem 'therubyracer'
bundle install
Works great :+1:


Kyle/motoko unit tests

Overview
This demonstrates a pattern for running unit tests on Motoko modules
Requirements
We should have a pattern for running a simple test suite with pass / fail conditions
Recommended Solution
This is a pattern that used to be in the Motoko base library as an example

thank you for your contribution. as this PR is quite outdated, I will be closing it. if you would still like to submit your changes, please do so in a new PR. thank you for your understanding.


[WIP] hydra ddp support

@omry any thoughts on how to handle the flag=val issue?
TODO: call Popen with arg=val command

@Borda can you give this last tod a shot?
We need to get Popen to call gpus=x when user is using hydra (instead of --gpus x)

@Borda @omry we need to make this release today. if we can't support --gpus x or find a way of passing gpus=x to the script this support will need to wait till the next release.

Quick comment on this, from browsing the code I see the --gpus flag is only added if it was not present in the original command. So, from one perspective of using Hydra with PL, I at the moment have a main configuration file which contains a subgroup trainer for use with PL. I then feed this subgroup into trainer by **OmegaConf.to_container(cfg.trainer, resolve=True). Hence, there is no --gpus flag in the command line but in my yaml file it is present.  Given this flow, I believe that the --gpus flag is unnecessary. In the end, I think this becomes a question of how Hydra should be supported in PL.

@anthonytec2 great. when it's hydra we won't pass the gpus command

that already happens.... it’s the rest of the command.

that already happens.... it’s the rest of the command.

Why the special treatment to --gpu?

because the gpus available may change depending on the environment and how we launch the scripts.
but anyhow, we need to push this 0.8.0 release, so let's table this for a future date haha. we need to shift focus back to the major parts of this release

Let's revisit this later.
PL should be agnostic to the command line of the user application.
If it needs to override them there should be a clearly defined protocol (not "best practices").


Cache API example not working as written?

Implemented the exact same code here on my Cloudflare Workers
https://github.com/cloudflare/workers-docs-engine/blob/master/src/content/examples/cache-api.md
Whenever I send a post request with my content body, I am getting an unknown error as following:
Uncaught (in response) TypeError: Body has already been used. It can only be used once. Use tee() first if you need to read it twice.
What am I doing wrong?

@adamschwartz It's not working as written in the example code. I initially referred the documentation and then tried to apply the code in example but still not able to make it work. (Trying it for POST Requests)


StatPoolsSystem not found

The documentation says that I should be able to call Game.StatPoolsSystem() as a global but running exactly that string in the console returns
Error: Function 'StatPoolsSystem' not found or is not a global.

Just to add to this, GetStatsSystem seems to have the same result, but GetStatsDataSystem seems to work fine


Show cancel button during data download part of sync

Allow user to cancel DataPullTask up until point where data has been downloaded and processing begins.
Don't allow user to cancel syncs that have been started automatically.
Avoid synchronization between button hiding and starting the data processing block. We avoid it because it isn't that bad if a user presses the cancel button after data processing has begun. If that happens, the task dialog says it is cancelling, butt the data processing still completes (because we don't check the AsyncTask.isCancelled() state in that block). Resulting in the user being slightly annoyed, if the data processing takes a long time.

Created a 2.27 version https://github.com/dimagi/commcare-odk/pull/1226


Add check for external redis ip address to prevent KeyError (#571)

As requested in issue #571 by @withsmilo

Can one of the admins verify this patch?

Jenkins ok to test

@DNCoelho Thanks for your contribution!

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Clipper-PRB/2062/
Test PASSed.


Change the package name...

...once we've decided on a new name.
How to do it: https://www.njtierney.com/post/2017/10/27/change-pkg-name/

It might be worth running any proposed names through available::available() before deciding on a name


Handle arguments to train script

This is a quick solution to allow arguments to be passed to the training script.
$ unweave python main.py -- --epochs=2 --lr=0.1

Will keep looking if it's possible to do this without the --

I didn't know this was possible! I thought we had to do something like unweave -- python train.py --lr 0.5 instead.


support eval() like python?

like this ---
>>> eval("1+2")
3

Starlark intentionally does not come with eval because it tends to invite metaprogramming uses that make programs very hard to read and reason about. If the purpose of this issue is to add eval as a standard built-in, you will need to propose a language change at https://github.com/bazelbuild/starlark. I plan to close this issue.
That said, it shouldn't be hard to implement the eval function as a built-in outside the interpreter. You will need to enumerate the the set of local and module-level variables that are defined at the call to eval, which can be done using the new API added in https://github.com/google/starlark-go/issues/538.


fix(instrumentation-perf-hooks): use TestMetricReader, unref timer

Which problem is this PR solving?

tests were failing as the utilization ended up being 1 (#1955) . This might have happened as the collection happened at a time where event loop was fully utilized. This PR attempts to fix this be replacing interval-based collection with on-demand collection, removing one variable that may push results into such a window. IIUC since collection is still async, it might happen that we still hit a window where event loop utilization is perfect when we collect (a 10ms sync window in the async collection cycle, which may happen if the machine we're running tests on is slow enough). In that case, we'll have to

fake time using sinon.useFakeTimers()
mock results from eventLoopUtilization() (using fake timers will have the returned utilization value always be 1, this is due to how the fake timer works in the background)
alternately, we can also allow <=1 as a value (because it's a valid value)

for me the process kept running as disable() was never called when the test failed. I can see this happening in apps too as it's rather uncommon to disable() instrumentations before shutting down. Therefore in unrefed the timer when it's created.

Fixes #1960

replacing interval-based collection with on-demand collection

@pichlermarc thanks for cleaning up these tests, they look much sturdier now

unref

Good call 👍


Many icons are missing from TTF font

Expected Behavior
Icon font files should include all icons from Orbit.
Current Behavior
The SVG font includes all icons, but TTF doesn't, so it seems like svg2ttf doesn't work well in our case and doesn't cover all icons. We can also assume that WOFF2 is missing those as well because we're generating them from the TTF file.
Possible Solution
Use a package like webfont, it's an all-in-one solution that not only works (I verified), but can also produce WOFF and EOT files, which I recommend, and the script simpler.
What do you think? We use a custom icon font on frontend and we would like to switch to Orbit's.

But does webfont support custom characters? I believe that tried it months ago, but there was not option to do it. (or maybe I tried something else, I'm not sure know 🤔 )

This issue is incorrect, it does (seem to) include all icons, I was viewing the font wrong.


Add annotations for volume type management/introspection

Let's add type management:
Type request:
spec.epc.altinity.com/type:
Actual type:
status.epc.altinity.com/type

Fixed in 0.3.0


Add sample implementation of alphaMode material setting

draft PR, pending discussion
This adds a way to set alphaMode for materials in GLTF. Creating transparent materials with GLTF is impossible without setting alphaMod=BLEND.
This seems to be very simple to implement, but I'm not sure what is the long term plan on aligning different interfaces and formats. If this approach is fine - I'll add tests and finalise the PR. If not - let me know what would be better way. I need this feature and happy to implement it in whatever way is best.

Good catch! This fix is fine, and I am happy to merge. Make sure this is the only change you need to make for your own purposes. I've been able to achieve transparency in three.js and babylon renderers while this bug has existed.
https://github.com/EliCDavis/polyform/blob/main/examples/ufo/main.go#L347-L380

Cool, I've switched to pointers and added alphaCutOff as requested. Also added relevant writer tests.
Yes indeed, the material settings of the UFO model work in Babylon and Three.js. I copied over the parts of GLTF JSON and it did render transparency. I'm not sure why tho, if I read the spec correctly - it should not work, as default is OPAQUE even if alphaMode is not set.
Btw, I also tested against 3dviewer.net and it did not work without BLEND, so there is clearly a disagreement between renderer implementations.
Anyway, it's part of the spec, so should exist, and now it does 🙂


Add common and tag dimensions across compliance queries closes #33

Checklist

[ ] Issue(s) linked

@khushboo9024 can you pls check this

Tested the control.pod_default_seccomp_profile_enabled after labelling the pod csi-azuredisk-node-hmb65 with Environment=test
Got proper results


Fix share icons

This PR fixes Android share icons as well as Toolbar title.
Resolves https://github.com/CovidShield/mobile/issues/63
Note:

In before image, the background push toolbar to the top which is wrong.
In after image, the content is overlapped with the background. It depends on device height. The content and share options are always stick to the bottom.
ScrollView is removed from this view because it isn't necessary.

Before

After

Ready for reviewing 🙇

@henrytao-me I think the text should always be below the image. It's okay if this page scrolls on smaller devices.

It's difficult to balance both. In this case, if we want to text is always below image, then this screen can scroll just a little bit which is odd.
One option is that we can make the svg connect-able on both sides (left and right), so that we can
scale the height to whatever space lefts and fill up the width with connected left and right. What do you think?

@henrytao-me I think let's keep things a little simpler and just let the view scroll as much as it needs to - keeping the illustration as is. Is that OK?

Share icons work great in Android simulator

Works great on Pixel 4 device as well.

Pushed the code to render top down. It looks like this at least on Pixel One.

Merge now. I will do a follow PR if needed.


Implement object-form keyframe syntax.

This patch extends the web-animations and web-animations-next
polyfills to accept the object-form syntax for specifying keyframes
(http://w3c.github.io/web-animations/#processing-a-frames-argument).
The capacity is added to the base polyfill, so loading the
web-animations polyfill will enable the new syntax.
It is possible that a browser has a native Web Animations
implementation (and hence does not load the base polyfill), but does
not natively accept object-form keyframes. To address this, this
patch also adds a file to the 'bonus' sources, so the object-form
keyframe syntax can be loaded independently of the base polyfill.

Done, PTAL.

lgtm


[Event Hubs] Add troubleshooting guide

Largely based on @conniey's https://github.com/Azure/azure-sdk-for-java/pull/28609. The PR also adds a sample that is helpful for troubleshooting.

API change check
API changes are not detected in this pull request.


relayPagination and network-only

Hi All
I have the following query
query FriendsQuery($after: String!) {
  me {
    id
    friends(after: $after) {
      edges {
        node {
          ...FriendshipFragment
        }
      }
      pageInfo {
        endCursor
        hasNextPage
      }
    }
  }
}

And the following hook

export const useFriends = () => {
  const client = useClient();
  const [{ fetching, error, data }] = useQuery<
    FriendsQueryQuery,
    FriendsQueryQueryVariables
  >({
    query: FRIENDS_QUERY,
    requestPolicy: "network-only",
    variables: {
      after: "",
    },
  });

  const friends =
    data === undefined ? undefined : data?.me?.friends ?? undefined;

  const fetchMore = React.useCallback(
    (cursor: string) => {
      return client
        .query<FriendsQueryQuery, FriendsQueryQueryVariables>(
          FRIENDS_QUERY,
          { after: cursor },
          { requestPolicy: "network-only" }
        )
        .toPromise();
    },
    [client]
  );

  return React.useMemo(
    () => ({
      fetching,
      error,
      friends,
      fetchMore,
    }),
    [fetching, error, friends, fetchMore]
  );

Results are merged using relayPagination
resolvers: {
  User: {
    friends: relayPagination()
  }
}

I have a component that does an infinite scrolling and shows all my friends.
When the component is unmounted and afterwards remounted i would like to fetch again the lists of friends (hence the network-only)
The problem is that due to the relay pagination the friends variable contains all the friends fetched from the previous mount with pageInfo.hasNextPage === false.
The only request that is requested again is the initial with after === ""
Thanks

I think the problem relies in the pageInfo of relayPagination. It always sets it to the last even though after the remount i have refetched the first page.
I am not sure if its wise to reset pageInfo to the currently fetched page results

Hey Zenios,
Good question but this isn't really the use case of a resolver, for us it would be impossible (and a waterfall) to refetch it all.
Let me explain, if we would refetch your first page we can't eagerly refetch your second page, we would have to check the pageInfo of your first response to go for your second one and so son.
The better solution for this situation would be to make a custom hook:
const useRevalidatingPagination = () => {
  const [list, setList] = useState([])

  const [data] = useQuery(query);

  useEffect(() => {
    if we go from fethcing to not fetching
      addToList
  }, [data]);

  return list;
}

So now we have a list that resets every time you navigate back, when you query you will receive the result with a stale flag so you can wait until it gets revalidated.

I dont expect urql to refetch all.
What i expect though is once the results are merged via relayPagination pageInfo should show the latest updated pageInfo not the last fetched from the previous results.

TO be more clear
Fetch page 1 results should be edges:[...page1] pageInfo:page1PageInfo
Fetch page 2 results should be edges:[...page1,...page2] pageInfo:page2PageInfo
Unmount / remoount and network only
Fetch page 1 results should be edges[...page1,...page2] pageInfo:page1PageInfo
while know its page2PageInfo
Hope its clear

The pageInfo is merged like all the pages are, so it'll appropriately merge the beginning and end bits.
Because you can fetch pages in "both directions" (after / before) we account for that and create a new pageInfo based on that. https://github.com/FormidableLabs/urql/blob/51ae6806a6adf5a9fc7f17c43ca2dbe066f9c9c3/exchanges/graphcache/src/extras/relayPagination.ts#L200-L214
So it will follow the logic you're describing, but unmount/remount is not what you're expecting here. Instead the result is identical no matter what page you ask for currently. We're thinking of maybe filtering it down, but that'd be an additional option on the resolver.
That being said even if you ask for the "first page" you'll get a result based on all the pages that Graphcache knows about.

That is understood but the way relayPagination is structtured right now there is no way once the whole list is fetched to refetch it (Throught the whole applications lifecycle)
Lets take facebook for example. If i view friends list once then go to another page and go back to friends list the full cycle is executed from the beginning while us with relayPagination i cannot do that.

You could try to write a resolver that's similar to relayPagination, I suppose, that is more strict. So you could walk the pages in the cache using pageInfo explicitly (by using endCursor as the input for after for instance) which means that if you refetch (using network-only or cache-and-network) the chain of pages can be broken by an earlier page changing.

The problem is that i didnt find a way to know which page was the last refreshed / fetched

I don't think you'll ever have to check which page was last fetched. The resolver just needs to get the requested page for your use-case and then "walk backwards" (so use the input arguments e.g. use the after cursor and find the page that has endCursor set to that).
When you're missing a page you can set info.partial = true to fetch the other pages. And when you don't have your current page you can either return the previous ones and set partial or just return undefined for a cache miss.

Thanks a lot

@zenios I’ll think about how much sense it makes to integrate this into the relayPagination helper btw 😇 it may just be very specific 😅

Think about it.my only concern is what i said before.once you use relay
pagination forget about refreshing. You have to hack your way around it
On Fri, 27 Mar 2020, 02:03 Phil Plückthun<EMAIL_ADDRESS>wrote:

@zenios https://github.com/zenios I’ll think about how much sense it
makes to integrate this into the relayPagination helper btw 😇 it may
just be very specific 😅
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FormidableLabs/urql/issues/672#issuecomment-604747414,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAV3YMWKPNFEBPNXPNGEVLRJPUNVANCNFSM4LUQKTKA
.

As for the very specific, I dont think i am the only person who want to use relayPagination with cache-and-network || network-only

I'm having what I think is the same problem described here. @zenios did you end up finding a solution/workaround?

I'm having what I think is the same problem described here. @zenios did you end up finding a solution/workaround?

Nope i did the merging manually without using relayPagination
On Wed, Jan 20, 2021 at 12:14 PM Vinícius Lemes<EMAIL_ADDRESS>wrote:

I'm having what I think is the same problem described here. @zenios
https://github.com/zenios did you end up finding a solution/workaround?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FormidableLabs/urql/issues/672#issuecomment-763496418,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAV3YMDTQV6IZBXIBVZ52DS22UJFANCNFSM4LUQKTKA
.

Nope i did the merging manually without using relayPagination
On Wed, Jan 20, 2021 at 12:14 PM Vinícius Lemes<EMAIL_ADDRESS>wrote:

I'm having what I think is the same problem described here. @zenios
https://github.com/zenios did you end up finding a solution/workaround?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/FormidableLabs/urql/issues/672#issuecomment-763496418,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAV3YMDTQV6IZBXIBVZ52DS22UJFANCNFSM4LUQKTKA
.

Feel sad, I think I have to manually merge the result too.

if you refetch (using network-only or cache-and-network) the chain of pages can be broken by an earlier page changing.

I have an issue probably related to this behavior, I mean lack of data after refetch.
Imagine the case of fetch page1, page2 , and add an item, and then fetch again.

fetch page1: [{id: 10}, {id: 9}, {id: 8}]
fetch page2: [{id: 10}, {id: 9}, {id: 8}, {id: 7}, {id: 6}, {id: 5}]
add item {id: 11}
fetch page1, page2:  [{id: 11}, {id: 10}, {id: 9}, {id: 7}, {id: 6}, {id: 5}]
so item of {id: 8} is gone in the paginated list.

Ideally I want step 4 to be [{id: 11}, {id: 10}, {id: 9}, {id: 7}, {id: 6}, {id: 5}] but at least I don't loose {id: 8}.
So I want to ask is there any way to clear or invalidate entire cache of the paginated query?
@kitten

After investigation, I found that I could invalidate all paginated query like in the example of invalidating-individual-fields in the doc. That way I won't loose the data in between pages.
https://formidable.com/open-source/urql/docs/graphcache/cache-updates/#invalidating-individual-fields


coff: undefined symbols use IMAGE_SYM_CLASS_EXTERNAL

Fixes #209
cc @wwylele

Coverage increased (+0.2%) to 54.148% when pulling 9c63000c21547cda120c36da6190f19acc1a999b on philipc:issue-209 into 267199e410ef2a06d955f7ee1ad6410e47e1a206 on gimli-rs:master.


dynamic QR code gives address http://<IP_ADDRESS>:80

Hello,
url with <IP_ADDRESS> is not going to work.
I'm using the docker version setup with portainer using the compose.yml
I have a cloudflare tunnel to the docker container and i can reach it via the URL.
If i change the http://<IP_ADDRESS>:80 etc..
with myURL https it does work, so the unique identifier is created but the correct URL is not created in the QR code.
What did i miss? or how can i correct this.

Can you attach here your qrcode please

Can you attach here your qrcode please

See above.
my URL is qr.sieliakus.nl
Also i deleted superuser after creating a new superuser, but now i have this message at the sample data:
`

Warning: Cannot modify header information - headers already sent by (output started at /var/www/html/dynamic_qrcodes.php:24) in /var/www/html/lib/Users/Users.php on line 130

`
I you want i can create you as user so you can see it for yourself?

Yes please create an user for me thanks

Yes please create an user for me thanks

giandonatoinverso
login and passwd

I solved it by inserting a new environment variable into the docker compose file which you will need to modify with your full hostname.
This allows you to have the same code for the self-hosted version and docker version.
Let me know if there are any problems. The files involved are also config.php and environment.php so you will have to delete and recreate the config volume (which among other things I removed from docker compose leaving only saved_qrcode)

Hi,
I completely deleted the stack and volumes and redeployed it with the new compose file.
But it stil points to http://<IP_ADDRESS>

`version: "3.2"
services:
php-dynamic-qrcode:
image: "giandonatoinverso/php-dynamic-qr-code:latest"
restart: "unless-stopped"
environment:
TYPE: "docker"
BASE_URL: "https://qr.sieliakus.nl"
DATABASE_HOST: "php-dynamic-qrcode-db"
DATABASE_PORT: "3306"
DATABASE_NAME: "qrcode"
DATABASE_USER: "qrcode"
DATABASE_PASSWORD: "R8w5nMuKsbbD%%SY4zvV"
DATABASE_PREFIX: ""
DATABASE_CHARSET: "utf8"
ports:
- 80:80
depends_on:
- php-dynamic-qrcode-db
volumes:
- php_dynamic_qrcode_config_data:/var/www/html/config
  - php_dynamic_qrcode_saved_qrcode_data:/var/www/html/saved_qrcode
networks:
  - php-dynamic-qrcode-network

php-dynamic-qrcode-db:
image: "giandonatoinverso/php-dynamic-qr-code-db:latest"
restart: "unless-stopped"
volumes:
- php_dynamic_qrcode_db_data:/var/lib/mysql
ports:
- '13306:3306'
environment:
MYSQL_ROOT_PASSWORD: "R8w5nMuKsbbD%%SY4zvV"
MYSQL_DATABASE: "qrcode"
MYSQL_USER: "qrcode"
MYSQL_PASSWORD: "R8w5nMuKsbbD%%SY4zvV"
networks:
- php-dynamic-qrcode-network
volumes:
php_dynamic_qrcode_db_data:
php_dynamic_qrcode_config_data:
php_dynamic_qrcode_saved_qrcode_data:
networks:
php-dynamic-qrcode-network:
driver: bridge`

Hi, I tried to log in and download the qrcode. The same redirect link comes out. I think everything is ok!
P.S. Thank you so much for the donation!

The download is ok, but the preview on the site is not the same. That is
with http://<IP_ADDRESS>
kr
Roy
Op wo 18 okt 2023 om 11:50 schreef Giandonato Inverso <
@.***>:

Hi, I tried to log in and download the qrcode. The same redirect link
comes out. I think everything is ok!
P.S. Thank you so much for the donation!
—
Reply to this email directly, view it on GitHub
https://github.com/giandonatoinverso/PHP-Dynamic-Qr-code/issues/71#issuecomment-1768087571,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYNNM6DUVNLHW6YD43MYNDX76Q4TAVCNFSM6AAAAAA6DQC32KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYGA4DONJXGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

What you mean for “preview on the site” ?

It's strange because i logged in your account and scanned those qrcodes.
My iphone gave me the correct url and then I went redirected to your linkedin account

That is really Strange, above screenshot also gives you the correct link?


Method name helpers (similar to BREAD and CRUD) ...

... but .post, .get, .patch,  and .del.

:+1:


IList.update

I'd like the method IList.update because I find myself often writing things like
myList.put(index, myFunction(myList.get(index)))

but it would be great to be able to write
myList.update(index, myFunction)

Done in version [8.2.0]:

IList.replaceBy method lets you define a function to transform an item at a specific index location.


Remove Binding from Page and PageModel

Part of #6071.

:up::date:

:up::date: @rynowak I think this is what we talked about.

5a184df808c1dfbca010900354eae9369fa488ea


Create fecea.txt

Adding FECEA

@fecea-ed You might miss my previous comment. Please react.

@fecea-ed I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for your understanding!


Fuzzing a list of all users

If a certain user exists, their folder URL is freecdn.lol/UserID. It will not redirect you back to the home page. If the ID does not have a folder, it redirects you to freecdn.lol/. Therefore, a list of all users could be fuzzed with enough time and effort.

it works :)


gnomeExtensions.night-theme-switcher: 40 -> 50

Motivation for this change
Night Theme Switcher has a new release with bug fixes and new features,
most notably GNOME Shell support for gnome 40 (v47)
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 125725 at f355c93d run on x86_64-linux 1

1 package built successfully:

gnomeExtensions.night-theme-switcher

1 suggestion:

warning: unclear-gpl
gpl3 is a deprecated license, please check if project uses gpl3Plus or gpl3Only and change meta.license accordingly.
Near pkgs/desktops/gnome/extensions/night-theme-switcher/default.nix:28:5:
   |
28 |     license = licenses.gpl3;
   |     ^


ci: Skip full CI for USERS.md file

Noticed in https://github.com/cilium/cilium/pull/34603

/test


German Translation of User Guides (Mining, Recovery)

German translation of the following user guides:

How to mine on a pool with XMRig
How to solo mine with the GUI
How to fix locked up funds
How to restore your account
Restoring wallet from keys

@stmax82 let me know if your review is complete here, so i can start mine.

@erciccione I'm done with this one and 4 out of 5 PRs. Only #1332 is left.. will do the last one soon.


add gcd

Codecov Report

Merging #111 (d47c618) into main (73df2a0) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #111      +/-   ##
==========================================
+ Coverage   89.09%   89.17%   +0.07%
==========================================
  Files          30       32       +2
  Lines        3743     3770      +27
==========================================
+ Hits         3335     3362      +27
  Misses        408      408

Impacted Files
Coverage Δ

optpy-runtime/src/lib.rs
100.00% <ø> (ø)

optpy-resolver/src/module/module_map.rs
95.45% <100.00%> (+1.01%)
:arrow_up:

optpy-runtime/src/stdlib.rs
100.00% <100.00%> (ø)

tests/test-stdlib.rs
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


New explorer: disallow dropping of root file onto empty space in multi root workspace?

Follow up from https://github.com/Microsoft/vscode/issues/68095
With my change you can now:

enter multi root with multiple folders
drag a root file of one folder onto empty space of explorer
drop it

I think we should probably not allow that and I verified this used to be like that before but now changed due to my recovery fix in https://github.com/Microsoft/vscode/issues/68095

I think we should allow. This would put the root at the end.
However when I now drop it to the empty space the order is not changed, it is a no-op. Assigning to february to investigate

@isidorn isn't that weird though? I drag a file from the first root folder all the way down to empty space (which is highlighted) and it ends up at the root of the last folder?

Aaaa I thought it only happens for the dragged roots, not for files. Yeah that is weird. Will fix that.

I can not repro. Please let me know what you are doing differently
Only when dropping roots do I get drop feedback and something actually happens

@isidorn it should not be possible to drag a FILE of one folder onto the empty space below the last root folder in a multi root workspace:

@isidorn I also see lots of issues trying to reorder root folders with drag, can you check?

@bpasero reordering issues I see and I can fix them. The original issue I do not see

@isidorn yeah now I also cannot reproduce those anymore


Backend cannot be loaded

I've tried this on both Slackware Linux and RHEL 9.2, not sure what I'm doing wrong.
I install asciidoctor-epub3 with gem:
$ gem install asciidoctor-epub3

Then I try to use it:
$ asciidoctor --backend asciidoctor-epub3 -r ~/bin/asciidoctor-epub3 book.adoc --trace
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- /home/klaatu/bin/asciidoctor-epub3 (LoadError)
        from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /usr/share/gems/gems/asciidoctor-2.0.15/lib/asciidoctor/cli/options.rb:277:in `block in parse!'
        from /usr/share/gems/gems/asciidoctor-2.0.15/lib/asciidoctor/cli/options.rb:275:in `each'
        from /usr/share/gems/gems/asciidoctor-2.0.15/lib/asciidoctor/cli/options.rb:275:in `parse!'
        from /usr/share/gems/gems/asciidoctor-2.0.15/lib/asciidoctor/cli/options.rb:34:in `parse!'
        from /usr/share/gems/gems/asciidoctor-2.0.15/lib/asciidoctor/cli/invoker.rb:24:in `initialize'
        from /usr/share/gems/gems/asciidoctor-2.0.15/bin/asciidoctor:13:in `new'
        from /usr/share/gems/gems/asciidoctor-2.0.15/bin/asciidoctor:13:in `<top (required)>'
        from /usr/bin/asciidoctor:23:in `load'
        from /usr/bin/asciidoctor:23:in `<main>'

I can confirm that the gem is installed:
$ file ~/bin/asciidoctor-epub3
/home/klaatu/bin/asciidoctor-epub3: Ruby script, ASCII text executable

I've tried a gem update but that fails to resolve the issue.
Am I missing some dependencies?
Standard backends (html5, docbook5, and so on) work as expected.

If you installed the gem, and the system package installs it correctly, then the command should be asciidoctor-epub3. There is no need to specify the backend or the require path. The bin script does all that for you.

$ asciidoctor-epub3 book.adoc

Worked like a charm.
I [obviously] had misunderstood the form a backend took.
Very cool, thanks!


Dunder methods to improve objects

How to use dunder methods to add behavior to objects. I add an example of how implement a french deck that can be used as a list.

@brennerm It's done!

Thanks!


Ktlint 1.3.0 release

https://github.com/pinterest/ktlint/releases/tag/1.3.0

@freemanjp ? see also #629

@freemanjp ping again

Hi @sanyarnd, sorry for the delay, this is resolved in version 3.3.0 of this plugin (allow a couple of hours for it to show up in Maven central).


az interactive doesn't work

seems to be regresion, because az interactive worked fine for a long time.
This is an autogenerated template. Please review and update as needed.
Describe the bug
Command Name
az interactive
Errors:
:open_mouth:

cannot import name 'DEFAULTS_SECTION' from 'azure.cli.core._config' (/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/_config.py)Traceback (most recent call last):
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/knack/cli.py", line 206, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/commands/__init__.py", line 326, in execute
    raise ex
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/commands/__init__.py", line 384, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/commands/__init__.py", line 377, in _run_job
    six.reraise(*sys.exc_info())
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/six.py", line 693, in reraise
    raise value
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/commands/__init__.py", line 354, in _run_job
    result = cmd_copy(params)
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/commands/__init__.py", line 145, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/__init__.py", line 451, in default_command_handler
    return op(**command_args)
  File "/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/command_modules/interactive/custom.py", line 31, in start_shell
    azext_interactive.start_shell(cmd, style=style)
  File "/Users/wawrzek/.azure/cliextensions/interactive/azext_interactive/__init__.py", line 10, in start_shell
    from .azclishell.app import AzInteractiveShell
  File "/Users/wawrzek/.azure/cliextensions/interactive/azext_interactive/azclishell/app.py", line 35, in <module>
    from azure.cli.core._config import DEFAULTS_SECTION
ImportError: cannot import name 'DEFAULTS_SECTION' from 'azure.cli.core._config' (/usr/local/Cellar/azure-cli/2.0.62/libexec/lib/python3.7/site-packages/azure/cli/core/_config.py)

To Reproduce:
Steps to reproduce the behavior. Note: Command arguments have been redacted.

Fill in additional info here
Run: az interactive

Expected Behavior
A clear and concise description of what you expected to happen.
Environment Summary
Darwin-18.2.0-x86_64-i386-64bit
Python 3.7.3
Shell: bash

azure-cli                         2.0.62

Extensions:
interactive                        0.4.1

Python location '/usr/local/Cellar/azure-cli/2.0.62/libexec/bin/python'
Extensions directory '/Users/wawrzek/.azure/cliextensions'

Python (Darwin) 3.7.3 (default, Mar 27 2019, 09:23:15)
[Clang 10.0.1 (clang-1<IP_ADDRESS>)]

Additional Context
Add any other context about the problem here.

You need to update your version of interactive. 0.4.1 is not compatible with CLI core 2.0.62.

az extension update --name interactive


FR: Voice Id

It would be nice, to attach the voice id on every request. It would make cool things possible...eg with kids, one could enable / disable internet.

Not possible with the version of the API the skill uses


not working in ubuntu 18.04

hi
iam trying this programme in ubuntu 18.04 when iam trying to build with make BUILD_OPENCL=1
nothing happening
my device is quadro fx 3800

What do you mean by "nothing happening"? Can you provide the output of make?

this is my windows iam trying this programme last 4 days but i dint got till now please help me
in windows i tried cuda and opencl but in open cl instalize device and stoping perform i dont understant my open cl image below in windows

please post open cl process its stoping when instaling device

пожалуйста, сообщите открытому процессу cl его остановку при установке устройства

NVIDIA Quadro FX 3800-And the error you see?Error-Cuda driver version is insufficient for cuda runtime version

but open cl version detected device pls help me that way

And why do you say that you have Ubuntu when the screens with Windows?

The Quadro FX 3800 GPU is no longer supported by CUDA 7.0 and beyond.The last CUDA version that supported that GPU was CUDA 6.5.And the minimum cuBitCrack is Cuda 8.Try to download it-https://developer.nvidia.com/opencl.If this does not help then you have a very old video card.


New-TestResources should warn when overwriting existing env variables with the subscription config

This will make it easier to diagnose and debug implicit issues where someone defines the env variables twice, once in the yaml or local shell, and again from the sub config+arm template outputs.
CC @christothes

This is fixed now @christothes


Using "homebrew.onActivation.cleanup" to make Homebrew Cask installation part declarative?

I just edited some packages in ~/nixos-config/modules/darwin/casks.nix, and realised that if I commented out a package, it didn't get uninstalled.
Anyway, I did some naive Googling (e.g. this article), and it seems the option I want is homebrew.onActivation.cleanup.
Does it make sense to make homebrew.onActivation.cleanup = "uninstall", (or = "zap") the default?
Or at least call it out in the README as a suggestion?
Also - it might be helpful to briefly mention in the README about enabling Full Disk Access around that point. WDYT?
==> Dispatching zap stanza
Error: Unable to remove some files. Please enable Full Disk Access for your terminal under System Settings → Privacy & Security → Full Disk Access.
Uninstalled 1 cask
Uninstalling mas... (7 files, 951.5KB)
Uninstalled 1 formula
Error: Refusing to untap homebrew/cask because it contains the following installed formulae or casks:
discord
docker
visual-studio-code
visual-studio-code@insiders
zoom
Setting up the Dock...

It's a good idea to mention this and have it available to the user, but setting it as default could possibly be intrusive if the user doesn't realize it and accidentally removes an app they really need. Although the config is declarative, under the hood it will be imperative mac terminal commands that do the installing/uninstalling, sometimes undocumented things happen, break, etc.
My preference is to always leave stuff sitting around and then reset my system from time to time.
Anyway, I think we add this commented line
# onActivation.cleanup = "uninstall";
in the areas that need it under templates/starter and templates/starter-with-secrets. Feel free to add this there and open a PR, or I can get to it eventually.

Merged this as part of your PR. Thanks!


Fix py.typed in namespace packages

Previously, the py.typed file was incorrectly put in the namespace package, instead of the non-namespace sub-packages.

Cc @Avasam

Could someone review this? The google packages are still broken.

Unfortunately, it seems that py.typed is now dropped completely from the package. I'll look into it later.

It's been a few weeks, but I also remember testing this locally where it seemed to work. Probably best to just work on #121 instead of trying to find this bug.

@srittau Should I wait before merging PRs relating to partial stubs? (ie https://github.com/python/typeshed/pull/11255/files )
iirc would affect type-checking users that also use modules missing from typeshed.

I would actually be interested if it also affects partial non-namespace packages.

I would actually be interested if it also affects partial non-namespace packages.

pyopenssl was released 20h ago and contains the marker file
https://pypi.org/project/types-pyOpenSSL/#history

I believe this PR was merged 29h ago?
If that's still too close/unsure, you'll know for sure once tensorflow (modified 3h ago) deploys.

I think I understand what you're thinking of: maybe only namespace packages are missing the py.typed file. Which could be caused, for example, by an incorrect path added to package_data[package]


Documentation for the new format strings feature

Hi!
I've recently noticed this PR with format strings: https://github.com/berry-lang/berry/pull/355
But I don't see any changes in the Wiki, website or short manual regarding this functionality.
BTW. What is the canonical documentation for the language? If I am pointed to one I can submit a PR or change to add documentation about this new feature.
Best regards,
Albert

It's in my TODO list to update the wiki, but not before weeks.
I also need to figure out documentation process to avoid divergence between wiki and published documentation


Notification not working as expected at certain time

Thats a funny one.. nearly pulled out whats left of my hair  last night.  Coudn't understand why suddenly the notifications were not working anymore both in emulator and physical phone.  Alarm sound still rings  but  the notifications don't display.  Then I woke up this morning, started testing and everything was working.  To reproduce I used the android  emulator.  In System -> Date & Time, disable "Set time automatically" then under time  set the time to a time between  8pm and 11:59pm.   Run the app with a notification,  notifications don't display.

It looks like it's an issue related to how the package handles time zones. I'll take a close look at it. Thank you for your message.

when manually changed?  it actually doesn't work for me between 8pm and 11:59pm  without changing the time manually.  Changing the time manually was just a step to reproduce without having to wait for that time frame to test.

Sorry @tiwwwi I guess I didn't understand well your issue...
In debug mode, are you able to see the Alarm logs ? Can you tell me what is the log you have when you schedule the alarm ? It can really help me. It should be something like this:
[Alarm] Notification with id $id scheduled successfuly at $zdt
zdt is the date and time the notification is scheduled in the Zulu timezone (GMT).

at a working time:
flutter run | grep Alarm
I/FlutterBackgroundExecutor( 6558): Starting AlarmService...
I/flutter ( 6558): [Alarm] Notification with id 48 canceled
I/flutter ( 6558): [Alarm] NotificationOnKillService set with success
I/flutter ( 6558): [Alarm] Notification with id 48 scheduled successfuly at 2023-03-27 04:14:50.000Z
I/flutter ( 6558): [Alarm] ring
I/flutter ( 6558): [Alarm] Alarm fadeDuration: {assetAudioPath: assets/audio/zedd.mp3, loopAudio: false, fadeDuration: 0}
I/flutter ( 6558): [Alarm] Alarm with id 48 starts playing.
at a not working:
flutter run | grep Alarm
I/FlutterBackgroundExecutor( 7450): Starting AlarmService...
I/flutter ( 7450): [Alarm] Notification with id 48 canceled
I/flutter ( 7450): [Alarm] NotificationOnKillService set with success
I/flutter ( 7450): [Alarm] Notification with id 48 scheduled successfuly at 2023-03-29 02:16:31.000Z
I/flutter ( 7450): [Alarm] ring
I/flutter ( 7450): [Alarm] Alarm fadeDuration: {assetAudioPath: assets/audio/zedd.mp3, loopAudio: false, fadeDuration: 0}
I/flutter ( 7450): [Alarm] Alarm with id 48 starts playing.

no difference in the logs when set to a working hours vs not working hours.   You can't replicate my issue?

I was able to reproduce your issue ! Fixing it is now my top priority. I'll let you know when it's done.

Hey @tiwwwi
I think I fixed the issue. Here are the changes I made:
https://github.com/gdelataillade/alarm/commit/6276b15a43104cf5a73f35a42fe3c1940b4d48d7#diff-bd754698c3da7e10df0d984e8c571502d2a7a7bf20e2c1b56d1949b6a9551d7e
Please update the alarm dependency to 0.2.4 and tell me if you still have the issue.

I confirm!  it works!!!!

I'm closing this issue then. Thank you very much for your help !