build tensorflow on tx2 with cuda8.0 and cudnn6.0

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Tx2
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): r1.6, r1.7, r1.8
Python version: 3.5
Bazel version (if compiling from source):.13.0- (@non-git)
GCC/Compiler version (if compiling from source):5.4.0
CUDA/cuDNN version: 8.0 / 6.0
GPU model and memory:  Denver2 8GB
Exact command to reproduce: bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.
./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::Eigen::half, int, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::Eigen::half, long long, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::Eigen::half, long long, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<float, int, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<float, int, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<float, long long, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<float, long long, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<double, int, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<double, int, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<double, long long, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel<double, long long, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<float> , int, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<float> , int, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<float> , long long, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<float> , long long, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<double> , int, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<double> , int, (bool)0> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<double> , long long, (bool)1> ") is not allowed

./tensorflow/core/kernels/gather_functor_gpu.cu.h(57): error: calling a __host__ function("__builtin_expect") from a __global__ function("tensorflow::GatherOpKernel< ::std::complex<double> , long long, (bool)0> ") is not allowed

20 errors detected in the compilation of "/tmp/tmpxft_0000294e_00000000-9_gather_functor_gpu.cu.compute_52.cpp1.ii".
ERROR: /home/nvidia/tensorflow/tensorflow/core/kernels/BUILD:1201:1: output 'tensorflow/core/kernels/_objs/gather_functor_gpu/tensorflow/core/kernels/gather_functor_gpu.cu.pic.o' was not created
ERROR: /home/nvidia/tensorflow/tensorflow/core/kernels/BUILD:1201:1: not all outputs were created or valid

and this is my configure settings
You have bazel 0.13.0- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python3.5

Found possible Python library paths:
  /usr/local/lib/python3.5/dist-packages
  /usr/lib/python3/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python3.5/dist-packages]

Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]:
jemalloc as malloc support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]:
Google Cloud Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Hadoop File System support? [Y/n]:
Hadoop File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]:
Amazon S3 File System support will be enabled for TensorFlow.

Do you wish to build TensorFlow with Apache Kafka Platform support? [Y/n]:
Apache Kafka Platform support will be enabled for TensorFlow.

Do you wish to build TensorFlow with XLA JIT support? [y/N]:
No XLA JIT support will be enabled for TensorFlow.

Do you wish to build TensorFlow with GDR support? [y/N]:
No GDR support will be enabled for TensorFlow.

Do you wish to build TensorFlow with VERBS support? [y/N]:
No VERBS support will be enabled for TensorFlow.

Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]:
No OpenCL SYCL support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 8

Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 6

Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:/usr/libaarch64-linux-gnu

Do you wish to build TensorFlow with TensorRT support? [y/N]:
No TensorRT support will be enabled for TensorFlow.

Please specify the NCCL version you want to use. [Leave empty to default to NCCL 1.3]:

Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 3.5,5.2]

Do you want to use clang as CUDA compiler? [y/N]:
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]:

Do you wish to build TensorFlow with MPI support? [y/N]:
No MPI support will be enabled for TensorFlow.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]:
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See tools/bazel.rc for more details.
	--config=mkl         	# Build with MKL support.
	--config=monolithic  	# Config for mostly static monolithic build.

Same issue here #19203

Same problem.

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 16.04
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): r1.7
Python version: 3.5
Bazel version (if compiling from source):.13.0- (@non-git)
GCC/Compiler version (if compiling from source):5.4.0
CUDA/cuDNN version: 8.0 / 6.0
GPU model and memory: Tesla P100, 16GB
Exact command to reproduce: bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

Indeed. Closing as a duplicate.


Slash Commands Doesnt work

hey hello my slash commands doesnt work i tried re inviting it doesnt work please help me

do node deploy-commands.js


WAVE XII

Please include updated images for wave XII and related info for this site.
Thank you

I don't make the icons, but I maintain the project that stores them and provides CSS.  We're currently waiting for someone to do image traces and put them into the TTF: https://github.com/geordanr/xwing-miniatures-font/issues/35
I'll get around to background images.
What other information ("related info") is missing?


Issues in StructuredMaterial2D::get_value

First of all, great work! Saw your talk at PW2020.
After importing a few layouts and seeing strange results, I became curious about the way the averaging is done when trying to get an average epsilon value. I think I understand the function correctly, but I also don't think it's doing what it's meant to be doing.
First of all, cell.get_area_ratio() will always either return 1, or 0, because it's defined from a rectangular grid.
if(cell.get_area_ratio() == 0) { 			break; 		} shouldn't be in the loop, but really should be a condition to go in the loop. It's not going to change between primitives. Of course that's very minor.
if(contains_p1 && contains_p2 &&
		   contains_p3 && contains_p4 &&
		   cell.get_area_ratio() == 1.0) {
				return prim->get_material(xd,yd);
		}

doesn't work if it's from a subsequent layer. Say you define a structure on top of a cladding. The first structure will give an overlap of say < 1 and correctly increments val and keeps on lopping, but on the second pass it will be overwritten by that return from the cladding. In the end, all the polygons must be kept so that a difference can be computed between them.
if(cell.get_area_ratio() > 0) { 		val += cell.get_area_ratio()*1.0; 	} will always increment val by 1 if gets through the condition. But is it really what it should do? It should really be the overlap that is "left" that contributes to val.
My final thought is that there should be a more efficient way for get_values than going grid point by grid point if you already have all the polygons in hand.

Nevermind. It wasn't obvious at first that cell shrinks at each iteration and remains the difference. I hadn't looked enough at the GridCell class.


Rebuild time is more than 20 seconds with large assets

What version of Remix are you using?
nightly
Are all your remix dependencies & dev-dependencies using the same version?

[X] Yes

Steps to Reproduce
Since updating an assets directory with a few larger files, Remix's build time is enormous. The assets contain two .mp4 files which are slightly bigger than 100MB. Upon removing the videos, the build time is back to normal.
What confuses me is, why is the rebuild time so consistently large when the assets are not changing? Seems like it might be some sort of issue where Remix is rewriting the videos with each build.
My config:
{
  cacheDirectory: './node_modules/.cache/remix',
  ignoredRouteFiles: ['**/.*', '**/*.test.{js,jsx,ts,tsx}'],
  serverModuleFormat: 'cjs',
  postcss: true,
  tailwind: true,
  serverDependenciesToBundle: [/^d3.*/, /^delaunator.*/, /^internmap.*/],
}

Expected Behavior
A usable build time.
Actual Behavior
I can't develop with the current rebuild times. Here is an excerpt from my Terminal:
 info  rebuilding... (~ app/components/Section.tsx)
 info  rebuilding... (~ app/components/Section.tsx)
 info  building...
The service is no longer running
 info  rebuilt (11.7s)
Remix App Server started at http://localhost:3000 (http://<IP_ADDRESS>:3000)
 info  app server ready (1.8s)
 info  hmr

 info  rebuilding... (~ app/root.tsx)
 info  rebuilding... (~ app/root.tsx)
 info  building...
The service is no longer running
 info  rebuilt (7.4s)
Remix App Server started at http://localhost:3000 (http://<IP_ADDRESS>:3000)
 info  app server ready (415ms)
 info  hmr

 info  rebuilding... (~ app/components/Section.tsx)
 info  rebuilding... (~ app/components/Section.tsx)
 info  building...
The service is no longer running
 info  rebuilding... (~ app/components/Section.tsx)
 info  rebuilding... (~ app/components/Section.tsx)
 info  building...
The service is no longer running
 info  rebuilt (23.7s)

I think Remix needs to check the video files to ensure they're the same, so this may be taking a long time.

This seems possible but an unnecessary workaround. Can't Remix just check file metadata to see if it's been modified? For development purposes this seems perfectly reasonable.

@j-palindrome I'm wondering if you're encountering the same problem with the vite compiler. Did you switch to it at some point?

I did, I don't know if this is an issue because I'm not using large videos anymore, but the vite compiler is generally much better.


Standalone support for OpenVino on Windows

install openvino:
pip install openvino

create a file 'main.py', content is:

from openvino.inference_engine import IECore
_ie = IECore()
print(_ie.available_devices[0])

output is: CPU

nuitka compile:

python -m nuitka --standalone --enable-plugin=numpy main.py

run main.exe, FAILED:

Traceback (most recent call last):
File "D:\test\TEST_N~1\TEST_O~1\MAIN~1.DIS\main.py", line 9, in
File "D:\test\TEST_N~1\TEST_O~1\MAIN~1.DIS\openvino\inference_engine_init_.py", line 30, in
ImportError

copy required files  to output folder:
C:\Users\zwp\AppData\Local\Programs\Python\Python38\Lib\site-packages\openvino\inference_engine\constants.pyd
C:\Users\Administrator\AppData\Local\Programs\Python\Python38\Lib\site-packages\openvino\libs

Always FAILED:

Traceback (most recent call last):
File "D:\test\TEST_N~1\TEST_O~1\MAIN~1.DIS\main.py", line 9, in
File "D:\test\TEST_N~1\TEST_O~1\MAIN~1.DIS\openvino\inference_engine_init_.py", line 30, in
ImportError

Environment:  python -m nuitka --version

0.9.6
Commercial: None
Python: 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)]
Flavor: Unknown
Executable: C:\Users\zwp\AppData\Local\Programs\Python\Python38\python.exe
OS: Windows
Arch: x86_64
WindowsRelease: 10

Manual copying is not supported and unlikely to work correctly. These will be hidden dependencies, that we need to define.

I was working my way through the hidden dependencies and adding data files, however it fails to run in the end. Enabling the plugin back then didn't cause numpy to be included, only to be handled if included. That dependency would have had to be specified manually. However, now it has hit a problem, in that rather than properly error exiting, it errors from the module load, but without an error. It seems it is crashing when IECore() is called.

So, applying an API monitor, I noticed it searches for plugins.xml in versioned paths, and top level. Moving it to top level of the dist folder helps it find it, and then of course the DLL paths are wrong, but changing it made things run. So that's great. Now I need to figure out why the PyPI package gets away with it being inside .libs near the DLLs and Nuitka standalone code does not.

Ok, deleting openvino.dll from top level dist folder, also solves this.
I believe when adding the DLLs, dependent DLLs go to the top level and not near the DLL using it, and the duplicate DLL is not even detected as a bad thing, when it gets added later. It of course should be only added in one spot, that by itself is another bug of Nuitka.
In this particular case, the DLL has to be near its data files, as it uses its own file path, so that is very much unwanted. We might have to revise how the DLL dependencies found for DLLs in folders are handled. I think they were put top level in an attempt to use that as the form of deduplication, to put them all there. It however should be easy to see that these DLLs are found near the inserted DLL, and therefore it should not be done that way.
This may require more thinking.

Thanks for your report, this is fixed on the factory branch, which is a development version under rapid development. You can try it out by going here: https://nuitka.net/doc/factory.html
Feedback if this is working is very welcome, just please do not share plans of doing it, but rather confirmations or denials of it working.

This is part of the hotfix release 1.5.1 that I just made.


fix permalink bug

when the permalink like this
permalink: :category/:abbrlink.html

the permalink will Display as https://som.im/seo/65d1869b.htmlamp

@DamonHu
I'm sorry, but that fix causes problems with current usage. I think you are using hexo-abbrlink.  Is it possible to change permalink as follows?
permalink: :category/:abbrlink/

Then it will work normally with the current version.
Edit your theme (e.g. head.ejs)
<% if (is_post() && config.generator_amp){ %>
  <link rel="amphtml" href="<%= config.url %><%= config.root %><%= page.path %>amp/index.html">
<% } %>

Yes, I used that plug-in. In order to be more unified, I've changed the amp to the front
 <link rel="amphtml" href="<%= config.url %><%= config.root %>amp/<%= page.path %>index.html">

My submission is to solve the problem that the permalink is not a directory format. when you set the permalink end with the .html.
Because I think it's better to SEO, so I always add .html to the back and find that problem. Maybe I'm an disparate😂.  And thank you for doing this

I'm sorry , I had forget custom path function. This request can solve by generateAmpPath option.
# _config.yml
generator_amp:
  generateAmpPath: amp/:category/:abbrlink.html

Please try it in current version.
I think that extension does not affect SEO.  It is more effective to create more articles than that you think to set the permalink end with the .html.


kubesec not running on alpine linux

Problem
Trying to run kubesec in alpine linux results in:
bash-4.4# ./kubesec
bash: ./kubesec: No such file or directory

How to reproduce
$ docker build -t kubesec_bug https://gist.githubusercontent.com/neerfri/08b3f87a6ad9f5bdb9bf771616175407/raw/e8b8d578122f499bfbb63a8c49811d25b159e64b/Dockerfile
$ docker run --rm -it kubesec_bug /bin/sh -c '/root/kubesec'
$ docker run --rm -it kubesec_bug /bin/sh -c 'ldd /root/kubesec'

Notes
I believe the reason is that it's dynamically linked against libraries (glibc) that don't exist in alpine (by default).
You can see a similar example and suggestions for solutions in:
https://github.com/gliderlabs/docker-alpine/issues/78
and https://github.com/blang/golang-alpine-docker

Fixed in 0.9.0.

@shyiko Thanks! You rock! 🤘


Attached data disk encryption

Added text for Attached data disk encryption, as per SQL Server team and Saket for the upcoming Ignite (2022) announcement

@RunCai @grbury pls review and sign-off

Learn Build status updates of commit 4a2e97b:
:warning: Validation status: warnings

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-overview.md
:warning:Warning

Details

articles/confidential-computing/confidential-vm-overview.md

Line 55, Column 97: [Warning: hard-coded-locale - See documentation] Link 'https://learn.microsoft.com/en-us/windows/security/information-protection/bitlocker/bitlocker-overview' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 55, Column 250: [Warning: hard-coded-locale - See documentation] Link 'https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/manage-bde-autounlock' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 55, Column 382: [Warning: hard-coded-locale - See documentation] Link 'https://learn.microsoft.com/en-us/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 55, Column 829: [Warning: hard-coded-locale - See documentation] Link 'https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/overview' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
Line 55, Column 97: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/en-us/windows/security/information-protection/bitlocker/bitlocker-overview' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 250: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/manage-bde-autounlock' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 382: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/en-us/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 382: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: https://learn.microsoft.com/en-us/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps
Line 55, Column 829: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/en-us/azure/key-vault/managed-hsm/overview' will be broken in isolated environments. Replace with a relative link.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Learn Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@mamccrea,
Can you review the proposed changes?
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Learn Build status updates of commit 2dcf683:
:bulb: Validation status: suggestions

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-overview.md
:bulb:Suggestion

Details

articles/confidential-computing/confidential-vm-overview.md

Line 55, Column 97: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows/security/information-protection/bitlocker/bitlocker-overview' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 244: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows-server/administration/windows-commands/manage-bde-autounlock' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 11ec5f1:
:bulb: Validation status: suggestions

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-overview.md
:bulb:Suggestion

Details

articles/confidential-computing/confidential-vm-overview.md

Line 55, Column 97: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows/security/information-protection/bitlocker/bitlocker-overview' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 244: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows-server/administration/windows-commands/manage-bde-autounlock' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit d134d8d:
:bulb: Validation status: suggestions

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-overview.md
:bulb:Suggestion

Details

articles/confidential-computing/confidential-vm-overview.md

Line 55, Column 97: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows/security/information-protection/bitlocker/bitlocker-overview' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 244: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows-server/administration/windows-commands/manage-bde-autounlock' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

#do-not-merge

Learn Build status updates of commit a847145:
:bulb: Validation status: suggestions

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-overview.md
:bulb:Suggestion

Details

articles/confidential-computing/confidential-vm-faq-amd.yml
:white_check_mark:Succeeded

articles/confidential-computing/confidential-vm-overview.md

Line 55, Column 97: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows/security/information-protection/bitlocker/bitlocker-overview' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 244: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/windows-server/administration/windows-commands/manage-bde-autounlock' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: docs-link-absolute - See documentation] Absolute link 'https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps' will be broken in isolated environments. Replace with a relative link.
Line 55, Column 370: [Suggestion: preserve-view-not-set - See documentation] You've pinned this link to a specific version of content with the view parameter. It's recommended not to pin a version unless that version is A) not the default view and B) the context is about that version specifically. To proceed with pinning a version add the &preserve-view=true to the URL.  Otherwise, remove the view parameter. URL: https://learn.microsoft.com/powershell/module/bitlocker/enable-bitlockerautounlock?view=windowsserver2022-ps

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@RunCai changes have been moved to FAQ now

Learn Build status updates of commit e753495:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-faq-amd.yml
:white_check_mark:Succeeded

articles/confidential-computing/confidential-vm-overview.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

Learn Build status updates of commit 74cbf1b:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

articles/confidential-computing/confidential-vm-faq-amd.yml
:white_check_mark:Succeeded

articles/confidential-computing/confidential-vm-overview.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel

@saketmicrosoft I'm closing this PR per Run's comments, the steps to data disk encryption need to be more verbose in this section and requires engineering review -
If you have a requirement to move the attached data disks to another virtual machine, you may want to save the bitlocker keys into a secure key management store like M-HSM & retrieve it manually or via APIs to re-attach the data disk on to a new virtual machine. Further to that please enable auto-uplock on the new VM/CVM as explained above.
Happy to help with a new PR


Error running with webpack: ENOENT: no such file or directory, open 'iam_service.json'

Environment details

OS: Linux & MacOS
Node.js version: v14.15.0
npm version: 6.14.8
@google-cloud/pubsub version: 2.10.0

Steps to reproduce

When running a pubsub consumer or producer with webpack, the following error occurs:

"err":{"message":"ENOENT: no such file or directory, open 'iam_service.json'"," stack":"Error: ENOENT: no such file or directory, open 'iam_service.json'\n     at Object.openSync (fs.js:476:3)\n     at Object.readFileSync (fs.js:377:35)\n     at fetch (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:403345:34) at Root.load (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:403379:13) at Root.loadSync (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:403420:17) at Object.loadSync (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:105027:29) at GrpcClient.loadFromProto (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:245393:48) at GrpcClient.loadProto (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:245434:21) at new IamClient (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:245684:38) at new SubscriberClient (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:86374:26)"
NOTE: This issue only happens when running with webpack, it works expected when webpack is not in use.

This problem can be worked around by creating directory ../../protos and place the following files in it:

$ ls ../../protos
iam_service.json
protos.json
Unfortunately ../../protos falls one level up of our install directory, so it's not a valid work around.

Using the {fallback: true} option in the PubSub constructor, as suggested in this post: https://github.com/googleapis/google-cloud-node/issues/2933, results in another error:

`"fetch is not a function","stack":"TypeError: fetch is not a function
at Service.newServiceStub. [as testIamPermissions] (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:244522:17)
at /Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:245730:29
at /Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:246661:16
at OngoingCall.call (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:244030:27)
at NormalApiCaller.call (/Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:246452:19)
at /Users/harrygardner/Documents/code/cribl/dist/sluice/bin/cribl.bundle.js:244163:30\n    at processTicksAndRejections (internal/process/task_queues.js:93:5)
Is there a work around for this issue?

@hgardneriv Thanks for the report! I think for now our take on webpack / rollup / et al is that we don't officially support it, but there is some talk about moving toward making it work at some point. We're putting the (badly named) web label on issues that have to do with that.

@feywind is there any way to escalate this issue? We are running into this same issue would like to know if there is a path forward from GCP / Google. Thanks again!

@amiracle The quickest way I know of to start some sort of escalation is through a support case, but I can say that there are actually some higher level discussions going on about the webpack support right now, so that might not be necessary. I can't promise anything, but it's more and more on the radar lately.

@alexander-fenster is taking a look at this now, he's our gax maintainer, so hopefully he'll have something for you.

Hi @hgardneriv,
Judging by your stack, I'm assuming you run webpack with target: "node", is this assumption correct?
In this case, don't use the fallback option (which is for browsers and other environments with no gRPC support).
The problem is caused by this snippet in google-gax:
    const nodejsProtoPath = path.join(
      __dirname,
      '..',
      '..',
      'protos',
      'iam_service.json'
    );

The default behavior of webpack changes how __dirname works (you can check it by doing console.log(__dirname) in your script and comparing the result with and without webpack), so this logic does not work properly when webpacked. The workaround is to add the following into your webpack config:
module.exports = {
  mode: 'production',
  target: 'node',
  node: {
    __dirname: true,  // this makes all the difference
  },
};

Please let us know if this workaround is acceptable for you.

To add to what I just said - since this code actually uses fs.readFileSync to read the JSON file, the webpack bundle produced won't be self contained: it will still need to be able to read<EMAIL_ADDRESS>and node_modules/google-gax/protos/iam_service.json from the filesystem so you will need to put at least these two files in your distribution along with the bundle.

Adding one more thing :) This is my test project that worked for me: https://gist.github.com/alexander-fenster/33c0142f0d62c2d78d0fa3e76559e26b

Hello @alexander-fenster thank your for the info. Yes, we are running webpack with 'target: "node"'.
Unfortunately enabling __dirname will not work in this case, we use the following:
      node: {
        __dirname: false,
        __filename: false,
      },

Something that would be nice is the ability to pass in the config directly vs. reading from the filesystem. With this option we could easily work around the issue. Given our webpack config, do you see any other viable work arounds?

@hgardneriv OK, I'll try to see if I can make an option for proto files root and will get back to you.

Also note that we have the same issue with protos.json which I think pulled from @google-cloud/pubsub.

Yes, those are very similar places in the code and the fix would affect all of them. The second place is in https://github.com/googleapis/nodejs-pubsub/blob/master/src/v1/publisher_client.ts#L156-L162 and the third is in https://github.com/googleapis/nodejs-pubsub/blob/master/src/v1/subscriber_client.ts#L157-L163.

@hgardneriv Just an update - I actually found a better way to make it work; I'll basically replace loading the proto JSON from the filesystem with a require call. This change will take some time to be released (since several packages are affected) but I'll be able to release the tagged next version of @google-cloud/pubsub with this change so you could evaluate it. I'll keep this issue updated when it's done (hopefully within a day or so).  Thank you!

@hgardneriv Before I start pushing the fix through several libraries, here's the test version for you to check:
$ rm -rf package-lock.json node_modules
$ npm install @google-cloud/pubsub@load-proto-json

The following command should give the pre-released versions for these two packages:
$ npm ls @google-cloud/pubsub google-gax
pubsub-repro@ /Users/fenster/pubsub-repro
└─┬<EMAIL_ADDRESS>  └──<EMAIL_ADDRESS>
With these versions, I'm able to run the test code both with and without webpack. Could you please check that this fix works for you?
Thank you!

@alexander-fenster - I'm happy to report that the build is working as expected, nice work!!!

Thank you! I'll go ahead and push this changes to the generator and google-gax. The wheels will start turning today and the releases will likely come out early next week.

I have the same problem with google-translate. The dist version of the app (builded by webpack) always return ENOENT: no such file or directory, open '/protos/protos.json

@alexander-fenster it looks like something similar to @hgardneriv case with  @google-cloud/pubsub, but right now it's for translation.

@dmarczydlo Please wait a few days and this will go away when I update the generator and the dependency chain to use require instead of the filesystem modules.

Awesome @alexander-fenster please let me know when the new approach will be available. I'm looking forward.

@alexander-fenster any update?

The code is ready, the releases are pending, please expect it soon. I'll update this issue.


Add .editorconfig with two-space indents to match code style

This will automatically configure many editors to use two space indents, making it easier for people to contribute who normally use four spaces for Python code.

GitHub account is now registered.


[Question] Why do distribution integration tests have dependencies?

I was looking over our tests manifest schema today and saw a build-dependencies node.  I was under the impression when we are running integration tests they are run against an OpenSearch service instance running the entire distribution.  Is this correct?
https://github.com/opensearch-project/opensearch-build/blob/main/manifests/1.3.0/opensearch-1.3.0-test.yml#L17-L23
  - name: anomaly-detection
    integ-test:
      build-dependencies:
        - job-scheduler
      test-configs:
        - with-security
        - without-security

@saratvemulapalli Do you have more context on this?  I can see that we copy some job scheduler artifacts, is that something that we should be doing for all test runs, or is there a deeper dependency chain that I am missing? https://github.com/opensearch-project/opensearch-build/blob/main/src/test_workflow/integ_test/integ_test_suite_opensearch.py#L55

I believe this is because the code used to run the tests gets compiled and needs these dependencies. In this example we check out anomaly-detection, then run ./gradlew integTest on it with some parameter to point to an external cluster. I think we could simplify this by downloading the entire build's maven artifacts and not cherry-pick what we need.

@peternied Anomaly Detection has a compile time dependency on Job Scheduler SPI. It is needed for the plugin to build and run the tests.
Ref: https://github.com/opensearch-project/anomaly-detection/blob/main/build.gradle#L566

@saratvemulapalli Seems like we should require the integ-test.sh script to handle this case rather than the shared test infra, what do you think?  I think if we populated local maven as dblock suggests it sounds like the setup portion is very straight forward.

Sorry I missed this. Thanks @bbarani for the tag.
@peternied sure. I couldn't really map how that works via integtest.sh. But simply as part of our bundle build, all dependency artifacts should be published to maven local for tests to run.

Closing this issue since the question has been answered as a comment. Feel free to raise this question in OpenSearch forums if you need any other information.


Variance Scaling Initializer

What do you guys think of using Variance Scaling Initializer like tensorflow does, and then defining He, Glorot and LeCun initilizers using it? I would be happy to write the PR if you are interested in this idea

This would indeed allow to streamline the definition of all those initializers. A PR would be welcome otherwise we can integrate that in the next few days.


Update to new plugin infrastructure

As proposed in https://github.com/feathersjs/feathers/issues/608
Drops support for Node 4 but makes it possible to install from Git directly without having to run a compilation step and removes all Babel dependencies.
I think the large diff is because the code transformation changed all the linefeeds. Only the import statements really changed.

I'll get on this as soon as I have time.

I'm still having sys admin issues, so I'm merging this blind. :)


Disease subclass

Break out disease annotation into geneDisease, alleleDisease, AGMDisease as an attempt to isolate kind-specific slots with their specific annotation type.  Also add back in constraints from ingest schema (ie: is_model_of only used with AGMDisease annotations).
For discussion in DQM next week :)

Thanks @sierra-moxon
Here are my comments:

'sgd_strain_background' slot should be declared in GeneDiseaseAnnotation class, not the AlleleDiseaseAnnotation class; question to consider: Does SGD intend to use this slot for anything other than gene-disease annotations?
'with' slot should be declared in parent class DiseaseAnnotation if RGD intends to use it for strains (as is currently being submitted); if only SGD is using it for gene-disease associations then maybe it should instead be declared in the GeneDiseaseAnnotation class; in any case, it shouldn't be restricted to the AlleleDiseaseAnnotation class.
'genetic_modifier' and 'genetic_modifier_relation' needs to be declared in the parent class DiseaseAnnotation and not just in AlleleDiseaseAnnotation; Maybe we need 'gene_genetic_modifier', 'allele_genetic_modifier' and 'agm_genetic_modifier' slots, each with an appropriate range: Gene, Allele, or AffectedGenomicModel?
In 'AGMDiseaseAnnotation' class definition, in slot_usage the subject should have range "AffectedGenomicModel" not "Allele" and the predicate range should be 'agm_disease_relation_enum' not 'allele_disease_relation_enum'


KAFKA-16413: add FileLockTest

Add unit tests for FileLock.
Ref: https://github.com/apache/kafka/pull/15568#pullrequestreview-1950676267
https://issues.apache.org/jira/browse/KAFKA-16413
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@FrankYang0529 Could you please add following use cases to UT?

call lock repeatedly - the later calls should throw exception
call unlock after calling lock repeatedly - unlock should work
ditto to tryLock, unlock, destory


Manage product level for deployment

User Story:
As a System Administrator, I want a way to toggle the product level for a deployment, so that I can manage what features users have access to without involving IT.
Description:
This story focuses on Unlocked Labs' ability to manage feature levels (Level 1, Level 2, or Level 3) available to users in a given deployment. It is motivated by the following needs:

Simplifying deployments by having one deployment with various features that can be locked or unlocked.
Demonstrating and selling the product based on customer interest by showcasing relevant features.
Making changes without requiring IT intervention.

In a SaaS product, this is analogous to a customer's product plan, which determines the features they see upon login. While there are no current requirements for different feature levels by user or facility, it would be ideal to avoid architectural decisions that might limit this ability in the future.
Tasks:

[ ]  Implement a back-end feature flag system that supports enabling or disabling features.
[ ]  Implement middleware/API route check framework to validate requests have access to feature flags.
[ ]  Implement front-end route guards to verify active user can access pages.
[ ]  Create an administration page to active/deactivate feature flags.

User Flow:

Modify Feature Access

System administrator logs into the system and sees navigation option for “Feature Management”
SA toggles feature on or off for the active instance and saves changes.
Users see updated navigation menus and access based on the new feature configuration.

Acceptance Criteria:

The system administrator can log in and enable/disable specific features such as Open Content, Course Management, Program Management.

Administrators cannot make these changes.

The changes take effect immediately without requiring a deployment.

#487 and #488 are directly dependent on this.


Ability to define metrics in dbt models

In order to use metriql in a dbt project, you need to be using YML files in dbt but not all the users are familiar with it. We should find a dbt-native way for SQL-heavy users to define them inside the SQL. One solution would be parsing column names such as metric__[MEASURE_NAME]__[OPTIONAL_AGGREGATION_TYPE] and creating measures automatically for these columns.
We need to create a macro so that the users can define them as follows:
{{config(materialized='metriql')}}

select c_nationkey,
    count(*) as {{measure('total_users', aggregation='sum')}},
    min(c_acctbal) as {{measure('min_balance', aggregation='min')}}
from {{source('tpch', 'customer')}}

The model above will be compiled as:
select c_nationkey,
  count(*) as metric__total_users__count,
  min(c_acctbal) as metric__min_balance__min
from snowflake_sample_data.tpch_sf1.customer group by 1

If there is no YML file that documents the model, we will get the table schema, create dimensions for each column, look for the columns that start with metric__, and create a measure for each of them. The dataset will look something like this:
target: source('tpch', "customer")
dimensions:
  c_nationkey:
        column: c_nationkey
  total_users:
        column: total_users
  min_balance:
        column: min_balance
measures:
  total_users_sum:
     aggregation: sum
     column: total_users
  min_balance_min:
     aggregation: min
     column: min_balance

FWIW, I think is a bit too clever to be relevant at this phase of metriql. The proposal isn't in harmony with the python principle of “prefer explicit over implicit”.

@dbrtly thanks for the input. The rationale behind this feature is making the switch to metriql for the existing dbt projects seamless. The power users usually have a convention for the metrics inside dbt models (here is an example: https://emilyriederer.netlify.app/post/convo-dbt/) and I'm trying to provide a way for them to use metriql with changing only a few lines of their existing files.
I agree with you on prefer explicit over implicit but reducing the friction is also important in this context. We're unlikely to implement such a solution without having a consensus anyways. :)

I hadn’t seen this post before. I get dbt errors if I leave out the yml files.
Is it feasible to go further than a naming convention and get better reach? Users don't necessarily having a naming convention in common but the ansi sql is common. Could metricql automatically add meticql config to a yml file after scanning the code for the full pattern in the second last cte:
select … sum(…) as column_name
and automatically classify all of those columns as measures?

I could not find a reliable way to do that yet and I don't think that I can. Here are the limitations:

The first option is parsing SQL queries but every database has a different dialect so it's not easy as it seems.  The parsers need to understand the full syntax of the database and I don't know any fault-tolerant parser that just skips the unknown parts of the query and just looks at the column definitions of the actual query.

The other solution would be running some metadata queries in the target database to find out the aggregation function of the metrics. If your materialization is view, a few databases provide a way to inspect the views but if it's table or incremental, then all the metrics become regular columns and even the database doesn't know if they're metric or not.

Hey @buremba,
To me the metrics should be part of metadata and YML files, so that analysts who don't want to break the data flows can easily inject and manage their metrics. From this aspect I like how cube.js defines complex metrics within "sql" meta (https://cube.dev/docs/direction-of-joins#transitive-join-pitfall). Maybe metriql can introduce a new dbt macro that generates/propagates metric SQL defined in YAML into model sql files. For e.g., in select section of a model so that all auto-generated metrics are strictly managed in YAML files:
select   .... some source to propagated expression,   {{ metriql_add_metrics(model_name, metric_group, put_comma_before=False, ... ) }}   ... from ... some_source ...
Moreover, analysts probably wouldn't want to define metrics only on the sources but also on the model outputs. This will require introduction of model yaml definitions. Maybe metriql should not place metrics in source YAML files, but introduce its own YAML directory to add metrics to sources or models, and developers can integrate such metrics into data flow by calling metriql macros within their transformation queries.

I would like to put forward naming conventions for metric and have a “generate_metriql_yaml” macro (like the dbt package codegen).
For example the following:

starts with:
count_of_
count_of_unique
approx_count_of_unique
total_
average_
maximum_
minimum_

That would be a start
Get Outlook for iOShttps://aka.ms/o0ukef

From: Mustafa Kıraç @.>
Sent: Saturday, August 7, 2021 10:04:12 PM
To: metriql/metriql @.>
Cc: Daniel Bartley @.>; Mention @.>
Subject: Re: [metriql/metriql] Ability to define metrics in dbt models (#17)
Hey @burembahttps://github.com/buremba,
To me the metrics should be part of metadata and YML files, so that analysts who don't want to break the data flows can easily inject and manage their metrics. From this aspect I like how cube.js defines complex metrics within "sql" meta (https://cube.dev/docs/direction-of-joins#transitive-join-pitfall). Maybe metriql can introduce a new dbt macro that generates/propagates metric SQL defined in YAML into model sql files. For e.g., in select section of a model so that all auto-generated metrics are strictly managed in YAML files:
select .... some source to propagated expression, {{ metriql_add_metrics(model_name, metric_group, put_comma_before=False, ... ) }} ... from ... some_source ...
Moreover, analysts probably wouldn't want to define metrics only on the sources but also on the model outputs. This will require introduction of model yaml definitions. Maybe metriql should not place metrics in source YAML files, but introduce its own YAML directory to add metrics to sources or models, and developers can integrate such metrics into data flow by calling metriql macros within their transformation queries.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/metriql/metriql/issues/17#issuecomment-894645987, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADLXPTJVVFZGK3XOFQXNL3TT3UOLZANCNFSM5BKSEPGQ.

Hey @muskirac, thanks for the insight. Indeed, metriql lets you create datasets from all your dbt resources including models, sources, and seeds. We also support sql relations and generate JOIN relations if you query a field from the using the dot notation just like Cube.js.
metriql generates all the SQL queries at runtime and all the consumers connect their data warehouse through our JDBC, REST API, etc. so I'm not sure if generating dbt models from metriql has a potential use-case with the metriql_add_metrics macro. How do plan to use the generated dbt model?
@dbrtly, clever idea! I agree with both of you, this approach seems to be a "black box" for the end-users and potentially be confusing since there will be multiple ways to define metrics. If people define the metrics using column names, it will be hard for them to switch to YML later on as the logic will be separated into two different parts of the codebase.
For now, I'm in favor of generating a macro just like dbt's codegen package as it will be a native dbt approach.

Thinking further is the dbt docs feature extensible? Can you do your own implementation of exposures and make it dance with the documentation website? Maybe:
metriql docs generate
metriql docs serve

@dbrtly, metriql CLI comes with a dashboard that lets you connect to BI tools and audit all the queries at the moment. We're planning to add a new page that visualizes all the data marts and lets you run queries in a simple interface for testing purposes. I believe that rather than introducing separate static documentation, it can be included in the dashboard. What do you think?

We support dbt's native metric definitions at the moment so so we don't need to introduce an alternative for serving docs, running tests, or defining the metrics itself.


[AMBARI-23351] The Ambari-DDL-Postgres-EMBEDDED-CREATE.sql file conta…

…ins incorrect filename inside it.
What changes were proposed in this pull request?
The file "Ambari-DDL-Postgres-EMBEDDED-CREATE.sql" contains some type in the filename inside it's content.
How was this patch tested?
Yes. Manual Testing.
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/1356/
Test FAILed.
Test FAILured.


Use clang-format-9 in CI

Upgrade CI to Ubuntu Bionic and to use clang-format-9 as our standard code formatter. Not using clang-format-9 should still be OK by contributors since other clang-format versions will probably be close, or manual adjustments can be made.
I saw Travis CI has an alpha feature of testing on other architectures now (https://docs.travis-ci.com/user/multi-cpu-architectures/) so I'm trying to enable a single arm64 test. They also support ppc64le and s390x in the same way.
edit: looks like it can take >10 mins for Travis CI to start an arm64 build, so we may want to wait on trying to enable other CPU architectures until they have lifted some of the concurrency limits.

Coverage increased (+0.006%) to 99.13% when pulling a6090439e9fff3302c1bf6648629520bcce39b69 on isaacbrodsky:bump-ci-version into 28a9b0e7807d1a7b7f08db8b3245f1e3c611f431 on uber:master.


[RFC] Adding an ONNX to Torch conversion

Hello everyone, in a recent RFC in torch-mlir a conversion from ONNX dialect to the Torch dialect in torch-mlir was proposed (https://github.com/llvm/torch-mlir/issues/1255). After feedback we are looking at adding that conversion pass to onnx-mlir.
Proposal

Add torch-mlir as a submodule: https://github.com/llvm/torch-mlir
Add a new conversion pipeline from ONNX to Torch (e.g. "--convert-onnx-to-torch-pipeline") to manage both op and type conversion.
Conversion tests in "test/mlir/conversion/onnx_to_torch/"
Ideally target the torch-mlir backend contract which comes after shape inference and maximizing value semantics

Proof of Concept
A proof of concept for the conversion can be found at: https://github.com/nod-ai/onnx-mlir/tree/convert-to-torch
A conversion for ONNXAddOp and ONNXConstantOp is included along with unit tests for each. This includes five passes in the conversion pipeline:

"convert-onnx-to-torch" handles the op by op lowering to torch dialect
"convert-function-types-to-torch-types" converts function arguments to torch types (e.g. torch.tensor) that wasn't converted in the previous pass
"finalize-torch-type-conversion" finalizes the conversion to torch types and appropriately removes any UnrealizedCastConversionOp's
"erase-onnx-entry-point" right now this entry point op gets removed for compatibility with torch-mlir. If this poses a problem on the onnx-mlir side feedback would be much appreciated.
"refine-torch-func-value-semantics" changes the types of tensors for public functions without uses to have value semantics. This is a compatibility requirement for torch-mlir but also a reflection of this PoC conversion targeting non-value semantic tensors. If tensors in onnx-mlir implicitly have value semantics then this pass can be removed and the type conversions handling in passes 1, 2, and 3 can be changed to match the type conversion in torch-mlir more closely.

Looking forward to any and all feedback!
cc @powderluv @silvasean @sstamenova @ashay

We are generally receptive, and we already have an effort from onnx to mhlo and onnx to tossa. Have you considered piggybacking on these? Namely from mhlo or toss to torch-mlid?
Adding new conversions is fine pending on these two conditions: one is that there is the support behind it to carry to a fully (or close to fully) supported pass. Our goal here is to avoid semi-complete solutions that add code burden to the project without providing useful service to our customers. Second that we resolve our dependences with LLVM in a somewhat concerted fashion. On that latter point, right now we attempt to synchronize our upgrade to LLVM on a weekly basis with MHLO... you can check our pinned issue #1624.

We are generally receptive, and we already have an effort from onnx to mhlo and onnx to tossa. Have you considered piggybacking on these? Namely from mhlo or toss to torch-mlid?

We have Torch -> {Linalg,MHLO,TOSA} conversions on the Torch-MLIR side, so ideally we can centralize all efforts on the ONNX -> Torch conversion and remove the need for ONNX -> {MHLO,TOSA} direct lowerings. Conversely, if significant effort is going to be invested in ONNX -> {MHLO,TOSA} then ONNX -> Torch makes less sense.

On that latter point, right now we attempt to synchronize our upgrade to LLVM on a weekly basis with MHLO... you can check our pinned issue #1624.

We are all basing off of the green commit in https://github.com/llvm/torch-mlir/issues/1178 so this should not be an issue.

5. "refine-torch-func-value-semantics" changes the types of tensors for public functions without uses to have value semantics. This is a compatibility requirement for torch-mlir but also a reflection of this PoC conversion targeting non-value semantic tensors. If tensors in onnx-mlir implicitly have value semantics then this pass can be removed and the type conversions handling in passes 1, 2, and 3 can be changed to match the type conversion in torch-mlir more closely.

This shouldn't be needed. Builtin tensors, which onnx-mlir appears to use, already have value semantics, so you should just directly convert builtin tensors to !torch.vtensor<...> appropriately.

Adding new conversions is fine pending on these two conditions: one is that there is the support behind it to carry to a fully (or close to fully) supported pass. Our goal here is to avoid semi-complete solutions that add code burden to the project without providing useful service to our customers.

Sounds good. We have few customers who would like to see this unified to avoid having two separate paths.  nod.ai team can contribute to support the required passes (as we do in torch-mlir) .
If this sounds reasonable we can get @qedawkins to refine the RFC with specific commits and start putting them up for review.

Right now, the MHLO is making rapid progress, which would integrate the ONNX->TF ecosystem. If there is sufficient support on your end, I can see the benefit of having a ONNX -> Torch ecosystem as well.  Can you comment on the technical merit of having a direct ONNX->Torch-MLIR vs ONNX->MHLO for your particular use?

Can you comment on the technical merit of having a direct ONNX->Torch-MLIR vs ONNX->MHLO for your particular use?

So for Torch-MLIR it would be a no-op technically. The idea is that since Torch-MLIR already has these lowerings written, then onnx-mlir could just use them. A few technical advantages of ONNX -> Torch -> MHLO for the ecosystem would be:

as MHLO evolves to StableHLO for frontends to target, onnx-mlir would be shielded from all those changes (additionally, as an ecosystem, we would only need to update one place)
onnx-mlir would benefit from all the e2e correctness testing that Torch-MLIR does for the lowerings
I see that some of the contributors to ONNX -> MHLO lowerings are from the same teams as Torch -> MHLO lowerings @Connor-XY @yaochengji (cc @fortianyou and @ZihengJiang for visibility from the TorchToMhlo side). This may reduce the work for those teams.
ONNX-mlir would get TOSA and direct Linalg lowerings "for free".

A few potential downsides:

onnx-mlir development would be more coupled to Torch-MLIR
onnx-mlir developers would have to be more aware of PyTorch op semantics to write ONNX -> Torch lowerings

Thanks for the feedback. We are definitely encouraging the MHLO onnx-mlir developers to migrate to StableHLO and I believe they agreed once it becomes stable... think it is almost there.
I agree also that with a stronger connection to MLIR via MHLO and possibly Torch, onnx to tosa might become redundant and since I have not seen lots of progress lately (apologies if mischaracterized) that might be a better way to go about that.
Given the small teams working on these projects, if there is a way to have common infrastructure (reusing when possible) between the MHLO and Torch, that would be great. Exploring structures to enable this kind of reuse would strengthen both effort and reduce redundancy, as AI models (ONNX, Torch, TF) do a lot of incremental changes that all need to be maintained and upgraded over time. So more reuse means less updating of redundant activities...
Along a similar way, we have a unified way to do shape inference to detect compile time shapes, and reuse the same code (in a different context) to generate the runtime code for runtime shapes. MHLO has started approaching custom code for dyn shapes, and we are encouraging to reuse the common infrastructure between shape inference for compile time and run time shapes.

Along a similar way, we have a unified way to do shape inference to detect compile time shapes, and reuse the same code (in a different context) to generate the runtime code for runtime shapes. MHLO has started approaching custom code for dyn shapes, and we are encouraging to reuse the common infrastructure between shape inference for compile time and run time shapes.

We have the exact same thing: https://github.com/llvm/torch-mlir/blob/main/docs/shape_lib.md :rofl:

Good to know... looks like we went through the same pain points, having 2 similar code for shape inference and later code gen, to unify both for lower maintenance.
We also have not generated guard yet, though it is also in the back of our mind.

As I understand the propose here is to integrate ONNX->Torch into onnx-mlir only and does not utilize onnx-mlir further e.g. to LLVM level.  Also this erase-onnx-entry-point indicates onnx-mlir driver is not needed. Then onnx-mlir can be a third_party somewhere for torch-mlir to import onnx models into torch-mlir dialect. In that sense, shape inference and other passes in onnx-mlir are not needed, and there is no need to unify them between torch-mlir and onnx-mlir.

The idea is that ONNX->Torch could save effort for the onnx-mlir community, by providing ONNX->Torch->{Linalg,MHLO,TOSA} which gives Linalg, MHLO, and TOSA "for free" (and eventually TCP, StableHLO, etc.). As Torch-MLIR we don't really have any interest per se in importing ONNX, so we would not take a dependency on onnx-mlir. We are simply offering to collaborate and provide a service if onnx-mlir doesn't want to maintain Linalg, MHLO, and TOSA (+ TCP, StableHLO, ...) lowerings yourselves. I'll let the onnx-mlir community decide if that is appealing, and we are willing to collaborate if you folks want that.

The idea is that ONNX->Torch could save effort for the onnx-mlir community, by providing ONNX->Torch->{Linalg,MHLO,TOSA} which gives Linalg, MHLO, and TOSA "for free" (and eventually TCP, StableHLO, etc.)

Just want to see the big picture. Could you elevate a bit about how end-users use this pipeline? For example,  how they prepare inputs for running the compiled model and get outputs. Your vision is to use onnx-mlir driver for that purpose or expect a new driver, or the driver already exists.

We are simply offering to collaborate and provide a service if onnx-mlir doesn't want to maintain Linalg, MHLO, and TOSA (+ TCP, StableHLO, ...) lowerings yourselves

Yes, sure, it makes sense.

Just want to see the big picture. Could you elevate a bit about how end-users use this pipeline? For example, how they prepare inputs for running the compiled model and get outputs. Your vision is to use onnx-mlir driver for that purpose or expect a new driver, or the driver already exists.

I'm imagining that onnx-mlir would just link to Torch-MLIR and run a few of Torch-MLIR's passes as an internal implementation detail to get from ONNX -> {MHLO,TOSA,Linalg} depending on the user wants. It shouldn't touch things like preparing inputs/drivers/etc.

We (nod.ai) have couple customers that require this - and we plan to support them somehow. And that is why we added the initial RFC with implementation in torch-mlir (which included onnx / onnx-mlir as deps) and then this RFC+implementation with torch-mlir as a dep in onnx-mlir.
Basically the customers want to avoid two paths to MHLO (which they care about) and two kinds / levels of ops/shape support.  We (nod.ai) team contribute a lot to torch-mlir to maintain the torch-dialect->{MHLO, TOSA, Linalg} and if we can leverage an ONNX entry point into torch-dialect and then leverage all the other lowerings it is one set of backend lowerings with two "frontends" torch and onnx.
The end product would like like a compiler/binary that can consume ONNX modules as an Onnxruntime EP or can consume Torch in some other serving solution like Triton Inference server.  The lowering to torch-dialect may be different but after that it is all the same allowing backend Accelerators to easily consume from both frameworks.

@Connor-XY @yaochengji Could you please chime in? Would like to hear your comments about ONNX->Torch->MHLO vs ONNX-MHLO. Thanks!

I think it would be great to have the conversion from ONNX to Torch, but I also believe that it is better to have the conversion from ONNX to MHLO directly. First, we could do the conversion directly so it isn’t necessary to use Torch as the bridge. Second, if we adopt ONNX->Torch->MHLO and there is a version upgrade, we need to modify both ONNX->Torch and Torch->MHLO. The conversion would rely on Torch (onnx-milr development would rely on Torch more).

To add what @Connor-XY said, thanks for the help of @tungld, @AlexandreEichenberger and all other community members, we are currently applying onnx-mlir on our business models in ByteDance.
We'll definitely try the ONNX->Torch route after it is mature. And before that, we'll keep maintaining and developing the ONNX-MHLO route.

We (at d-Matrix) support the idea of a robust conversions between ONNX and Torch front-ends. It will be interesting to see how this RFC evolves. It seems reasonable to avoid duplicating the effort to support intermediate dialects and features like shape inference. This will likely make a convenient solution (for us) to maintain ML front-ends in our stack.

Do you think you could present the benefits of this direct approach at our Tu evening meeting, either next week or in 2 weeks?

Do you think you could present the benefits of this direct approach at our Tu evening meeting, either next week or in 2 weeks? And if you could give us a good link to read about the MLIR/Torch effort, that would be great

We can aim for a presentation + discussion at next week's meeting if that works for you. Torch-MLIR has documentation you can reference for the time being here: https://github.com/llvm/torch-mlir/tree/main/docs
The architecture document (https://github.com/llvm/torch-mlir/blob/main/docs/architecture.md) might be a good place to start.

@AlexandreEichenberger What is the status on this? Is tomorrow's meeting still a good time to discuss this?

Absolutely, you can have a 15 min slot at our 8-9pm est meeting.
https://github.com/onnx/onnx-mlir/wiki/Informal-meeting-agenda-and-notes

Would be great if you could explain the value add of the onnx-torch lowering in the picture of the torch-mlir effort:
https://github.com/llvm/torch-mlir/blob/main/Torch-MLIR.png

There is no value-add for Torch-MLIR (it is actually a minor cost for us to support this). It is purely ONNX-MLIR reusing work already done for Torch-MLIR. It is a win for the ecosystem to maintain fewer lowerings, in theory.

Got it. Found your https://github.com/llvm/torch-mlir/blob/main/docs/architecture.md instructing. If you could give us an idea of where you would intersect ONNX dialect with the dialects in that doc, that would let us focus our attention a bit more on the relevant dialect, tx

ONNX-MLIR would produce the torch dialect in the "backend contract" form.

This diagram shows a preview of how the conversion pass slots in on the Torch-MLIR side for reference before the meeting.

The ONNX-MLIR side in this diagram leaves out some components that can be filled in if we want.

Is there a similar effort to have a "stable torch dialect", like "stableMHLO"?

No, our dialect is only as stable as the torch op set (which is fairly stable, but not guaranteed)

@qedawkins would you be able to add a pdf of your presentation yesterday? for these that were not able to be present. Tx

Sure!
Here it is: ONNX To Torch Conversion.pdf

I have a summary of our Tuesday evening discussion in the wiki pages from this project here.
As a follow on to this discussion, we would like to have feedback from the teams developing onnx->tosa and onnx->mhlo converters, as this RFC may also be able to generate tossa or mhlo output via the torch-mlir backends currently available.
I am listing below developers that I see associated with these two projects; if you know of any other, please add references here and/or contact them directly.
Thanks
@Connor-XY @yaochengji @chenchongsong @BraunPhilipp  @sjarus Please provide feedback as you see fit within a week, would be great to have it before the next meeting where we will take a position on this RFC. You are obviously welcome to participate at the meeting too, now open to the whole onnx-mlir community. See wiki page for info on how to join.

Adding @eric-k256 @stephenneuendorffer and @ljfitz for input on ONNX-TOSA in the context of the likely presence of ONNX-Torch and the existing Torch-TOSA.

Please provide feedback as you see fit within a week

ONNX2Torch route should be useful, especially when we are considering changing to stablehlo. The biggest concern we have is that ONNX -> Torch -> MHLO route is more complex than ONNX -> MHLO, which introduces more potential issues.
And the ONNX->MHLO route is already applied on the business models in Bytedance. The model coverage is quite high ( about 50%), considering we've only worked on this for 2 months.  Therefore we will continue to work on the ONNX->MHLO route to increase the model coverage in Bytedance. For the ONNX -> Torch -> MHLO route, if we later find that it is mature enough, we will switch to it.

Thanks for your feedback @yaochengji, this is very useful. Primary concern to us is to have mature implementation of dialects, so knowing that your team will continue working on MHLO is good to know. Hopefully you can also resolve the handling of dynamic shapes in a way that reuses onnx-mlir infra would be great and unblock some of the MHLO PRs that have been blocked for longer than I like.

Hopefully you can also resolve the handling of dynamic shapes in a way that reuses onnx-mlir infra would be great and unblock some of the MHLO PRs that have been blocked for longer than I like.

Yes, we're actively working on this. I've already discussed with @Connor-XY offline and figured out how to use this correctly. I guess @Connor-XY could resolve this soon.

I mentioned that we have a way to annotate code/makefile/xxx and then gather annotations to generate a report of coverage. We do this for our support of lowering to CPU and to our NNPA based accelerator. This same mechanism can be used to generate a report of coverage of a converter from ONNX to TOSA/MHLO/Torch-MLIR.
PR describes format and will let you know which file/makefile to edit. We added link to reports on main README.md page.
PR #1475

I mentioned that we have a way to annotate code/makefile/xxx and then gather annotations to generate a report of coverage.

Thanks. We'll take a look at this.

Any additional feedback? So far, there is mostly positive feedback.
One thing we learned from MHLO effort is that it is nice to have a switch that disable building the MHLO part. CI all work with everything on, but certain users may elect to not have parts that are not needed. This would also be a feature that would be good to have for torch-mlir converter.

Yes, Torch-MLIR has a build flag to disable building the MHLO path.

Yes, Torch-MLIR has a build flag to disable building the MHLO path.

Was asking if you would also support a flag disabling building Torch-MLIR within onnx-mlir

Ah yes, that's a good idea.

I believe that unless we hear any additional concerns by today, we should go ahead with your RFC.

I believe you should go ahead and start PRs to implement your converter. Good luck.

Thank you all for the feedback and helpful discussion. I've opened a PR #1731 where we can begin ironing out the details and get an initial version of this merged.


Update manifest.json to support BU 570

This change enables the detecting of the BU 570. I did some local testing with a custom script and the parser used in the component also seems to work fine with the BU 570, but for some reason home assistant is not able to connect to the device. Therefore, isn't able to read and parse the data.
the error:
[custom_components.medisanabp_ble] A4:C1:38:**:**:**: Bluetooth error whilst polling: A4:C1:38:**:**:** - A4:C1:38:**:**:**: Failed to connect after 5 attempt(s): Timeout waiting for connect response while connecting to A4:C1:38:**:**:** after 20.0s, disconnect timed out: False, after 20.0s

Some device details:
AdvertisementData(local_name='1872B', service_uuids=['00001810-0000-1000-8000-00805f9b34fb'], rssi=-64)

0000180a-0000-1000-8000-00805f9b34fb (Handle: 12): Device Information
   00002a29-0000-1000-8000-00805f9b34fb (Handle: 13): Manufacturer Name String
   00002a24-0000-1000-8000-00805f9b34fb (Handle: 15): Model Number String
   00002a25-0000-1000-8000-00805f9b34fb (Handle: 17): Serial Number String
   00002a27-0000-1000-8000-00805f9b34fb (Handle: 19): Hardware Revision String
   00002a26-0000-1000-8000-00805f9b34fb (Handle: 21): Firmware Revision String
   00002a28-0000-1000-8000-00805f9b34fb (Handle: 23): Software Revision String
   00002a23-0000-1000-8000-00805f9b34fb (Handle: 25): System ID
00001810-0000-1000-8000-00805f9b34fb (Handle: 27): Blood Pressure
   00002a35-0000-1000-8000-00805f9b34fb (Handle: 28): Blood Pressure Measurement
   00002a36-0000-1000-8000-00805f9b34fb (Handle: 31): Intermediate Cuff Pressure
   00002a49-0000-1000-8000-00805f9b34fb (Handle: 34): Blood Pressure Feature
0000180f-0000-1000-8000-00805f9b34fb (Handle: 36): Battery Service
   00002a19-0000-1000-8000-00805f9b34fb (Handle: 37): Battery Level
00001805-0000-1000-8000-00805f9b34fb (Handle: 40): Current Time Service
   00002a2b-0000-1000-8000-00805f9b34fb (Handle: 41): Current Time

Thank you for helping with the project!


Can't reuse a UUID after deletion

I've just emptied a Weaviate but when I add a new Thing that with an ID that previously existed, I get a 422.
It seems that removing a UUID keeps it somewhere.

Please add exact steps on how to reproduce this. I just tried the following on 0.18.1 and it works fine:
➜  weaviate git:(master) ✗ curl -H "Content-Type: application/json" -d '{"id": "9837590a-96e6-4804-a7ac-67a135a682e5", "class": "Car"}' localhost:8080/weaviate/v1/things/
{"class":"Car","id":"9837590a-96e6-4804-a7ac-67a135a682e5"}
➜  weaviate git:(master) ✗ curl -H "Content-Type: application/json" -d '{"id": "9837590a-96e6-4804-a7ac-67a135a682e5", "class": "Car"}' localhost:8080/weaviate/v1/things/
➜  weaviate git:(master) ✗ curl -X DELETE localhost:8080/weaviate/v1/things/9837590a-96e6-4804-a7ac-67a135a682e5
➜  weaviate git:(master) ✗ curl -H "Content-Type: application/json" -d '{"id": "9837590a-96e6-4804-a7ac-67a135a682e5", "class": "Car"}' localhost:8080/weaviate/v1/things/
{"class":"Car","id":"9837590a-96e6-4804-a7ac-67a135a682e5"}
➜  weaviate git:(master) ✗ curl -X DELETE localhost:8080/weaviate/v1/things/9837590a-96e6-4804-a7ac-67a135a682e5
➜  weaviate git:(master) ✗ curl -H "Content-Type: application/json" -d '{"id": "9837590a-96e6-4804-a7ac-67a135a682e5", "class": "Car"}' localhost:8080/weaviate/v1/things/
{"class":"Car","id":"9837590a-96e6-4804-a7ac-67a135a682e5"}
➜  weaviate git:(master) ✗ curl -X DELETE localhost:8080/weaviate/v1/things/9837590a-96e6-4804-a7ac-67a135a682e5
➜  weaviate git:(master) ✗ curl -H "Content-Type: application/json" -d '{"id": "9837590a-96e6-4804-a7ac-67a135a682e5", "class": "Car"}' localhost:8080/weaviate/v1/things/
{"class":"Car","id":"9837590a-96e6-4804-a7ac-67a135a682e5"}
➜  weaviate git:(master) ✗

It's always the same ID and I keep deleting and re-adding it without any issues.

@bobvanluijt is this still an issue?

Need to validate this, will let you know asap

Nope, just validated this.


c_src

Handle c_src.
First (prototype):
copy c_src, do not package compiled, cache locally.
Second (prospect):
package compiled c_src and remember system for which they were compiled.

Done copying


backport cluster evaluation permissions #1885 to opendistro-1.11

Description
Backport of #1885 to opendistro-1.11
Issues Resolved
Backport PR #1885
Check List

[x] New functionality includes testing
[x] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@peternied we need to revert these changes since it's causing es-7.9 to fail.


UI does not show timestamps on otel indexes

Currently, we don't store the timestamp field for otel indexes (otel-logs-v0 and otel-traces-v0).
Consequently, the UI can't display them as the documents returned do not contain the timestamp field value.

@guilload I think this is ok as long as Quickwit UI is not heavily used by users.


[Flips] Introducing Soul-Bound Token (SBT) to cadence and flow exosystem

This proposal is to introduce a standard for Soul-Bound Token to the flow ecosystem.
Soul-Bound Token means tokens that are Non-Fungible yet not transferrable to the others.
Soul-Bound Token (SBT) should be differentiated from NFTs for long-run to faciliate easy
integration of SBTs.

I think we need different collection for SBTs, instead of each SBT having its own collection.
So NFT contract as a template can be not the best idea.
I see different parties minting SBTs:
TSSBT, allDaySBT, FLOAT etc but I think users would like to store them in single collection.
Otherwise I can sell my TSSBT collection, and forcing soulblund can be tricky.

Yeah, if we truly want to have a type of token that can't be transferred between accounts, then we'd have to implement some sort of central ledger, or introduce a new feature to Cadence that prevents transfers

I have a question: How to define soul-bound in Flow/Cadence?
Token can be transferred, and also, Collection can be transferred, and even the Account itself can be swapped by replacing keys.

@LanfordCai I think account = identity is good starting point. I mean we can skip account swap scenario. Collection can be swapped, but we can make collection to work only when it is stored in one specific account.

Closing as abandoned


Dropdownlist on IPhone

Context
I use dropdownlist on my webpage with options greater than 30 characters. When I'm using Chrome on IOS mobile I can't read all the selected content, but on Android (Tested on Galaxy S7 edge and J7 Metal) I can by scrolling sideways, even if the "option" content is larger than the width.  Is there any solution to scroll horizontally a selected dropdownlist content on IPhone?
Your Environment
Chrome on Iphone 6 or 7

Do you have codepen reproducing the problem? I'm assuming by dropdownlist, you mean any dropdown?

I'm using the standard Materialize Select.

Btw, It will only happen if you select the last option of the dropdown (the longest one)

Also, it's published at: http://teste.crediloja.com.br/cliente/teste.html


perf: cache computations in WithRequestDocumentRoot

Alternative to #1152.
Improve performance by ~10% on my Mac with the default config.

In the meantime, here are some quick benchmarks to put you at ease 😄
Docker Linux amd64 20 CPU cores i9 wrk -t4 -c180 -d30 http://localhost/worker-with-watcher.php
In dunglas/frankenphp:1.25
php_server 40 threads: ~52000 RPS
php 40 threads: ~70000 RPS
php 8 threads: ~69000 RPS
In dunglas/frankenphp:1.3
php_server 40 threads: ~66000 RPS
php 40 threads: ~93000 RPS
php 8 threads (ideal): ~122000 RPS
The benchmark script is the one in testdata/worker-with-watcher.php. This doesn't capture all performance improvements though. Improvements to the $_SERVER import scale linearly with the size of the environment. And dispatching to workers in order should improve IO with external services (hard to quantify unless you're benchmarking a prod environment - which I plan to do with k6 and AWS ECS)

@AlliBalliBaba thank you very much!!
@mholt the working directory can change if the chdir syscall is used: https://www.man7.org/linux/man-pages/man2/chdir.2.html

Ah... I forgot about that. Thanks


Error in composer install

Getting below error when running composer install via Ansible. But when run composer install manually everything works fine.
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Dependency resolution completed in 0.000 seconds
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for kylekatarnls/update-helper 1.2.1 -> satisfiable by kylekatarnls/update-helper[1.2.1].
    - kylekatarnls/update-helper 1.2.1 requires composer-plugin-api ^1.1.0 || ^2.0.0 -> no matching package found.
  Problem 2
    - kylekatarnls/update-helper 1.2.1 requires composer-plugin-api ^1.1.0 || ^2.0.0 -> no matching package found.
    - nesbot/carbon 1.39.1 requires kylekatarnls/update-helper ^1.1 -> satisfiable by kylekatarnls/update-helper[1.2.1].
    - Installation request for nesbot/carbon 1.39.1 -> satisfiable by nesbot/carbon[1.39.1].

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Composer version: 1.10.21

Are you sure, PHP version and composer version are the same when installing manually and in Ansible?
As the error states that composer-plugin-api ^1.1.0 || ^2.0.0 -> no matching package found. (so composer stuff missing), then you should first try to:
composer diagnose
Fix failures that are given there.
Else, if all OK:
composer update -vvv


Analyse where the errors are happening in the map

This PR adds a evaluation on the TSDF value where the error is happening

Observations:

False negatives appear on the boundary of the object
Highest is slightly inside the object
Flase negatives can be decreased when the TSDF threshold when generating bin is increased
Flase negatives can be decreased by adjusting the occupancy threshold


🚧 Dev ➾ rewrite the e2e tests for the contract types plugin

🔗 Epic|Feature Link
No response
🚥 Definition of Done

[ ] 🎯 Planning and Design
[ ] 🏎 Dev Implementation Complete
[ ] ♾ PR Merged
[ ] 🏆 Product Acceptance

🧰 Work Description
The e2e tests are being rewritten using a cli testing library. It is the turn of contract types to be revised.
✅ To Do
rewrite the tets
make them pass or prove the reason they don;t is a bug
⚖ Acceptance Criteria
✨ Functional:

[ ] passing tests or issues logged to explain failed tests
[ ]

🔭 Scope
🙅♂ Out of Scope:

...

other plugns
Code of Conduct

[X] I agree to follow this project's Code of Conduct

this is done


Fix Step 3 in section - Debugging with Stetho on Android

The change should be made in MainActivity.java and not MainApplication.java.
Thanks for submitting a pull request! Please provide enough information so that others can review your pull request:

Unless you are a React Native release maintainer and cherry-picking an existing commit into a current release, ensure your pull request is targeting the master React Native branch.

Explain the motivation for making this change. What existing problem does the pull request solve?
Prefer small pull requests. These are much easier to review and more likely to get merged. Make sure the PR does only one thing, otherwise please split it.
Test plan (required)
Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes UI.
Make sure tests pass on both Travis and Circle CI.
Code formatting
Look around. Match the style of the rest of the codebase. See also the simple style guide.
For more info, see the "Pull Requests" section of our "Contributing" guidelines.

I don't think that is correct. In general Stetho supports initialization in the Application object so it seems like we should encourage people to do that same thing in React Native. See http://facebook.github.io/stetho/ for details. I am going to close this pull request but if I am wrong here please feel free to let me know!


Fix NS line for the zone file

What this PR does / why we need it:
There was a missing @ at the beginning of the NS record, which caused the line to be invalid, and we didn't get any alerts about it on zone re/load.
This also broke recourse query for the NS, as the query will be passed to the internal DNS, but the record is missing.
Special notes for your reviewer:
When asking the parent (forcing no recursive query) DNS for the NS, we get a response,
❯ dig NS vm.cnv2.example.com @ns1.eng.example.com +norecurse

; <<>> DiG 9.18.17 <<>> NS vm.cnv2.example.com @ns1.eng.example.com +norecurse
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32195
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;vm.cnv2.example.com.        IN      NS

;; AUTHORITY SECTION:
vm.cnv2.example.com. 300 IN  NS      ns.vm.cnv2.example.com.

;; ADDITIONAL SECTION:
ns.vm.cnv2.example.com. 300 IN A     <IP_ADDRESS>

;; Query time: 9 msec
;; SERVER: <IP_ADDRESS>#53(ns1.eng.example.com) (UDP)
;; WHEN: Thu Sep 14 16:10:56 IDT 2023
;; MSG SIZE  rcvd: 92

But when the server redirects us to our internal DNS, we get not respond.
❯ dig NS vm.cnv2.example.com @ns1.eng.example.com

; <<>> DiG 9.18.17 <<>> NS vm.cnv2.example.com @ns1.eng.example.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21606
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;vm.cnv2.example.com.        IN      NS

;; Query time: 24 msec
;; SERVER: <IP_ADDRESS>#53(ns1.eng.example.com) (UDP)
;; WHEN: Thu Sep 14 16:11:04 IDT 2023
;; MSG SIZE  rcvd: 59

I tested this by manually patching the zone file to include the missing @.

thanks
something is weird, it doesnt run tests / git actions

/ok-to-test

right thanks, but git action still not running

It looks like the GH workflow has to be approved to run - I can see it here in the UI - I can try and kick it off if ye want?

yes thanks, but we might want to add Rabin to kubevirt group so it wont happen ?

we might want to bump soon kubevirtci, as 1.25 is too old

@rabin-io maybe worth to run please a few times the e2e to see its not flaky ?
there was a failure one time according emails

/test pull-kubesecondarydns-e2e-k8s

/test pull-kubesecondarydns-e2e-k8s

/test pull-kubesecondarydns-e2e-k8s

we are good to go, anyhow it was a cluster-up that fails one time, not tests related

\approve
Thanks!

/approve


Added my name to list.

Thanks for your contribution 🚀

Thank you, today I learned more about the pull request.😄

Thank you, today I learned more about the pull request.😄

Nice work!

@Leonardovalle33, if you want to contribute to this project, I recommend you to first

Join to the GitHub organization.
Join our amazing Discord server.
If you want to contribute to this project, you can

Read the instructions on how to add your profile
Review other people's requests to add their profile. Don't forget to check if it's in alphabetical order. :)

Start looking out for good opportunities to contribute.
Check out for people asking for help in cases where you can help them.
Don't ask to ask: just ask your question, and someone will help you. 🙂

I hope that helps. If you need any more help, let us know! 😄


Rewrite in Rust

This project needs to be rewritten in Rust, as borrow checker would solve many issues you have on every stream.
Not to mention the code will be safer and memory efficient.

@aodhneine I'm really sorry, but this is outside of the scope of the project. The initial idea was a challenge to implement a game entirely in C.


Fix ping checking for memif

Found while #284
Signed-off-by: Artem Glazychev<EMAIL_ADDRESS>

Under what circumstances are we seeing 0 packets sent for ping?

@edwarnicke
Fox example, I'm catching:
vpp# ping <IP_ADDRESS>
Failed: no source address for egress interface
Failed: no source address for egress interface
Failed: no source address for egress interface
Failed: no source address for egress interface
Failed: no source address for egress interface

Statistics: 0 sent, 0 received, 0% packet loss

Right... but that's a legit failure we should fix not paper over :)

Sure. This PR is for this purpose.
Without this fix all test will be passed.


Cannot read properties of undefined (reading '__H')

Hey there guys, I am trying to test some of my custom hooks that work in prod, but don't work with @testing-library/preact-hooks.
I am trying to write some unit tests where I check the height of a banner. Initially I used jest, but now I'm using vitest, which has an API almost identical to jest.
All the tests pass besides the ones on the custom hooks. This seems like a package issue.
Thanks!

Stack Info:
Preact: 10.11.3
Vite: 4.0.0
Vitest: 0.27.0
@testing-library/preact-hooks: 1.1.0

we have a similar issue...have you found a solution @pitis

@ositowang I believe the issue is that useState is not mocked. So if you create a way to mock useState, it may work for the other hooks

https://preactjs.com/guide/v10/preact-testing-library#testing-preact-hooks using renderHook makes the difference.

I spent all of yesterday trying to figure out how to get this working. This issue is quite old now, but on the off chance it might help anyone else with this problem here's what I've found.
@testing-library/preact-hooks seems to be intended to be used with older versions of Preact. Because of conflicting peer dependencies, it can't be used alongside the latest version of @testing-library/preact.
However, if you're using Preact 10, then you can import renderHook from @testling-library/preact instead. For my setup, at least, this resolved my issue and I'm not able to test my hooks as expected.
Unfortunately, at the time I'm writing this, the testing documentation for the latest version of Preact currently recommends importing renderHook from the outdated @testing-library/preact-hooks library.


Some style fixes

I picked this file at random, and have made some fixes for conformity with our style guide. There shouldn't be any changes to functionality here, it's all superficial.

Thanks @willtebbut, looks good to me!

I will take another look soon.

Good catches @xukai92

Thanks for correcting them so quickly! I will merge the PR once the CI passes.


Fix bad dates & missing vaccine types for Publix

This code is untested! I don't have a good setup for running this, but these changes match what I'm seeing from the Publix website. Please run and test before merging.
I’m seeing a lot of Publix entries with unknown vaccine type and bad appointment entries like:
{
  "appointments": [
    {
      "time": null,
      "type": "",
      "vaccine_types": [],
      "appointment_types": []
    },
    {
      "time": null,
      "type": "",
      "vaccine_types": [],
      "appointment_types": []
    }
  ]
}

This should fix the issues, as far as I can tell from looking at Publix's site.

@Mr0grog: Thanks! Your fix got things working again, and I've also gone ahead and updated how vaccine types are determined based on this new endpoint (https://github.com/GUI/covid-vaccine-spotter/commit/697582542e62f7803969e9a655dfcea99d91c0bc). Thanks for the sleuthing!

Happy I could help! 😄