HDDS-2592 Add Datanode command to allow the datanode to persist its admin state

What changes were proposed in this pull request?
When the datanode state is change in SCMNodeManager a Datanode Command is created to transmit the new datanode state to the datanode. The datanode will then persist this state in the "datanodeID.yaml" file.
The current datanode.yaml file simply persists a DatanodeDetails object in YAML format. Therefore two new fields have been added to DatanodeDetails to support this:
persistedOpState,
persistedOpStateExpiryEpochSec
The first is the operational state and the second is the seconds from the epoch the setting should expire (for maintenance mode).
When a datanode starts up, it will read these values and supply them in the heartbeat to SCM.
If SCM notices the DN was registered before, SCM is seen as the source of truth and will update the DN state to that set in SCM if they differ.
If the DN was never registered with SCM (eg if SCM was restarted), then the DN state is seen as the source of truth and SCM will be updated accordingly.
This change also moves the state expiry time into the NodeStatus object, so it now carries operationalState, healthState and State expiry.
Note that the DN does not do anything with the persisted operational state. They are simply persisted and reported in the heartbeat. In this respect the DNs are being used as a form of distributed storage for the operational state so the decommission process can survive a restart.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-2592
How was this patch tested?
Tested manually using a docker build. This change still needs unit tests added if the current approach is agreed upon.

@anuengineer Thanks for the review on this. I probably need to add some tests around these changes. I did not do that initially, as I wanted to be sure we were happy with the approach (storing the state in the datanode details, allowing it to be included in the heartbeat and the DN not making further use of the value). Now you have reviewed and are happy with this direction, I will get some tests added and check your comments in more detail.

/pending "I will get some tests added and check your comments in more detail."

We have parked the decommission work at the moment. Anu approved this change, but I feel we need some unit tests around the changes.
As this is only going onto a branch which we will need to review in more detail later, I have raised HDDS-3303 to log the fact we need some tests and then I propose we commit this to the branch as is to get it out of the PR queue.

/ready


Debug mode: disable multi-threading

For debug mode we can add parameter debug: boolean
noroutine.init({
  modules: [module1, module2],
  debug: true,
});

this will disable multi-threading and ballancing.

@tshemsedinov Does it mean if debug mode is turned on - we should execute our modules in only one thread? As regular nodejs application flow?

Yes @Bortnyak

For debug mode we can add parameter debug: boolean
noroutine.init({
  modules: [module1, module2],
  debug: true,
});

this will disable multi-threading and ballancing.

For debug mode we can set monitoring: 246060*1000 or even more


windows build issues(finally made it)

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): windows 10 x64
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): source
TensorFlow version: 2.4.0
Python version: 3.8.3 x64
Installed using virtualenv? pip? conda?: N/A
Bazel version (if compiling from source): 3.4.1
GCC/Compiler version (if compiling from source): visual studio 2019
CUDA/cuDNN version: 11.0/8.0.1
GPU model and memory: RTX2070 GDDR6 8GB

Describe the problem
many problems with windows build, and I made a few fixes
Provide the exact sequence of commands / steps that you executed before running into the problem

cudnn_stub  dependency problem ( https://github.com/tensorflow/tensorflow/issues/41057 )
I simply add dependency in that BUILD

then thrust version unmatch error
I delete thrust version - cub version comparison codes from
bazel-out\x64_windows-opt\bin\external\local_config_cuda\cuda\cuda\include\thrust\system\cuda\config.h

finally, big old numpy problem
this has been very long and unsolvable error, I made same fix as before( simply pip uninstall numpy ( uninstall conda numpy ) and pip install numpy )

and still there are too many "ignoring unknown options" from cl
anyway now I succeeded build and have my wheel file
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

anyway now I succeeded build and have my wheel file

@alanpurple,
Is this still an issue? Please feel free to close the issue if resolved. Thanks!

this is 4 issues, 3 I fixed myself,  1(warnings) I can't fix

think of this post as kinda PR since I made fixes

@alanpurple,
In this case, could you please submit a new PR for the fix. Thanks!

@amahendrakar
Is conda numpy problem unsolvable?

Is conda numpy problem unsolvable?

Could you please describe this issue and provide the steps to reproduce it, so that we can look into it? Thanks!

@amahendrakar
#35977

@alanpurple,
Can we close this issue since it is already being tracked there? Thanks!

@amahendrakar
ok

@surak I believe this is what he did:
https://github.com/tensorflow/tensorflow/pull/41822/commits/d480a3ad713e50057a2b1a611e7cacd6bbca1eb5
Didn't realize that cudnn_version.h was already defined somewhere in the file. Anyhow, tried it again with fresh source for 2.3.0 and with this change and it builds successfully on Windows now.

I still get a directory without the correct file. All I get is
In file included from tensorflow/stream_executor/cuda/cudnn_stub.cc:15:
bazel-out/k8-py2-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cudnn_header/third_party/gpus/cudnn/cudnn.h:59:10: fatal error: cudnn_version.h: No such file or directory
   59 | #include "cudnn_version.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
SUBCOMMAND: # //tensorflow/core:function_ops_op_lib [action 'Compiling tensorflow/core/ops/function_ops.cc']
(cd /tmp/eb-w9wkEo/tmpSnNZgZ-bazel-build/execroot/org_tensorflow && \

And this directory has
ls -la bazel-out/k8-py2-opt/bin/external/local_config_cuda/cuda/_virtual_includes/cudnn_header/third_party/gpus/cudnn
total 12
drwxrwxr-x 2 strube1 jusers 4096 Aug 25 19:27 .
drwxrwxr-x 3 strube1 jusers 4096 Aug 25 19:27 ..
lrwxrwxrwx 1 strube1 jusers  139 Aug 25 19:27 cudnn.h -> /tmp/eb-w9wkEo/tmpSnNZgZ-bazel-build/execroot/org_tensorflow/bazel-out/k8-py2-opt/bin/external/local_config_cuda/cuda/cudnn/include/cudnn.h

I don't really understand Bazel's syntax. Can you help me with that?

@isleong @alanpurple turns out that it's somewhere else. cudnn 8 has different headers from that cudnn.h and bazel fails to identify the correct version. It's at third_party/gpus/cuda_configure.bzl line 1164:
This is the problematic part:
    if [int(x) for x in cuda_config.cudnn_version.split(".")] < [8, 0]:
      cudnn_headers = ["cudnn.h"]
    else:
      cudnn_headers = ["cudnn_adv_infer.h",
                       "cudnn_adv_train.h",
                       "cudnn_cnn_infer.h",
                       "cudnn_cnn_train.h",
                       "cudnn_ops_infer.h",
                       "cudnn_ops_train.h",
                       "cudnn.h",
                       "cudnn_version.h",
                      ]

This cudnn version is somehow wrong.
This is my patch:
--- third_party/gpus/cuda_configure.bzl.orig	2020-08-26 18:55:08.405838123 +0200
+++ third_party/gpus/cuda_configure.bzl		2020-08-26 19:24:47.466666178 +0200
@@ -1136,10 +1136,11 @@
         out_dir = "cuda/bin",
     ))

-    if [int(x) for x in cuda_config.cudnn_version.split(".")] < [8, 0]:
-      cudnn_headers = ["cudnn.h"]
-    else:
-      cudnn_headers = ["cudnn_adv_infer.h",
+    #if [int(x) for x in cuda_config.cudnn_version.split(".")] < [8, 0]:
+    #  cudnn_headers = ["cudnn.h"]
+    #else:
+    cudnn_headers = ["cudnn_adv_infer.h",
+                       "cudnn_backend.h",
                        "cudnn_adv_train.h",
                        "cudnn_cnn_infer.h",
                        "cudnn_cnn_train.h",

This will break with older cuDNNs, but turns out that because check fails, I really don't care. I only need this version of TensorFlow for the A100 gpus, which only support CUDA 11 and cuDNN 8+ anyway.


fix(bg/openPayments): redirect with right error on key-add failure

@raducristianpopa to make changes to /welcome screen first.
Part of https://github.com/interledger/web-monetization-extension/issues/613

Fixed in https://github.com/interledger/web-monetization-extension/pull/714


New App: Skordle Scores

Description
This is a new app that pulls information from Skordle.com about high school sporting events in the Oklahoma area. The Skordle Scores app pulls all the games and their data for the selected sport and class. The data includes team names, logos, status, scores, and dates/times. The app always pulls the games from the current week or day (may change later). It then displays which game (1/something) it's on, the two teams, status (scheduled, final, etc.), and their scores or scheduled date depending. A user can change the desired sport and or class in the mobile app, causing the app to refresh its data.
Copilot

🤖 Generated by Copilot at 2c613b4
Summary
📝🏈📦

Add a new app for displaying high school sports scores from Skordle. The app consists of a manifest file and a Python script that fetches and formats the data.

manifest file
Skordle Scores app metadata
autumn harvest time

Walkthrough

Add the manifest file for the Skordle Scores app (link)

I have read the CLA Document and I hereby sign the CLA


CUDA Setup failed despite GPU being available

System Info
===================================BUG REPORT===================================
/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
warn(msg)
/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/targets/x86_64-linux/lib/libcudart.so')}.. We select the PyTorch default libcudart.so, which is {torch.version.cuda},but this might missmatch with the CUDA version that is needed for bitsandbytes.To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variableFor example, if you want to use the CUDA version 122BNB_CUDA_VERSION=122 python ...OR set the environmental variable in your .bashrc: export BNB_CUDA_VERSION=122In the case of a manual override, make sure you set the LD_LIBRARY_PATH, e.g.export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2
warn(msg)
/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /opt/amazon/efa/lib64:/opt/amazon/openmpi/lib64:/opt/aws-ofi-nccl/lib:/usr/local/cuda/lib:/usr/local/cuda:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/targets/x86_64-linux/lib:/usr/local/lib:/usr/lib:/lib did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('() {  eval /usr/bin/modulecmd bash $*\n}')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary /root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/libbitsandbytes_cuda118.so...
libcusparse.so.11: cannot open shared object file: No such file or directory
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=118 make cuda11x
python setup.py install
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/opt/conda/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/opt/conda/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
from . import cuda_setup, utils, research
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 1, in
from . import nn
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/nn/init.py", line 1, in
from .modules import LinearFP8Mixed, LinearFP8Global
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in
from bitsandbytes.optim import GlobalOptimManager
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/optim/init.py", line 6, in
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "/root/kohya_ss/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 20, in
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following command to get more information:
    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

Reproduction
run train task in Kohya_ss on Amazon linux 2023
Expected behavior
Successful training

facing the same issue

I have the same issue when running on the: AWS Deep learning OSS Nvidia Driver AMI GPU PyTorch 2.0.1 (Ubuntu 20.04) 20240102 OS.
Is there a workaround?


The ellipsis in the function parameters of challenge intermediate/literalstring causes challenge failed

In challenge intermediate/literalstring, function execute_query has a parament accepted Iterable[str]，and set default value .... The same pattern is observed in the provided solution.
def execute_query(sql, parameters: Iterable[str] = ...):
    """No need to implement it"""

This will cause a type check error by pyright

error: Expression of type "ellipsis" cannot be assigned to parameter of type "Iterable[str]"

It seems like I must access type check by changing Iterable[str] to Iterable[str] | EllipsisType , like
from typing import Iterable, LiteralString
from types import EllipsisType

def execute_query(sql: LiteralString, parameters: Iterable[str] | EllipsisType = ...):
    """No need to implement it"""

or change ... to  () for type check.
from typing import Iterable, LiteralString
from types import EllipsisType

def execute_query(sql: LiteralString, parameters: Iterable[str] = ()):
    """No need to implement it"""

I don't think either of the above solutions to be the best way for this issue, but I don't know any better way to use ellipsis for default parameter with type hint.

May I ask which type checker are you using? Python-Type-Challenges uses pyright and it works as expected, see playground. It is normal that different type checkers behave differently here and there.
parameters: Iterable[str] = () works too, if you make a PR I'll accept it.

I'm using Pyright version 1.1.344, which you can reproduce in this playground.
I found a related discussion in this issue of pyright and the corresponding  release note of version 1.1.329.  The key change mentioned is:

Behavior Change: Added check for the use of an ellipsis for a default argument value for a function that is not in a stub, not overloaded, and without a placeholder implementation.

When an ellipsis is used in a function body, means not been implemented, Pyright treats the ellipsis as Any, for example:
from typing import LiteralString, Iterable
def execute_query(sql: LiteralString, parameters: Iterable[str] = ...):
     ...

will get:
> pyright --pythonversion=3.12  test.py
0 errors, 0 warnings, 0 informations

When using docstring, pass, and NotImplementedError,  Pyright will check the type of ellipsis used as a default parameter. This behavior causes the type check error.
from typing import LiteralString, Iterable
def execute_query(sql: LiteralString, parameters: Iterable[str] = ...):
    """Not implemented yet."""

>  pyright --pythonversion=3.12  test.py
test.py:3:67 - error: Expression of type "ellipsis" cannot be assigned to parameter of type "Iterable[str]"
    "ellipsis" is incompatible with protocol "Iterable[str]"
      "__iter__" is not present (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations

I think adding an ellipsis within the function execute_query's body of question file is a better way to fix this. The solution file can pass the type check (I made a mistake for this).

Thanks for digging into this. You're right about the current behavior (this comment describes it).
I'll make a change to question.py to use ellipsis in function body.


Feature/crmsh implementation

Update the way crm installation is checked.
Update LICENSE to apache (salt uses apache license)

I think the job died during the execution. I have restarted that. Let's see...


Unable to build dawn

Hey,
I'm trying to build dawn on iOS but unfortunately Im encoutering a build issue.
Steps:

Clone the repo
git submodule update --init
cd package && yarn
yarn build-dawn <- fails

Is there anything I missed?
The error:
🍏 arm64 iphoneos [264/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/escaping.cc.o
🍏 arm64 iphoneos [265/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/internal/damerau_levenshtein_distance.cc.o
FAILED: third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/internal/damerau_levenshtein_distance.cc.o
/usr/bin/g++  -I/Users/okwasniewski/workspace/react-native-webgpu/externals/dawn/third_party/abseil-cpp -O3 -DNDEBUG -std=c++17 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -fPIC -Wno-array-parameter -Wno-deprecated-builtins -Wno-unknown-warning-option -Wall -Wextra -Wc++98-compat-extra-semi -Wcast-qual -Wconversion -Wdead-code-aggressive -Wdeprecated-pragma -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wfor-loop-analysis -Wformat-security -Wgnu-redeclared-enum -Winfinite-recursion -Winvalid-constexpr -Wliteral-conversion -Wmissing-declarations -Woverlength-strings -Wpointer-arith -Wself-assign -Wshadow-all -Wshorten-64-to-32 -Wsign-conversion -Wstring-conversion -Wtautological-overlap-compare -Wtautological-unsigned-zero-compare -Wundef -Wuninitialized -Wunreachable-code -Wunused-comparison -Wunused-local-typedefs -Wunused-result -Wvla -Wwrite-strings -Wno-float-conversion -Wno-implicit-float-conversion -Wno-implicit-int-float-conversion -DNOMINMAX -MD -MT third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/internal/damerau_levenshtein_distance.cc.o -MF third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/internal/damerau_levenshtein_distance.cc.o.d -o third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/internal/damerau_levenshtein_distance.cc.o -c /Users/okwasniewski/workspace/react-native-webgpu/externals/dawn/third_party/abseil-cpp/absl/strings/internal/damerau_levenshtein_distance.cc
fatal error: error in backend: live register clobbered by inserted prologue instructions
clang++: error: clang frontend command failed with exit code 70 (use -v to see invocation)
Apple clang version 16.0.0 (clang-16<IP_ADDRESS>)
Target: arm64-apple-darwin24.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang++: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang++: note: diagnostic msg: /var/folders/nc/lg2x6s_x2tz9q0zxs01j1qfc0000gn/T/damerau_levenshtein_distance-92bc71.cpp
clang++: note: diagnostic msg: /var/folders/nc/lg2x6s_x2tz9q0zxs01j1qfc0000gn/T/damerau_levenshtein_distance-92bc71.sh
clang++: note: diagnostic msg: Crash backtrace is located in
clang++: note: diagnostic msg: /Users/okwasniewski/Library/Logs/DiagnosticReports/clang_<YYYY-MM-DD-HHMMSS>_<hostname>.crash
clang++: note: diagnostic msg: (choose the .crash file that corresponds to your crash)
clang++: note: diagnostic msg:

********************
🍏 arm64 iphoneos [266/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/numbers.cc.o
🍏 arm64 iphoneos [267/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/str_cat.cc.o
🍏 arm64 iphoneos [268/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/str_replace.cc.o
🍏 arm64 iphoneos [269/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/str_split.cc.o
🍏 arm64 iphoneos [270/903] Building CXX object third_party/abseil/absl/strings/CMakeFiles/absl_strings.dir/substitute.cc.o
🍏 arm64 iphoneos [271/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_civil_time.dir/internal/cctz/src/civil_time_detail.cc.o
🍏 arm64 iphoneos [272/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_time_zone.dir/internal/cctz/src/time_zone_fixed.cc.o
🍏 arm64 iphoneos [273/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_time_zone.dir/internal/cctz/src/time_zone_if.cc.o
🍏 arm64 iphoneos [274/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_time_zone.dir/internal/cctz/src/time_zone_impl.cc.o
🍏 arm64 iphoneos [275/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_time_zone.dir/internal/cctz/src/time_zone_format.cc.o
🍏 arm64 iphoneos [276/903] Building CXX object third_party/abseil/absl/time/CMakeFiles/absl_time_zone.dir/internal/cctz/src/time_zone_info.cc.o
🍏 arm64 iphoneos ninja: build stopped: subcommand failed.
/Users/okwasniewski/workspace/react-native-webgpu/package/scripts/build/util.ts:39
        reject(new Error(`${label} exited with code ${code}`));
               ^
Error: 🍏 arm64 iphoneos exited with code 1
    at ChildProcess.<anonymous> (/Users/okwasniewski/workspace/react-native-webgpu/package/scripts/build/util.ts:39:16)
    at ChildProcess.emit (node:events:519:28)
    at ChildProcess.emit (node:domain:488:12)
    at maybeClose (node:internal/child_process:1105:16)
    at Socket.<anonymous> (node:internal/child_process:457:11)
    at Socket.emit (node:events:519:28)
    at Socket.emit (node:domain:488:12)
    at Pipe.<anonymous> (node:net:338:12)

Are you using the latest XCode version tooling?

These steps are also executed automatically on our github actions: https://github.com/wcandillon/react-native-webgpu/blob/main/.github/workflows/build-dawn.yml so this might help as well.
And you can download the prebuilt binaries directly if you want to skip this step altogether: https://github.com/wcandillon/react-native-webgpu/actions/runs/10156718895
Maybe it would be fun to have a script in the example so if the prebuild binaries are not available, download them from Github. Maybe it's something you could setup/contribute.

If you replace the -O3 with -O2 on the command line, does that fix it? This is ultimately an Internal Compiler Error in Xcode’s clang, so the issue is there. That said, there is hopefully a workaround that doesn’t dramatically impact end-to-end performance.

@wcandillon Yeah, this would be really useful. I'll try to implement this once I get the build working locally :) For now I've created a draft PR adding visionOS support to see if maybe the CI will build it properly:  https://github.com/wcandillon/react-native-webgpu/pull/88

Are you using the latest XCode version tooling?

Im running Xcode 16 beta, but I've also tested out Xcode 15, no luck on both.

If you replace the -O3 with -O2 on the command line, does that fix it? This is ultimately an Internal Compiler Error in Xcode’s clang, so the issue is there. That said, there is hopefully a workaround that doesn’t dramatically impact end-to-end performance.

Looks like it's passed to some dependency of dawn but one of their build scripts, I'll try to check if but if its working for William then it must be something on my machine

Oh finally I've been able to run this on iOS. Dawn is now successfully building for iOS and visionOS but something is still wrong in the copy headers step.. I've took the cpp folder from the Github CI/CD and only then it worked! Im going to investigate this

I guess we can close this right?


[storage] Updated install instructions

Current install instructions install the track1 SDK.

Can one of the admins verify this patch?


ModuleNotFoundError: No module named 'copy_reg\r'

When I try to run the file recognize_video.py, I get an error:
Traceback (most recent call last):
File "C:/Users/%username%/Desktop/face-recognition-using-deep-learning/recognize_video.py", line 22, in
recognizer = pickle.loads(open("output/recognizer.pickle", "rb").read())
ModuleNotFoundError: No module named 'copy_reg\r'.
'copy_reg\r' - its a first line of text from 'recognizer.pickle' - file

Change this line:
22 in that file
recognizer = pickle.loads(open("output/recognizer", "rb").read())

I've changed the line 22 to
recognizer = pickle.loads(open("output/recognizer", "rb").read())
but it's not working.

It worked, thanks a ton!


How to associate an image with each Category tag?

Greetings.  I am trying to figure out a way to associate an image with each blog article Category tag. My ideas seem like sub-optimal workarounds.  Can anyone offer guidance?  We are using blog article categories as filters, and we need their associated images to be content managed.

Hi, this functionality doesn't exist within Umbraco or Articulate. Articulate just uses Umbraco tags, it doesn't do anything fancy with them. If you wanted to add an icon or other metadata to an Umbraco tag, you'll need to extend the database and create an editor for that.
For example, you could create a new DB table that is a 1:1 relation with the cmsTags table that adds columns for whatever metadata you want. You'll then need to create some sort of tags editor UI to fill in this data.


Where is gone the Firebase Admin SDK ?

Until yesterday, we could find the Firebase Admin SDK on Maven repository. We can still install it but the GitHub repo has disappeared.
I have an issue with the GSON dependency used by Firebase Admin SDK. It is using GSON version 2.6.2 which has an configuration issue. Please, update it to 2.7.0 or higher if possible.
Also, can I find some news about Firebase Admin Java SDK somewhere? I can't think Firebase team leaving its lovely community in the dark. 🕵

@jwngr what happened to that github repo?

Hey @LM450N, let's see if I can get you some answers:

Until yesterday, we could find the Firebase Admin SDK on Maven repository.

The link you shared in not actually the real Maven repository, it just a mirror of it. The link you want is this one.

We can still install it but the GitHub repo has disappeared.

That repo is a private repo which is why you cannot see it. How did you even get the URL for that in the first place?

I have an issue with the GSON dependency used by Firebase Admin SDK. It is using GSON version 2.6.2 which has an configuration issue. Please, update it to 2.7.0 or higher if possible.

I will look into upgrading it.

Also, can I find some news about Firebase Admin Java SDK somewhere? I can't think Firebase team leaving its lovely community in the dark.

The release notes are here, a docs guide starts here, the API reference is here, and releases are tweeted from @FirebaseRelease. I even wrote a blog post on the Admin SDKs just last week. Is there something else you'd like to see from us?

Also, I should note, the source code for the Admin Java SDK is not open source, which is why you cannot access that.

Thank you for your answers. It's perfect! 💯
Today I tried to use this SDK on App Engine and I don't really understand why it's throwing the "IllegalStateException: This feature is only available to backend instances."
I mean, why App Engine instance is not considered as a backend? I'm using it only to expose the datastore through a REST API. My purpose is get user information from Firebase on App Engine. This could be a solution but it allows only to verify the OAuth token.
There is a setting parameter to define or it's simply not possible? I guess it's not the place to discuss about it 😄 .

What version of the Admin Java SDK are you using? We released a fix today with the 4.0.2 release which added back in support for Java 7, which is what GAE Standard runs on. Can you try using the latest release and see if that works for you?

In fact I was using the "firebase-server-sdk" meanwhile the new library to be compatible with Java 7. I just now tried with the "firebase-admin-sdk" (4.0.2) and I got the same error. Here a part of the StackTrace.

Ahhh, I think I know what is going on. Are you by chance using GAE automatic scaling? Unfortunately, the Firebase Java SDK currently only supports manual scaling. You can read this Stack Overflow question for some more info.

Yes my App Engine app is automatically scaled. I've read threads talking about scaling but not detailed as this one. I'll try it asap. Please, as soon as the teleportation will be available, call me, I'll send you a 🍺 .
Thank you a lot for your help!

Cheers, glad we got everything figured out 🍻


Add "Toggle Names" functionality to epics page

https://github.com/jolewis-ddn/jira-status-reporter/blob/2412fa54e81265a091d9ea8128b70371c0266092/JiraStatusServer.js#L442

Fixed in Master now


Please list supported major versions near the top of the README

Description:
I want to know which Node versions I can test with. The only thing that looks like an answer to that question is this part of the README:

Examples:

Major versions: 14, 16, 18

and so I assumed for the last few months (until I saw 19 in someone else's Action) that these are the only versions that were supported, because trying to figure out from the code would take a few minutes (I don't know anything about how GitHub Actions are implemented) and trying a version would take even longer and would be a waste of time if it turned out to not be supported.

Hello @verhovsky!
As mentioned in the README, the action can download Node.js versions from the dist so you can specify any of the versions presented there:

The action will first check the local cache for a semver match. If unable to find a specific version in the cache, the action will attempt to download a version of Node.js. It will pull LTS versions from node-versions releases and on miss or failure will fall back to the previous behavior of downloading directly from node dist.

You also can specify canary, nightly and rc versions.
I am closing the issue, but feel free to contact us if you have any questions.


Release v0.23.12

If anyone has unmerged issues they really want in this release, please milestone them or mention them here.  I would like to crank through this today, if possible.

A few PRs are waiting for review for a relatively long-time and targeting the 0.23 series. I'd be happy if we consider them for inclusion in this release: https://github.com/http4s/http4s/pull/6204 https://github.com/http4s/http4s/pull/6316

Oh, shoot, I almost forgot, I thought more and I decided we should keep all the JS-only APIs private. PR forthcoming.

Btw, since these release notes are describing the schism so nicely, seems a shame not to include blaze on that list as well. We're only blocked by https://github.com/http4s/http4s/pull/6204 and https://github.com/http4s/http4s/pull/6387 and then we can schism it as well.

The retryKey, but I don't see any of the other changes mentioned. So I figure that is still incoming?

CI ran on 3966296 to prove that Laika can digest the duplicate headers.  I skipped it on the subsequent tweaks for expediency.


Image File Locations in New Google Sites Wiki

In the design journal entry for 2024-03-21, I found that the image files for the New Google Sites version are all stored outside of my Wiki.
I need to decide whether to leave there or import them into the Git Repository.

Only three (3) pages were affected. These were converted manually.


Wrong mixedlayer on covered=colonnade + tunnel=building_passage

This is wrong "fix": http://www.openstreetmap.org/changeset/27025584
Pathways are on the same layer (photo proof: http://mosday.ru/photos/61_184.jpg ), so they should not be disconnected.
Such cases should not be fixed automatically.

This changeset was reverted:

Close  this ticket here!

@OverQuantum thanks for reporting. The data comes from http://keepright.at and we have no way of filtering based on tags.


Fast math flags for LLVM backend

This PR adds support for fast math flags in LLVM backend. Currently, the user can specify them via command-line (this approach was chosen for easier benchmarking). The specified flags are named exactly  the same as in LLVM.
Example:
// fma.mod
FUNCTION fma(a, b, c) {
    fma = (a * b) + c
}

$ ./nmodl fma.mod --verbose debug llvm --ir --fmf nnan contract afn --opt

define double @fma(double %a, double %b, double %c) {
  %1 = fmul nnan contract afn double %a, %b
  %2 = fadd nnan contract afn double %1, %c
  ret double %2
}

This is useful to enable previously unsafe FP-math optimizations. For example, fused-multiply-add instructions can now be generated when lowering LLVM IR to assembly/executing it via JIT.
For specification of FMF see LLVM LangRef: https://llvm.org/docs/LangRef.html#fastmath

I can confirm that I see the fma instructions being generated in the "compute-bound" kernel


Color Mode Feature - Add ColorModeScript Next.js of TypeScript version

Subject
Color Mode
Description
In chakra-ui source code has the ColorModeScript of Next.js in TypeScript version.
The docs don't have, which cause confusion because theme.ts already had .js and .ts version + People are using TypeScript more and more nowadays.
export default class Document extends NextDocument {
  static getInitialProps(ctx: DocumentContext) {
    return NextDocument.getInitialProps(ctx)
  }

  render() {
    return (
      <Html>
        <Head />
        <body>
          <ColorModeScript />
          <Main />
          <NextScript />
        </body>
      </Html>
    )
  }
}

Hi @tannguyencse19. Recently we've made a docsite restructure and made sure to add this in the Next.js guide here: https://chakra-ui.com/guides/getting-started/nextjs-guide#color-mode-script. Thank you for opening this issue.


Dependency with redis stack server

Configuration
To use this package, you need to have a Redis instance running. You can configure the connection by setting the following environment variable:

export REDIS_URL="redis://username:password@localhost:6379"

In this configuration are we expecting redis or redis stack server to be run at localhost?

This integration should work as long as there's search and JSON support which comes preconfigured in Stack and Stack Server.


All Kafka CI tests failing due to proxymap error "Unable to lookup original destination"

Note that this is causing basic Kafka to fail. Hence marked as a 1.0 blocker.
root@k8s1:~# cilium bpf proxy list | grep <IP_ADDRESS> | grep 14107 | wc -l
810
root@k8s1:~# cilium bpf proxy list | grep <IP_ADDRESS> | grep 14107 | head
<IP_ADDRESS>:57898 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2569
<IP_ADDRESS>:59186 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2611
<IP_ADDRESS>:57956 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2571
<IP_ADDRESS>:58256 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2581
<IP_ADDRESS>:59180 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2611
<IP_ADDRESS>:57972 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2572
<IP_ADDRESS>:59170 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2610
<IP_ADDRESS>:58590 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2592
<IP_ADDRESS>:58604 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2592
<IP_ADDRESS>:58122 (6) => 14107   <IP_ADDRESS>:9092 identity 32541 lifetime 2576

If you see the output of the entries, when a new produce message happens on L7 the number increase in k8s1
BeforeAnyTraffic: Pod "cilium-2gnpq" entries in the proxy 1
BeforeAnyTraffic: Pod "cilium-dcb6x" entries in the proxy 1
---------------------------------------------------
---------------------------------------------------
TrafficWithoutL7: Pod "cilium-2gnpq" entries in the proxy 1
TrafficWithoutL7: Pod "cilium-dcb6x" entries in the proxy 1
---------------------------------------------------
AfterAppliedL7: Pod "cilium-2gnpq" entries in the proxy 1
AfterAppliedL7: Pod "cilium-dcb6x" entries in the proxy 1
---------------------------------------------------
---------------------------------------------------
AfterL7Traffic: Pod "cilium-2gnpq" entries in the proxy 789
AfterL7Traffic: Pod "cilium-dcb6x" entries in the proxy 1

https://gist.github.com/eloycoto/4f234ae22876870e33151e30ab536b2e
Also Eloy tested this on Kafka tests(runtime as well as k8s) and it was failing.
Envoy is working fine.

When this error occurs, a Proxy4Map.Dump returns an empty map.

The proxy map size was different between BPF (524288) and userspace (:
$ grep -r 'define.*PROXY_MAP_SIZE' bpf
bpf/node_config.h:#define PROXY_MAP_SIZE 524288

var (
	// Proxy4Map represents the BPF map for IPv4 proxy
	Proxy4Map = bpf.NewMap(Proxy4MapName,
		...,
		...,
		...,
		8192,
		...,
		...).WithNonPersistent()
)

After setting the same value in userspace, the failure doesn't occur anymore.


Typo on string #07

Thank you! I've taken the opportunity to fix the corresponding typo in the sister project.


feat(api): Environment discovery

Introduces tanka.ListEnvs for discovering all nested environments starting from any given directory on the filesystem.
This is now implemented by concurrently testing each subfolder of the given directory as to whether it is a valid environment. The recursion stops if any directory is a valid environment.
Switches all occurences of FindBaseDirs to the more performant ListEnvs algorithm.
A quick tk env list on our config repo gave the following:
Before: tk env list  25.40s user 0.69s system 946% cpu 2.757 total
After:  tk env list  1.61s  user 0.37s system 811% cpu 0.243 total

Needs rebasing on #467 once merged

#467 is approved, please rebase this PR so we can review the changes properly.

#467 is approved, please rebase this PR so we can review the changes properly.


please add libpq-dev and sqlite_devel to /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-dev-sl7:latest

DUNE is requesting to add postgresql-devel and sqlite-devel to the development container in INC000001172471.

Installed. Even if libpq-devel is not found in the RPM list ....

The actual package needed was postgresql-devel that provides /usr/include/libpq-fe.h.
This package is already included in the container.


Query message get wrong result when tls enabled

The tls is enabled in my rocketmq cluster.
The QueryMsgTraceById throw a MQClientException(Code: 208, DESC: query message by key finished, but no message), in fact, the message I queried has a sub trace and a pub trace.
I notice that the QueryMessageTransfer calcute the transferred bytes depends on the return value of target.write method, but the target object defined in FileRegionEncoder return the wrong value, ByteBuffer's capacity is always greater than it's readable bytes, which may cause some bytes in QueryMessageTransfer not read to the ByteBuf in FileRegionEncoder, so super.decode in NettyDecoder get the wrong bytes.

I add some logs to describe the bug
RocketMQ version: 4.9.3
The message I QueryMsgTraceById has a sub trace and a pub trace
QueryMessageProcessor

QueryMessageTransfer

FileRegionEncoder

NettyDecoder

Console log:

The [QueryMessageProcessor] total bytes: 1021 indicate that 1021 bytes should be transferred, and [QueryMessageProcessor] message count: 2 indicate that 2 messages was found. [QueryMessageTransfer] header total bytes: 1017 1017 is the length of data used by LengthFieldBasedFrameDecoder, in QueryMessageTransfer, the header is 193 bytes but 256 transferred, and the first message is 378 bytes but 1024 transferred, 1280(capacity 256 + capacity 1024) is greater than 1021, so the 2nd message will not be transferred, in actual, 571(header 193 + 1st message 378) bytes is transferred,  but LengthFieldBasedFrameDecoder should get 1021 bytes
NettyDecoder log

NettyDecoder get 571 bytes, and not more byte transferred.


floorp: 11.14.1 -> 11.15.0

Description of changes
Release notes: https://blog.ablaze.one/4397/2024-07-21/
Git changelog: https://github.com/Floorp-Projects/Floorp/compare/v11.14.1...v11.15.0
Fixes #325785.
Regarding #312383 and their licensing shenigans, I've checked it everything looks fine to me that it really is licensed under MPL-2.0 and we do not need to do anything.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4231

Result of nixpkgs-review pr 326101 run on x86_64-linux 1

2 packages built:

floorp-unwrapped
floorp-unwrapped.debug


Changes for v1.1

Follow-up from #1 these are the changes I want to make.

Add component values to silkscreen, easier to assemble.

Add through-holes to the Base board to Radio board interface. Castelated isn't working great on its own.  Adding through-holes that line up to allow someone to put a wire or pin through would be better.

Add outline on Base board for the Shim board, partially to help the user place it, but also to make sure we designers remember not to put anything there.  Maybe even a keep-out zone.

Swap M3x10mm with M3x12mm.  Adding the board thickness to the stack up leaves very few threads for the stand-off to grab.

The hardware here is the same as used for the User Adapter.  The 12mm will work in the User adapter too, even though it's not needed.  The advantage is that we can make a single pile of M3 hardware packets and use them for both the User and Radio adapter kits.

GX16 connector riser boards are taller than the brass standoffs. They cause the top board to not lay down flat.  Push the GX16 out past the edge of the top board.  Or find some better way to address this.  (A slot in the top board to hold the riser board?)

Does it make sense to revisit the over-all layout, do away with the "hand held" shape I developed for the user kits?  Maybe something more square?  Better suited to mounting next to a radio?  Maybe even with an optional metal case?

I'm not sure how difficult this would be, but with how much clearance is underneath the GX16 it might be possible to lower it down and get away with using the existing standoffs. That might make connecting it to the radio board a little trickier though, since going down much further would put GX16 pin 4 very close to the radio board. Since the connection from pin 4 to the vertical slot is straight down anyway, that might not be a big issue...

As for the overall layout, I think it'd be interesting to do a version with a female GX16 socket and design the whole adapter to sit almost flush with the radio itself, similar to the PowerPole conversion blocks I've seen for the Yaesu FT-891 and other radios. I know the obvious issue here is that every radio has their mic port in a different place, but fewer cables means a neater setup and fewer things to forget when going portable or helping with an event.

I'm not sure how difficult this would be, but with how much clearance is underneath the GX16 it might be possible to lower it down and get away with using the existing standoffs. That might make connecting it to the radio board a little trickier though, since going down much further would put GX16 pin 4 very close to the radio board. Since the connection from pin 4 to the vertical slot is straight down anyway, that might not be a big issue...

The vertical space is the concern. I lowered the GX16 as much as was convenient at the time not thinking about this problem. I'll spend some time lowering it as much as possible instead of as convenient. See what I can do.

As for the overall layout, I think it'd be interesting to do a version with a female GX16 socket and design the whole adapter to sit almost flush with the radio itself, similar to the PowerPole conversion blocks I've seen for the Yaesu FT-891 and other radios. I know the obvious issue here is that every radio has their mic port in a different place, but fewer cables means a neater setup and fewer things to forget when going portable or helping with an event.

That is an interesting idea. I like it. My immediate questions are: I don't think PCB mount GX16s are common, or possibly available at all. I don't REMEMBER seeing them, but I also wasn't looking for them so maybe they are. Have you looked?
I'd also worry about the headphone/speaker connection. Physical spacing between the GX16 and 3.5mm won't be consistent across radios, so the 3.5mm will still need to be a cable for radios that don't provide audio out through the mic port.
But if we can find PCB mount female GX16, that would work pretty well for Icom radios (which do provide receive audio on the GX16.)

Icom GX16 Radio Adapter notes
Tested with an Icom IC7300 with a Heil Pro Set Elite-Ic (electret) headset.

No issues with PTT

Audio is brought in through the GX16, but without a cable plugged into the PHONES port on the front of an ICOM, the IC7300 speaker will not disable.

Audio coming in through the 3.5mm audio jack is attenuated relative to the audio from the GX16 port (with no audio cable plugged into the GX16 adapter board.

I think the R6/R8 resistors are only needed if using GX16 audio? The IC7300 PHONE board has the 100ohm resistors inside the IC7300.

I'm confused a bit by the layout. Some install notes could be handy. This layout for the radio base board might need to be reviewed.

Standoff notes

Radio adapter PCB screws with the hex standoff spacing.  I replaced the M3x10 mm screw with an M3x12 mm screw and added an M3 nut to adjust the offset a little bit.  Suggest some PCB spacers.
The spacers are overall too short. I used 20mm long M3 hex spacers. 18mm would probably work well.

I did not think to remove the nut from the connector.  Suggest notes to do so.
Schematic shows red X over R7 and R5, but the PCB has those two ref des on the silk screen.  Suggest moving the rest X or change the ref des

Icom GX16 Radio Adapter notes Tested with an Icom IC7300 with a Heil Pro Set Elite-Ic (electret) headset.
* No issues with PTT

* Audio is brought in through the GX16, but without a cable plugged into the PHONES port on the front of an ICOM, the IC7300 speaker will not disable.

* Audio coming in through the 3.5mm audio jack is attenuated relative to the audio from the GX16 port (with no audio cable plugged into the GX16 adapter board.

  * I think the R6/R8 resistors are only needed if using GX16 audio? The IC7300 PHONE board has the 100ohm resistors inside the IC7300.

I'm confused a bit by the layout. Some install notes could be handy. This layout for the radio base board might need to be reviewed.

* Standoff notes

  * Radio adapter PCB screws with the hex standoff spacing.  I replaced the M3x10 mm screw with an M3x12 mm screw and added an M3 nut to adjust the offset a little bit.  Suggest some PCB spacers.
  * The spacers are overall too short. I used 20mm long M3 hex spacers. 18mm would probably work well.

* I did not think to remove the nut from the connector.  Suggest notes to do so.

* Schematic shows red X over R7 and R5, but the PCB has those two ref des on the silk screen.  Suggest moving the rest X or change the ref des
  ![image](https://private-user-images.githubusercontent.com/25909885/396332523-2b3d4dd1-b99e-4093-8cdd-9927ae2d4635.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzQ0MDQ0NzcsIm5iZiI6MTczNDQwNDE3NywicGF0aCI6Ii8yNTkwOTg4NS8zOTYzMzI1MjMtMmIzZDRkZDEtYjk5ZS00MDkzLThjZGQtOTkyN2FlMmQ0NjM1LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDEyMTclMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQxMjE3VDAyNTYxN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTNhNDgwYWQ2OGZlOWVmYjE2NjViZmU3NmEwZWU4YjI0YzU1NDM3NWZmM2FmZTNhNzY2ZDljMWRjMTllMTEwODAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.egFFU2nr2yB9rjKh_kxhzWqHTmoNfrMEwE312c1XH4E)

Maybe an Icom specific configured board should have some of the following notes:

Do not install: C1, C2, RV2, R4
Install jumper wire on J3 from pin 2 to pin 4
Unsure about R7 & R5 on headphones

Audio is brought in through the GX16, but without a cable plugged into the PHONES port on the front of an ICOM, the IC7300 speaker will not disable.

This is a known issue.  I'm not sure whether the built-in speaker can be disabled in software while still allowing audio out the GX16.  It might require a dummy-plug in the headphones port or something.

Audio coming in through the 3.5mm audio jack is attenuated relative to the audio from the GX16 port (with no audio cable plugged into the GX16 adapter board. I think the R6/R8 resistors are only needed if using GX16 audio? The IC7300 PHONE board has the 100ohm resistors inside the IC7300.

The 100R resistors are bypassable.  That's what the jumpers on J6, pins 1-2, 3-4.  They short across the 100R resistors.  You need the 100R if you're feeding from a speaker output, and don't need them if feeding from a headphone output.  So putting jumpers on that side of the header shorts out the 100R series resistors.
Does that change the audio levels for you?

I'm confused a bit by the layout. Some install notes could be handy. This layout for the radio base board might need to be reviewed.

And how...  Yes, this needs assembly instructions, which I haven't written yet.  I probably should have thrown together some notes before sending the hardware to y'all.
I'm all for changes to the board layout if it'll make it easier to understand what's going on.  Any specific suggestions? (other than what you've already presented here, that is.)

Radio adapter PCB screws with the hex standoff spacing. I replaced the M3x10 mm screw with an M3x12 mm screw and added an M3 nut to adjust the offset a little bit. Suggest some PCB spacers.

M3x12mm: I came to that conclusion too.  I've updated the BOM.
Spacer: Already done, and almost ready to ship to me.

These are the same thickness as the radio-specific board and go on the other stand-offs to even them out.  Yes, I made sure the dog-bone would fit. :-)

The spacers are overall too short. I used 20mm long M3 hex spacers. 18mm would probably work well.

Yeah, the stand-offs were sized for RJ45s; I didn't think about the GX16 being taller.

Schematic shows red X over R7 and R5, but the PCB has those two ref des on the silk screen. Suggest moving the rest X or change the ref des

Good catch. That was the intent from the beginning, I just missed these.  Thanks.  Fixed.

Maybe an Icom specific configured board should have some of the following notes:
Do not install: C1, C2, RV2, R4
Install jumper wire on J3 from pin 2 to pin 4

That's exactly what the jumper header does.  Note the silkscreen on the board shows you the orientation to install a jumper to bypass the mic attenuation circuit.

Your jumpers are installed correctly (horizontally).  Does adjusting RV1 make a difference to your mic audio level?  Does removing those components change anything?
Or, are you just saying because they will NEVER be needed, don't bother populating them and just hard jumper the board instead?
That's actually a compelling argument:  Since the radio-specific board is soldered in place, any given kit will only ever work in one radio configuration.  Having it be reconfigurable doesn't make as much sense as it does on the User Adapter.

Bah. @gx1400 ^^  Forgot to mention you. Sorry.

Audio is brought in through the GX16, but without a cable plugged into the PHONES port on the front of an ICOM, the IC7300 speaker will not disable.

This is a known issue. I'm not sure whether the built-in speaker can be disabled in software while still allowing audio out the GX16. It might require a dummy-plug in the headphones port or something.

I feel like maybe those 100ohm resistors should live on the GX16 add-on board, then get disconnected (to use the ones that are inside of the IC7300) when an audio cable is inserted.
I think for the ICOM IC7300, I'd recommend cutting the trace (or putting a ohm jumper pad there) so that it needs to the headphone audio cable for audio.

Audio coming in through the 3.5mm audio jack is attenuated relative to the audio from the GX16 port (with no audio cable plugged into the GX16 adapter board. I think the R6/R8 resistors are only needed if using GX16 audio? The IC7300 PHONE board has the 100ohm resistors inside the IC7300.

The 100R resistors are bypassable. That's what the jumpers on J6, pins 1-2, 3-4. They short across the 100R resistors. You need the 100R if you're feeding from a speaker output, and don't need them if feeding from a headphone output. So putting jumpers on that side of the header shorts out the 100R series resistors.
Does that change the audio levels for you?

I finally figured that out last night.  It's just some clever layout that's hard to follow.
It does change the audio level, But maybe it's the 100ohm resistors in the IC7300 vs the lack thereof over the GX16 audio path that's throwing me for a loop.

Spacer: Already done, and almost ready to ship to me.
These are the same thickness as the radio-specific board and go on the other stand-offs to even them out. Yes, I made sure the dog-bone would fit. :-)

Looks nice!

Maybe an Icom specific configured board should have some of the following notes:
Do not install: C1, C2, RV2, R4
Install jumper wire on J3 from pin 2 to pin 4

That's exactly what the jumper header does. Note the silkscreen on the board shows you the orientation to install a jumper to bypass the mic attenuation circuit.

Your jumpers are installed correctly (horizontally). Does adjusting RV1 make a difference to your mic audio level? Does removing those components change anything?
Or, are you just saying because they will NEVER be needed, don't bother populating them and just hard jumper the board instead?
That's actually a compelling argument: Since the radio-specific board is soldered in place, any given kit will only ever work in one radio configuration. Having it be reconfigurable doesn't make as much sense as it does on the User Adapter.

I think RV1 circuit is totally by-passed in the Electret configuration?  I think your last statement is the takeaway.  If the boards are soldered for specific radio configurations, maybe all of the user configurable jumpers would add confusion.
I think the README for any specific radio or family of radios could tell you (and show you) which jumpers and components to install where.
Removing the right angle connectors and pin jumpers might help bring down bom cost a bit too

I feel like maybe those 100ohm resistors should live on the GX16 add-on board, then get disconnected (to use the ones that are inside of the IC7300) when an audio cable is inserted.
If not, I think for the ICOM IC7300, I'd recommend cutting the trace (or putting a ohm jumper pad there) so that it needs to the headphone audio cable for audio.

I remembered now.  I'm not sure that all Icom radios put out audio on GX16.  I know its common with modern gear, but I don't know how universal that has been over the years.  Similarly, I'm pretty sure that not all radios have Headphones output, some might only have Speaker.  The 100R are required there.
So the question isn't: What's needed to support THIS PARTICULAR ICOM, but what's needed to support ALL Icoms?  I don't think it's a good idea to limit the options.

I finally figured that out last night. It's just some clever layout that's hard to follow.

Yeah, it took me a bit to figure it out again today.  I could draw that better, that's for sure.

It does change the audio level, But maybe it's the 100ohm resistors in the IC7300 vs the lack thereof over the GX16 audio path that's throwing me for a loop.

Your jumpers are installed correctly (horizontally). Does adjusting RV1 make a difference to your mic audio level? Does removing those components change anything?
Or, are you just saying because they will NEVER be needed, don't bother populating them and just hard jumper the board instead?
That's actually a compelling argument: Since the radio-specific board is soldered in place, any given kit will only ever work in one radio configuration. Having it be reconfigurable doesn't make as much sense as it does on the User Adapter.

I think RV1 circuit is totally by-passed in the Electret configuration? I think your last statement is the takeaway. If the boards are soldered for specific radio configurations, maybe all of the user configurable jumpers would add confusion.

I think the right answer here is to keep the header outline on the board, but document how to assemble it hard wired for a given config so the jumpers aren't needed.  And say things like: "If you configure for electret, you're bypassing this whole circuit so you can just leave those parts off the board if you want."
But I do want to keep the configuration options available.

I think the README for any specific radio or family of radios could tell you (and show you) which jumpers and components to install where.

Agreed.  The documentation on these prototypes I sent you was ... well, "lacking" is too kind a word.  "Entirely non-existent" is more accurate.

Removing the right angle connectors and pin jumpers might help bring down bom cost a bit too

They are a tiny fraction of the BOM cost.  I'd keep them in the kit, but document how to not use them if you don't want to.

For Yaesu RJ45 kits - the kits you shipped have one RJ45 with LED leads, the other without. If plan it to sell kits with one of each, be sure to emphasize that the two RJ45s are different and that the LED should be noted and installed in the correct location


Using Cucumber 7 adapter with TestNG shows gives wrong number of tests in Report

Hi,
I have a Cucumber-TestNG-Maven-Java project in which I used cucumber7 adapter to generate the extent reports. The number of Scenarios that the report is capturing is 1 more than the actual number of scenarios.
This is my Runner file setup.

This is the Feature file.

Cucumber Version: 7.2.3
TestNG Version: 7.4.0
Cucumber Adapter Version: 1.7.0
Below is the report. As can be seen report has captured the Scenario Outline twice although the run is Once.

Use the latest version in maven - 1.9.2. This will fix the issue in the tag details.
The test page displays the feature, scenario outline, scenario, step hierarchy.  The scenario outline name is reused for the scenario name by default. To differentiate between these have a look at this example feature file - https://github.com/grasshopper7/cuke7-extent-adapter-report/blob/ac7c232d01bcc6a60b01507cb0affbaa9caf708e/cuke7-extent-adapter-report/src/test/resources/stepdefs/scenario%26outline.feature#L8

have the same problem when using tags for a cucumber -junit gradle project .I am using the latest version but still seeing the issue.

Are you referring to the tags issue - plus 1 for scenario outline? What version? Gradle or maven should not make any difference.
Plz add a screenshot. thx

Hi @grasshopper7 -unfortunately working on office project ,adding screenshot might be difficult but the issue is similar as mentioned above in the screenshot.
Suppose i have a scenario and i put 2 tags @a @b on it.
Now , in runner file i pass only @a in tags and run it ,the output html/pdf report shows 2 passed TC - one with @a and another with @b while only @ is run .
Plus in html report the passed percentage is shown as 0%.
Will try to provide screenshots by setting up project in my personal machine ,till then i hope i was able to provide some information

Cucumber tag filtering for scenarios is based on the criterion that the tag is present for that scenario. It is like 'contains' rather than 'only has'. The report will output the data that it receives from cucumber and not perform any automatic filtering.
For the 0 pass percentage I will need more details of the feature files.

so this behavior is like this since both tags are present in cucumber.json which is although weird
.
for 0 percentage i could not replicate it in my local machine .not sure what is happening in the work project and why the behaviour is like that

The behavior of the tag filtering by cucumber is reflected correctly in the report.


changes in contest page

added few new images to contest page

@rahul-n18 can you provide a screenshot of the changes you made?

Below are the changed pics

Good work @rahul-n18!

Thank you,
One thing i just needed to ask you, I am very new to this Hacktoberfest, so as now you have merged my request will I get a badge?


azcopy(V10.17.0/V10.18.0) sync crash when there are some read only files in destination folder.

Which version of the AzCopy was used?
V10.17.0/V10.18.0
Which platform are you using? (ex: Windows, Mac, Linux)
Alpine/Windows
What command did you run?
Job-Command sync https://xxxxxxxx.file.core.windows.net/xxx?se=2023-04-11t05%3A26%3A45z&sharesnapshot=2023-04-10t05%3A26%3A45.0000000z&sig=-REDACTED-&sp=rl&sr=s&sv=2019-07-07 https://xxxxxx.file.core.windows.net/anp?se=2023-04-11t05%3A26%3A46z&sig=-REDACTED-&sp=rcwdl&sr=s&sv=2019-07-07 --preserve-smb-info=true --preserve-smb-permissions=true --recursive=true --log-level=Error --delete-destination=true
What problem was encountered?
azcopy crashed with below logs:
Job 62eea6df-8db2-ec46-5303-fa7ea26970dc has started
Log file is located at: d:\azcopy_logs\62eea6df-8db2-ec46-5303-fa7ea26970dc.log
INFO: Deleting extra object: xxxxxxx.zip
INFO: error -> github.com/Azure/azure-storage-file-go/azfile.newStorageError, /home/vsts/go/pkg/mod/github.com/!azure/azure-storage-file-go@v0.6.1-0.20201111053559-3c1754dc00a5/azfile/zc_storage_error.go:42
===== RESPONSE ERROR (ServiceCode=ReadOnlyAttribute) =====
Description=The specified resource is read-only and cannot be modified at this time.
RequestId:217fa45c-901a-0081-4c2a-6c5759000000
How can we reproduce the problem in the simplest way?
mark a file as read-only and start sync job with delete-detination=true
Have you found a mitigation/solution?
move back to azcopy V10.16.1

Hi @wingfeng, can you post the backtrace for the crash?

sorry. Can you told me how to get "backtrace" ?
I have post all message what I can get from screen.
and here is the 62eea6df-8db2-ec46-5303-fa7ea26970dc.log contents:
2023/04/11 03:56:05 AzcopyVersion  10.18.0
2023/04/11 03:56:05 OS-Environment  windows
2023/04/11 03:56:05 OS-Architecture  amd64
2023/04/11 03:56:05 Log times are in UTC. Local time is 11 Apr 2023 11:56:05

it did not closed as wanted.

If azcopy crashed, it would leave a statement on the cli, with the trace of where it failed. Can you post that? Similar to one below

goroutine 7041 [running]:
github.com/Azure/azure-storage-azcopy/v10/ste.(*jpptFolderTracker).ShouldSetProperties(0xc001d640a0, {0xc0017d9f10, 0x69}, 0x2, {0x100c8aa80, 0xc000972000})
	/Users/runner/work/1/s/ste/folderCreationTracker.go:126 +0x2ff
github.com/Azure/azure-storage-azcopy/v10/ste.(*blobFolderSender).EnsureFolderExists(0xc001512400)
	/Users/runner/work/1/s/ste/sender-blobFolders.go:124 +0x2ba
github.com/Azure/azure-storage-azcopy/v10/ste.anyToRemote_folder({_, _}, {{0xa9cefaa2, 0xaf0c, 0xba4c, {0x70, 0x68, 0xa4, 0xdf, 0x43, ...}}, ...}, ...)
----
created by github.com/Azure/azure-storage-azcopy/v10/ste.NewJobMgr
	/Users/runner/work/1/s/ste/mgr-JobMgr.go:202 +0xc37

here is the screenshot of azcopy running cli. nothing left before I run next command.

Hi @wingfeng
The latest version of AzCopy supports a flag named --force-if-read-only that can be used for situations like this.
Could you try upgrading, use the new flag and let us know if it resolves your issue?

Closing due to inactivity in the thread. Feel free to reopen if you are still experiencing the issue after trying out the above suggestion.


Move some unused code to a new unused.cpp file

There is lots of code in StrobeAlign that is unused. It is either commented out or it exists as a function that is never called. Since I would like to focus on the relevant code when working on the program, I would like to somehow get rid of it. I usually just delete old code, knowing that I can find it again in the Git history if the need arises.
However, I can see that you may be reluctant to do that as it appears that you sometimes run experiments by quickly commenting/uncommenting a function call (for example, to switch out the hash function). So to still have the code somewhere more easily accessible, I suggest to move it to a file "unused.cpp" for the moment. This PR adds that file and moves some unused functions from index.cpp into it. Additionally, this adds an unused-code target to CMake, which is only there to ensure that the code in unused.cpp is compiled when running "make". This ensures that the functions are runnable should they be needed.

I’ve gone through aln.cpp as well and moved everything I found that was unused over to unused.cpp as well. With this, aln.cpp is now down to 3000 lines (from 4000 lines when I started a couple of weeks ago).


dotnet test does not detect tests using Test SDK 16.9.1 and .Net Core 3.1

Steps to reproduce:
I created a new NUnit project with Visual Studio and .net core 3.1. It contained the following dependencies:
Microsoft.NET.Test.Sdk (16.9.1)
NUnit (3.13.0)
NUnit3TestAdapter

The project is created with a simple empty test, which I can see in the test explorer and execute through it.
However, I have CLI which uses dotnet test to run these tests. When I run the following command, i get no tests:
dotnet test -t MyFirstTestProject.dll
Downgrading Test Sdk to 16.8.3 solves this issue.
AB#1287840

@Haplois do we need to wait for .NET 6 release for a proper fix?

No, We will fix this with our next release.


Loss of Precision when Serializing floats

Hi,
When using this code:
Console.WriteLine(JSON.Serialize(100921.688f));

I am getting this output:

100921.7

I would like to preserve the precision but it seems like the number is being rounded. This behaviour is explained here.
I can see this is happening in this method of ThunkWriter.cs:
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public void Write(float f)
    {
        Write(f.ToString(CultureInfo.InvariantCulture));
    }

In fact simply doing 100921.688f.ToString() gives the same result shown earlier.
I've also tried serializing the same thing with JSON .NET and it does not have this problem.

I expect the same issue.
I want to switch from JSON.NET to Jil.JSON (faster in installation of my cache).
But it crash while storing double.MinValue and double.MaxValue.
Due to this loss of precision.

This should be ok:
value.ToString("R", CultureInfo.InvariantCulture)

@dandago I created a PR for this issue. Feel free to use my fork if you need this fix.

@Jerome2606 thanks for looking into this.
It has been over a month since I opened this issue, which seems like a stretch for something so small. Is Jil still maintained at all?

It seems no, and that's so weird.
I'm using Jil.JSON and Newtonsoft in my production environment.
Jil.JSON is so much faster for bulk operation (cache initialization) while Newtonsoft is faster for small serialization/deserialization
@kevin-montrose Can you check on the pull request or give the right to anybody to maintain this project ?


cicd: improvements

:tada: This PR is included in version 2.16.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Datastore emulator orphans child process when run in background

I noticed this when running tox -e datastore-emulator in gcloud-python.
$ date "+%H:%M:%S"
11:47:18
$ ps
  PID TTY          TIME CMD
25498 pts/9    00:00:05 bash
26411 pts/9    00:00:00 ps
$ gcloud beta emulators datastore start &> log.txt &
[1] 26433
$ date "+%H:%M:%S"
11:47:29
$ ps
  PID TTY          TIME CMD
25498 pts/9    00:00:05 bash
26433 pts/9    00:00:00 gcloud
26442 pts/9    00:00:00 python2
26472 pts/9    00:00:00 bash
26474 pts/9    00:00:00 java
26486 pts/9    00:00:02 java
26515 pts/9    00:00:00 ps
$ date "+%H:%M:%S"
11:47:39
$ kill 26433
[1]+  Terminated              gcloud beta emulators datastore start &> log.txt
$ date "+%H:%M:%S"
11:47:49
$ ps
  PID TTY          TIME CMD
25498 pts/9    00:00:05 bash
26442 pts/9    00:00:00 python2
26472 pts/9    00:00:00 bash
26474 pts/9    00:00:00 java
26486 pts/9    00:00:02 java
26551 pts/9    00:00:00 ps

/cc @jgeewax @tseaver

@vilasj I think this issue applies to any emulator in the gcloud emulator group that is started as a subprocess.

I can confirm it also holds for pubsub. I'm addressing for now by manually corralling all the child processes via psutil
proc = psutil.Process(pid)
for child_proc in proc.children(recursive=True):
    try:
        child_proc.kill()
        child_proc.terminate()
    except psutil.NoSuchProcess:
        pass
proc.terminate()
proc.kill()

Note that the top level parent process here is gcloud.sh, which starts a python process that executes gcloud.py, which starts a bash process that executes gcd.sh, which starts a java process which is the actual emulator!
When you send SIGTERM to the background parent process, it just kills itself. It does not propagate the SIGTERM to its children. That is why you still see the child processes running.

 I had a PR out to add Python code that manually kills the child processes and have updated it with the recommendation to use kill -- -${PID}.
This helps the users of gcloud-python that manage to read our CONTRIBUTING, but doesn't do much for other users of the emulators.
Do you expect a typical user to know they need to clean up all children after
gcloud beta emulators foo start &

This is not specific to gcloud beta emulators. It happens when you background any long running process that starts sub-processes.
Perhaps we can put something in the help text for the command about running the emulators in the background...

Yes I realize (did I give an indication that I thought child processes were specific to gcloud beta emulators?).
But I'm not concerned about me, I'm concerned about other users.
Why would it be clear to the average user that the process spawns 5 more processes? The fact that start has no option to be background-ed makes users do it manually and in turn makes them orphan the subprocesses they didn't know about.

Of course, this is not about you or me, it is about our users. Did you read the second sentence of my comment?
I agree that it may not be clear to users that start creates sub-processes. That is why I have filed a bug to add help text to clarify this.
The alternative is to run all emulators within the same process. This is not a reasonable alternative in this case because gcloud is written in python and the emulators are written in Java.

Great, that's a start. You should trap the SIGINT/SIGTERM/SIGKILL signal(s) for the parent and try to clean up the children (I used psutil in Python to do this in a cross-platform way, though could've used Python's atexit as well).
It's unclear why being able to background the emulator and send the output to a log file is not a feature.
Closing this issue out since we are no longer chatting about google-cloud-datastore.

Agree with all your suggestions.

We can look into trapping signals in gcloud.sh and performing some sort of cleanup.
We hope to add backgrounding capability in the future along with emulator 'process management'.

Good to hear!


Random changes in code will cause MQTT connection to fail..

I was using the ThingSpeak MQTT example which includes pubsubclient to connect before sending data.  For some reason which seems not to be logical very minor changes I make in the code will cause the connection to fail (rc -2 or -4).  The change can be one within the reconnect() or somewhere unrelated.
My basic code: https://pastebin.com/7GRUF6cP using ESP12-F and Wemos D1 Mini with latest PubSub library.
Examples where one minor changes causes the issue:
I remove this line and the connection no longer works - Serial.print("Attempting MQTT connection...");
Moving everything to setup() or loop() and the connection no longer works
Any help is appreciated.

I fixed it by wiping and reinstalling the ide and libraries.  Must have been some weird conflict


Document all queries, configuration and result formats

Both in-code and in documentation.

Completed by a600bbfa2058ed395e629329f56a99fc0ca200bf


Addressing issue 145

Closes #145. Added checks to the __init__ for both classes in the green_wave_env file. Edited the experiment runner code in green_wave.py to express these changes

Did you add the required params to the rllib examples as well?

@kjang96 I think all of the grid experiments everywhere will fail unless you add these params to their additional_env_params now


removing hmac signature verification feature

Description
As stated, see CI-5037
Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[x] I have signed the Adobe Open Source CLA.
[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

@shikhartanwar : please review this

Thank you, @purplecabbage


Trouble getting anything besides text when using V2 and get_users_tweets() method

Hello,
Currently trying to migrate over to API v2, but I'm having difficulty getting the usual extras when querying user timelines. API v1.1 automatically returns fields such as "created_at", and retweet/reply/quote statuses, but apparently V2 needs to be told explicitly to return those items. The issue I'm running into is that the procedures for returning that data as outlined in official documentation do not seem to be working for Tweepy.
Here is my code, attempting to get both "created_at" and "public_metrics" in addition to the tweet text:
import Tweepy
TOKEN = '12345678912345678912345567889etc'
id = '1234567890'

tweepy_client = authorize_twitter_oauth2(TOKEN) #just a custom login function
tweets = tweepy_client.get_users_tweets(id=id, tweet_fields=['created_at,public_metrics']).data

The above code just returns the tweet id and the tweet text, ignoring my requests for "created_at" and "public_metrics". I've also tried formatting the "tweet_fields" argument like this, but still to no avail:
tweet_fields=['created_at' ,'public_metrics']

Am I doing something wrong or is this a current limitation of Tweepy?
Thanks for any support.

https://tweepy.readthedocs.io/en/latest/faq.html#why-am-i-not-getting-expansions-or-fields-data-with-api-v2-using-client

https://tweepy.readthedocs.io/en/latest/faq.html#why-am-i-not-getting-expansions-or-fields-data-with-api-v2-using-client


[SPARK-33128][SQL] Remove EXCEPT, INTERSECT, MINUS, UNION from strictNonReserved

What changes were proposed in this pull request?
This pr removes EXCEPT, INTERSECT, MINUS and UNION from strictNonReserved.
Why are the changes needed?

This is a regression since Spark 3.0.
EXCEPT, INTERSECT and UNION are all reserved, Please see https://www.postgresql.org/docs/12/sql-keywords-appendix.html for more details.

Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit test.

Test build #130928 has started for PR 30331 at commit 15759d9.

Test build #130928 has finished for PR 30331 at commit 15759d9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130928/
Test FAILed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35533/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35533/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/35533/
Test PASSed.

Test build #132296 has started for PR 30331 at commit 15759d9.

Test build #132296 has finished for PR 30331 at commit 15759d9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132296/

@wangyum, why did you close? just curious.


Separate Testing Suite

Idea
Right now the project is growing in size at an incredible rate, but the only downside is that the testing suite is just a single file. In most cases, this would work fine, but this monolithic file needs to be cut down and separated into a bunch of separate testing files. We can keep the same file system structure to organize the files but we just need to break up the testing suite.
Todo

[x] #54
[x] #55
[x] #56
[x] #57
[x] #58

@William-McGonagle Reporting for duty, Sir.

Hey, @NikitaLejnev- would you like to join our Organization? It's a non-profit and just starting out so there wouldn't really be any pay, but we would love to have you!

@William-McGonagle I would love to help out. My programming skill leaves much to be desired but I hope to proove useful.

No- you seem do be doing a great job! I just invited you to the organization if you would like to join up.


net-glb uses wrong type for serve_while_stale attribute in google_compute_backend_bucket

https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/blob/v21.0.0/modules/net-glb/variables.tf#L40
module "glb_forms" {
  source     = "github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/net-glb?ref=v21.0.0"
  project_id = "project_id"
  name       = "test-glb"
  address    = module.lb_ext_ip_forms.global_addresses["global-address"].self_link
  protocol   = "HTTPS"

  https_proxy_config = {
    ssl_policy = google_compute_ssl_policy.xyz.self_link
  }

  backend_buckets_config = {
    forms = {
      bucket_name          = "bucket-xyz"
      edge_security_policy = google_compute_security_policy.xyz.self_link
      enable_cdn           = true

      cdn_policy = {
        default_ttl       = 300
        serve_while_stale = false
      }
    }
  }
...

Results:
│ Error: Incorrect attribute value type
│
│   on .terraform/modules/glb_forms/modules/net-glb/backends.tf line 40, in resource "google_compute_backend_bucket" "default":
│   40:       serve_while_stale            = p.value.serve_while_stale
│
│ Inappropriate value for attribute "serve_while_stale": number required.

Leaving serve_while_stale out has same result, and using integer as a value produces different error.

serve_while_stale is a number. What error do you get when you specify an integer?

Closing this. Please try setting a number like Julio points out above.


relative $ref not resolving

What version of Ajv are you using? Does the issue happen if you use the latest version?
6.5.4
When trying to resolved relative $ref, the algorythm returns there error:
MissingRefError: can't resolve reference identifier_info_schema.json# from id https://w3id.org/dats/schema/person_schema.json#

JSON Schema

see: https://raw.githubusercontent.com/datatagsuite/schema/master/person_schema.json
the json input being a bit big, i don't want to overflow this thread
Your code

code is available here: https://runkit.com/terazus/5bdb2df069012300129cdc73
I'm either doing something wrong in my code, or in my schemas, i guess.
Ty

I assume the expectation is that this this schema should be loaded from URL https://w3id.org/dats/schema/identifier_info_schema.json#? The standard usage is to add all necessary schemas using addSchema. Ajv does not do any I/O unless you use compileAsync method and loadSchema option (see https://github.com/epoberezkin/ajv#asynchronous-schema-compilation)


Allow select component to skip the label

:mushroom:
But please don't change versions in PRs, it will get messy if we merge another PR before this.

Also, we don't do versions in ui_components. It's not published to rubygems and only references via github, so no need for that.

Changing the version in the PR shouldn't really be an issue. If someone else changes it on master before, there will be a conflict and it will be resolved manually.


Command specifc usage messages on hyprctl

It'd be nice to have usage information on commands in hyprctl, especially dispatch. Most times I try to use it from memory I make mistakes with its usage and end up keeping the wiki open. The commit in this PR is just a first start/draft, and I'm aiming to open the discussion on how you would feel it would be best included. Perhaps this is too much to keep in the executable and we could instead extend / add a further man page, or perhaps you think it's not worth the extra effort maintaining the new offline usage docs as well as the wiki. I'd be very interested to hear which you think would be more appropriate and would happily start work on it.

this will get outdated really quickly. Why not redirect to the wiki?

Offline documentation is preferable where possible as it does not require an internet connection. After giving it some further thought though I don't like what I've committed here and am now much more strongly on the side of expanding the man page(s). The first tool I reach for when using a shell program I'm not 100% familiar with is the program itself (-h), but very shortly after that I look it up on man long before I open my browser. Hyprctl already does have a man page, but it is far from usable as a standalone document and just provides some general hints. Having it populated with much more information with a disclaimer that it may not be 100% up to date, and to fallback to the wiki as truth in those cases, would be much more preferable to me and I'm sure many other consumers of the project.
In the case of dispatchers as an example, the dispatchers and parameter tables are the main bits of information I'd like to be able to more quickly reference. A script generating a man page out of the markdown in the wiki wouldn't be difficult, but I'm not confident enough in deployment to comfortably assert that it wouldn't be difficult to actually put in place.
Final say is yours, but is definitely something I personally would like to use.

my two cents: having to open a web browser, navigate to the wiki, then find where the thing I need is located gets old fast.
In terms of it being outdated, it might (since it makes you have to update in two locations), but the top level commands are already in the help so you already have to do that.
also, the error messages for the dispatch command are less than helpful at the current moment. I'd be nice to just have the help as part of the error.

I will be working on updating the manpages soon.

Closing this issue in that case. @fufexan feel free to tag me here when you start if you want to split the work.


thread 'main' panicked at 'index out of bounds: the len is 1 but the index is 1', limitador/src/limit.rs:131:29

thread 'main' panicked at 'index out of bounds: the len is 1 but the index is 1', limitador/src/limit.rs:131:29

This is due to a condition of a Limit that isn't properly formatted. Currently the format is pretty strict: variable == value where even the space matter. I have some work on-going that catches those when the limit is loaded, unlike now where it fails when the limit is evaluated. I'm also working on making the parsing more robust...
This should be addressed by the next release


Avoided potential panic due to divide by 0

Signed-off-by: Aditi Ahuja<EMAIL_ADDRESS>

[ ] I added CHANGELOG entry for this change.
[x] Change is not relevant to the end user.

Changes
Earlier, since fraction was divided by sampleFactor without checking if it was equal to 0, there was the possibility of a panic. Now, it is first checked to be equal to 0 and that case is handled separately.
Verification

Quickest reviews and approval ever! Thanks @saswatamcode and @squat!


Is it possibile to use CorceRedirectTo200 in master?

Issue opened for the creation of a wiki page that summarizes the doubts and problems for newbies (https://github.com/volatiletech/authboss/issues/210).

Is it possibile to use CorceRedirectTo200 in master?
I'm using it like this:
ab.Config.Paths.RootURL = os.Getenv("SITE_URL")
ab.Config.Storage.Server = database
ab.Config.Storage.SessionState = sessionStore
ab.Config.Storage.CookieState = cookieStore
ab.Config.Modules.LogoutMethod = "GET"
ab.Config.Core.ViewRenderer = defaults.JSONRenderer{}
ab.Config.Core.Redirector = &defaults.Redirector{CorceRedirectTo200: true}
defaults.SetCore(&ab.Config, true, false)

but still I have a GET -> localhost/auth/logout -> redirected to -> /.
Why?
How to disable redirect and just use 200?

Your problem is that the call to defaults.SetCore overrides your redirector you just set up :p

@aarondl long time to respond I know.
I inverted lines from this:
ab.Config.Paths.RootURL = os.Getenv("SITE_URL")
ab.Config.Storage.Server = database
ab.Config.Storage.SessionState = sessionStore
ab.Config.Storage.CookieState = cookieStore
ab.Config.Modules.LogoutMethod = "GET"
ab.Config.Core.ViewRenderer = defaults.JSONRenderer{}
ab.Config.Core.Redirector = &defaults.Redirector{CorceRedirectTo200: true}
defaults.SetCore(&ab.Config, true, false)

to this:
ab.Config.Paths.RootURL = os.Getenv("SITE_URL")
ab.Config.Storage.Server = database
ab.Config.Storage.SessionState = sessionStore
ab.Config.Storage.CookieState = cookieStore
ab.Config.Modules.LogoutMethod = "GET"
ab.Config.Core.ViewRenderer = defaults.JSONRenderer{}
defaults.SetCore(&ab.Config, true, false)
ab.Config.Core.Redirector = &defaults.Redirector{CorceRedirectTo200: true}

and the problem is still there:
I still have a GET -> localhost/auth/logout -> redirected to -> /.
Why?

Isn't ab.Config.Core.Redirector = &defaults.Redirector{CorceRedirectTo200: true} setting a config in the &ab.Config I will use after in defaults.SetCore(&ab.Config, true, false)?
Authboss is great but this config is sometimes too complicated for newbies and simple people like me. :(

Ok. After small investigation I understood the problem is this line: https://github.com/volatiletech/authboss/blob/master/defaults/responder.go#L44.
The func isAPIRequest() because I was trying with Postman without Content-Type.
The problem now is I have errore on login, on logout (line 103 of responder.go: body, mime, err := r.Renderer.Render(req.Context(), "redirect", data)) because I think my Renderer struct is missing.

Little news:
I was able to make it work like this:
defaults.SetCore(&ab.Config, true, false)
ab.Config.Core.Redirector = &defaults.Redirector{Renderer: &defaults.JSONRenderer{}, CorceRedirectTo200: true}

but still don't know why with defaults.SetCore() I have FormValueName: "redir".
Is there a doc somewhere?
I can write it if I can understand.

https://github.com/volatiletech/authboss/blob/master/defaults/defaults.go#L25
https://github.com/volatiletech/authboss/blob/master/response.go#L11-L14
Do these links help?


Node Removal Malloc/Object Leaks

Hey all! I'm building a project that dynamically connects and disconnects nodes at runtime. Functionally everything works wonderfully, but I'm noticing some leaks as nodes are disconnected and garbage collected in swift. Each time a node is removed, I rebuild the entire audio graph. detach() is called on all existing nodes, as well as AudioKit.disconnectAllInputs(). Then new AKNodes are created and the entire audio graph rebuilt from scratch. From a user perspective everything works as it should, but I'm noticing a lot of leaked DSP objects. Here for example is after one complete rebuild of the graph. You can see the old DSP objects are still sticking around despite having no references:

I dug around in the source code and found that DSP objects are never actually freed. See the following code from AKAudioUnitBase.mm, for example:
- (instancetype)initWithComponentDescription:(AudioComponentDescription)componentDescription
                                     options:(AudioComponentInstantiationOptions)options
                                       error:(NSError **)outError {

    // Omitted for readability

    _kernel = (AKDSPBase*)[self initDSPWithSampleRate:defaultFormat.sampleRate
                                         channelCount:defaultFormat.channelCount];

    // Omitted for readability
}

- (BOOL)allocateRenderResourcesAndReturnError:(NSError **)outError {

    // Omitted for readability

    _inputBus.allocateRenderResources(self.maximumFramesToRender);
    _kernel->init(self.outputBus.format.channelCount, self.outputBus.format.sampleRate);
    _kernel->reset();
    return YES;
}

- (void)deallocateRenderResources {
    _inputBus.deallocateRenderResources();
    _kernel->deinit();
    [super deallocateRenderResources];
}

The AKDSPBase object is created in initWithComponentDescription and initialzed in allocateRenderResourcesAndReturnError, but it is never deallocated. _kernel->deinit(); doesn't actually do anything -- it is an unimplemented virtual function of AKDSPBase. AKSoundpipeDSPBase and any DSP classes that subclass it do provide a destroy() function to deallocate memory created during init(), but this function is never called by any part of AudioKit and is not available for AKAudioUnitBase.mm to call directly because it is not declared in the top AKDSPBase class.
One potential solution to this would be to rename all destroy() functions in subclasses of AKDSPBase to overridden deinit() functions. Because destroy() is never called it would not break any part of the code, and deinit() is a virtual function available in AKDSPBase that could be called in deallocateRenderResources as it is now. Another possibility would be to include a virtual destructor in AKDSPBase, which would allow delete _kernel to be called in deallocateRenderResources. Child classes could call their destroy() functions from a virtual destructor.
I tried a combination of the above in a few audio units that I am working with: I changed destroy() to deinit() in AKPanner, AKKorgLowpassFilter and AKWhiteNoise, which I call from deallocateRenderResources. Additionally, I added an empty virtual destructor to AKDSPBase so that I could follow the deinit() call in deallocateRenderResources with a delete _kernel call. The good news is that objects are no longer leaked upon rebuilding the audio graph, but I'm not sure if this is really the proper fix for this, and doing a complete fix in this way would require modifying every DSP class that has a destroy() function. I just started using AudioKit and am not very familiar with the source as a whole, I'm sure y'all will know the best way to go about this!

This is a lot of write up, we're working on making dynamically creating, connecting, and disconnecting nodes easier.  Perhaps @dave234 has something to say about this?

So, looking at your write up again, I would say it would be great if you could make a pull request with your fix for at least one of the nodes. I can make the update to all the rest.

I'm happy to :) Though I'd like to discuss the best course of action before creating a pull request, as these are somewhat high level changes that may require changes in all child classes.
There's two parts to this leak. The first is that delete is never called on the  AKDSPBase* _kernel object in the AKAudioUnitBase and AKGeneratorAudioUnitBase classes. This must happen, otherwise the DSP objects are leaked (as they are created with a new call). The second part is that, currently, all Soundpipe DSP classes do their memory cleanup in a function called destroy which is never called.
The solution to the first is to add a virtual destructor to AKDSPBase by adding virtual ~AKDSPBase() {} to the header. This is a one line fix for subclasses that define their own destructor (automatic override), or subclasses that declare no destructor. However, for some reason there are many DSP subclasses that declare a destructor in the header, but never actually define the destructor. This will cause linking errors with a virtual destructor in AKDSPBase unless the destructor declarations are removed in any subclasses without a corresponding definition (we could also just define an empty destructor in these classes, but if it's currently undefined what's the point of having a destructor at all?).
For the Soundpipe cleanup problem, there are two solutions that I can think of and I don't know which you'd prefer to do. The first would be to rename all destroy functions in soundpipe DSP classes to deinit (including the one in AKSoundpipeDSPBase), thus overriding the virtual and currently empty deinit function in AKDSPBase. deinit is already being called in AKAudioUnitBase despite being an empty function, so it seems like destroy was intended to be deinit in the first place? The second option would be to make destroy a private function and call it from the destructor (which will actually be called now that we are calling delete). We would need to make it private to ensure that someone couldn't unknowingly call it manually and then delete the object, thus calling it twice. The first seems to be the cleanest method but I wanted to ask as this will require changing pretty much all of the soundpipe classes.

The cleanest method is what I want.  I don't mind doing the busywork for the rest.

This should be addressed by the recent PR and follow up commits. Any other helpful changes would be most welcomed. Thanks!


Issue in play event

Describe the bug
I'm trying to play https://youtu.be/f1tN1MSWs20 this song. It says video unavailable but it still sent play message.
Expected behavior
When it cannot play a song, it should not send play message . It should just send the error message.
Additional context
I've tried many times to play the song, but every time same problem.

maybe video is unavailable in your(or vps) region

I know the video is unavailable. But it would be better , if you fix the play event

maybe video is unavailable in your(or vps) region

He means it shouldn't emit playSong with unavailable videos.

I'm trying to play https://youtu.be/f1tN1MSWs20 this song. It says video unavailable but it still sent play message.

Idk if I can improve this since this error emits on ytdl-core stream, getInfo cannot detect it. I'll read ytdl-core code and make a pull or make a function to check unplayable videos before creating the stream. Should be on v3.


added predictor app code

Community article: https://community.datarobot.com/t5/general-knowledge-base/building-a-serverless-predictor-app/ta-p/7870

ah, i had added the filled-in policy.json instead of the template version! just fixed it

approved


Calib prep repeat search

This PR introduces code to search for repeats in the list of calls to the pipeline. It also looks for entries where one call is completely present within another call. strun commands have also been added to the main output table.
In order to store this information, several new columns have been added to the main table of information. The index column is simply a row counter. The repeat_of_index_counter columns lists, for a given row, the row index number where exactly the same strun call is made. In the index_contained_within column, the index number of the row that contains the complete strun call for that row is listed. For a row that is not repeated anywhere nor contained within any other rows, a value of -1 is given.
for example:

index
repeat_of_index_number
index_contained_within
strun_command

0
-1
3
strun file1.fits --options1

1
-1
4
strun file2.fits --options2

2
-1
-1
strun file3.fits --options3

3
-1
-1
strun file1.fits --options1 --options4

4
-1
-1
strun file2.fits --options2 --options5

5
-1
-1
strun file4.fits --options6

6
5
-1
strun file4.fits --options6

In this case:
Indexes 5 and 6 are exactly the same. This is indicated in row 6 where repeat_index_number is 5.
The command in row 0 is fully contained within the command for row 3. This is shown in index_contained_within for row 0. Similarly, the command in row 1 is fully contained within the command in row 4.
In cases like that of rows 0 and 3, the code takes the output filename for row 0 and adds it as an intermediate output in row 3.
What this means is that the only strun commands from this table that must be executed in order to produce the full set of outputs are those in rows where both repeat_of_index_number and -index_contained_within` are -1.

@arminrest do you want to review this, or should I just merge it and you can see if what it does makes sense?