Background android notification

If I sent the notification to an individual devices as shown in the documentation, if the app is background and we click on it in the noitification section the app does not open.
I have tried everything, please help!

How are sending the notification? Can you paste your code?
FCM documentation states that only Data Notifications are caught by the app when in background. Instead Message Notifications are only when app in in foreground.

Like that:
        $client = new Client();
        $client->setApiKey($this->key);
        $client->injectGuzzleHttpClient(new ClientHttp());
        $message = new Message();
        $message->setPriority('high');
        $notif = new Notification($title, $messageText);
        $notif->setClickAction('MainApplication');
        $notif->setSound("default");
        $message->addRecipient(new Device($user->getFirebaseToken()));
        $message
            ->setNotification($notif)
            ->setData($data);
        $response = $client->send($message);

The issue is in the fact that you send the notification as a Message Notification instead of Data Notification.
 $notif = new Notification($title, $messageText);

Here you are setting the title and body of the notification and this is handled automatically by the device to show the notifications. Instead if you need to make the notification show also when the app is in background you need to leave this 2 properties empty and set title and body in the Data property.
 $notif = new Notification();
.... // add all your settings of Notification then:
$message
            ->setNotification($notif)
            ->setData([
                'title' => 'this is a title',
                'body' => 'this is a body'
            ]);

This will require you to create a listener in the mobile app which will get the title and body and manually generate the notification.


Python processes still running after closing VSCode

Applies To

[X] Notebooks (.ipynb files)
[X] Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?
I am running a Python script with GridSearchCV and n_jobs > 1. This spawns N threads. After closing VSCode, these threads remain in memory.
This only happens with the Jupyter extension for VSCode (Interactive Window / .ipynb files). If I run the script in Terminal ("Run Python File in Terminal") or directly in Jupyter Notebook (outside of VSCode), the threads are closed properly.
I have tried some suggestions on similar issues (#5485, #1626, microsoft/vscode-python #19347, conda/conda #11569, microsoft/vscode #72769, scikit-learn/scikit-learn #12817) but none have worked for me.
Before closing VSCode

After closing VSCode

VS Code Version
1.69.2
Jupyter Extension Version
v2022.6.1201981810
Jupyter logs
Visual Studio Code (1.69.2, undefined, desktop)
Jupyter Extension Version: 2022.6.1201981810.
Python Extension Version: 2022.10.1.
No workspace folder opened.
info 9:21:57.461: ZMQ install verified.
El usuario pertenece al grupo de experimentos 'jupyterTestcf'
El usuario pertenece al grupo de experimentos 'jupyterEnhancedDataViewer'
info 9:21:57.843: LSP Notebooks experiment is disabled -- not in treatment group
info 9:21:57.982: Experiment status for python is {"enabled":true,"optInto":[],"optOutFrom":[]}
info 9:22:2.416: Process Execution: > c:\ProgramData\Anaconda3\python.exe -m pip list
> c:\ProgramData\Anaconda3\python.exe -m pip list
info 9:22:20.901: Starting interactive window for resource 'c:\Users\federico.azzi\Desktop\tree\arbol_decision.py'
info 9:22:21.26: Preferred Remote kernel for Interactive-1.interactive is undefined
info 9:22:21.204: Starting Jupyter Session id = 'startUsingPythonInterpreter:.jvsc74a57bd0ad2bdc8ecc057115af97d19610ffacc2b4e99fae6737bb82f5d7fb13d2f2c186.c:\ProgramData\Anaconda3\python.exe.c:\ProgramData\Anaconda3\python.exe.-m#ipykernel_launcher' (Python Path: c:\ProgramData\Anaconda3, EnvType: Conda, EnvName: 'base', Version: 3.9.12) for 'Interactive-1.interactive' (disableUI=false)
info 9:22:21.206: Creating raw notebook for resource 'c:\Users\federico.azzi\Desktop\tree\arbol_decision.py'
info 9:22:21.206: Computing working directory for resource 'c:\Users\federico.azzi\Desktop\tree\arbol_decision.py'
info 9:22:21.220: Process Execution: > c:\ProgramData\Anaconda3\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2022.6.1201981810\pythonFiles\normalizeSelection.py
> c:\ProgramData\Anaconda3\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2022.6.1201981810\pythonFiles\normalizeSelection.py
info 9:22:21.226: Starting raw kernel 'base (Python 3.9.12)' for interpreter c:\ProgramData\Anaconda3\python.exe
info 9:22:21.250: Process Execution: > c:\ProgramData\Anaconda3\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> c:\ProgramData\Anaconda3\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 9:22:21.280: Kernel launching with ports 9000,9001,9002,9003,9004. Start port is 9000
info 9:22:21.302: Creating daemon process for c:\ProgramData\Anaconda3\python.exe with env variables count 61
info 9:22:21.316: Process Execution: > c:\ProgramData\Anaconda3\python.exe -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_interrupt_daemon -v --ppid 3976
> c:\ProgramData\Anaconda3\python.exe -m vscode_datascience_helpers.daemon --daemon-module=vscode_datascience_helpers.kernel_interrupt_daemon -v --ppid 3976
info 9:22:21.597: Registering dummy command feature
info 9:22:21.653: get interrupthandle daemon
info 9:22:21.665: Process Execution: > c:\ProgramData\Anaconda3\python.exe -m ipykernel_launcher --ip=<IP_ADDRESS> --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"da1cfb1e-1b2a-4ce0-87b1-8bca0202c5eb" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\federico.azzi\AppData\Roaming\jupyter\runtime\kernel-v2-39765MEwELewjoOT.json
> c:\ProgramData\Anaconda3\python.exe -m ipykernel_launcher --ip=<IP_ADDRESS> --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"da1cfb1e-1b2a-4ce0-87b1-8bca0202c5eb" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\federico.azzi\AppData\Roaming\jupyter\runtime\kernel-v2-39765MEwELewjoOT.json
info 9:22:21.665: Process Execution: cwd: ~\Desktop\tree
cwd: ~\Desktop\tree
info 9:22:21.915: ipykernel version 6.9.1 for c:\ProgramData\Anaconda3\python.exe
info 9:22:21.915: ipykernel location c:\ProgramData\Anaconda3\lib\site-packages\ipykernel\__init__.py for c:\ProgramData\Anaconda3\python.exe
warn 9:22:23.953: StdErr from Kernel Process c:\ProgramData\Anaconda3\lib\site-packages\traitlets\traitlets.py:2202: FutureWarning: Supporting extra quotes around strings is deprecated in traitlets 5.0. You can use 'hmac-sha256' instead of '"hmac-sha256"' if you require traitlets >=5.
  warn(

warn 9:22:23.953: StdErr from Kernel Process c:\ProgramData\Anaconda3\lib\site-packages\traitlets\traitlets.py:2157: FutureWarning: Supporting extra quotes around Bytes is deprecated in traitlets 5.0. Use 'da1cfb1e-1b2a-4ce0-87b1-8bca0202c5eb' instead of 'b"da1cfb1e-1b2a-4ce0-87b1-8bca0202c5eb"'.
  warn(

info 9:22:24.37: Kernel Output: NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

To read more about this, see https://github.com/ipython/ipython/issues/2049

To connect another client to this kernel, use:
    --existing c:\Users\federico.azzi\AppData\Roaming\jupyter\runtime\kernel-v2-39765MEwELewjoOT.json

info 9:22:24.254: Se inició el kernel base (Python 3.9.12)., (Raw session started and connected)
info 9:22:24.255: Finished connecting 31d765e2-0562-40a8-92b6-5871213e0687
info 9:22:24.258: UpdateWorkingDirectoryAndPath in Kernel
info 9:22:24.259: Executing silently Code (idle) = import types as _VSCODE_types\nimport os\nimport hashlib as _VSCODE_hashlib\nfrom IPython import get_ip
info 9:22:24.307: Executing silently Code (completed) = import types as _VSCODE_types\nimport os\nimport hashlib as _VSCODE_hashlib\nfrom IPython import get_ip with 1 output(s)
info 9:22:24.307: Waiting for idle on (kernel): f8315e67-d9d1-45e7-bd9e-8cf4ee0cb757 -> idle
info 9:22:24.307: Finished waiting for idle on (kernel): f8315e67-d9d1-45e7-bd9e-8cf4ee0cb757 -> idle
info 9:22:24.313: Executing silently Code (idle) = __file__ = 'c:\\Users\\federico.azzi\\Desktop\\tree\\arbol_decision.py'
info 9:22:24.319: Widget Message: Received IPyWidgets_kernelOptions in ScriptManager
info 9:22:24.319: Widget Message: Received IPyWidgetMessages.IPyWidgets_kernelOptions
info 9:22:24.323: Executing silently Code (completed) = __file__ = 'c:\\Users\\federico.azzi\\Desktop\\tree\\arbol_decision.py' with 0 output(s)
info 9:22:24.332: Generated code for 1 = <ipython-input-1-03c6b825e39d> with 206 lines
info 9:22:24.389: Widget Message: data-base-url set to https://file%2B.vscode-resource.vscode-cdn.net/c%3A/Users/federico.azzi/.vscode/extensions/ms-toolsai.jupyter-2022.6.1201981810/tmp/scripts/a8afd25f292aa84774f0bcc2eab57fcaf24523317a2271647f91f5b9f6c47a9c/jupyter
info 9:22:35.618: Cell 1 executed with state Success

Coding Language and Runtime Version
Python 3.9.12
Language Extension Version (if applicable)
v2022.10.1
Anaconda Version (if applicable)
conda 4.12.0
Running Jupyter locally or remotely?
Local

Thanks for the issue.
Do you have some example code that repros the problem?

My only guess as to how this might be a VS code issue is that we don't wait for the kernel to shutdown on exit.  Jupyter may be waiting for shutdown.
The way to verify that would be to kill the jupyter server (when using just jupyter) and see if the processes leak there too.

Thanks for the issue.
Do you have some example code that repros the problem?

Thanks for the quick reply. This is a short example that can be used to reproduce the problem:
import numpy as np
import pandas as pd
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split, GridSearchCV, RepeatedKFold
from sklearn.ensemble  import RandomForestRegressor
import multiprocessing
import warnings
warnings.filterwarnings('ignore')

boston = load_boston(return_X_y=False)
df = np.column_stack((boston.data, boston.target))
df = pd.DataFrame(df,columns = np.append(boston.feature_names, "MEDV"))
X_train, X_test, y_train, y_test = train_test_split(
                                        df.drop(columns = 'MEDV'),
                                        df['MEDV'],
                                        test_size = 0.33,
                                        random_state = 123
                                    )
param_grid = {
    'max_features': [5, 7, 9],
    'max_depth' : [1, 3, 5]
    }

grid = GridSearchCV(
        estimator  = RandomForestRegressor(random_state=123),
        param_grid = param_grid,
        scoring    = 'neg_root_mean_squared_error',
        n_jobs     = multiprocessing.cpu_count() - 1,
        cv         = RepeatedKFold(n_splits = 5, n_repeats = 2, random_state = 123),
        refit      = True,
        verbose    = 0,
        return_train_score = True
       )

grid.fit(X = X_train, y = y_train)

result = pd.DataFrame(grid.cv_results_)
result.filter(regex = '(param.*|mean_t|std_t)') \
    .drop(columns = 'params') \
    .sort_values('mean_test_score', ascending = False) \
    .head(4)

Thanks that helps. I'm reproing. The processes don't die during shutdown in VS code but do in Jupyter regardless of whether or not I kill them in the UI or just shutdown jupyter altogether.


[Question] [Optimization] async support?

I noticed we are using fastapi to build micro services, but we aren't using async for anything?
For example, llamaindex has full async support on nearly every component. Yet all the methods are defined using synchronous methods.
We can greatly increase throughput but using async with fastapi

Thanks for reminding, we already use async

@kevinintel for sure, but shouldn't the components be using async? For example for embeddings, we have await embed_model.aget_text_embedding() or with llms await llm.achat() or with vector stores vector_store.aquery() and so on
I don't see this being utilized

@logan-markewich，
we currently have async support in some microservices, but not all. We'll need to evaluate each case individually. Thank you for highlighting this issue; addressing it can significantly improve the performance of the entire pipeline.


buildah: 1.19.8 -> 1.20.0

Motivation for this change
https://github.com/containers/buildah/releases/tag/v1.20.0
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 117638 at 96c7efbf run on x86_64-linux 1

2 packages built successfully:

buildah
buildah-unwrapped

3 suggestions:

warning: missing-patch-comment
Please add a comment on the line above, explaining the purpose of this patch.
Near pkgs/development/tools/buildah/default.nix:29:5:
   |
29 |     ../../../applications/virtualization/podman/remove-unconfigured-runtime-warn.patch
   |     ^

warning: missing-phase-hooks
buildPhase should probably contain runHook preBuild.
Near pkgs/development/go-modules/generic/default.nix:145:5:
    |
145 |     buildPhase = args.buildPhase or ''
    |     ^

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/development/go-modules/generic/default.nix:220:5:
    |
220 |     installPhase = args.installPhase or ''
    |     ^


RIFEX-58 Fixing node version

Current node version on package.json is not working, basically throws a bunch of errors, that's because node version was updated but not the dependencies

This affects the compilation process on the project

Nice catch!
Please use the hotfix branch prefix the next time you find a bug on master branch

You are right i made that mistake and later i just realize about it, my bad


fluentd kafka max_send_limit_bytes

I am getting error on fluentd Kafka::MessageSizeTooLarge" , I can't seem to find where to set "max_send_limit_bytes" on kafka to less than 1M (kafka default)
Environment details:

Kubernetes version (e.g. v1.15.2): 1.19.9
Cloud-provider/provisioner (e.g. AKS, GKE, EKS, PKE etc): rancher 2.5.7
logging-operator version (e.g. 2.1.1): 3.94 , tried 3.9.0
Install method (e.g. helm or static manifests): helm
Logs from the misbehaving component (and any other relevant logs): fluentd

/kind bug

Hi @ahma
I tried to remove/reinstall logging-operator 3.9.4 , but still it doesn't download updated CRD yamls.
Is there any manual steps I need to do ?
Thanks

It depends on how you installed it. Helm doesn't upgrade crds for you, so you have to do that for yourself.

This PR isn't released yet. We waiting some other fixies, but we l'll make a new releasa soon. Thanks for your patience!

Hi, @ahma
It looks like max_send_limit_bytes param mentioned in this issue is not added. Is there any reason?

max_send_limit_bytes - default: nil - Max byte size to send message to avoid MessageSizeTooLarge. For example, if you set 1000000(message.max.bytes in kafka), Message more than 1000000 byes will be dropped.

Ref. https://github.com/fluent/fluent-plugin-kafka#buffered-output-plugin


SSL_connect: Unexpected EOF (OpenSSL::SSL::Error)

my code
CAT = File.read("somefile.txt")

loop do
  category = CAT.split("\n")
  category.each do |v|
    is_f = v.split("|")
    is_f == "c" ? fork { get(v) } : get(v)
  end
end

def get()
  # http client
  # n = number from http get
  (0...n).each do |i|
    # http client get
    # json parsing
    # http client post
  end
end

my app has been running for 1 hours, thanks ^^
error
SSL_connect: Unexpected EOF (OpenSSL::SSL::Error)
0x437105: ??? at ??
0x46e2f2: ??? at ??
0x46a779: ??? at ??
0x4695f7: ??? at ??
0x464e64: ??? at ??
0x4636a4: ??? at ??
0x4102c8: ??? at ??
0x430c6b: main at ??
0x7f3bad1ed830: __libc_start_main at ??
0x40e889: _start at ??
0x0: ??? at ??

https://github.com/crystal-lang/crystal/blob/3c6c75e68f326bb91be35f71ae30672fd454776e/src/openssl.cr#L110

Please reopen this with some reproducible code.

@RX14 I guess I can not give it all, and this error comes after 1 hour running, so I'm not sure where exactly the problem

We can't investigate anything without a working code sample. You can either give the whole code + instructions, or preferably you can remove the bits of code that are unneccesary to reproduce.
It appears to me that your code requests multiple URLs, perhaps it's the same URL which causes the error every time?


The global_step update in optimization_gpu.py (line 74-75) is redundant.

You didn't override the apply_gradients function in class AdamWeightDecayOptimizer, Normally the global step update is done inside of apply_gradients.

Thank you.

Fixed.


[Usage]: LLM with tensor_parallel_size larger than n. gpus in one node

Your current environment

The output of `python collect_env.py` on a single node:

Register this system with Red Hat Insights: insights-client --register
Create an account or view all your systems at https://red.ht/insights-dashboard
Last login: Fri Sep 27 16:49:32 2024 from <IP_ADDRESS>
[gpuccett@lrdn0487 ~]$ cd /leonardo_scratch/large/userexternal/gpuccett/Repos/MGT2025-private/
[gpuccett@lrdn0487 MGT2025-private]$ conda activate conda_venv/
(/leonardo_scratch/large/userexternal/gpuccett/Repos/MGT2025-private/conda_venv) [gpuccett@lrdn0487 MGT2025-private]$ python collect_env.py
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Red Hat Enterprise Linux release 8.7 (Ootpa) (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-16)
Clang version: Could not collect
CMake version: version 3.20.2
Libc version: glibc-2.28

Python version: 3.12.6 | packaged by conda-forge | (main, Sep 22 2024, 14:16:49) [GCC 13.3.0] (64-bit runtime)
Python platform: Linux-4.18.0-425.19.2.el8_7.x86_64-x86_64-with-glibc2.28
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA A100-SXM-64GB
GPU 1: NVIDIA A100-SXM-64GB
GPU 2: NVIDIA A100-SXM-64GB
GPU 3: NVIDIA A100-SXM-64GB

Nvidia driver version: 530.30.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  1
Core(s) per socket:  32
Socket(s):           1
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
Stepping:            6
CPU MHz:             2601.000
CPU max MHz:         2601.0000
CPU min MHz:         800.0000
BogoMIPS:            5200.00
Virtualization:      VT-x
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-15
NUMA node1 CPU(s):   16-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==<IP_ADDRESS>
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==<IP_ADDRESS>
[pip3] nvidia-cufft-cu12==<IP_ADDRESS>
[pip3] nvidia-curand-cu12==<IP_ADDRESS>
[pip3] nvidia-cusolver-cu12==<IP_ADDRESS>
[pip3] nvidia-cusparse-cu12==<IP_ADDRESS>
[pip3] nvidia-ml-py==12.560.30
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] nvidia-nvjitlink-cu12==12.6.68
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] pyzmq==26.2.0
[pip3] torch==2.4.0
[pip3] torchvision==0.19.0
[pip3] transformers==4.45.1
[pip3] triton==3.0.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-cublas-cu12        <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
[conda] nvidia-cudnn-cu12         <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cufft-cu12         <IP_ADDRESS>                pypi_0    pypi
[conda] nvidia-curand-cu12        <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusolver-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusparse-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-ml-py              12.560.30                pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.68                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
[conda] pyzmq                     26.2.0                   pypi_0    pypi
[conda] torch                     2.4.0                    pypi_0    pypi
[conda] torchvision               0.19.0                   pypi_0    pypi
[conda] transformers              4.45.1                   pypi_0    pypi
[conda] triton                    3.0.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.6.1.dev238+ge2c6e0a82
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    GPU2    GPU3    NIC0    NIC1    NIC2    NIC3    CPU Affinity    NUMA Affinity
GPU0     X      NV4     NV4     NV4     PXB     NODE    NODE    NODE    0-15    0
GPU1    NV4      X      NV4     NV4     NODE    PXB     NODE    NODE    0-15    0
GPU2    NV4     NV4      X      NV4     NODE    NODE    PXB     NODE    0-15    0
GPU3    NV4     NV4     NV4      X      NODE    NODE    NODE    PXB     0-15    0
NIC0    PXB     NODE    NODE    NODE     X      NODE    NODE    NODE
NIC1    NODE    PXB     NODE    NODE    NODE     X      NODE    NODE
NIC2    NODE    NODE    PXB     NODE    NODE    NODE     X      NODE
NIC3    NODE    NODE    NODE    PXB     NODE    NODE    NODE     X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1
  NIC2: mlx5_2
  NIC3: mlx5_3

How would you like to use vllm
In version 0.5.4 I used to be able to initialize and run Llama 3.1 405b instruct hf when running on 4x4A100 - 64Gb by first initializing ray on all nodes and then calling:
llm = LLM(tensor_parallel_size=16, ...)
Now I am trying the same script on 0.6.2 and it looks like this is not possible anymore, the code gets stuck at
ESC[33m(autoscaler +21m12s)ESC[0m Error: No available node types can fulfill resource request {'node:<IP_ADDRESS>': 0.001, 'GPU': 1.0}. Add suitable node types to this cluster to resolve this issue.
INFO 09-27 16:47:50 ray_utils.py:183] Waiting for creating a placement group of specs for 1270 seconds. specs=[{'node:<IP_ADDRESS>': 0.001, 'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}, {'GPU': 1.0}]. Check ray status to see if you have enough resources
Is it intentionally deprecated to use a tensor_parallel_size larger than the number of gpus on a single node or should I change something to adjust to the newer version of vllm?
Thank you in advance.
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Maybe you can try TP=4 PP=4? As mentioned here

Maybe you can try TP=4 PP=4? As mentioned here

I think that will raise an error since that is not supported for offline inference.

@youkaichao do you know whether PP is planned for offline inference?

PP does not support offline inference yet. contribution is welcome

PP does not support offline inference yet. contribution is welcome

Unfortunately I don't think I have enough knowledge/skill to do that.
But is tensor_parallel_size spanning more than one node not supported anymore a choice?


List::Util missing 'uniq' on some common distros

duprule appears to have a dependency on a newer version of List::Util than the one that ships with Ubuntu 16.04 LTS. I'm using the version of List::Util that ships with perl 5.22.1:
$ perl ./duprule.pl
"uniq" is not exported by the List::Util module
Can't continue after import errors at ./duprule.pl line 4.
BEGIN failed--compilation aborted at ./duprule.pl line 4.
For better backward compatibility, some options might be:

switching to a standalone implementation of uniq

requiring List::Util 1.44 and giving a warning

using List::MoreUtils (which is in the Ubuntu repositories as liblist-moreutils-perl)

Closed by #30.


Make //tensorflow/cc:cc_ops compile for Android

It seems that the bazel target //tensorflow/cc:cc_ops is currently not compilable for Android. From what I saw, this is at least because of the missing platform specific headers (e.g. in tensorflow/core/platform/gif.h).
Making the cc_ops compilable for Android would allow to fully use tensorflow on Android. In particular, I'm trying to build tensorflow for Android with the operations to be able to provide javacpp-presets which allow an automated generation of complete Java bindings. Therefore they can be used to completely omit the jni/C++ code in Android applications that want to use tensorflow.

The Android version of TF uses a different build tree so that we can concentrate on binary size, etc. You'll find that most of cc_ops is provided by //tensorflow/core:android_tensorflow_lib


Define core auth abstractions with docs

Using the existing auth contracts and implementing the glue for an
example authentication endpoint revealed several issues. To resolve
these, the commit includes:

fixes to auth related to an actual implementation
the first circe integration (of more to come)
finagle-only filters (as opposed to finch Routers in server module)
brings ServiceMatcher into borderpatrol package
brings ServiceIdentifier into borderpatrol package
adds tests for ServiceMatcher

:+1:

Current coverage is 63.34%

Merging #72 into master will decrease coverage by -6.88% as of b0054ec

@@            master     #72   diff @@
======================================
  Files           21      24     +3
  Stmts          262     311    +49
  Branches        14      16     +2
  Methods          0       0
======================================
+ Hit            184     197    +13
  Partial          0       0
- Missed          78     114    +36

Review entire Coverage Diff as of b0054ec

Powered by Codecov. Updated on successful CI builds.


Add support for Azure Functions

It will be very important to choose what host type the app can be deployed.

Hey @murilocurti !
Can you elaborate more on that? How do you think the reverse-proxy would be useful on Az Func or serverless workloads like that?
I'm curious to understand use cases for it in the serverless scenario.

I see Azure Functions like an Asp.Net Core hosting diversification when used with http triggers.
Serverless has some advantages because don't require all the scaling options as standard App Service Plans.
If the availability of reverse proxy is limited to regular Asp.Net Core apps, developers will be forced to work with App Service Plans or VMs.
It's more to let developers free to choose what is best for each scenario. I know that is perfectly possible to use the regular Asp.Net Core hosting options.
Here at Space Needle we rely more on Azure Functions than the other host options. And it definitely worse using.
Is it enough details about the request?
Best Regards

Hey,
As we discussed offline, I understand your points and desires with it. It would be nice to have the reverse-proxy capabilities on Azure Functions even though I'm pretty sure this project would be able to run (with some adaptations) in Azure Function project, we have some drawbacks from the serverless approach that may ruin the whole benefits of it (or any reverse-proxy per se) brings to the table.
To name a few:

Cold-start time of functions will cause poor performance handling requests;
Every time a function start, it would start the reverse-proxy, it would have to load all the routing configuration which would introduce a lot of latency to the request;
On-the-fly routing changes or configs would be useless;
Probes to Azure Function based backends would be a cost problem, and in the end of the day useless;
Azure Functions host implementation and hosting model limitations makes it a bit hard to use (i.e. no proper DI, unable to change configuration sources, etc.)

If you want to go this route, there is a way... If you use Azure Functions Premium (or greater) SKU, you can put your functions project into a container (just a simple Dockerfile) and you can also inject your own custom Azure Functions Host process, which will give you more control and the ability run the reverse-proxy on it.
In that case, most of the limitations I've mentioned of the Az Func are worked around, including the cold-start, since in that SKU, you have at least 1 warm container always active (you can configure to have more) which means you don't suffer of cold-start and you still get the auto-scalling capabilities of Az Func.
Another option that I see, is if the Azure Functions team decides to "embrace" this as a feature inside the product extending the current Proxy Function functionalities to use this. That would bring you the full serverless model and at same time, abstract away the reverse-proxy deployment. This is basically what is happening with SignalR Services when they are hosting SignalR for you as a fully managed service and allow you to have backends that handle Hub method calls.
I hope that gives you a light.

@galvesribeiro I dont agree with any of your 5 drawbacks.
Azure functions runs on aspnet core and uses DI, it can also keep singleton objects across function invocations.
So its definite possible to make it work on azure functions with the exact same benefits/drawbacks as any other app has running on azure function if the azure function team wanted to.

@pksorensen I didn't said it was impossible. Just saying that it is very limited and bring you drawbacks. I've been using Az Functions since its inception. If you look at the SDK/Host/Tooling repository for the issues, and search for anything related to DependencyInjection, you will see that it has a lot of limitations and problems that a normal Asp.Net projections doesn't.
To start with, it doesn't honor the API surface from Microsoft.Extensions.DependencyInjection.Anstractions. That is because they have their own IServiceProvider implementation that rely on a custom IoC container that is very limited as you may see on many issues (just check the the WebJobs Host source code, it is OSS and you will see it is not just an AspNet Core project). But that is just one topic. I could spend an 1+ hour talking about the difficulties we had when we dived into Az Functions but that is not the case. Again, it is not that the product is bad. It is just that it was not meant to support such scenarios IMHO, even if I don't agree with the design of the whole product technically speaking.

Azure functions runs on aspnet core and uses DI,

Again, the DI implementation is not near close to Asp.Net Core one. You can't even change things like IConfiguration and other infrastructure components. There are some hackery to create another DI container and then after build, re-registrer everything on the Functions one, but that has a a huge chance of overwriting the Host services already being registered which affects the way the auto-scaler work, etc, so I wouldn't go that path (believe me, I've tried).

it can also keep singleton objects across function invocations.

Like I said, it is technically possible BUT, if you have the bad luck to get have to wait for the host to start, and even worse, if the host start is pulling the reverse-proxy (or any other) configuration from an external source (i.e. Database, Azure App Configuration, etc), you are doomed. Your performance will be hurt a lot. One of the major points on this project is to have the proxy configuration dynamically loaded AND reloaded, so you can read in production to changes based on external circumstances and that would be totally killed if you have a cold start.

So its definite possible to make it work on azure functions with the exact same benefits/drawbacks as any other app has running on azure function if the azure function team wanted to.

Like I said, it is possible, but without bringing the proxy to the platform, in this case Azure Functions, it will not be a good thing IMHO. Look for SignalR Services as I mentioned before. If they didn't brought it inside Azure as a product but instead let you send a SignalR Hub connection request directly to an Azure Function, how would maintain the statefulness? You would be basically create 1 connection per Hub method Invokation which totally blow the whole purpose of stateful connections with WebSockets.
Again, there are ways to do it. Not technically impossible. The project is very well designed and abstracted BUT, you have to take in consideration the constraints that serverless hosts bring to you. By definition, serverless functions, like Azure Functions, aren't meant to keep stateful connections.
A reverse proxy doesn't just handle REST APIs... What about HTTP/2? WebSockets? Session/cookie Affinity? Where do you put the state (ofc, besides having an external state storage like Redis or something else which again, bloat the use case)?
Those are the answers that you don't have for this particular scenarion.
But again, like I said before, perhaps using the Azure Functions Premium SKU would reduce you problems with cold-start (without a custom Functions Host) or, fully account for the DI and custom configuration dependencies that I've mentioned (with a custom host in a Docker container).

Just to give an idea o how messy is the DI implementation on it, look at https://github.com/Azure/azure-functions-host/issues/4413 and like I said, have a look on DependencyInjection as a search in the Azure Functions related repositories and you will see that DI is a hot topic of problems on it. You can now guess why! 💃

Running the proxy within Azure functions is not a scenario we plan to support in 1.0. Adding to the backlog to reconsider later.

I am running it on azure function now for my hobby projects, consumption plan using a wrapper.

The startup is all the same as a normal startup of aspnetcore app, and if a project becomes popular enough that azure functions do not performance well enough, its just a switch to deploy to a normal webapp.
Not having any of the issues @galvesribeiro is mentioning - maybe only a few extra ms overheads of running on azure function vs raw IIS.

@pksorensen do you have a repository you could share?

I am running it on azure function now for my hobby projects, consumption plan using a wrapper.
The startup is all the same as a normal startup of aspnetcore app, and if a project becomes popular enough that azure functions do not performance well enough, its just a switch to deploy to a normal webapp.
Not having any of the issues @galvesribeiro is mentioning - maybe only a few extra ms overheads of running on azure function vs raw IIS.

@pksorensen can you share a sample? I think it deserves an article :) Thanks!


Bump Flutter packages

Closes #2158 #2160

Summary by Sourcery
This pull request introduces several new features and enhancements across multiple components. Key additions include advanced tooltip configurations for charts, new event handling for markdown selection changes, and improved image handling in markdown controls. Additionally, the audio recorder functionality has been extended with a cancel recording feature, and several dependencies have been updated.

New Features:

Added support for various tooltip configurations in BarChart and LineChart, including rounded radius, margin, padding, max content width, rotate angle, horizontal offset, border side, fit inside horizontally and vertically, and show on top of chart box area.
Introduced new MarkdownSelectionChangeEvent and MarkdownSelectionChangedCause classes in Python SDK for handling markdown selection change events.
Added new properties and event handlers for Markdown control in Python SDK, including shrink_wrap, fit_content, soft_line_break, img_error_content, on_tap_text, and on_selection_change.
Implemented imageBuilder in MarkdownControl to handle base64 and URL images, including SVG support and error handling.

Enhancements:

Refactored MarkdownControl to use parseMarkdownExtensionSet for setting markdown extension sets.
Enhanced audio recorder functionality by adding cancel_recording method in both Dart and Python SDKs.
Updated dependencies in pubspec.yaml for various packages including flutter_markdown, file_picker, shared_preferences, and fl_chart.

@sourcery-ai review


OAuth 2.0: Allow theme with OAuth 2.0

Brief description of changes applied
Theme with OAuth (I reckon OIDC & SAML as well) is not functional at the moment (I tested it with 5.2.x)
This PR consist a change that can let OAuth have theme, do note that this PR is very rough.
For seeing how to code apply, continue below:
For example:
{
"@class" : "org.apereo.cas.support.oauth.services.OAuthRegisteredService",
"clientId": "awesomeoauth",
"clientSecret": "secr3t",
"serviceId" : "^https://www.example.com/.*",
"name" : "awesomeservice",
"id" : 1000,
"evaluationOrder" : 1000,
"supportedResponseTypes" : [ "java.util.HashSet", [ "code" ] ],
"supportedGrantTypes" : [ "java.util.HashSet",  [ "authorization_code" , "refresh_token"] ],
"theme" : "awesomeoauththeme"
}
When accessing the above with OAuth, we are not accessing the below url:

https://mycas.example.com?service=https://www.example.com/

Instead we are accessing something like this:

https://mycas.example.com/cas/login?service=https%3A%2F%mycas.example.com%3A443%2Fcas%2Foauth2.0%2FcallbackAuthorize%3Fclient_name%3DCasOAuthClient%26client_id%3Dawesomeoauth%26redirect_uri%3Dhttps%253A%252F%252Fwww.example.com%252F12345%26response_type%3Dcode

Which does not trigger the OAuth theme, which is not ideal
There probably are many ways to implement OAuth theming, but in this PR I used the following method to make OAuth with theme works:

Create a new theme resolver "CasOAuthThemeResolver"
Check if the service starts with "https://mycas.example.com:443/cas/oauth2.0/callbackAuthorize", if yes proceed, no then stop
Get the parameter REDIRECT_URI inside the service, (e.g. https://www.example/com)
Find the original service by comparing the REDIRECT_URI
Extract the theme parameter from the above service ("name" : "awesomeservice") (i.e. "awesomeoauththeme")
Return this theme parameter as the resolved theme for this resolver

See if the above flow seems ok or not for resolving the CAS theme?
CAS OAuth with theme is used multiple time in my project, so it would be awesome if it was supported officially, thanks!
Any documentation on how to configure, test
Any possible limitations, side effects, etc
I have only tested this in CAS 5.2.0, and I do not know if this will have any sort of performance hit
Also the code here is very messy, but I can try clean it up if this behavior is approve to be added to CAS
this PR is aiming for 5.3.0 or later.
Reference any other pull requests that might be related.

This change is unnecessary since the behavior you're after is already provided by 5.3. Please test with 5.3 and report back.

@mmoayyed Sorry for not testing with the latest version before submitting this PR.
I have now tested it with the 5.3.0-RC3 and CAS OAuth theme definitely works, thanks!
I will close this ticket now

Wonderful. No problem. Thank you very much for the feedback.


requirements.lock will not be updated

Output of helm version:
Client: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.0", GitCommit:"f6025bb9ee7daf9fee0026541c90a6f557a3e0bc", GitTreeState:"clean"}

Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-13T22:27:55Z", GoVersion:"go1.9.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): none
helm dependency update . works like expected and loads the version 2.1.0, but in the requirements.yaml is still the old 2.0.2 version written. So its not possible to use helm dependency build during CI/CD.
Our requirements.yaml:
dependencies:
  - name: common
    repository: "@sedo"
    version: ^2.0.0

It will also not work with helm 2.7.0.

Seeing exact the same issue on our deployments ...

I presume that deleting the lock file and rewriting will add the correct version? Sounds like something is not correctly regenerating the lock.

Your right. The lock file will also be updated if you change the version.

any news on this?

I'm experiencing the same issue.

I'm also experiencing this, working on debugging it and proposing a fix.

The issue is here: https://github.com/helm/helm/blob/master/pkg/downloader/manager.go#L159-L163
It checks if the hash of requirements.yaml changed, not requirements.lock when deciding if it should write requirements.lock which doesn't make sense if you run with loose dependencies in requirements.yaml; for example we use the format ^N.N.x to pull in the latest minor release of a chart automatically in our CI/CD environment.  I'll work on a PR to fix this, until then the workaround as mentioned above is to delete requirements.lock before run helm dep up.

duplicate of #2731.


To support longer assembly names in the core dbs mapping_session table.

VARCHAR(20) is too short for some assembly names.
Requirements

Filling out the template is required. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion;
Review the contributing guidelines for this repository; remember in particular:

do not modify code without testing for regression
provide simple unit tests to test the changes
if you change the schema you must patch the test databases as well, see Updating the schema
the PR must not fail unit testing

Description
Using one or more sentences, describe in detail the proposed changes.
Increased length of old_assembly and new_assembly columns in the mapping_session table in the core databases.
Use case
Describe the problem. Please provide an example representing the motivation behind the need for having these changes in place.
Xenopus tropicalis assembly name would appear truncated as "Xenopus_tropicalis_v" in the current database schema.
Benefits
If applicable, describe the advantages the changes will have.
Longer assembly names would be supported.
Possible Drawbacks
If applicable, describe any possible undesirable consequence of the changes.
More space would be required.
Testing
Have you added/modified unit tests to test the changes?
No. I think the existing unit tests should work.
If so, do the tests pass/fail?
N/A.
Have you run the entire test suite and no regression was detected?
No.

that's a lot of pushing...

done now I think, sorry I thought there would be fewer files to modify in the test databases directory.

reminder to whoever merges this:
do a squash and merge instead of the usual merge commit

I couldn't find that option on github.

sql/table.sql is included.
Yes, I updated the test dbs files manually. I thought it was easier than running the script. If you could provide me with some example command line that would be helpful.

this should still be valid:
https://www.ebi.ac.uk/seqdb/confluence/display/ENSCORE/Patching+test+databases
by default the patches does all dbs, so you can restrict to core only with -types core flag
that would be something like  perl ../ensembl-test/scripts/patch_test_databases.pl -types core -curr_dir modules/t
all test db patching would then be applied in a single commit

Thank you. I'll do that then.


peacecorps.oig.gov has incorrect agency field

It's listed as the USPS OIG for the Agency field (the Organization field is correct).

Where do you see peacecorps.oig.gov?


Use today's date by default

You say in your TODO you want to add a default date.
Why not adding an argument parser and add a calculation for a date instead of using sys.argv[i]?

This would be a useful improvement. Currently DATE is defined as a bash variable so it can be used in the git commit message (see https://github.com/tomwhite/covid-19-uk-data#daily-workflow), so I'm not sure the best way to keep these consistent.

I'd try this
parser.add_argument("-s", "--start-date", dest="date",
default=date.today() - timedelta(days = 1),
type=lambda d: datetime.strptime(d, '%Y%m%d').date(),
help="Date in the format yyyymmdd")
You could then call the argument directly at the end with the date variable

I fixed this by creating a little wrapper script (update.sh).


"server_state" is full, but "complete_ledgers" is empty (Version: 2.1.0-rc1)

Issue Description
On a host with no existing db, installing rippled and waiting for the db to sync up, results in "server_state" full, but "complete_ledgers" empty.
Steps to Reproduce

Install rippled 2.1.0-rc1
Bring up rippled to join devnet
Wait for db to sync up

seq matching with public devnet dns
server_state=full/proposing

Expected Result
As in https://xrpl.org/connect-your-rippled-to-the-xrp-test-net.html#5-verify-that-your-server-syncs, wait for "seq" from local host to match with public network dns. Ideally the db should be synced now.
Additionallywaiting for "server_state" to be "full" confirms the db is synced
Actual Result
After "seq" matched with public dns, and "server_state" is full, the issue is, "complete_ledgers" is "empty"
Wait for rippled db to sync up (attempt: 1/2)...
  [Time elapsed: 0 seconds] Sequence:  2 (local) vs  4077157 (devnet - <IP_ADDRESS>)
  [Time elapsed: 20 seconds] Sequence:  4 (local) vs  4077164 (devnet - <IP_ADDRESS>)
  [Time elapsed: 40 seconds] Sequence:  11 (local) vs  4077171 (devnet - <IP_ADDRESS>)
  [Time elapsed: 60 seconds] Sequence:  11 (local) vs  4077177 (devnet - <IP_ADDRESS>)
  [Time elapsed: 80 seconds] Sequence:  11 (local) vs  4077184 (devnet - <IP_ADDRESS>)
  [Time elapsed: 100 seconds] Sequence:  13 (local) vs  4077191 (devnet - <IP_ADDRESS>)
  [Time elapsed: 120 seconds] Sequence:  17 (local) vs  4077197 (devnet - <IP_ADDRESS>)
  [Time elapsed: 140 seconds] Sequence:  17 (local) vs  4077204 (devnet - <IP_ADDRESS>)
  [Time elapsed: 160 seconds] Sequence:  17 (local) vs  4077211 (devnet - <IP_ADDRESS>)
  [Time elapsed: 180 seconds] Sequence:  24 (local) vs  4077217 (devnet - <IP_ADDRESS>)
  [Time elapsed: 200 seconds] Sequence:  24 (local) vs  4077224 (devnet - <IP_ADDRESS>)
  [Time elapsed: 220 seconds] Sequence:  24 (local) vs  4077231 (devnet - <IP_ADDRESS>)
  [Time elapsed: 240 seconds] Sequence:  26 (local) vs  4077237 (devnet - <IP_ADDRESS>)
  [Time elapsed: 260 seconds] Sequence:  31 (local) vs  4077244 (devnet - <IP_ADDRESS>)
  [Time elapsed: 280 seconds] Sequence:  31 (local) vs  4077251 (devnet - <IP_ADDRESS>)
  [Time elapsed: 300 seconds] Sequence:  31 (local) vs  4077258 (devnet - <IP_ADDRESS>)
  [Time elapsed: 320 seconds] Sequence:  36 (local) vs  4077264 (devnet - <IP_ADDRESS>)
  [Time elapsed: 340 seconds] Sequence:  38 (local) vs  4077271 (devnet - <IP_ADDRESS>)
  [Time elapsed: 360 seconds] Sequence:  38 (local) vs  4077278 (devnet - <IP_ADDRESS>)
  [Time elapsed: 380 seconds] Sequence:  38 (local) vs  4077284 (devnet - <IP_ADDRESS>)
  [Time elapsed: 400 seconds] Sequence:  44 (local) vs  4077291 (devnet - <IP_ADDRESS>)
  [Time elapsed: 420 seconds] Sequence:  44 (local) vs  4077297 (devnet - <IP_ADDRESS>)
  [Time elapsed: 440 seconds] Sequence:  44 (local) vs  4077304 (devnet - <IP_ADDRESS>)
  [Time elapsed: 460 seconds] Sequence:  49 (local) vs  4077311 (devnet - <IP_ADDRESS>)
  [Time elapsed: 480 seconds] Sequence:  49 (local) vs  4077317 (devnet - <IP_ADDRESS>)
  [Time elapsed: 500 seconds] Sequence:  49 (local) vs  4077324 (devnet - <IP_ADDRESS>)
  [Time elapsed: 520 seconds] Sequence:  52 (local) vs  4077331 (devnet - <IP_ADDRESS>)
  [Time elapsed: 540 seconds] Sequence:  54 (local) vs  4077337 (devnet - <IP_ADDRESS>)
  [Time elapsed: 560 seconds] Sequence:  54 (local) vs  4077344 (devnet - <IP_ADDRESS>)
  [Time elapsed: 580 seconds] Sequence:  55 (local) vs  4077351 (devnet - <IP_ADDRESS>)
  [Time elapsed: 600 seconds] Sequence:  59 (local) vs  4077357 (devnet - <IP_ADDRESS>)
  [Time elapsed: 620 seconds] Sequence:  59 (local) vs  4077364 (devnet - <IP_ADDRESS>)
  [Time elapsed: 640 seconds] Sequence:  59 (local) vs  4077371 (devnet - <IP_ADDRESS>)
  [Time elapsed: 660 seconds] Sequence:  4077377 (local) vs  4077377 (devnet - <IP_ADDRESS>)

**** server_state: "full", complete_ledgers: "empty",

Environment
devnet
Supporting Files
rippled.log: https://gist.github.com/sgramkumar/ad5e49ad9ffd3648799aa3f7155bb45f
server_info output
Loading: "/opt/ripple/etc/rippled.cfg"
2024-Feb-09 08:27:07.771282325 UTC HTTPClient:NFO Connecting to <IP_ADDRESS>:51234

{
   "result" : {
      "info" : {
         "build_version" : "2.1.0-rc1",
         "complete_ledgers" : "empty",
         "hostid" : "runner-6s2nbvqf-project-1346-concurrent-0",
         "initial_sync_duration_us" : "665368053",
         "io_latency_ms" : 1,
         "jq_trans_overflow" : "0",
         "last_close" : {
            "converge_time_s" : 2,
            "proposers" : 10
         },
         "load" : {
            "job_types" : [
               {
                  "in_progress" : 1,
                  "job_type" : "clientRPC"
               },
               {
                  "job_type" : "untrustedValidation",
                  "per_second" : 3
               },
               {
                  "in_progress" : 1,
                  "job_type" : "updatePaths"
               },
               {
                  "avg_time" : 1,
                  "job_type" : "ledgerData",
                  "peak_time" : 26,
                  "per_second" : 29
               },
               {
                  "job_type" : "advanceLedger",
                  "peak_time" : 2,
                  "per_second" : 3
               },
               {
                  "job_type" : "trustedValidation",
                  "per_second" : 2
               },
               {
                  "job_type" : "trustedProposal",
                  "per_second" : 1
               },
               {
                  "job_type" : "peerCommand",
                  "per_second" : 37
               },
               {
                  "job_type" : "SyncReadNode",
                  "per_second" : 28351
               },
               {
                  "job_type" : "AsyncReadNode",
                  "peak_time" : 5,
                  "per_second" : 13467
               },
               {
                  "job_type" : "WriteNode",
                  "peak_time" : 1,
                  "per_second" : 6125
               }
            ],
            "threads" : 6
         },
         "load_factor" : 1,
         "network_id" : 2,
         "node_size" : "medium",
         "peer_disconnects" : "19",
         "peer_disconnects_resources" : "0",
         "peers" : 1,
         "ports" : [
            {
               "port" : "51234",
               "protocol" : [ "http" ]
            },
            {
               "port" : "51235",
               "protocol" : [ "peer" ]
            },
            {
               "port" : "6006",
               "protocol" : [ "ws" ]
            }
         ],
         "pubkey_node" : "n9JZ4YtPxFfD9o3VsrqZrXaKyBCEB7p1f3Hafh6WzBKhCtAND7cM",
         "pubkey_validator" : "none",
         "server_state" : "full",
         "server_state_duration_us" : "1024720",
         "state_accounting" : {
            "connected" : {
               "duration_us" : "174130926",
               "transitions" : "15"
            },
            "disconnected" : {
               "duration_us" : "490236184",
               "transitions" : "15"
            },
            "full" : {
               "duration_us" : "1024720",
               "transitions" : "1"
            },
            "syncing" : {
               "duration_us" : "1000922",
               "transitions" : "1"
            },
            "tracking" : {
               "duration_us" : "5",
               "transitions" : "1"
            }
         },
         "time" : "2024-Feb-09 08:27:07.771664 UTC",
         "uptime" : 666,
         "validated_ledger" : {
            "age" : 3,
            "base_fee_xrp" : 1e-05,
            "hash" : "331471338D1CE8D3FA81836C609C005952529DE1A70B02CD47E5D43F6BB791D4",
            "reserve_base_xrp" : 10,
            "reserve_inc_xrp" : 2,
            "seq" : 4077377
         },
         "validation_quorum" : 8,
         "validator_list" : {
            "count" : 1,
            "expiration" : "2031-Aug-07 19:36:22.000000000 UTC",
            "status" : "active"
         }
      },
      "status" : "success"
   }
}

Setup pipeline to capture the transition from "empty" to "non-empty" when server_state is full
Example 1:
..
  [Time elapsed: 160 seconds] Sequence:  4494078 (local) vs  4494078 (devnet - <IP_ADDRESS>)
**** Sat Feb 24 03:05:12 UTC 2024<PHONE_NUMBER>.531117) server_state: "full", complete_ledgers: "empty",
**** Wait for 'complete_ledgers' to be non-empty
..
..
**** Sat Feb 24 03:06:28 UTC 2024<PHONE_NUMBER>.929838) server_state: "full", complete_ledgers: "4494056-4494103",

Time taken for the transition: 3:05:12 to 3:06:28. That is 1 minutes 16 seconds
Example 2:
  [Time elapsed: 240 seconds] Sequence:  4517165 (local) vs  4517165 (devnet - <IP_ADDRESS>)
**** Sat Feb 24 22:49:55 UTC 2024<PHONE_NUMBER>.562676) server_state: "full", complete_ledgers: "empty",
**** Wait for 'complete_ledgers' to be non-empty
..
**** Sat Feb 24 22:50:50 UTC 2024<PHONE_NUMBER>.555848) server_state: "full", complete_ledgers: "4517153-4517155",

Time taken for the transition: 22:49:55 to 22:50:50. That is 55 seconds

Thank you for finding that it can take a significant amount of time (about a minute) for a synced server to get complete ledgers.
When using rippled, clients should generally wait for complete_ledgers to be non-empty before proceeding with RPC calls/queries. This should be clarified in the docs.
Alternatively, we could make a change to the definition of server_state: full. The potential change would be to not report full until complete_ledgers is non-empty.

I may have a little insight that I can contribute here.  I'm going to guess that the syncing @sgramkumar is doing is without a pre-existing database.  If that's incorrect, then the following conjecture is off base.
I very seldom run rippled without a pre-existing database.  But just today rippled told me that my database was corrupt, so I wiped my database.  The startup behavior I saw, without a pre-existing database, was surprising to me.  The server reported that it was proposing very quickly, like under a minute.  But it wasn't stable.  Before it got around to having a "published_ledger" it had 270 transitions out of "full".  I never see anything like that when I have a pre-existing database.  If I have a pre-existing database I'll have at most 2 or 3 transitions out of "full".
I suspect this is relatively new behavior?  Like, new this year?  I have started rippled with a fresh database in the past and it takes a while to sync, but it didn't jump to "full" until it was actually ready.
What's relevant for this issue is that we may be looking at behavior that is new and only occurs when starting without a database.


In local vS it is authenticating the User , when deploying to azure it is not

In local VS it is authenticating the User and in webApi able to get the claims for the loggedInuser , when deploying it to Azure environment (Web on cloudservice and Service on WebSites) it is not authenticating the user and throwing the user "Authentication has been denied for this user".

if the client and the web API are on different URLs, the basic logon won't work. See https://github.com/AzureADSamples/SinglePageApp-WebAPI-AngularJS-DotNet for an example in which a SPA invokes a service on a different origin

Thank you ,for quick response ,
I followed the AzureADSamples/SinglePageApp-WebAPI-AngularJS-DotNet sample , integrating with my actual solution on local VS it worked fine but on deployment to azure it is not sending the token to webApi service.
also I am using adal.js v 0.0.1 not 1.0.0 do you think it can be problem ?

Hi. I'm not sure if anyone is still watching this issue but if so could you clarify whether this scenario of having different origins the SPA and API can be made to work?

Possibly related to this issue: https://github.com/Azure-Samples/active-directory-angularjs-singlepageapp-dotnet-webapi/issues/12 ?

Closing this as we are archiving this sample. Please check out the new sample at ms-identity-javascript-angular-spa-aspnetcore-webapi.


Switch for calldeflection

Hello.
Thank you for working on this integration.
Would it be possible for you to add a switch, to turn on/off the calldeflection?
See here:
https://gist.github.com/fake666/75c035c74b7be448dd9b3d1316ea6cfe

Thanks for your suggestion.
Your linked component uses the same library (fritzconnection) as we do, so it should be pretty easy to integrate it here.
Pull requests welcome :blush:

Note the following comment from the gist:

removing and adding calldeflections while homeassistant is running will break this ;-)

This should probably be avoided if possible.

Thank you very much for your fast response.
Do you mean, you can do the integration?
I would be happy to help on this with testing.
But I have really no clue from coding.

Do you mean, you can do the integration?

It should be easily doable from a technical point of view.
I'm not quite sure whether we should add it here, or rather wait until we merged this integration into HA core... (related #29)

I think, you should add it here.
So we can test it, before merging in the HA core.


Using script tags with absolute URLs as src attribute require a webc:keep

Hello @zachleat,
I used a <script> tag in the <head> of my main page with a type="module" and an absolute URL as source src="https://example.com". This didn't work and I had to add a webc:keep.
I think I expected WebC to let them as is as it already does with link tags referring to external CSS files.
WDYT?

Yeah, this is definitely a bug!

This will ship with 0.2.0 tomorrow!

You will be able to use <script src> and <link rel="stylesheet" href> to reference external assets and they’ll be compiled back into the JS and CSS for the components!
Or, you can use webc:keep to leave them as-is.


English translation!

Can't understand what is written! google translate doesn't work well
the only thing I could understand from the messy translation google provided is you made an ISO of around 600 DOS games
is that right?
if so thank you but please also provide English Instructions!

This is a 64 in 1 DOS game collection, not 600 in 1.
The instruction is quite simple, just decompress the DOS.rar to any where you like and run:
DFend.exe

Beware:

half of the games are not in English.
game info and descriptions are not in English.

half of the games are not in English.
game info and descriptions are not in English.

That's sad! any way to translate them!

haha, foreign friends Welcome!


Land cover standardization

First Stab at the land cover standardization PR.
Fine-tuning, testing, and some pruning still to do, but I am at a stage where PR can be submitted.
Description:
This PR primarily adds the reclassify_land_cover.R file containing 1 function reclassify_land_cover().
Functions works with NLCD and the FORESCE input datasets.
Takes a land cover dataset and reclassifies it's land cover classes to a standardized classification, detailed in a pre-defined reclassification table (current classification has 9 classes, vs 20+ in NLCD). The reclassification tables have been also added to 1_fetch/in/
Output example:
NLCD:
# A tibble: 458 × 11
   PRMS_segid AREASQKM_PRMS prop_NLCD11_1 prop_NLCD11_4 prop_NLCD11_5 prop_NLCD11_6 prop_NLCD11_7 prop_NLCD11_8 prop_NLCD11_2 prop_NLCD11_3 prop_NLCD11_9
   <chr>              <dbl>         <dbl>         <dbl>         <dbl>         <dbl>         <dbl>         <dbl>         <dbl>         <dbl>         <dbl>
 1 1_1                65.0           0             0.03           0            0.85          0             0             0.08          0.02          0.01
 2 10_1                5.47          0.02          0.05           0            0.76          0             0.01          0.09          0.02          0.05
 3 11_1                5.94          0.04          0.03           0.1          0.72          0             0.01          0.09          0             0.01
 4 111_1              41.6           0             0.02           0            0.62          0.01          0             0.28          0.07          0
 5 112_1              84.1           0             0.05           0            0.6           0             0.01          0.23          0.08          0.02
 6 113_1               3.40          0             0.2            0            0.63          0.02          0.01          0.03          0.07          0.02
 7 114_1              54.5           0             0.04           0            0.74          0.01          0.02          0.15          0.03          0.01
 8 115_1              41.7           0.01          0.07           0            0.56          0             0.01          0.24          0.07          0.04
 9 116_1              41.8           0             0.03           0            0.74          0             0.01          0.2           0.02          0
10 117_1              12.2           0             0.01           0            0.87          0             0.01          0.1           0.01          0

FORESCE:
# A tibble: 761 × 22
    POI_ID hru_id hru_segment_orig hru_segment_comment    hru_x   hru_y hru_lat hru_area hru_segment region Shape_Length Shape_Area    ID prop_lcClass_3 prop_lcClass_2 prop_lcClass_1 prop_lcClass_4 prop_lcClass_5 prop_lcClass_6
     <int>  <int>            <int> <chr>                  <dbl>   <dbl>   <dbl>    <dbl>       <int> <chr>         <dbl>      <dbl> <int>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>
 1 8077930   2846             1263 NA                  1776975.  1.94e6    38.8   15316.        1263 02           64863.  61982133.     1          0.376        0.0156         0.0233         0.126         0.00476          0.155
 2 8078046   2847             1264 NA                  1772627.  1.95e6    38.8     387.        1264 02            8369.   1566006.     2         NA           NA              0.0682         0.00573       0.00115         NA
 3 8077930   2848             1263 NA                  1774358.  1.95e6    38.8    1210.        1263 02           21943.   4896969.     3         NA           NA              0.0872        NA             0.0112           0.0167
 4 8078046   2849             1264 NA                  1766858.  1.94e6    38.8   39139.        1264 02          106082. 158390574.     4          0.604        0.0203         0.0116         0.0381        0.00320          0.126
 5 8078046   2850             1264 NA                  1765460.  1.95e6    38.8    5557.        1264 02           47628.  22488083.     5         NA            0.0273         0.0399         0.0184        0.00168          0.0703
 6 8077030   2851             1260 NA                  1744986.  1.95e6    38.9     544.        1260 02           11827.   2199987.     6         NA            0.153         NA             NA             0.000408         0.107
 7 8077030   2852             1260 NA                  1745716.  1.95e6    38.9    1843.        1260 02           19498.   7457258.     7          0.699        0.0769        NA              0.0112       NA                0.0386
 8 8077776   2853             1262 NA                  1742164.  1.96e6    39.0    1756.        1262 02           16598.   7105953.     8         NA            0.0533         0.0199         0.0210        0.000127         0.0951
 9 8077776   2854             1262 NA                  1741778.  1.96e6    39.0     433.        1262 02            7937.   1752067.     9         NA            0.00874        0.0802         0.0612        0.000514         0.0514
10 8076836   2856             1259 NA                  1752208.  1.95e6    38.9   18485.        1259 02           72087.  74805296.    10          0.612        0.0153         0.00883        0.106         0.00259          0.106

Next steps:

[x] Check p2_NLCD_lc_PRMS_proportions dims, appears to be missing 287_1 despite fixing that in #38 merge.
[x] Handling the NA land cover class values. --> proposing to change NA to 0
[x] Simplification function and combine dplyr piping
[ ] FORESCE should be aggregated to PRMS_segid vs. POI_id. To fix after or before as added target in pipeline
[ ] Generalize naming etc - this can be handy for other PUMP tasks

Handling the NA land cover class values.

Are the pixel values with NA the same in every year?

Are the pixels with NA the same in every year?

@jds485  - No. There are different number of NA pixels in each of the processed FORESCE raster, 1 every decade.
For example, take looking at the difference between the Land Cover FORESCE tif file for 1970 an 1980 (sample called: final_raster_table_70 and final_raster_table_80 respectively):
final_raster_table_70[6, lcClass 10] is NA in 1970 while final_raster_table_80[6, lcClass 10] is 0.001633320 in 1980.
> final_raster_table_70
     lcClass0    lcClass1  lcClass2    lcClass3   lcClass4   lcClass5   lcClass6   lcClass8   lcClass9    lcClass10    lcClass11    lcClass12   lcClass13    lcClass14    lcClass15   lcClass16  lcClass17  lcClass18    lcClass19 ID
4 0.015057845 0.051992181 0.2444883 0.023512967 0.03370116 0.05142396 0.12584381 0.05828238 0.01170534 0.0127167761 0.0018012592 0.0027785985 0.110467759 0.0173136805 0.0117223902 0.013671387 0.07488579 0.08506262 0.0535718344  4
5          NA 0.009005764 0.2690922 0.009045789 0.03081972 0.06187960 0.10138489 0.07152578 0.04002562 0.0109269933 0.0010806916 0.0020813321 0.074247518 0.0017211015 0.0013208453 0.004042587 0.10826929 0.06123919 0.1422910663  5
6          NA 0.020416497 0.1784402 0.078399347 0.15353205 0.21804818 0.12372397         NA         NA           NA           NA 0.0004083299 0.062066149 0.0073499388 0.0028583095 0.008574929 0.03429971 0.11147407 0.0004083299  6

> final_raster_table_80
   lcClass0   lcClass1  lcClass2   lcClass3   lcClass4   lcClass5   lcClass6   lcClass8   lcClass9   lcClass10    lcClass11    lcClass12   lcClass13    lcClass14   lcClass15   lcClass16  lcClass17  lcClass18    lcClass19 ID
4 0.015063527 0.06722049 0.2526593 0.06701593 0.02631429 0.04976476 0.11475782 0.05828238 0.01160875 0.016239744 0.0019717253 0.0025285815 0.078783781 0.0172398118 0.007920995 0.014410073 0.06097007 0.08653431 0.0507136850  4
5          NA 0.02713737 0.2694524 0.03542267 0.03045949 0.04370797 0.10138489 0.07152578 0.03994557 0.012287864 0.0016810759 0.0016410503 0.047030099 0.0017211015 0.001360871 0.004042587 0.10758886 0.06560199 0.1380083253  5
6          NA 0.02041650 0.1784402 0.07839935 0.15353205 0.21804818 0.12127399         NA         NA 0.001633320           NA 0.0004083299 0.061249490 0.0073499388 0.003266639 0.008574929 0.03429971 0.11269906 0.0004083299  6

The raster_to_catchment_polygons() function that processes the FORESCE tif files uses terra::extract() to output a vector of all raster pixel values within a given polygon:
i.e. hru segment :: 2163 is associated with a vector of raster pixel values c(19, 9,6,4,5,19,19,19,....)
This is then summarized to get the frequency of each class type and then the proportion of that class with respect to the total number of pixels in the polygon. When summarizing, the classes that are not found in given polygon get value of NA.
I will adjusted the raster_to_catchment_polygons.R to change NA to 0, and will push with other updates the LandCover_standardization branch and PR

Thanks for checking this. Did you check why the NAs are appearing for FORESCE (looks like NLCD is producing 0s)? Does NA mean that the land cover was not present in that HRU that year?

Thanks for checking this. Did you check why the NAs are appearing for FORESCE (looks like NLCD is producing 0s)? Does NA mean that the land cover was not present in that HRU that year? Sorry, you said that. Sounds good to convert NA to 0

Yup, NLCD which is already in a tabular format for HRU catchments, handles classes that are not present that year as 0. I've fixed the rbind.filll.list function we use to rbind the summarized land cover classes per polygon
into dataframe, fills empty classes with 0 not NA.

This PR is ready for Review:
Not sure who has more availability for review but
This PR standardizes the NLCD land Cover data and the FORE-SCE land cover data.
add a couple new targets.
Description of PR above.

Improvements since initial PR submission:

rbind.fill.list() in raster_to_catchment_polygons.R   now fill missing vals (i.e. classes that dont exist in given polygon) with 0 rather than NA
grouped FORE-SCE dataset to PRMS_segid scale. This was done by aggregating to hru_segment, and mutating new PRMS_segid just by concatenating hru_segment id + _1. (too easy ? 😆 )
simplified reclassify_land_cover() by combine two pipes.
streamlines rd_salt processing steps by removinf p2_rdsalt_per_catchment_grped and adding that process step to target p2_rdsalt_per_catchment

Output:
head(p2_PRMS_NLCD_lc_proportions_reclass)

# A tibble: 459 × 11
   PRMS_segid AREASQKM_PRMS prop_lcClass_1 prop_lcClass_4 prop_lcClass_5 prop_lcClass_6 prop_lcClass_7 prop_lcClass_8 prop_lcClass_2 prop_lcClass_3 prop_lcClass_9
   <chr>              <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>
 1 1_1                65.0            0              0.03            0             0.85           0              0              0.08           0.02           0.01
 2 10_1                5.47           0.02           0.05            0             0.76           0              0.01           0.09           0.02           0.05
 3 11_1                5.94           0.04           0.03            0.1           0.72           0              0.01           0.09           0              0.01
 4 111_1              41.6            0              0.02            0             0.62           0.01           0              0.28           0.07           0
 5 112_1             271.             0              0.03            0             0.64           0              0.01           0.25           0.05           0.01
 6 113_1              21.1            0              0.07            0             0.55           0.01           0.01           0.32           0.03           0

head(p2_FORESCE_LC_per_catchment_reclass[5])

[[5]]
# A tibble: 416 × 11
   PRMS_segid hru_segment prop_lcClass_3 prop_lcClass_2 prop_lcClass_1 prop_lcClass_4 prop_lcClass_5 prop_lcClass_6 prop_lcClass_7 prop_lcClass_8 prop_lcClass_9
   <chr>            <int>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>          <dbl>
 1 1_1                  1         0.127       0               0.00408         0.0443       0.000112           1.78         0.0138       0.00126          0.0252
 2 2_1                  2         0.0809      0               0.00110         0.0373       0.000193           1.85         0.00873      0.000166         0.0175
 3 3_1                  3         0.158       0.0000211       0.00347         0.128        0.000162           2.65         0.0111       0.0000528        0.0528
 4 4_1                  4         0.0183      0               0.0146          0.0911       0.000818           1.80         0.00644      0                0.0723
 5 5_1                  5         0           0               0.144           0.0691       0.0779             1.61         0.00564      0.0429           0.0495
 6 6_1                  6         0.0988      0               0.00311         0.0180       0.0000415          0.855        0.00914      0.00414          0.0122

The column names should include the year.

grouped FORE-SCE dataset to PRMS_segid scale. This was done by aggregating to hru_segment, and mutating new PRMS_segid just by concatenating hru_segment id + _1

Some of the PRMS segment IDs have _2, so I another approach may be needed here. I'd need to remind myself of the segment ID indexing to be sure. Did you check this?

From @lekoenig: Will this function eventually use the processed catchment polygons? There's a step to group_by HRU - does it make sense to add a step to dissolve those boundaries in the processing code?

I was expecting the row sum over all land cover classes to = 1 for p2_PRMS_NLCD_lc_proportions_reclass. Do you know what's causing the values that are not 1? Maybe it's how rounding is applied?

I explored this and noticed the same range. There is rounding that is applied so due to that. Let's me test this however.

Same question for FORESCE. Wondering why some proportions are 2 and 3
I did not get a chance to check sums actually, thanks for checking and catching this. I will look back at original file and see what's going on.

Should we use mean instead of sum when aggregated hru_segments?

Good catch! The aggregated proportion should be a weighted sum by area: (LCHRU1Area1 + LCHRU2Area2)/(Area1 + Area2)

@jds485

Some of the PRMS segment IDs have _2, so I another approach may be needed here. I'd need to remind myself of the segment ID indexing to be sure. Did you check this?

So you are correct, we have 3 PRMS_segid's that have a _2 suffix.
This makes the matching tricky, because technically some of our hru_segments have two PRMS_segids. This is the case for hru_segment:50 (PRMS_segid 50_1 and 50_2) as well as hru_segment:3, and hru_segment:8
Amny suggestions? (@lekoenig possibly you can speak to this ? )

Good catch! The aggregated proportion should be a weighted sum by area: (LCHRU1Area1 + LCHRU2Area2)/(Area1 + Area2)

[ ] Update raster group_by of reclassified FORESCE data taking both hru_segment and hru_area and use hru_area to calculate area weighted average when aggregating. Worth creating a mini function for the aggregation step and giving the option to change summarizing function. (link to code lines)

@jds485 see latest commit with new aggregation of lc class proportion that accounts for change in catchment area.
Can you review and let me know what needs changing?
Couple items to note.

Per my comment above, the dfs in p2_FORESCE_LC_per_catchment_reclass are larger after merging with the p2_PRMS_hru_segment target because some 2 hru_segemnts have >1 PRMS_segid. To resolve in the p2_PRMS_hru_segment target.
total_PRMS_area and hru_segment cols are kept in the dfs. E.g. p2_FORESCE_LC_per_catchment_reclass[[1]]$total_PRMS_area. Should they be removed ultimately? Note that p2_PRMS_NLCD_lc_proportions_reclass has an AREA col but not a hru_segment col.
How should we rbind the list of dfs for FORESCE? I was planning to add new col with year id and then rbind. So PRMS_segids will duplicated (5 times for the 5 decades).

Did you address adding the year to the output column names? And keeping the low intensity and med-high intensity developed areas as separate classes?

Yes, there are 10 classes, vs before there were 9.

For future commits with only comment or variable name changes, can you describe them as something like "update comment" or "update variable names"?

Replies to your comments:

handling total_PRMS_area and hru_segment cols in the dfs for p2_FORESCE_LC_per_catchment_reclass p2_PRMS_NLCD_lc_proportions_reclass

Both of these have the PRMS segment ID, so I don't think we need either area or hru segment columns in these final dfs.

How should we rbind the list of dfs for FORESCE

I think an rbind with a year column will be helpful to create land cover timeseries for each PRMS segment. Can you also add a year column to the NLCD data? I can file an issue to create the land cover timeseries for each segment

I think an rbind with a year column will be helpful to create land cover timeseries for each PRMS segment. Can you also add a year column to the NLCD data? I can file an issue to create the land cover timeseries for each segment

Add year col in 2_process.R#L145 mutate(Year = .y)

This results from reclassify_table_reclass_col being a character vector. all_of is needed in select when specifying character vectors. I made the same edit to the above select

I' ve fixed this now with {{}}

Correction: commit 693de50 removes p2_rdsalt_per_catchment_grped, not p2_rdsalt_per_catchment_allyrs


Not able to deploy an app on azure using App service

Does this occur consistently?
Repro steps:

Action: appService.Deploy
Error type: Error
Error Message: Failed to get status of deployment.
Version: 0.24.7
OS: linux
OS Release: 5.19.0-35-generic
Product: Visual Studio Code
Product Version: 1.76.0
Language: en

Call Stack
extension.bundle.js:2:2208177
rejected extension.bundle.js:2:2206767

Are you able to successfully deploy if you try the deployment again?

That's what I was looking for. Thanks for your help. It helped me a lot.
On Fri, Mar 17, 2023 at 4:25 AM Alex Weininger @.***>
wrote:

Are you able to successfully deploy if you try the deployment again?
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-azureappservice/issues/2415#issuecomment-1472867447,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AUC6P2IRJ3EW4DHPUJXDUJ3W4OK45ANCNFSM6AAAAAAVXULN3U
.
You are receiving this because you authored the thread.Message ID:
@.***>

Sorry that you are experiencing this issue, could you please add additional repro steps, thank you

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Cannot obtain good results for NLST dataset

Hi,
I am segmenting the openly available NLST lung CT dataset obtained from https://cloud.imi.uni-luebeck.de/s/pERQBNyEFNLY8gR .
The resulting segmentations of the ribs and vertebrae do not look good, is this to be expected?

Thank you for your help!

The images are not in standard orientation. The vertebrae should be at the lower end of the image. If you flip along the y-axis it works as expected.

ah thanks a lot! it works now


v0.11.1 installs image versions 0.11.0 by default

What happened:
[root@9ca498e2e26e shipyard]# subctl show versions --kubeconfig output/kubeconfigs/kind-config-cluster1
Cluster "cluster1"
COMPONENT                       REPOSITORY                                            VERSION
submariner                      quay.io/submariner                                    0.11.0
submariner-operator             quay.io/submariner                                    0.11.0
service-discovery               quay.io/submariner                                    0.11.0
 ✓ Showing versions

What you expected to happen:
[root@9ca498e2e26e shipyard]# subctl show versions --kubeconfig output/kubeconfigs/kind-config-cluster1
Cluster "cluster1"
COMPONENT                       REPOSITORY                                            VERSION
submariner                      quay.io/submariner                                    0.11.1
submariner-operator             quay.io/submariner                                    0.11.1
service-discovery               quay.io/submariner                                    0.11.1
 ✓ Showing versions

How to reproduce it (as minimally and precisely as possible):
make clusters
curl -sL https://get.submariner.io | VERSION=0.11.1 bash
subctl deploy-broker --kubeconfig output/kubeconfigs/kind-config-cluster1
subctl join --kubeconfig output/kubeconfigs/kind-config-cluster1 broker-info.subm
subctl show versions --kubeconfig output/kubeconfigs/kind-config-cluster1

Anything else we need to know?:
This will probably necessitate a bump to 0.11.2 as we dont want to overwrite existing 0.11.1 tags, so technically subctl version 0.11.2 should install 0.11.2
Best and easiest way to do so is simply initiate a 0.11.2 release since image version updating logic was fixed by https://github.com/submariner-io/releases/pull/273
Environment:

Diagnose information (use subctl diagnose all):
Gather information (use subctl gather)
Cloud provider or hardware configuration:
Install tools:
Others:

Do we know what caused this?

This is now fixed with 0.11.2.


Gradients with TensorArray fail after scatter of 0 size

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): ('v1.3.0-rc2-20-g0787eee', '1.3.0')
Python version: 2.7.12
Bazel version (if compiling from source): N/A
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: see below

Describe the problem
The following fails:
import tensorflow as tf

x_orig = tf.zeros([3], tf.float32)

ta = tf.TensorArray(tf.float32, size=1, element_shape=[3])
ta = ta.write(0, x_orig)
ta = ta.scatter(tf.range(0, 0), ta.gather(tf.range(0, 0)))
x = ta.read(0)
g, = tf.gradients(tf.reduce_sum(x), [x_orig])

sess = tf.Session()
print sess.run(g)

with this error:
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape <unknown> is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
         [[Node: gradients/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGatherV3 = TensorArrayGatherV3[dtype=DT_FLOAT, element_shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"](gradients/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/TensorArrayGradV3, range, gradients/TensorArrayScatter/TensorArrayScatterV3_grad/TensorArrayGrad/gradient_flow)]]

The above code works if ta = ta.scatter(tf.range(0, 0), ta.gather(tf.range(0, 0))), which gathers a 0x3 tensor and then scatters it into zero indices in ta, is removed.
This seems to happen because _TensorArrayScatterGrad does not (and cannot?) copy over the element_size from the Python TensorArray object that .scatter was called on.
I submitted #12742 as a fix, which works for the above snippet..

Assigning to @ebrevdo who is reviewing the related PR.

It has been 14 days with no activity and this issue has an assignee.Please update the label and/or status accordingly.


not_an_alternate error when assigning a player to play down a board using chesster

The captains guidelines for the 4545 league have the following instructions for assigning a player to play up or down a board:

If no alternate can be found, but a player can play up or down a board from your team (in line with the rules), you can assign them to play up using the command "assign <player> to board <b> during round <r> on <team name>" in #captains - additionally captains must inform the opposing captain and opponent player of the new assignment.

When attempting to use this command to assign a player to play down a board, however, we get the not_an_alternate error:

david252525 [10:21 AM]
@ chesster assign @david252525 to board 4 during round 7 on More Zero Than Alpha

chesster APP [10:23 AM]
I failed to update alternate assignment: Error: not_an_alternate
Please contact a moderator.

This error comes from heltour, more specifically api.py around line 315 (in the current master branch). I'm not familiar with the code base, but at first glance it looks like maybe it's checking if the player is playing up a board, but not playing down a board.
Expected behavior: captains should be able to assign their players to play down a board, as specified in the captains guidelines. Currently this seems to require moderator intervention.

Discussed with the mods and they would prefer to keep it this way. The captains guidelines should be updated.


feat: Add Gotify alerts

This pull request adds Gotify as an alert target. It's common in homelab environments but we also use it with Proxmox VE for alerts. Adding this would reduce the number of places we have to check for alerts. This request should also close #2383.
These changes are very similar to those for Rocket.Chat(#1836) and DingTalk(#2363). It does not include any changes to setup.go for semaphore setup because I saw they were removed from the DingTalk pull request.
This request adds three new config variables.
GotifyAlert  bool   `json:"gotify_alert,omitempty" env:"SEMAPHORE_GOTIFY_ALERT"`
GotifyUrl    string `json:"gotify_url,omitempty"   env:"SEMAPHORE_GOTIFY_URL"`
GotifyToken  string `json:"gotify_token,omitempty" env:"SEMAPHORE_GOTIFY_TOKEN"`

It also adds a template for the json request (services/tasks/templates/gotify.tmpl) and a function TaskRunner.sendGotifyAlert() which is called the same as other alerts in TaskRunner.SetStatus().
This has been tested in a local development version of Semaphore UI with our production version of Gotify.
Gotify Example

Hi @tjleach98 did you test it?

Yes! I setup the development version of Semaphore UI and used the built in tasks for testing. It was setup to alert using our production version of Gotify. The screenshot included in my request message is the notification on Gotify after running the 'Ping Site' task.


Sum type example

Fixes #13.

Thank you very much for this contribution.
In hindsight I am not convinced it is a good idea to add this to the readme.
Sum types are a bother, and they are well documented in the documentation.
This readme only serves to show off the project, not to show how to use it.
What do you think @ysangkok ?


ClrPopoverContent rootNodes of undefined

Describe the bug
Using the smartToggleService to toggle the open flag is causing an issue with an undefined reference.
ERROR TypeError: Cannot read property 'rootNodes' of undefined
        this.checkCollector.pipe(Object(rxjs_operators__WEBPACK_IMPORTED_MODULE_4__["debounceTime"])(0)).subscribe(() => {             this.alignContent();             this.shouldRealign = false;             this.renderer.setStyle(this.view.rootNodes[0], 'opacity', '1');         }));
How to reproduce
Mousing quickly over the item which changes visibility state, or rapidly changing the visibility state of the item seems to cause this error.  Noticed first in the button group menu items popover, and then in a custom component i created.
Clicking the icon to toggle the popover menu very quickly while pressing the enter key with the button in focus to allow for a faster change rate will produce the error.
https://stackblitz.com/edit/clarity-light-theme-v3-next-4qjkr8
Versions
App

Angular: 9
Node: 12.11.1
Clarity: 3.0.0-next.8

Hi @mykah89 can you supply a simple stackblitz reproduction? I updated one to use 3.0.0-next.8 and Angular 9 here: https://stackblitz.com/edit/clarity-light-theme-v3-next

Thanks for the updated stackblitz.
Here is a link to the reproducible issue, as a side note, it also seems to expose a bug in the button group which opens the in menu items when clicking any button.
It was difficult to reproduce in the stackblitz environment, i'm guessing it has some sort of speed/timing element to it.
I had to continuously click the cog icon as fast as i could while simultaneously pressing the enter key quickly. It was much easier to reproduce in my application
https://stackblitz.com/edit/clarity-light-theme-v3-next-4qjkr8

Moving to backlog for investigation.

The button group issue has been identified, but we need a better reproduction for rootNodes. I'm not clear, are you creating your own popover implementation or seeing this error when using a Clarity component?

I have a notification component that opens a popover on hover using the ClrPopoverToggleService to toggle the open property.
In my application even slowly hovering back and forth over the component will cause the error at some point.
Using the stackblitz i was unable to reproduce in this fashion, i assume its because the stackblitz executes faster, which is why i used the button as an example.  Even in my application toggling the button slower causes the issue.
I have updated the stackblitz with a toggle button that seems to consistently provide the same error.
https://stackblitz.com/edit/clarity-light-theme-v3-next-4qjkr8
Please let me know if there is anything additional i can provide to help here since this error prevents me from updating my production application to angular 9 with clarity.

While the button overflow issue seems to be resolved, this underlying issue is still not resolved, toggling the popover with the ClrPopoverToggleService as per the stackblitz still causes an error while referencing the rootNodes.

You are trying to toggle with an internal service (which has to be public unfortunately), which isn't clear to me why you are trying to do this. However, the issue is that you should use the toggleWithEvent($event) method instead and pass in the click/keyboard event. Setting the open state directly is causing the issue for you, since this is an event based toggle example.
This is not really a supported use case and could change in the future. I don't know the context by which you want to do this, but a better long term solution would be to request an input binding to toggle the state of the overflow menu (currently not available).

Thanks for the feedback, i have a notifications component utilizing the popover service.. this button example was just the easiest way i could display it at the time.
Here is an updated stack blitz with my implementation, also using the toggleWithEvent as you outlined.
The error does not occur all of the time, hovering quickly over and off of the notifications indicator will eventually result in the console error i have described, again this occurs fairly quickly in my larger application, it seems to occur less often in the faster running stackblitz but it does occur.
ERROR
Error: Cannot read property 'rootNodes' of undefined
https://stackblitz.com/edit/clarity-light-theme-v3-next-bpxtfg