How to cancel the background download of Remotepdf

I have large files in remote, is there any option to cancel the background download while downloading through remotepdfviewpager.

Hi,
for the moment it is not possible to cancel Pdf download but you can implement your own downloader and add it a cancel() method that you can call to cancel your current download.
All you have to do is pass your custom DownloadFile implementation to RemotePdfViewPager through this constructor
public RemotePDFViewPager(Context context,
                              DownloadFile downloadFile,
                              String pdfUrl,
                              DownloadFile.Listener listener) {
    ...
}

example
remotePdfViewPager = new RemotePDFViewPager(this, MyCustomDownloadFileImpl(), "http://path.to.my.pdf.file", downloadFileListener);

you can paste DownloadFileUrlConnectionImpl as an example and add your cancel() method

thank you...


[Wallet] Users SBAT see updated gold exchange screen

Expected Behavior
Users see local currency in exchange flow and commit to direction(buy/sell gold) before entering the flow. UI matches mocks below

Figma
Current Behavior
Current exchange screen looks like this:

Separating local currency to another issue, as it will build on top of https://github.com/celo-org/celo-monorepo/issues/1484


Bug: Forgefinder export has too low skill values

When I export my recipes to import them on ForgeFinder I get incorrect skill values, looks like the skill bonus from my profession gear isn't included (missing exactly 26 skill for tailoring recipes which is the combined skill on my profession gear).
Tested using version 8.3.2

It seems I missed to include the profession gear! Ill fix and hotfix asap

Fixed with 8.3.3 (currently deploying)


Debuging alternative address device

Im trying to setup a second ADS1115 in my code. So im using the alternate address.
Using an i2c scanner it detects the device with the correct address. However when I attempt to do anything with the device and this driver, it fails....
The code bellow always returns error state. But the same code with the default address and other device works...
    let address = SlaveAddr::Alternative(true, false); // new_sda(); //// sda 0x4A
    let mut adc = Ads1x1x::new_ads1115(i2c, address);
    let result = adc.set_full_scale_range(FullScaleRange::Within4_096V); // set range to 4.096v
    match result {
        Ok(()) => {},
        Err(_e) => {
            info!("{}: device error", TASK_ID);
            return
        }, // unable to communicate with device
    }

How do I debug this? How can I confirm the address is the same as my scanner I2C SCAN: device: 0x4A

Well it's started working!


more tests

Description
Testing headers and for v2 newsletter feature.
This pull request includes:

[ ] Bug Fix
[ ] Feature
[ ] Documentation Update
[x] Tests

The following changes were made:

@ashleygraf101 please get tests to pass and I will merge

The TDD for newsletter signup won't pass.


Change link to web.archive.org for former Hortonworks blog

Hi @epogrebnyak
Thank you so much for pointing me the dead link and creating the pull request. But I think it is better to just remove it, as I want to keep the links always be updated. I am removing it now.
Have a good day!
Cheers,
Binh.

Removing the link is a good decision - it is a bit old. Deleting it was my first reaction, but I thought you weere more attached to having it.
You also inspired me to make several subtopics based on your financials/banks list and add more links I remembered about from before. They are too detailed to merge into original list  and hard to do so by one change per PR, but I'm htinking on how to develop this list:
https://github.com/epogrebnyak/awesome-scalability/issues/1

Closing the PR as original correction implemented.


[BUG] deepspeed inference error on 2 node.

my transformer is 4.21.2 and my deepspeed is 0.8.1. And I run the  code bloom-ds-zero-inference.py on two node with the following command

torchrun  --node_rank=0  --nnodes 2 --nproc_per_node=8 --master_addr  xxxxx --master_port 6000 bloom-inference-scripts/bloom-ds-inference.py --name bigscience/bloom > 1.log 2>&1

torchrun  --node_rank=1  --nnodes 2 --nproc_per_node=8 --master_addr  xxxxx --master_port 6000 bloom-inference-scripts/bloom-ds-inference.py --name bigscience/bloom > 1.log 2>&1

I don't use the deepspeed command like below  becasue I forget my password.
deepspeed --num_gpus 8 --num_nodes 2  --master_addr XXXX --hostfile hostfile  bloom-inference-scripts/bloom-ds-inference.py --name bigscience/bloom > 1.log 2>&1

The below is  my error log.

Node 0

WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
[2023-03-03 02:21:44,523] [INFO] [comm.py:657:init_distributed] Initializing TorchBackend in DeepSpeed with backend nccl
*** Loading the model bigscience/bloom

Fetching 197 files:   0%|          | 0/197 [00:00<?, ?it/s]
Fetching 197 files: 100%|██████████| 197/197 [00:00<00:00, 4156.77it/s]
PHLRR4036:24139:24139 [0] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24139:24139 [0] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24139:24139 [0] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24139:24139 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24139:24139 [0] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.706248] [PHLRR4036:24139:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24139:24139 [0] NCCL INFO cudaDriverVersion 11080
NCCL version 2.14.3+cuda11.7
PHLRR4036:24141:24141 [2] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24150:24150 [7] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24148:24148 [6] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24146:24146 [5] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24140:24140 [1] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24142:24142 [3] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24139:24253 [0] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24139:24253 [0] NCCL INFO P2P plugin IBext
PHLRR4036:24144:24144 [4] NCCL INFO cudaDriverVersion 11080
PHLRR4036:24141:24141 [2] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24142:24142 [3] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24150:24150 [7] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24140:24140 [1] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24146:24146 [5] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24148:24148 [6] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24141:24141 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24141:24141 [2] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24141:24141 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24141:24141 [2] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.720920] [PHLRR4036:24141:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24142:24142 [3] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24142:24142 [3] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24142:24142 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24142:24142 [3] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR4036:24150:24150 [7] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24150:24150 [7] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24150:24150 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24150:24150 [7] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.723766] [PHLRR4036:24142:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24141:24258 [2] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
[1677810113.723912] [PHLRR4036:24150:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24141:24258 [2] NCCL INFO P2P plugin IBext
PHLRR4036:24144:24144 [4] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR4036:24140:24140 [1] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24140:24140 [1] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24140:24140 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24140:24140 [1] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.725985] [PHLRR4036:24140:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24148:24148 [6] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24148:24148 [6] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24148:24148 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24148:24148 [6] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR4036:24146:24146 [5] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24146:24146 [5] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24146:24146 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24146:24146 [5] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.726464] [PHLRR4036:24148:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
[1677810113.726557] [PHLRR4036:24146:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24142:24263 [3] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24142:24263 [3] NCCL INFO P2P plugin IBext
PHLRR4036:24150:24264 [7] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24150:24264 [7] NCCL INFO P2P plugin IBext
PHLRR4036:24144:24144 [4] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR4036:24144:24144 [4] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR4036:24144:24144 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR4036:24144:24144 [4] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
[1677810113.729624] [PHLRR4036:24144:f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
PHLRR4036:24139:24253 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24140:24267 [1] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24140:24267 [1] NCCL INFO P2P plugin IBext
PHLRR4036:24139:24253 [0] NCCL INFO Using network IBext
PHLRR4036:24146:24268 [5] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24146:24268 [5] NCCL INFO P2P plugin IBext
PHLRR4036:24148:24269 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24148:24269 [6] NCCL INFO P2P plugin IBext
PHLRR4036:24144:24272 [4] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR4036:24144:24272 [4] NCCL INFO P2P plugin IBext
PHLRR4036:24141:24258 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24141:24258 [2] NCCL INFO Using network IBext
PHLRR4036:24142:24263 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24142:24263 [3] NCCL INFO Using network IBext
PHLRR4036:24150:24264 [7] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24150:24264 [7] NCCL INFO Using network IBext
PHLRR4036:24146:24268 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24146:24268 [5] NCCL INFO Using network IBext
PHLRR4036:24148:24269 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24148:24269 [6] NCCL INFO Using network IBext
PHLRR4036:24140:24267 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24140:24267 [1] NCCL INFO Using network IBext
PHLRR4036:24144:24272 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR4036:24144:24272 [4] NCCL INFO Using network IBext
PHLRR4036:24148:24269 [6] NCCL INFO Setting affinity for GPU 6 to 0fe03f80
PHLRR4036:24141:24258 [2] NCCL INFO Setting affinity for GPU 2 to 1fc07f
PHLRR4036:24146:24268 [5] NCCL INFO Setting affinity for GPU 5 to 0fe03f80
PHLRR4036:24139:24253 [0] NCCL INFO Setting affinity for GPU 0 to 1fc07f
PHLRR4036:24142:24263 [3] NCCL INFO Setting affinity for GPU 3 to 1fc07f
PHLRR4036:24140:24267 [1] NCCL INFO Setting affinity for GPU 1 to 1fc07f
PHLRR4036:24150:24264 [7] NCCL INFO Setting affinity for GPU 7 to 0fe03f80
PHLRR4036:24144:24272 [4] NCCL INFO Setting affinity for GPU 4 to 0fe03f80
PHLRR4036:24139:24253 [0] NCCL INFO Channel 00/02 :    0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
PHLRR4036:24141:24258 [2] NCCL INFO Trees [0] 3/10/-1->2->-1 [1] 3/-1/-1->2->10
PHLRR4036:24140:24267 [1] NCCL INFO Trees [0] 4/-1/-1->1->0 [1] 4/-1/-1->1->0
PHLRR4036:24150:24264 [7] NCCL INFO Trees [0] -1/-1/-1->7->6 [1] -1/-1/-1->7->6
PHLRR4036:24148:24269 [6] NCCL INFO Trees [0] 7/-1/-1->6->5 [1] 7/-1/-1->6->5
PHLRR4036:24146:24268 [5] NCCL INFO Trees [0] 6/-1/-1->5->4 [1] 6/-1/-1->5->4
PHLRR4036:24142:24263 [3] NCCL INFO Trees [0] 0/-1/-1->3->2 [1] 0/-1/-1->3->2
PHLRR4036:24144:24272 [4] NCCL INFO Trees [0] 5/-1/-1->4->1 [1] 5/-1/-1->4->1
PHLRR4036:24139:24253 [0] NCCL INFO Channel 01/02 :    0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
PHLRR4036:24139:24253 [0] NCCL INFO Trees [0] 1/-1/-1->0->3 [1] 1/-1/-1->0->3
PHLRR4036:24144:24272 [4] NCCL INFO Channel 00/0 : 3[13000] -> 4[83000] [receive] via NET/IBext/1/GDRDMA
PHLRR4036:24139:24253 [0] NCCL INFO Channel 00/0 : 15[91000] -> 0[5000] [receive] via NET/IBext/0
PHLRR4036:24140:24267 [1] NCCL INFO Channel 00 : 1[8000] -> 2[d000] via SHM/direct/direct
PHLRR4036:24146:24268 [5] NCCL INFO Channel 00 : 5[89000] -> 6[8e000] via SHM/direct/direct
PHLRR4036:24140:24267 [1] NCCL INFO Channel 01 : 1[8000] -> 2[d000] via SHM/direct/direct
PHLRR4036:24146:24268 [5] NCCL INFO Channel 01 : 5[89000] -> 6[8e000] via SHM/direct/direct
PHLRR4036:24141:24258 [2] NCCL INFO Channel 00/0 : 2[d000] -> 3[13000] via P2P/IPC
PHLRR4036:24148:24269 [6] NCCL INFO Channel 00/0 : 6[8e000] -> 7[91000] via P2P/IPC
PHLRR4036:24141:24258 [2] NCCL INFO Channel 01/0 : 2[d000] -> 3[13000] via P2P/IPC
PHLRR4036:24148:24269 [6] NCCL INFO Channel 01/0 : 6[8e000] -> 7[91000] via P2P/IPC
PHLRR4036:24150:24264 [7] NCCL INFO Channel 00/0 : 7[91000] -> 8[5000] [send] via NET/IBext/1
PHLRR4036:24142:24263 [3] NCCL INFO Channel 00/0 : 3[13000] -> 4[83000] [send] via NET/IBext/1
PHLRR4036:24139:24253 [0] NCCL INFO Channel 01/0 : 15[91000] -> 0[5000] [receive] via NET/IBext/0
PHLRR4036:24144:24272 [4] NCCL INFO Channel 01/0 : 3[13000] -> 4[83000] [receive] via NET/IBext/1/GDRDMA
PHLRR4036:24139:24253 [0] NCCL INFO Channel 00/0 : 0[5000] -> 1[8000] via P2P/IPC
PHLRR4036:24144:24272 [4] NCCL INFO Channel 00/0 : 4[83000] -> 5[89000] via P2P/IPC
PHLRR4036:24139:24253 [0] NCCL INFO Channel 01/0 : 0[5000] -> 1[8000] via P2P/IPC
PHLRR4036:24144:24272 [4] NCCL INFO Channel 01/0 : 4[83000] -> 5[89000] via P2P/IPC
PHLRR4036:24140:24267 [1] NCCL INFO Connected all rings
PHLRR4036:24140:24267 [1] NCCL INFO Channel 00 : 1[8000] -> 4[83000] via SHM/direct/direct
PHLRR4036:24140:24267 [1] NCCL INFO Channel 01 : 1[8000] -> 4[83000] via SHM/direct/direct
PHLRR4036:24146:24268 [5] NCCL INFO Connected all rings
PHLRR4036:24146:24268 [5] NCCL INFO Channel 00/0 : 5[89000] -> 4[83000] via P2P/IPC
PHLRR4036:24146:24268 [5] NCCL INFO Channel 01/0 : 5[89000] -> 4[83000] via P2P/IPC
PHLRR4036:24150:24264 [7] NCCL INFO Channel 01/0 : 7[91000] -> 8[5000] [send] via NET/IBext/1
PHLRR4036:24139:24287 [0] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR4036:24142:24263 [3] NCCL INFO Channel 01/0 : 3[13000] -> 4[83000] [send] via NET/IBext/1
PHLRR4036:24148:24269 [6] NCCL INFO Connected all rings
PHLRR4036:24148:24269 [6] NCCL INFO Channel 00 : 6[8e000] -> 5[89000] via SHM/direct/direct
PHLRR4036:24148:24269 [6] NCCL INFO Channel 01 : 6[8e000] -> 5[89000] via SHM/direct/direct
PHLRR4036:24141:24258 [2] NCCL INFO Connected all rings
PHLRR4036:24144:24291 [4] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR4036:24141:24258 [2] NCCL INFO Channel 00/0 : 10[d000] -> 2[d000] [receive] via NET/IBext/0/GDRDMA
PHLRR4036:24139:24253 [0] NCCL INFO Connected all rings
PHLRR4036:24150:24264 [7] NCCL INFO Connected all rings
PHLRR4036:24150:24264 [7] NCCL INFO Channel 00/0 : 7[91000] -> 6[8e000] via P2P/IPC
PHLRR4036:24139:24253 [0] NCCL INFO Channel 00 : 0[5000] -> 3[13000] via SHM/direct/direct
PHLRR4036:24142:24263 [3] NCCL INFO Connected all rings
PHLRR4036:24139:24253 [0] NCCL INFO Channel 01 : 0[5000] -> 3[13000] via SHM/direct/direct
PHLRR4036:24150:24264 [7] NCCL INFO Channel 01/0 : 7[91000] -> 6[8e000] via P2P/IPC
PHLRR4036:24150:24264 [7] NCCL INFO Connected all trees
PHLRR4036:24150:24264 [7] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24150:24264 [7] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24148:24269 [6] NCCL INFO Connected all trees
PHLRR4036:24148:24269 [6] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24148:24269 [6] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24141:24258 [2] NCCL INFO Channel 01/0 : 10[d000] -> 2[d000] [receive] via NET/IBext/0/GDRDMA
PHLRR4036:24142:24263 [3] NCCL INFO Channel 00 : 3[13000] -> 0[5000] via SHM/direct/direct
PHLRR4036:24142:24263 [3] NCCL INFO Channel 01 : 3[13000] -> 0[5000] via SHM/direct/direct
PHLRR4036:24144:24272 [4] NCCL INFO Connected all rings
PHLRR4036:24144:24272 [4] NCCL INFO Channel 00 : 4[83000] -> 1[8000] via SHM/direct/direct
PHLRR4036:24144:24272 [4] NCCL INFO Channel 01 : 4[83000] -> 1[8000] via SHM/direct/direct
PHLRR4036:24142:24263 [3] NCCL INFO Channel 00/0 : 3[13000] -> 2[d000] via P2P/IPC
PHLRR4036:24142:24263 [3] NCCL INFO Channel 01/0 : 3[13000] -> 2[d000] via P2P/IPC
PHLRR4036:24141:24258 [2] NCCL INFO Channel 00/0 : 2[d000] -> 10[d000] [send] via NET/IBext/0
PHLRR4036:24140:24267 [1] NCCL INFO Channel 00/0 : 1[8000] -> 0[5000] via P2P/IPC
PHLRR4036:24140:24267 [1] NCCL INFO Channel 01/0 : 1[8000] -> 0[5000] via P2P/IPC
PHLRR4036:24139:24253 [0] NCCL INFO Connected all trees
PHLRR4036:24140:24267 [1] NCCL INFO Connected all trees
PHLRR4036:24140:24267 [1] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24139:24253 [0] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24139:24253 [0] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24140:24267 [1] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24144:24272 [4] NCCL INFO Connected all trees
PHLRR4036:24144:24272 [4] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24144:24272 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24146:24268 [5] NCCL INFO Connected all trees
PHLRR4036:24146:24268 [5] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24146:24268 [5] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24141:24258 [2] NCCL INFO Channel 01/0 : 2[d000] -> 10[d000] [send] via NET/IBext/0
PHLRR4036:24141:24285 [2] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR4036:24141:24258 [2] NCCL INFO Connected all trees
PHLRR4036:24141:24258 [2] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24141:24258 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24142:24263 [3] NCCL INFO Connected all trees
PHLRR4036:24142:24263 [3] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR4036:24142:24263 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR4036:24141:24258 [2] NCCL INFO comm 0x55ee705f7690 rank 2 nranks 16 cudaDev 2 busId d000 - Init COMPLETE
PHLRR4036:24144:24272 [4] NCCL INFO comm 0x564611e72a30 rank 4 nranks 16 cudaDev 4 busId 83000 - Init COMPLETE
PHLRR4036:24146:24268 [5] NCCL INFO comm 0x55822f760a60 rank 5 nranks 16 cudaDev 5 busId 89000 - Init COMPLETE
PHLRR4036:24150:24264 [7] NCCL INFO comm 0x555644620380 rank 7 nranks 16 cudaDev 7 busId 91000 - Init COMPLETE
PHLRR4036:24140:24267 [1] NCCL INFO comm 0x5612afe990e0 rank 1 nranks 16 cudaDev 1 busId 8000 - Init COMPLETE
PHLRR4036:24142:24263 [3] NCCL INFO comm 0x5562191160f0 rank 3 nranks 16 cudaDev 3 busId 13000 - Init COMPLETE
PHLRR4036:24148:24269 [6] NCCL INFO comm 0x55cede6d3870 rank 6 nranks 16 cudaDev 6 busId 8e000 - Init COMPLETE
PHLRR4036:24139:24253 [0] NCCL INFO comm 0x5634028d2ec0 rank 0 nranks 16 cudaDev 0 busId 5000 - Init COMPLETE

PHLRR4036:24141:24296 [2] ib_plugin.c:978 NCCL WARN NET/IB : Got completion from peer <IP_ADDRESS><48598> with error 12, opcode 0, len 0, vendor err 129
PHLRR4036:24141:24296 [2] NCCL INFO include/net.h:35 -> 2
PHLRR4036:24141:24296 [2] NCCL INFO transport/net.cc:1034 -> 2
PHLRR4036:24141:24296 [2] NCCL INFO proxy.cc:520 -> 2
PHLRR4036:24141:24296 [2] NCCL INFO proxy.cc:684 -> 2 [Proxy Thread]
[E ProcessGroupNCCL.cpp:456] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data.
[E ProcessGroupNCCL.cpp:461] To avoid data inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  NCCL error: unhandled system error, NCCL version 2.14.3
ncclSystemError: System call (e.g. socket, malloc) or external library call failed or device error.
Last error:
NET/IB : Got completion from peer <IP_ADDRESS><48598> with error 12, opcode 0, len 0, vendor err 129
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24139 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24140 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24142 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24144 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24146 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24148 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 24150 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 2 (pid: 24141) of binary: /opt/conda/bin/python
Traceback (most recent call last):
  File "/opt/conda/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==1.13.0a0+d0d6b1f', 'console_scripts', 'torchrun')())
  File "/home/lambda7xx/.local/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/home/lambda7xx/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/home/lambda7xx/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/home/lambda7xx/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/lambda7xx/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
bloom-inference-scripts/bloom-ds-inference.py FAILED
------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-03-03_02:22:01
  host      : PHLRR4036.corp.microsoft.com
  rank      : 2 (local_rank: 2)
  exitcode  : -6 (pid: 24141)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 24141
======================================================

Node 1

WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
PHLRR3088:25515:25515 [2] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25526:25526 [7] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25517:25517 [3] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25523:25523 [6] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25513:25513 [0] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25521:25521 [5] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25514:25514 [1] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25519:25519 [4] NCCL INFO cudaDriverVersion 11080
PHLRR3088:25513:25513 [0] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25521:25521 [5] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25513:25513 [0] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25513:25513 [0] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25513:25513 [0] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25513:25513 [0] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25517:25517 [3] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25521:25521 [5] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25521:25521 [5] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25521:25521 [5] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25521:25521 [5] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25513:25585 [0] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25513:25585 [0] NCCL INFO P2P plugin IBext
PHLRR3088:25521:25586 [5] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25521:25586 [5] NCCL INFO P2P plugin IBext
PHLRR3088:25517:25517 [3] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25517:25517 [3] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25517:25517 [3] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25517:25517 [3] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25523:25523 [6] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25517:25590 [3] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25517:25590 [3] NCCL INFO P2P plugin IBext
PHLRR3088:25526:25526 [7] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25515:25515 [2] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25513:25585 [0] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25513:25585 [0] NCCL INFO Using network IBext
PHLRR3088:25523:25523 [6] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25523:25523 [6] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25523:25523 [6] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25523:25523 [6] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25521:25586 [5] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25521:25586 [5] NCCL INFO Using network IBext
PHLRR3088:25515:25515 [2] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25515:25515 [2] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25515:25515 [2] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25515:25515 [2] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25523:25596 [6] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25523:25596 [6] NCCL INFO P2P plugin IBext
PHLRR3088:25526:25526 [7] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25526:25526 [7] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25526:25526 [7] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25526:25526 [7] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25514:25514 [1] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25519:25519 [4] NCCL INFO Bootstrap : Using ens9f1:<IP_ADDRESS><0>
PHLRR3088:25517:25590 [3] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25517:25590 [3] NCCL INFO Using network IBext
PHLRR3088:25515:25600 [2] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25515:25600 [2] NCCL INFO P2P plugin IBext
PHLRR3088:25526:25601 [7] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25526:25601 [7] NCCL INFO P2P plugin IBext
PHLRR3088:25514:25514 [1] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25514:25514 [1] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25514:25514 [1] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25514:25514 [1] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25519:25519 [4] NCCL INFO NET/Plugin: Failed to find ncclNetPlugin_v6 symbol.
PHLRR3088:25519:25519 [4] NCCL INFO NET/Plugin: Loaded net plugin NCCL RDMA Plugin (v5)
PHLRR3088:25519:25519 [4] NCCL INFO NET/Plugin: Failed to find ncclCollNetPlugin_v6 symbol.
PHLRR3088:25519:25519 [4] NCCL INFO NET/Plugin: Loaded coll plugin SHARP (v5)
PHLRR3088:25523:25596 [6] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25523:25596 [6] NCCL INFO Using network IBext
PHLRR3088:25514:25607 [1] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25514:25607 [1] NCCL INFO P2P plugin IBext
PHLRR3088:25519:25608 [4] NCCL INFO Plugin Path : /opt/hpcx/nccl_rdma_sharp_plugin/lib/libnccl-net.so
PHLRR3088:25519:25608 [4] NCCL INFO P2P plugin IBext
PHLRR3088:25515:25600 [2] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25515:25600 [2] NCCL INFO Using network IBext
PHLRR3088:25526:25601 [7] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25526:25601 [7] NCCL INFO Using network IBext
PHLRR3088:25514:25607 [1] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25514:25607 [1] NCCL INFO Using network IBext
PHLRR3088:25519:25608 [4] NCCL INFO NET/IB : Using [0]mlx5_0:1/IB [1]mlx5_2:1/IB [RO]; OOB ens9f1:<IP_ADDRESS><0>
PHLRR3088:25519:25608 [4] NCCL INFO Using network IBext
PHLRR3088:25514:25607 [1] NCCL INFO Setting affinity for GPU 1 to 1fc07f
PHLRR3088:25515:25600 [2] NCCL INFO Setting affinity for GPU 2 to 1fc07f
PHLRR3088:25513:25585 [0] NCCL INFO Setting affinity for GPU 0 to 1fc07f
PHLRR3088:25521:25586 [5] NCCL INFO Setting affinity for GPU 5 to 0fe03f80
PHLRR3088:25517:25590 [3] NCCL INFO Setting affinity for GPU 3 to 1fc07f
PHLRR3088:25526:25601 [7] NCCL INFO Setting affinity for GPU 7 to 0fe03f80
PHLRR3088:25519:25608 [4] NCCL INFO Setting affinity for GPU 4 to 0fe03f80
PHLRR3088:25523:25596 [6] NCCL INFO Setting affinity for GPU 6 to 0fe03f80
PHLRR3088:25526:25601 [7] NCCL INFO Trees [0] -1/-1/-1->15->14 [1] -1/-1/-1->15->14
PHLRR3088:25523:25596 [6] NCCL INFO Trees [0] 15/-1/-1->14->13 [1] 15/-1/-1->14->13
PHLRR3088:25521:25586 [5] NCCL INFO Trees [0] 14/-1/-1->13->12 [1] 14/-1/-1->13->12
PHLRR3088:25519:25608 [4] NCCL INFO Trees [0] 13/-1/-1->12->9 [1] 13/-1/-1->12->9
PHLRR3088:25515:25600 [2] NCCL INFO Trees [0] 11/-1/-1->10->2 [1] 11/2/-1->10->-1
PHLRR3088:25514:25607 [1] NCCL INFO Trees [0] 12/-1/-1->9->8 [1] 12/-1/-1->9->8
PHLRR3088:25517:25590 [3] NCCL INFO Trees [0] 8/-1/-1->11->10 [1] 8/-1/-1->11->10
PHLRR3088:25513:25585 [0] NCCL INFO Trees [0] 9/-1/-1->8->11 [1] 9/-1/-1->8->11
PHLRR3088:25514:25607 [1] NCCL INFO Channel 00 : 9[8000] -> 10[d000] via SHM/direct/direct
PHLRR3088:25514:25607 [1] NCCL INFO Channel 01 : 9[8000] -> 10[d000] via SHM/direct/direct
PHLRR3088:25519:25608 [4] NCCL INFO Channel 00/0 : 11[13000] -> 12[83000] [receive] via NET/IBext/1/GDRDMA
PHLRR3088:25513:25585 [0] NCCL INFO Channel 00/0 : 7[91000] -> 8[5000] [receive] via NET/IBext/0
PHLRR3088:25521:25586 [5] NCCL INFO Channel 00 : 13[89000] -> 14[8e000] via SHM/direct/direct
PHLRR3088:25521:25586 [5] NCCL INFO Channel 01 : 13[89000] -> 14[8e000] via SHM/direct/direct
PHLRR3088:25523:25596 [6] NCCL INFO Channel 00/0 : 14[8e000] -> 15[91000] via P2P/IPC
PHLRR3088:25515:25600 [2] NCCL INFO Channel 00/0 : 10[d000] -> 11[13000] via P2P/IPC
PHLRR3088:25523:25596 [6] NCCL INFO Channel 01/0 : 14[8e000] -> 15[91000] via P2P/IPC
PHLRR3088:25515:25600 [2] NCCL INFO Channel 01/0 : 10[d000] -> 11[13000] via P2P/IPC
PHLRR3088:25526:25601 [7] NCCL INFO Channel 00/0 : 15[91000] -> 0[5000] [send] via NET/IBext/1
PHLRR3088:25517:25590 [3] NCCL INFO Channel 00/0 : 11[13000] -> 12[83000] [send] via NET/IBext/1
PHLRR3088:25519:25608 [4] NCCL INFO Channel 01/0 : 11[13000] -> 12[83000] [receive] via NET/IBext/1/GDRDMA
PHLRR3088:25519:25608 [4] NCCL INFO Channel 00/0 : 12[83000] -> 13[89000] via P2P/IPC
PHLRR3088:25519:25608 [4] NCCL INFO Channel 01/0 : 12[83000] -> 13[89000] via P2P/IPC
PHLRR3088:25513:25585 [0] NCCL INFO Channel 01/0 : 7[91000] -> 8[5000] [receive] via NET/IBext/0
PHLRR3088:25513:25585 [0] NCCL INFO Channel 00/0 : 8[5000] -> 9[8000] via P2P/IPC
PHLRR3088:25513:25585 [0] NCCL INFO Channel 01/0 : 8[5000] -> 9[8000] via P2P/IPC
PHLRR3088:25521:25586 [5] NCCL INFO Connected all rings
PHLRR3088:25514:25607 [1] NCCL INFO Connected all rings
PHLRR3088:25514:25607 [1] NCCL INFO Channel 00 : 9[8000] -> 12[83000] via SHM/direct/direct
PHLRR3088:25514:25607 [1] NCCL INFO Channel 01 : 9[8000] -> 12[83000] via SHM/direct/direct
PHLRR3088:25521:25586 [5] NCCL INFO Channel 00/0 : 13[89000] -> 12[83000] via P2P/IPC
PHLRR3088:25521:25586 [5] NCCL INFO Channel 01/0 : 13[89000] -> 12[83000] via P2P/IPC
PHLRR3088:25526:25601 [7] NCCL INFO Channel 01/0 : 15[91000] -> 0[5000] [send] via NET/IBext/1
PHLRR3088:25517:25590 [3] NCCL INFO Channel 01/0 : 11[13000] -> 12[83000] [send] via NET/IBext/1
PHLRR3088:25523:25596 [6] NCCL INFO Connected all rings
PHLRR3088:25513:25624 [0] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR3088:25515:25600 [2] NCCL INFO Connected all rings
PHLRR3088:25519:25626 [4] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR3088:25515:25600 [2] NCCL INFO Channel 00/0 : 2[d000] -> 10[d000] [receive] via NET/IBext/0/GDRDMA
PHLRR3088:25526:25601 [7] NCCL INFO Connected all rings
PHLRR3088:25526:25601 [7] NCCL INFO Channel 00/0 : 15[91000] -> 14[8e000] via P2P/IPC
PHLRR3088:25526:25601 [7] NCCL INFO Channel 01/0 : 15[91000] -> 14[8e000] via P2P/IPC
PHLRR3088:25523:25596 [6] NCCL INFO Channel 00 : 14[8e000] -> 13[89000] via SHM/direct/direct
PHLRR3088:25513:25585 [0] NCCL INFO Connected all rings
PHLRR3088:25517:25590 [3] NCCL INFO Connected all rings
PHLRR3088:25523:25596 [6] NCCL INFO Channel 01 : 14[8e000] -> 13[89000] via SHM/direct/direct
PHLRR3088:25513:25585 [0] NCCL INFO Channel 00 : 8[5000] -> 11[13000] via SHM/direct/direct
PHLRR3088:25513:25585 [0] NCCL INFO Channel 01 : 8[5000] -> 11[13000] via SHM/direct/direct
PHLRR3088:25526:25601 [7] NCCL INFO Connected all trees
PHLRR3088:25526:25601 [7] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25526:25601 [7] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25523:25596 [6] NCCL INFO Connected all trees
PHLRR3088:25523:25596 [6] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25523:25596 [6] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25515:25600 [2] NCCL INFO Channel 01/0 : 2[d000] -> 10[d000] [receive] via NET/IBext/0/GDRDMA
PHLRR3088:25519:25608 [4] NCCL INFO Connected all rings
PHLRR3088:25517:25590 [3] NCCL INFO Channel 00 : 11[13000] -> 8[5000] via SHM/direct/direct
PHLRR3088:25517:25590 [3] NCCL INFO Channel 01 : 11[13000] -> 8[5000] via SHM/direct/direct
PHLRR3088:25519:25608 [4] NCCL INFO Channel 00 : 12[83000] -> 9[8000] via SHM/direct/direct
PHLRR3088:25519:25608 [4] NCCL INFO Channel 01 : 12[83000] -> 9[8000] via SHM/direct/direct
PHLRR3088:25517:25590 [3] NCCL INFO Channel 00/0 : 11[13000] -> 10[d000] via P2P/IPC
PHLRR3088:25517:25590 [3] NCCL INFO Channel 01/0 : 11[13000] -> 10[d000] via P2P/IPC
PHLRR3088:25515:25600 [2] NCCL INFO Channel 00/0 : 10[d000] -> 2[d000] [send] via NET/IBext/0
PHLRR3088:25514:25607 [1] NCCL INFO Channel 00/0 : 9[8000] -> 8[5000] via P2P/IPC
PHLRR3088:25514:25607 [1] NCCL INFO Channel 01/0 : 9[8000] -> 8[5000] via P2P/IPC
PHLRR3088:25513:25585 [0] NCCL INFO Connected all trees
PHLRR3088:25513:25585 [0] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25513:25585 [0] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25514:25607 [1] NCCL INFO Connected all trees
PHLRR3088:25514:25607 [1] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25514:25607 [1] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25519:25608 [4] NCCL INFO Connected all trees
PHLRR3088:25519:25608 [4] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25519:25608 [4] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25521:25586 [5] NCCL INFO Connected all trees
PHLRR3088:25521:25586 [5] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25521:25586 [5] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25515:25600 [2] NCCL INFO Channel 01/0 : 10[d000] -> 2[d000] [send] via NET/IBext/0
PHLRR3088:25515:25622 [2] NCCL INFO NCCL_IB_TIMEOUT set by environment to 300.
PHLRR3088:25515:25600 [2] NCCL INFO Connected all trees
PHLRR3088:25515:25600 [2] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25515:25600 [2] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25517:25590 [3] NCCL INFO Connected all trees
PHLRR3088:25517:25590 [3] NCCL INFO threadThresholds 8/8/64 | 128/8/64 | 512 | 512
PHLRR3088:25517:25590 [3] NCCL INFO 2 coll channels, 2 p2p channels, 2 p2p channels per peer
PHLRR3088:25515:25600 [2] NCCL INFO comm 0x5614942e7890 rank 10 nranks 16 cudaDev 2 busId d000 - Init COMPLETE
PHLRR3088:25517:25590 [3] NCCL INFO comm 0x55b9aef1d5c0 rank 11 nranks 16 cudaDev 3 busId 13000 - Init COMPLETE
PHLRR3088:25519:25608 [4] NCCL INFO comm 0x55cdde437fd0 rank 12 nranks 16 cudaDev 4 busId 83000 - Init COMPLETE
PHLRR3088:25526:25601 [7] NCCL INFO comm 0x55c8fb11f310 rank 15 nranks 16 cudaDev 7 busId 91000 - Init COMPLETE
PHLRR3088:25513:25585 [0] NCCL INFO comm 0x561baa3cd5c0 rank 8 nranks 16 cudaDev 0 busId 5000 - Init COMPLETE
PHLRR3088:25514:25607 [1] NCCL INFO comm 0x560b6c952590 rank 9 nranks 16 cudaDev 1 busId 8000 - Init COMPLETE
PHLRR3088:25523:25596 [6] NCCL INFO comm 0x563f29166010 rank 14 nranks 16 cudaDev 6 busId 8e000 - Init COMPLETE
PHLRR3088:25521:25586 [5] NCCL INFO comm 0x5582407e5d50 rank 13 nranks 16 cudaDev 5 busId 89000 - Init COMPLETE

PHLRR3088:25515:25633 [2] socket.c:505 NCCL WARN Net : Call to recv from <IP_ADDRESS><45256> failed : Connection reset by peer
PHLRR3088:25515:25633 [2] NCCL INFO socket.c:522 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO ib_plugin.c:584 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO ib_plugin.c:883 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO include/net.h:33 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO transport/net.cc:1018 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO proxy.cc:520 -> 2
PHLRR3088:25515:25633 [2] NCCL INFO proxy.cc:684 -> 2 [Proxy Thread]
[E ProcessGroupNCCL.cpp:455] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
terminate called after throwing an instance of 'std::runtime_error'
  what():  NCCL error: unhandled system error, NCCL version 2.15.1
ncclSystemError: System call (e.g. socket, malloc) or external library call failed or device error. It can be also caused by unexpected exit of a remote peer, you can check NCCL warnings for failure reason and see if there is connection closure by a peer.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25513 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25514 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25517 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25519 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25521 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25523 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 25526 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 2 (pid: 25515) of binary: /opt/conda/bin/python
Traceback (most recent call last):
  File "/opt/conda/bin/torchrun", line 33, in <module>
    sys.exit(load_entry_point('torch==1.13.0a0+d0d6b1f', 'console_scripts', 'torchrun')())
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 345, in wrapper
    return f(*args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/run.py", line 762, in main
    run(args)
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/run.py", line 753, in run
    elastic_launch(
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 132, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/opt/conda/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 246, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
bloom-inference-scripts/bloom-ds-inference.py FAILED
------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-03-03_02:22:01
  host      : PHLRR3088.corp.microsoft.com
  rank      : 10 (local_rank: 2)
  exitcode  : -6 (pid: 25515)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 25515
======================================================

Any updates? I have the same issue?

same here

Any updates? I have the same issue?  some NCCL operations have failed or timed out

Same error, any updates?


Enhance Presto url_extract_*  functions to use RE2 instead of boost::regexp

Description
Currently the presto url_extract_* functions use boost::regexp which are slower than RE2 . Some research online says that the speed up is close to 10x. We already use RE2 in velox , so we will not be adding a new dependency.

Closing due to #7668


Use Singelton design in node (MDStore for example)

Is your feature request related to a problem? Please describe.
Today we do not utilize the way that node js implements Singelton Pattern (class), for more information.
Describe the solution you'd like
Utilize the way that node js implements Singelton Pattern (class).
Describe alternatives you've considered
Leave it as is.
Additional context
For example MDStore class:
https://github.com/noobaa/noobaa-core/blob/6ec43ac7cb3591307fb67d7841b5b3cb28b7b005/src/server/object_services/md_store.js#L63-L69
We can change to:
module.exports = new MDStore()
Except for the elegancy of code, I wonder if it improves performance (and if it is measurable).

@shirady The main drawback against this pattern is that the constructor is called regardless of using the singleton, just upon requiring the module. This can have side effects. For example the current pattern allows to use cyclic requires, but the new pattern will fail . Another example is when the constructor has side effects such as setting globals or connecting to the database. So I think we cannot use this pattern without distinction. Performance is probably so negligible that cannot even be measured.


Run As Administrator

Lots of folks mentioning a notification/display that one is running as Administrator. But, how about getting there! I've had to create a PS script to elevate from standard PS cli. There should be an easier way to enable "Run as Administrator", like there is today with PowerShell alone.
Keep up the great work! Loving the new PowerToys!!!

/dup #1032

Copy that. Sorry I missed the “Megathread”. 😉
Cheers…
Cary R. Wagner
ケーリーR.ワグナー
Director, Global Engineering Cloud & Automation Services
GECAS | Engineering Operation Services | PPD
Hitachi Vantara
+1<PHONE_NUMBER> ‐ Everywhere
+1<PHONE_NUMBER> – Fax
Call Cary with Skype 4<EMAIL_ADDRESS>Cary.Wagner<EMAIL_ADDRESS>

Open an EOS ServiceNow Incidenthttps://hitachivantara.service-now.com/askit?id=sc_cat_item&sys_id=0f37a20c4f21a7004ec39efd0210c7c0 <<<

This message and any attachment are proprietary and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you are hereby notified that you have received this transmittal in error. In this case, any review, dissemination, distribution, or copying of this transmittal is strictly prohibited. If you have received this communication in error, please notify us immediately by reply e-mail and immediately delete this message and all its attachments.
From: Dustin L. Howett (MSFT<EMAIL_ADDRESS>Sent: Thursday, December 12, 2019 12:05 PM
To: microsoft/terminal<EMAIL_ADDRESS>Cc: Cary Wagner<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [microsoft/terminal] Run As Administrator (#3939)
***** EXTERNAL EMAIL *****
/dup #1032https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fterminal%2Fissues%2F1032&data=01|01|Cary.Wagner%40HitachiVantara.com|2f48e4f4054745bd8a7908d77f3e8a98|18791e1761594f52a8d4de814ca8284a|0&sdata=rxJMGGWp1A%2F%2BLzLLFzBetF0AXa8hzgBtX4AcUAeihdc%3D&reserved=0
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fterminal%2Fissues%2F3939%3Femail_source%3Dnotifications%26email_token%3DABWSQUAZ2XOSJX7YWG5P5ZLQYKKN7A5CNFSM4J2DT5J2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGX3RMQ%23issuecomment-565164210&data=01|01|Cary.Wagner%40HitachiVantara.com|2f48e4f4054745bd8a7908d77f3e8a98|18791e1761594f52a8d4de814ca8284a|0&sdata=U4aFhvnTAYQMEAf0QuBScvf6XjA%2FCl0mSuA3B6FZCb8%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABWSQUHPPEQIWPXKC7MNWQLQYKKN7ANCNFSM4J2DT5JQ&data=01|01|Cary.Wagner%40HitachiVantara.com|2f48e4f4054745bd8a7908d77f3e8a98|18791e1761594f52a8d4de814ca8284a|0&sdata=Llb8rdTYw%2FvlAIhJD3IUJH6RZrJwRWfIR%2B7w9qy8q9I%3D&reserved=0.


Publish on pip

I am getting ready to publish a package that depends on this one. It would be nice if all dependencies could be installed through pip. README.md says this is 'coming soon.'  Let's make it happen. Is there anything I can do to help? What steps are necessary before this happens?
-- Nate

I suspect, given the age of that file, that this is not being actively
worked on. I'd be comfortable submitting a pull request.
--
Benjamin Mako Hill
https://mako.cc/
Creativity can be a social contribution, but only in so far
as society is free to use the results. --GNU Manifesto

Looking at the code, there seem to be some incomplete parts (e.g. NotImplemented in State.update). I am hoping to learn if there is some existing vision for what state the project should be in before it is published. Of all the parts of python-mediawiki-utilities that were refactored into this mediawiki-utilities project, this component seems to have diverged the most.


JDK Download does not work anymore

The JDK download does not work anymore.
The direct download in the browser only works after accepting the license prior to same and the generated download link contains some kind of session id that is necessary for a successful download too.
So it seems as if Oracle implemented both a cookie and a session gate.

What version of java se cookbook are you using?

8.112.0


checkGenieo should return different exit codes depending on detection

It'd be super cool if checkGenieo.sh returned 0 or 1 depending on if it found malware, that way you can easily chain it together with the removal tool: checkGenieo.sh && removeGenieo.sh

Done. I'm eventually just going to merge these into one script.


Failed to verify cifar_dm-large_2_255 from CROWN-IBP

Hi there,
I was trying to reproduce Table 2 in https://openreview.net/pdf?id=Skxuk1rFwB
I was able to produce the same results as in CROWN-IBP on three models, but failed on this one:
$ mkdir -p model_weights
$ wget -O model_weights/models_crown-ibp_dm-large.tar.gz https://download.huan-zhang.com/models/crown-ibp/models_crown-ibp_dm-large.tar.gz
$ tar xvf ./model_weights/models_crown-ibp_dm-large.tar.gz -C ./model_weights

$ python examples/vision/cifar_training.py --model cnn_7layer --load model_weights/models_crown-ibp_dm-large/cifar_dm-large_2_255/IBP_large_best.pth --verify --eps .00784313725490196078
resume from eps=0.007843137255
[ 1:  79]: eps=0.007843137255 CE=0.8538 Err=0.2918 Loss=8.0311 Robust_CE=8.0311 Verified_Err=0.9978 Time=0.0234

Did I do anything wrong here? Or is there a bug?

This is not a bug. cifar_training.py by default uses IBP to verify all models. However, for the CIFAR eps=2/255 model, you need to use CROWN-IBP to verify (IBP for all intermediate neurons, CROWN for last layer). You will need to slightly modify cifar_training.py to verify this model.
You need to change https://github.com/KaidiXu/auto_LiRPA/blob/master/examples/vision/cifar_training.py#L313, replace 'IBP' with 'CROWN-IBP'. You probably also need to make minor changes to a few other places.

I forgot to mention that you also need to set the factor in CROWN-IBP to 1.0:
https://github.com/KaidiXu/auto_LiRPA/blob/f4492caea9d7f1e6bcee52e70dbcda6b747f43da/examples/vision/cifar_training.py#L101
A dirty fix is just to set factor=1.0.

Hi @huanzhang12 , thanks for your prompt reply. The two fixes do lower the verified error from 99.78% to 45.50%, but the verified error in other cases get increased. Is it expected? Shall we try both configurations and report the lower error in practice?

Dataset
Eps
Err
IBP Verified_Err
CROWN-IBP factor=1.0 Verified_Err

MNIST
0.1
1.05%
2.30%
3.70%

0.2
1.80%
3.80%
13.19%

0.3
1.80%
6.68%
29.53%

0.4
1.80%
12.46%
57.34%

CIFAR-10
2/255
29.18%
99.78%
45.50%

8/255
54.60%
67.11%
79.80%

@mzweilin This is normal and expected. Except for the CIFAR 2/255 case, other models are eventually trained using the IBP bounds (factor becomes 0 at the end of training) so they should be verified using IBP. For each model, we should report the lowest verified error, but usually the lowest error is obtained using the corresponding training method.

@mzweilin This is normal and expected. Except for the CIFAR 2/255 case, other models are eventually trained using the IBP bounds (factor becomes 0 at the end of training) so they should be verified using IBP. For each model, we should report the lowest verified error, but usually the lowest error is obtained using the corresponding training method.

Thanks for the clarification. I have found the difference from https://github.com/huanzhang12/CROWN-IBP/tree/master/config
I'm closing this issue.


To be closed - Implementation task: Downloads flow

Task/Issue URL: https://app.asana.com/0/0/1203712351528756/f
Description
Added the Notify Me component to the Downloads screen.
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

Closing this we we're going with the alternative implementation in https://github.com/duckduckgo/Android/pull/2785.


Mypy precommit

Add a precommit hook for mypy and silence errors. Some of these are significant issues which suggest the code would not have run (e.g. pass missing argument, no return value from a function leading to a codepath being impossible)

 Add mypy to enrichment repo

Ticket added for Abbreviation Detector bug: https://trello.com/c/38Ee0vZD/704-abbreviation-pipeline-not-working


Adding support for a REST API endpoint as a datasource

As mentioned in #1235, there is a need for being able to pull data from REST API endpoints.
Documentation changes are visible at: https://jess-lowe.github.io/osv.dev/.
Changed files can be found at:

https://jess-lowe.github.io/osv.dev/rest-api-contribution/
https://jess-lowe.github.io/osv.dev/architecture/
https://jess-lowe.github.io/osv.dev/data/

Testcases utilise a localhost instance to mock the behaviour of the external HTTP request.

nice work btw!


How to log hparams to Tensorboard?

Hello! I'm trying to view my hparams on tensorboard, but can't actually see them there. As I understood from documentation, to log hparams one should add self.hparams in the init of the LightningModule. Here's what I'm doing:
class MyModule(pl.LightningModule):
    def __init__(self,hparams):
        super().__init__()
        self.hparams =  Namespace(**{'learning_rate': hparams.learning_rate,
                                    'batch_size':hparams.batch_size,
                                    'normalize_embeddings':hparams.normalize_embeddings,
                                    'normalize': hparams.normalize,
                                    'k_neighbors':hparams.k_neighbors,
                                    'melspec_dir':hparams.melspec_dir})

My hparams also contain Trainer hparams, so to log only the right ones, I wrote this workaround. But anyway, I could not see any of my hparams. What could be the problem?

Hi,
I am trying to figure the same thing out since 2 days now.
So, for just writing the hparams to tensorboard, you can do this:
class MyModule(pl.LightningModule):

    def __init__(self, hparams: dict):
        super(MyModule, self).__init__()
        self.hparams = hparams

It seems that the logger.log_hyperparams is called with those self.hparams before the training begins.

However, I have no idea what to do to add metrics (e.g. validation loss of best epoch) to a set of hyperparameters. Can i extend the question to that?
Currently, I am changing the logger like this:
class MyTensorBoardLogger(TensorBoardLogger):

    def __init__(self, *args, **kwargs):
        super(MyTensorBoardLogger, self).__init__(*args, **kwargs)

    def log_hyperparams(self, *args, **kwargs):
        pass

    @rank_zero_only
    def log_hyperparams_metrics(self, params: dict, metrics: dict) -> None:
        params = self._convert_params(params)
        exp, ssi, sei = hparams(params, metrics)
        writer = self.experiment._get_file_writer()
        writer.add_summary(exp)
        writer.add_summary(ssi)
        writer.add_summary(sei)
        # some alternative should be added
        self.tags.update(params)

And then I'm writing the hparams with metrics in a callback:
def on_train_end(self, trainer, module):
        module.logger.log_hyperparams_metrics(module.hparams, {'val_loss': self.best_val_loss})

But that doesn't seem right.
Is there a way to write some metric together with the hparams as well?

Thanks for the reply!

Did you try defining your Namespace outside, then passing it as hparams?

hparams, that I pass to init of my module is already a Namespace that contains many hyperparameters, including those that I want to log in tensorboard.

I never tried it with a namespace, I always use dicts.
Did you try it with a dict?

No, I didn't
According to #651 hparams should be a Namespace object, so I assume that will not work, but I'll check.

#1228 is not related

Do you know when self.logger.log_hyperparams(hparams) is called? Does it happen at the end of training or every epoch?

when you overwrite it with a print you can see that it gets called at the very beginning of the training (not training step or epoch, the whole training).

I'm still not able to log any hparams to Tensorboard, although I exactly follow the documentation and do this:
class MyModule(pl.LightningModule):
  def __init__(self, hparams):
    super().__init__()
    self.hparams = hparams

,where hparams, that I'm passing to init are parser.parse_args()
Does anybody have similar problems?

I tried explicitly add hparams to tb like this:
 self.logger.experiment.add_hparams(hparam_dict=
                                            {'learning_rate': self.hparams.learning_rate},
                                           metric_dict = dict())

But that didn't work.

Ok, I it. It looks like that TensorBoard won't display hparams if some previous runs were without them. So I had to start a new tb process with a clean logs directory.
Everything works normal now.

Oh Jesus, I've been wasting so much time debugging Lightning. Thank @RafailFridman .

@Borda
Excuse me, could you please mention this behaviour in the docs? Thank you

Excuse me, could you please mention this behaviour in the docs? Thank you

That sounds as a good idea, mind send a PR with suggestion?


Microphone plugin in iOS 11 not working (example page)

I am having trouble implementing the microphone plugin on iOS 11. It works great on every other platform I have tried, including Chrome in Android, but iOS is a problem.
I have tried many different implementations myself, but the fact that the demo page for the microphone plugin doesn't work for me on iOS 11.3 makes me wonder if this is a problem across the board, not just for me.
https://wavesurfer-js.org/plugins/microphone.html
For me, the page asks for access to the microphone as expected and indicates that mic access is live (console states Device ready!), but no waveform is displayed when audio input is presented. And when I tap the button to stop the mic, I get this error in the console:
TypeError: this.stream.stop is not a function. (In 'this.stream.stop()', 'this.stream.stop' is undefined)

I am assuming that this error is because I am trying to stop the stream when no stream has previously been created?
Can anyone get this to work properly? Is this a known issue related to iOS?
Thanks.

@hburgund I think this should work, but take a look at safari issues: https://github.com/katspaugh/wavesurfer.js/search?q=safari&type=Issues

Hi @thijstriemstra...thx for the response. I have looked at some of these issues already with no luck, but I will look again to make sure I haven't missed anything.
Can you confirm that the official example for the mic plugin does in fact work in mobile safari for you? I only have one iOS device running iOS 11, so it is certainly possible that for some reason that device is behaving abnormally. I am working on getting more devices to test and will report back ASAP.

I don't have a device to test it on (atm).

ah, no problem! I will do more testing and if anyone else who sees this wants to add data points, that would be great too. :-)

sure, I can do this and see what I can come up with.
FWIW, I have already done experiments with using wavesurfer to display the waveform of an audio file that was just recorded in the browser using mic access. I have gotten this to work on all major platforms, including mobile Safari in iOS 11. On all other platforms, the mic plugin displays the real-time audio input, but in mobile Safari, nothing is displayed in real-time though the waveform is displayed after the recording is completed.
This is a very rough dev situation right now, but here is my test page:
https://halseyburgund.com/dev/rw-web-framework/example/
Scroll down to #2 Make Recording; after initializing the recorder, you can make a recording and see wavesurfer, both the mic plugin and the core waveform display.

FWIW, I have already done experiments with using wavesurfer to display the waveform of an audio file that was just recorded in the browser using mic access.

Also check the videojs-record (and videojs-wavesurfer) projects!

thx for those links. looks like videojs-record and videojs-wavesurfer unfortunately have the same problem in that they don't visualize the mic input in mobile safari. The recorder does work, like my example, and the waveform is displayed after recording is completed; just no real-time display. The console prints out statements that suggest it is working like "wavesurfer.js microphone plugin enabled." and "Start microphone". Weird.
It's looking more and more like this is a universal problem; I will see if I can figure it out and will post back here if I make any substantive progress. Would love any ideas anyone else might have to try.

@hburgund check #1509, should work..

looks like videojs-record and videojs-wavesurfer unfortunately have the same problem in that they don't visualize the mic input in mobile safari.

with new wavesurfer.js release that should be fixed there as well.


Word overflowing into 3rd line

The word "details" should come in the second line itself.

Text alignment fixed in design. Dev to make the changes further in app
Fig File: (frame is in "Manage Keys Flow" Section in the middle of file)
https://www.figma.com/design/m5eMFTKD391AiJAXaEzgxB/Bitcoin-Keeper?node-id=2120-4353&t=VSd47RZBUA8y7PcM-1
S.S:

#5152  - Add Learn more on this screen
#5100 - Manage Key Icon Changes


Backmerge: Release 2.11.0 to development

This PR backmerges 2.11.0 release to development

Travis automatic deployment:
https://pr1310--safereact.review.gnosisdev.com/app


Report as "Idle" instead of "Off"

Is there a way to change the thermostat in settings or otherwise to report as "Idle" instead of "Off" when thermostat is in Auto Mode and neither heating nor cooling?  When neither heating or cooling, the device isn't really off, it's just waiting for a temperature to trigger either heating or cooling.

Please try the release v1.2.5.


Create first storefront release on zeroheight

When we have entered content in the zeroheight and are ready to share with users we need to "turn on" the releases.

[x] Create first release
[x] Make new release default
[x] Make Latest release, hidden from viewers since that will be our draft version
[x] Share with the team how this works

Created our first release Nov 1. Need to share with the team how this works.

Had a sharing session with the team today about how releases work.


Improve code reusability

The code in each function is highly repetitive. How about creating a general function that validates parameters, builds the query and download the data? This way, it would be easy to replace the body of the functions used in the future if needed.

The bulk of the user-facing functions are auto-generated from the mustache templates, which is why the functions are repetitive.
It would make sense to make these templates available, also.

@ivan-krukov I didn't clearly explain myself. I know that the functions are auto-generated, but even then it's good practice to avoid code reuse as much as possible. So instead of repeating the same lines of code that go into each function, we can create a general function and just call it from all these specific functions.
This model also promotes information hiding where FirebuildeR is only used to build new specific functions and it is abstracted from what the functions actually do. Therefore, if you want to change what the functions do, you'll just need to change this general function in the same project where those functions are defined (which is FirebrowseR and not FirebuildeR).
Check this example:
# General function
getFirebrowseData <- function(parameters, to.Validate, invoker, method {
  validate.Parameters(params = parameters, to.Validate = to.Validate)

  url = build.Query(parameters = parameters,
                    invoker = invoker,
                    method = method)
  ret = download.Data(url, parameters$format, parameters$page)
}

# Specific function after the code has been auto-generated
Analyses.CopyNumber.Genes.Thresholded = function(format = "json",
                             cohort = "",
                             gene = "",
                             tcga_participant_barcode = "",
                             page = "1",
                             page_size = "250",
                             sort_by = "cohort"
                             ){

  parameters = list(format = format,
                    cohort = cohort,
                    gene = gene,
                    tcga_participant_barcode = tcga_participant_barcode,
                    page = page,
                    page_size = page_size,
                    sort_by = sort_by)
  to.Validate = c("gene","tcga_participant_barcode")
  ret <- getFirebrowseData(parameters, to.Validate,
                           invoker = "Analyses",
                           method = "CopyNumber/Genes/Thresholded")
  return(ret)
}

Ah, I see what you mean. That is definitely a better design.
This also gives one central (and rather transparent) place to deal with the implementation. Also better from the version control point of view, since only one file would change (as opposed to updating all the functions).

That's a good idea, even if it's just three lines. I'll get this started asap, also feel free to contribute to FirebuildeR.


Sensor device classes

Description:

Added the light device class for light intensity sensors. light is also the name of a binary_sensor device class, so we keep the consistency. Furthermore it was already been used unofficially by the bh1750 sensor
I moved the device_class names to homeassistant.const
Added device_classes to a few additional sensors

https://github.com/home-assistant/home-assistant-polymer/pull/1143
https://github.com/home-assistant/home-assistant.github.io/pull/5306
https://github.com/home-assistant/developers.home-assistant/pull/10
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] Documentation added/updated in home-assistant.github.io

cc: @OttoWinter

Breaking changes
To unify the sensor light units, lux has been replaced with lx. That covers the following sensor platforms: homematic, isy994, miflora, mysensors, tahoma, vera and template sensors.

I've removed the unnecessary icons, opened a doc and frontend PR (see description).
The remaining question would be which icon to choose for light. I've seen both weather-sunset and lightbulb.

So I've added this new rule to the docs last week. Not going to enforce it for this PR, but just for the future:
https://developers.home-assistant.io/docs/en/entity_index.html#changing-the-entity-model

If you want to add a new feature to an entity or any of its subtypes (light, switch, etc), you will need to propose it first in our architecture repo.

Will also require an update to the dev docs: https://developers.home-assistant.io/docs/en/entity_sensor.html#available-device-classes

I'm no expert for all the sensor components, but just changing the unit shouldn't result in a breaking change. The only one would be for template sensors or customize entries that use lux.

Can you add a paragraph to the PR intro about which platforms have breaking changes?

It's a breaking change for people that export their data externally

Added the breaking changes note to the description.

Just a thought: light as a device class is not very descriptive. Especially if we want to use a unit system in the future, IMO it would be good if we would use the physical terms for these things.
So maybe luminosity? I mean light wouldn't be a problem either, but I just feel like luminosity makes more sense here.

What about brightness?

IMO light is enough, since it's used mostly internally and I don't think we need a second device_class with a comparable name that would require such a change.
However I'll change it if that's the consensus.

IMO light is enough, since it's used mostly internally and I don't think we need a second device_class with a comparable name that would require such a change.
However I'll change it if that's the consensus.

The consensus in the dev channel was that we should call the new device class illuminance


Archive files tasks suddenly failing

My builds suddenly stopped working today. The Archive Files tasks is failing to find any of the files to put into the archive. I saw issue #2646 which is the same issue and is now closed. The build task on my machine was updated to 1.0.5 this morning which caused the builds to fail. It looks like the issue has been fixed in 1.0.6 - how do I get that version (or go back to 1.0.4)?
Sorry for the newbie question - I didn't find any instructions on how to specify a version number for a task.

We are having the same issue. Builds working yesterday are now failing.
We are trying to zip files published with dotnet publish, but the Archive Files Task CWD's into the wrong folder and will not find all the required files:
For example:
dotnet publish outputs the files to src/$(SolutionName)/bin/$(BuildConfiguration)/net451/win7-x64/publish
The Archive Files task CWD's into src/$(SolutionName)/bin/$(BuildConfiguration)/net451/win7-x64 and will not find the files required.
This was working yesterday.

I'm seeing the same issue with a Hosted agent and it started between 10:48 AM and 4:02 PM CDT yesterday (9/27).
If I check "Prefix root folder name to archive paths" the ArchiveFiles task works, but then my Azure Web Site deploy is deployed under the "publish" folder.

I am also having this issue. It appears that it is running 7z.exe from the parent directory instead of the rootFolder.

I believe Yang has a fix to patch.  Getting confirmation on patch would be great.  I believe they are pushing today.

The patch is being pushed out now.  Could you please try your build again?

My builds are fixed. Thanks

I can confirm that the Archive Task now works as expected. Thanks!

I confirmed my build is also now working.
Are you making any process changes to stop broken builds being pushed into VSTS? This issue was found and the code change already made well before 1.0.5 was pushed into my environment and broke my build.
My company is considering moving more projects into VSTS and reliability of builds is important. I understand bugs will exist, I just would hope that known bugs that break basic functionality don't get into production.

@janecos First of all sorry about this breakage, I totally agree with you that bugs which break basic functionalities should not get into production.  We are actively working on improving both our unit tests coverage as well as E2E testing infrastructures and we hope to catch those show stoppers before they hit production.  We also dogfood our tasks in production and have the ability to apply patches quickly.  We understand our current limitation and are actively working on updating the release process to reduce those occurrences.  Again sorry for the problem this has caused.

In this case the issue was found (and fixed) before it was pushed to production (see #2646). I guess it wasn't identified as a show stopper?

Hi @janecos.  Unfortunately, in this case, it was an oversight.  We fixed the bug, but missed cherry-picking the commit into the branch from master that had already been created for deployment to production.  The fix didn't make it into the deployment branch.


thinc: fix darwin build

Motivation for this change
thinc uses Accelerate on macOS as a blas library. Add the necessary
dependencies to allow building on unsandboxed Darwin machines.
With this change, spacy should also build on Darwin.
ZHF #45961
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build thinc

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: thinc
Partial log (click to expand)
Cannot nix-instantiate `thinc' because:
error: attribute 'thinc' in selection path 'thinc' not found

No attempt on x86_64-linux (full log)
The following builds were skipped because they don't evaluate on x86_64-linux: thinc
Partial log (click to expand)
Cannot nix-instantiate `thinc' because:
error: attribute 'thinc' in selection path 'thinc' not found

I admit I didn't check, but the commit message should read pythonPackages.thinc.... :smile:
@GrahamcOfBorg build python36Packages.thinc python27Packages.thinc

Success on x86_64-linux (full log)
Attempted: python36Packages.thinc, python27Packages.thinc
Partial log (click to expand)
copying path '/nix/store/wiviadg6f35l14r8ggp9a3v5zg67180g-python3.6-toolz-0.9.0' from 'https://cache.nixos.org'...
copying path '/nix/store/vy17vfn8v8z0jj2yq0ajdbwlvygsb115-python3.6-tqdm-4.25.0' from 'https://cache.nixos.org'...
copying path '/nix/store/v4ac3a0plwivhzly1p1sfzcm0nhqr0cc-python3.6-wrapt-1.10.11' from 'https://cache.nixos.org'...
copying path '/nix/store/ahdhxr700nl8xy0cmac2cp23m92hk8lg-python3.6-mock-2.0.0' from 'https://cache.nixos.org'...
copying path '/nix/store/g256w45wk6fx2fg03x700s38gj3pa9q0-python3.6-more-itertools-4.3.0' from 'https://cache.nixos.org'...
copying path '/nix/store/k82faygdwff4a8nd0xa0j5xa30fn6dmm-python3.6-pytest-3.7.4' from 'https://cache.nixos.org'...
copying path '/nix/store/bkp9yldkb2gx4877g92nlmgp9gsh0qmq-python3.6-cytoolz-<IP_ADDRESS>' from 'https://cache.nixos.org'...
copying path '/nix/store/k7qd5m9bd9ich64pjqa55vsvbhdnl00w-python3.6-thinc-6.11.2' from 'https://cache.nixos.org'...
/nix/store/k7qd5m9bd9ich64pjqa55vsvbhdnl00w-python3.6-thinc-6.11.2
/nix/store/n39kky1h2b4a21psxbv4323ndddiyh06-python2.7-thinc-6.11.2

Failure on aarch64-linux (full log)
Attempted: python36Packages.thinc, python27Packages.thinc
Partial log (click to expand)
    e.build_objects(self.compiler, src_dir)
  File "setup.py", line 83, in build_objects
    compiler, src_dir, 'cblas_s%s' % iface, iface))
  File "setup.py", line 195, in compile_interface
    if self.is_haswell:
  File "setup.py", line 65, in is_haswell
    return self.cpu_info['model'] >= 63
KeyError: 'model'
builder for '/nix/store/3xj8sci324bcbrkqdicfkk0pz4vkvhj9-python3.6-thinc-6.11.2.drv' failed with exit code 1
error: build of '/nix/store/3xj8sci324bcbrkqdicfkk0pz4vkvhj9-python3.6-thinc-6.11.2.drv', '/nix/store/a01pfrxzqx9mflklgzhwq4pyc1rxn4pz-python2.7-thinc-6.11.2.drv' failed

Ah, oops, I'll do that the next time ;).

no worries, I can do magic when merging...
was broken on aarch64 before, so let's ignore that for now and just look at darwin.

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: thinc
Partial log (click to expand)
Cannot nix-instantiate `thinc' because:
error: attribute 'thinc' in selection path 'thinc' not found

Success on x86_64-darwin (full log)
Attempted: python36Packages.thinc, python27Packages.thinc
Partial log (click to expand)
Requirement already satisfied: coverage>=4.0 in /nix/store/cs1vfk8fmxbx98bapb9vvrp45wpkrgmh-python3.6-coverage-4.5.1/lib/python3.6/site-packages (from hypothesis>=2->thinc==6.11.2) (4.5.1)
Installing collected packages: thinc
Successfully installed thinc-6.11.2
/private/tmp/nix-build-python3.6-thinc-6.11.2.drv-0/thinc-6.11.2
post-installation fixup
strip is /nix/store/df6k4mgdjxciy0f637lryp7c9ln7n1m3-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/zby9r9rxy7d01d62v2pysmhn33032abb-python3.6-thinc-6.11.2/lib
patching script interpreter paths in /nix/store/zby9r9rxy7d01d62v2pysmhn33032abb-python3.6-thinc-6.11.2
/nix/store/zby9r9rxy7d01d62v2pysmhn33032abb-python3.6-thinc-6.11.2
/nix/store/8nj0vx3hv6fqaggg1nsccj2g8ab3zvsz-python2.7-thinc-6.11.2

@GrahamcOfBorg build python36Packages.spacy python27Packages.spacy

Failure on aarch64-linux (full log)
Attempted: python36Packages.spacy, python27Packages.spacy
Partial log (click to expand)
  File "setup.py", line 83, in build_objects
    compiler, src_dir, 'cblas_s%s' % iface, iface))
  File "setup.py", line 195, in compile_interface
    if self.is_haswell:
  File "setup.py", line 65, in is_haswell
    return self.cpu_info['model'] >= 63
KeyError: 'model'
builder for '/nix/store/3xj8sci324bcbrkqdicfkk0pz4vkvhj9-python3.6-thinc-6.11.2.drv' failed with exit code 1
cannot build derivation '/nix/store/8p2iwrkhsip703j9gx5ah75q8wjzdsma-python3.6-spacy-2.0.12.drv': 1 dependencies couldn't be built
error: build of '/nix/store/8p2iwrkhsip703j9gx5ah75q8wjzdsma-python3.6-spacy-2.0.12.drv', '/nix/store/9m3d22ymnm2hc8yvlrfpddgv149mfhwd-python2.7-spacy-2.0.12.drv' failed

Success on x86_64-linux (full log)
Attempted: python36Packages.spacy, python27Packages.spacy
Partial log (click to expand)
copying path '/nix/store/kh44p1x4z8cwwhav29csrdig0jskzndr-python3.6-webencodings-0.5.1' from 'https://cache.nixos.org'...
copying path '/nix/store/1bs6dqd8393fc6gy13w9zdm4s2vx1r4w-python3.6-cryptography-2.3.1-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/3hqfvm2s3lmai5gw27lxpm02zi0m0xg1-python3.6-html5lib-1.0.1' from 'https://cache.nixos.org'...
copying path '/nix/store/834v3x9lz1vbrrlnxbdpnyrmxqraj15d-python3.6-pyOpenSSL-18.0.0-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/wphh2s31x39mqhb7188zyjv7sp08l5xn-python3.6-urllib3-1.23' from 'https://cache.nixos.org'...
copying path '/nix/store/byn3cjaz1cqifnq27xwvcr58njznnw8v-python3.6-requests-2.19.1-dev' from 'https://cache.nixos.org'...
copying path '/nix/store/ky65g67gk2c8qzhiragrl3zldzdl3mqp-python3.6-ftfy-4.4.3' from 'https://cache.nixos.org'...
copying path '/nix/store/d9c148kzzvh5r9vshdysmymdhdmdmbx4-python3.6-spacy-2.0.12' from 'https://cache.nixos.org'...
/nix/store/d9c148kzzvh5r9vshdysmymdhdmdmbx4-python3.6-spacy-2.0.12
/nix/store/cdpa7gjg6pxl5d0x3hvnr38j2l2182f7-python2.7-spacy-2.0.12


Feature/issues 3347

Thanks for opening a Pull Request!
Here's a handy checklist to ensure your PR goes smoothly.

[ ] I signed Google's Contributor License Agreement
[ ] My code conforms to Google's python style.
[ ] My PR at a minimum doesn't decrease unit-test coverage (if applicable).
[ ] My PR has been functionally tested.
[ ] All of the tests pass
[ ] I have submitted a corresponding PR in the Forseti Terraform module (if applicable).

These guidelines and more can be found in our contributing guidelines.

[root@a2da39b2370f workspace]# kitchen verify --test-base-path="integration_tests/tests"
-----> Starting Kitchen (v1.24.0)
-----> Verifying ...
$$$$$$ Running command terraform workspace select kitchen-terraform-forseti-local in directory /workspace/integration_tests/fixtures/forseti
$$$$$$ Running command terraform output -json in directory /workspace/integration_tests/fixtures/forseti
inventory-module: Verifying host <IP_ADDRESS>
Skipping profile: 'inspec-gcp' on unsupported platform: 'ubuntu/18.04'.
...
Profile: forseti_integration_testing
Version: (not specified)
Target:  ssh://ubuntu@<IP_ADDRESS>:22
✔  inventory - cai gsc export file: Command: gsutil ls gs://forseti-cai-export-a110fc59/organizations-92932930834-resource-1572993132795378.dump | grep gs://forseti-cai-export-a110fc59/organizations-92932930834-resource-1572993132795378.dump
✔  Command: gsutil ls gs://forseti-cai-export-a110fc59/organizations-92932930834-resource-1572993132795378.dump | grep gs://forseti-cai-export-a110fc59/organizations-92932930834-resource-1572993132795378.dump stdout should match "gs://forseti-cai-export-a110fc59/organizations-92932930834-resource-1572993132795378.dump"
✔  Command: forseti model delete 5fa33e291fd04686911c2c25445b5dac exit_status should eq 0
✔  Command: forseti inventory delete<PHONE_NUMBER>795378 exit_status should eq 0
Profile Summary: 1 successful control, 0 control failures, 0 controls skipped
Test Summary: 3 successful, 0 failures, 0 skipped
Finished verifying  (0m51.29s).
-----> Kitchen is finished. (0m53.48s)


Release v0.16.0

Release of the v0.16.0 version of the OTLP.
Executed make sync VERSION=v0.16.0.

@MrAlias the release procedure says to do make verify, but there is no such target. Should it be make verify-versions?

I wasn't sure if I need to update versions.yaml in this PR or need to do it in a separate PR.

@MrAlias the release procedure says to do make verify, but there is no such target. Should it be make verify-versions?

Ah, yeah, it should be make verify-versions.

I wasn't sure if I need to update versions.yaml in this PR or need to do it in a separate PR.

Yeah, I missed that. We should include a version bump in versions.yaml with this PR.

versions.yaml updated.

@MrAlias PTAL.

@MrAlias I am trying to run the next steps in the release procedure. I tried:
make add-tags MODSET=all

but it complains:
MODSET unset: set to taget module set from versions.yaml

What am I doing wrong?
(It may be easier for you to do the release, if you prefer that instead of teaching me how to do it).

Sorry about the delay. I've pushed tags for this release and made this: https://github.com/open-telemetry/opentelemetry-proto-go/releases/tag/v0.16.0
Should be good to go.
I'll touch base with you in chat about the failure. I would like to not have you blocked from making releases in the future!


Make title of updated Advanced Policy tutorial more descriptive

Description
Now that k8s network policy supports ingress and egress policy, the previous title no longer describes the content of this tutorial (no longer shows the policy features that Calico offers above and beyond k8s). This PR adjust the title to be more descriptive.
Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

Thanks @caseydavenport. I can see what you mean. Certainly, when we add the additional tutorial to discuss unique Calico policy features, we'll need to get some better names here...maybe before.


Add index component

Added a new "SearchIndex" component that populates from sampleManifest.ts.

Added backdoor to the SPA if search params match a "?getSearchIndex" that renders "SearchIndex" component
The index is constructed from "sampleSpec.ts" per sample
"?getSearchIndex" page will be scraped from a backend indexer/builder.

Note that this is the most unintrusive way I could generate the index without doing a refactor of the existing code.
If there is a way to generate a "samplesIndex.json" post-build and dump it into the builds directory, this would be preferable.
I tried using ts-node and running "sampleManifest.ts" independently but this requires several configurations to sampleSpec and tsconfig due to the dynamic loader.
Considering how clean this implementation is, I'm not sure if that would be any better depending on how much the repo must change.

So you're just indexing name and group?

So you're just indexing name and group?

Currently just name and group name since these are the only attributes that makes sense in sampleSpec.ts.
We could add label maybe? README.md is way too much content.
I think adding a "tags" and "short description" might improve it.

Just like sandboxes, each showcase sample should have a short description that contains tags.  We should index those tags.

I assume we'll want to wait for all samples to have a new attribute called "shortDesc", which includes a description and optional tag before we can merge this then.

We should write a few short descriptions but no need to wait for them all, right?  If the description is missing, then there are no tags to add to the index.

Yeah I can do that. For now I won't index the contents README.md. I'll add a new index for short description and moving forward we can add these in our sampleSpec.ts.


:bug: fix Docker remediations for unpinned GHA dependencies

What kind of change does this PR introduce?
Bug fix

[x] PR title follows the guidelines defined in our pull request documentation

What is the current behavior?
As both the check for unpinned dependencies in GitHub Actions and the check for unpinned Docker dependencies contribute to d.Dependencies, the loop that created remediations for Docker dependencies would also create try to create Docker remediations for the unpinned GitHub Actions dependencies.
This could get really slow, especially when scanning a repo with many GitHub Actions such as https://github.com/apache/beam.
What is the new behavior (if this is a feature change)?**
Only create Docker remediations for Docker dependencies (and similar for GitHub workflow remediations)

[x] Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes
NONE
Special notes for your reviewer
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the release-note
(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

Improve Pinned-Dependencies remediation creation

/scdiff generate Pinned-Dependencies

How did the speed-up look on your end?

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Is there still need for improvement

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

scorecard --repo=https://github.com/apache/beam --checks Pinned-Dependencies went from 16 minutes to 24 seconds on my machine ;)

Fantastic. Thanks for catching this!

Faster is always better, and I haven't done any further profiling, but I suspect the bottlenecks will be elsewhere now.

This continues with collectDockerfileInsecureDownloads, collectShellScriptInsecureDownloads, and collectGitHubWorkflowScriptInsecureDownloads.

These don't iterate over the Dependencies, though, as far as I noticed

I did a quick and dirty test to avoid going over the GHA deps again in collectDockerfilePinning and got another 20% speedup: 24s to 20s on my machine forgithub.com/apache/beam, which is better but like you said probably no longer a bottleneck.

/scdiff generate Pinned-Dependencies


Added ReportUnit Alias

@gep13 need to be refactored to use Cake.Core.Tooling.Tool<TToolSettings>

@devlead said....
@gep13 need to be refactored to use Cake.Core.Tooling.Tool

Yip, I rebased and pushed all the PR's earlier today, working on fixing them just now
@patriksvensson @devlead would you be able to review the refactoring that I have done in the new commit, and if you are both happy, I can squash it.
Thanks!

Left comment. Other than that this looks good to me :+1:

Good catch, updated and rebased this.

Looks great @gep13 :+1: , could you just rebase into 1 commit.

@devlead done :smile:

@gep13 thanks:+1:


[BD-46] docs: add info about xs min-width

Description
Add question mark icon and tooltip with text for the xs min-width breakpoint
Deploy Preview
Include a direct link to your changes in this PR's deploy preview here (e.g., a specific component page).
Merge Checklist

[ ] If your update includes visual changes, have they been reviewed by a designer? Send them a link to the Netlify deploy preview, if applicable.
[ ] Does your change adhere to the documented style conventions?
[ ] Do any prop types have missing descriptions in the Props API tables in the documentation site (check deploy preview)?
[ ] Were your changes tested using all available themes (see theme switcher in the header of the deploy preview, under the "Settings" icon)?
[ ] Were your changes tested in the example app?
[ ] Is there adequate test coverage for your changes?
[ ] Consider whether this change needs to reviewed/QA'ed for accessibility (a11y). If so, please add wittjeff and adamstankiewicz as reviewers on this PR.

Post-merge Checklist

[ ] Verify your changes were released to NPM at the expected version.
[ ] If you'd like, share your contribution in #show-and-tell.
[ ] 🎉 🙌 Celebrate! Thanks for your contribution.

Thanks for the pull request, @monteri!
When this pull request is ready, tag your edX technical lead.

@viktorrusakov Yes, there is an issue https://github.com/openedx/paragon/issues/2375, we discussed on the meeting to implement tooltip in order to specify which min breakpoint is supported in Paragon. Since this breakpoint is not used in the code then we decided to put it in the tooltip as info.

@monteri I see, thanks! LGTM.

@monteri 🎉 Your pull request was merged!  Please take a moment  to answer a two question survey so we can improve your experience in the future.

:tada: This PR is included in version 20.45.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Avoid Models Field Caching

The TestModel does not refresh the ModelStack instance whenever the current_model changes. As a result, the TestModel fails to validate input data correctly when current_model is dynamically updated. Specifically, updating current_model from Model to Model2 should allow the TestModel to use Model2 for validation, but it continues to reference Model1.
Code to reproduce:
from pydantic import BaseModel

class Model1(BaseModel):
    name: str

class Model2(BaseModel):
    numbers: list[int]

current_model = Model1

class ModelStack:
    def __getattr__(self, item):
        return getattr(current_model, item)

class TestModel(BaseModel):
    submodel: ModelStack()

print(TestModel.model_validate({"submodel": {"name": "John Doe"}}))  # Expected: submodel=Model1(name='John Doe')

current_model = Model2
print(TestModel.model_validate({"submodel": {"numbers": [1, 2, 3]}}))  # Expected: submodel=Model2(numbers=[1, 2, 3])
# Actual output:
#
# pydantic_core._pydantic_core.ValidationError: 1 validation error for TestModel
# submodel.name
#   Field required [type=missing, input_value={'numbers': [1, 2, 3]}, input_type=dict]
#     For further information visit https://errors.pydantic.dev/2.9/v/missing

Expected Behavior: The TestModel should refresh its ModelStack submodel whenever current_model is updated. Therefore, after updating current_model to Model2, TestModel should recognize it and validate {"submodel": {"numbers": [1, 2, 3]}} correctly.
Actual Behavior: The TestModel does not refresh the submodel and continues to expect the fields defined in Model1, resulting in a validation error.
Environment (if needed):
Python: 3.12.1
Pydantic: 2.9.2

Hi @scaui0,
Thanks for your question. Indeed, you need to do a manual schema rebuild to get the desired behavior here. See:
from pydantic import BaseModel

class Model1(BaseModel):
    name: str

class Model2(BaseModel):
    numbers: list[int]

current_model = Model1

class ModelStack:
    def __getattr__(self, item):
        return getattr(current_model, item)

class TestModel(BaseModel):
    submodel: ModelStack()

print(TestModel.model_validate({"submodel": {"name": "John Doe"}}))  # Expected: submodel=Model1(name='John Doe')

current_model = Model2
TestModel.model_rebuild(force=True)

print(TestModel.model_validate({"submodel": {"numbers": [1, 2, 3]}}))  # Expected: submodel=Model2(numbers=[1, 2, 3])


[SPARK-13904] Add exit code parameter to exitExecutor()

What changes were proposed in this pull request?
This PR adds exit code parameter to exitExecutor() so that caller can specify different exit code.
How was this patch tested?
Existing test
@rxin @hbhanawat

Jenkins, test this please.

Test build #56074 has started for PR 12457 at commit df565e4.

@hbhanawat
What do you think ?

Test build #56074 has finished for PR 12457 at commit df565e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56074/
Test PASSed.

Looks good to me.

It seems this PR can be merged.

LGTM. Merging into master. Thanks!


Collector as a service

Hi,
I am trying to create a new collector and is it possible to exposed as a service. My idea is that one og my process will invoke this service and insert the data into MongoDB.
Thanks
Bimal

If you want to "push" data into hygieia, all you need is the corresponding API. Share your use case and we can collaborate on the design.

Thanks @tabladrum
My requirement is to have a service to store the data. Because it is on demand, means whenever I have some data I need to store in MongoDb and I would like to have a service in API project to display in UI.
I have a process which will invoke the collector/data collector service to persist the data. Later user can configure the UI widget to display the data.
Please let me know if you are looking for more details.
Thanks,
Bimal

One more question regarding the relationship. When I create a new collector, which are the  documents/tables I need to care about?.
I am getting an issue in error in below mentioned service call in dashboard-data.js
var route = widgetId ? $http.put(dashboardRoute + '/' + dashboardId + '/widget/' + widgetId, widget) : $http.post(dashboardRoute + '/' + dashboardId + '/widget', widget);
Thanks,
Bimal

@bimalvv A collector is a micro service to "pull" the data from data source (e.g. Stash,SubVersion) and push to mongodb. so depending on the source, you may or may not be able to reuse existing model classes and may need to add new ones to core, api and collector projects. Hope it helps.

Collectors are the microservices such as github collector or jenkins collector etc.
CollectorItems are documents like Jenkins jobs, Github Repo etc. These point to the Collector.
The documents such as Builds, Commit etc has references to corresponding CollectorItems.
Dashboard document has Widget that has ComponenId.
Component document has a set of collector items.


Disable login not working

I am using this ui a lot, and it is really working great.
My network is closed from external access, so I have set the authentication to ‘None’ in advanced ui settings.
However, this seems to only work for a short period of time. After initial login it keeps showing the login screen after some time - often around 1 hour.
I am running newest version in Docker, and I have multiple instances.

Did you restart after changing the authentication option?

I've made some changes, so that the login page will automatically redirect the / when auth is disabled, and the socket connections will automatically re-connect when refreshing a session while auth is disabled.


feat(inference): support generation using trainer

Motivation
Support generation using trainer/engine, required in RLHF.
Modification

inference.py: support trainer/engine generation. refact code in getting inference params.
scheduler: support more input arguments like scalar and inference params.

Checklist
Before PR:

[x] Pre-commit or other linting tools are used to fix the potential lint issues.
[x] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
[x] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.

以及补充一个测例，某条数据拼起来做batch generation的结果需要与这条数据单独generation的结果一致（在推理参数一致的情况下）

已添加在test_generate.py