Issues using deepspeed transformer kernel

Hello,
I have been training models using deepspeed to train bert models.
The code I use is adapted from bing bert example.
I use --deepspeed_transformer_kernel=True when training and after training load the same model and perform prediction.
I am facing 4 issues when performing prediction using trained model:
Some minor changes made to the model and forward function:
def forward(self, input_ids, valid_length=None,  masked_lm_labels=None,  embedding_mode=True):

When I load a deepspeed model without kernel into a model with transformer kernel (I verified that the weights loaded are exact by double transformation) the embedding generated by model with transformer kernel and without transformer kernels are very different.

When using model with transformer kernel the embeddings generated by 2 inputs, input_ids1.shape[1] = 16 and input_ids2.shape[1] = 128, and valid_length = [10], and input_ids1[:, :10] == input_ids2[:, :10] the embeddings generated are not the same they may vary at the first decimal itself.

For some awkward number for input_ids.shape[1] like 9: the embedding generated are all NANs.

When input_ids.shape[1] < 5 :  the program crashes with cuda error (does not display the error most of the time).

NOTE:
These issues do not exist if I use a model without deepspeed kernel.
Please let me know if this is expected and if so why and if not how can it be potentially fixed.

Hi @sriramiscool
Thanks for pointing out these issues.
We are aware of some of these issues and are constantly working on fixing them asap. For the last two issues you mentioned, I pushed a PR, https://github.com/microsoft/DeepSpeed/pull/587. Could you please check it the issues are resolved?
I also think this PR would solve your first issue. But, please let me know if it does not and I further investigate this.
I am not sure if I understand the second issue. Are you saying that the data only exists in [:, :10] and the rest is zero? Can you please explain this further?
Thanks.
Reza

@RezaYazdaniAminabadi Yes. We mean the results become very different when we add more paddings. We expected a small difference due to masked softmax, but the difference is actually non-negligible.

Can you try something here? Pad the mask to be a high negative value (like -10000) for the part of the input that is padded, and let's see if the results are still too much different. This will reduce the effect of this padded part when going through the softmax by making them a high negative value, which result in near-zero softmax output.

Also, let's use random values on the padded input instead of zero (this should not impact the result).
Thanks,
Reza

Can you try something here? Pad the mask to be a high negative value (like -10000) for the part of the input that is padded, and let's see if the results are still too much different. This will reduce the effect of this padded part when going through the softmax by making them a high negative value, which result in near-zero softmax output.

We did this in the model.

We converted 1 to 0 and 0 to -10000 in the mask in the model.

I see, thanks for pointing this.
Would you mind double-checking this with the new PR (https://github.com/microsoft/DeepSpeed/pull/587)?
For the padding, I have seen similar issue and there seems to be a problem when all the padded data are zeros or ones (similar). I could solve that by changing the padding data to random values. I wonder if the same behavior exists in your testing.
Looking forward to seeing your result.
Thanks,
Reza

Thank you for pointing to the PR, we will try it out and let you know soon.
And we did try random padding tokens as well, the result is still the same.
Thank you,
Sriram
On Tue, Dec 8, 2020, 12:18 AM Reza Yazdani<EMAIL_ADDRESS>wrote:

I see, thanks for pointing this.
Would you mind double-checking this with the new PR (#587
https://github.com/microsoft/DeepSpeed/pull/587)?
For the padding, I have seen similar issue and there seems to be a problem
when all the padded data are zeros or ones (similar). I could solve that by
changing the padding data to random values. I wonder if the same behavior
exists in your testing.
Looking forward to seeing your result.
Thanks,
Reza
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/DeepSpeed/issues/589#issuecomment-740461405,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADUIPRG6YGATAITI6SELVLLSTXONHANCNFSM4URJ5UBA
.

Hello,
I tried the PR.
Looks like it fixed the Issue 3 and Issue 4 but the Issue 1 and 2 still seem exist.
Below is the output for the same code as earlier using the PR you pointed me to. Please let me know what can be done next.
Thanks.
-----------------------------------ISSUE 1-------------------------------------------------
EMBEDDINGS DIFFERENT FOR MODEL WITH AND WITHOUT TRANSFORMER KERNEL
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 0.9900880455970764
-----------------------------------ISSUE 2-------------------------------------------------
EMBEDDINGS FOR MODEL WITH TRANSFORMER KERNEL: 2 INPUTS DIFFERENT SIZE BATCH BUT SAME DATA
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 0.0124969482421875
-----------------------------------ISSUE 3-------------------------------------------------
EMBEDDINGS FOR MODEL WITH TRANSFORMER KERNEL: INPUTS WITH ODD BATCH LENGTH
EMBEDDINGS GENERATED : [[[ -3.377   -1.132    3.041  ...  -3.508   -3.05    -4.23  ]
[ -4.402    1.008    1.122  ...  -1.098   -0.9053  -1.595 ]
[ -2.691    2.217    1.026  ...   7.35     4.28    -3.467 ]
...
[ -0.654   -1.821    2.586  ...   3.568    6.945   -0.9917]
[ -2.203   -3.877   -4.36   ... -19.33    -0.8467  -0.373 ]
[  5.758   -4.156    5.67   ...  -6.32   -12.195    0.2412]]]
-----------------------------------ISSUE 4-------------------------------------------------
EMBEDDINGS FOR MODEL WITH TRANSFORMER KERNEL: INPUTS WITH VERY SMALL BATCH LENGTH
SYSTEM HAS CRASHED IF NOTHING IS PRINTED AFTER THIS...
EMBEDDINGS GENERATED : [[[-2.16   -1.155   3.951  ... -3.059  -2.559  -2.781 ]
[-2.053   1.162   2.062  ... -1.6045 -2.066  -1.168 ]]]

Also found some interesting pattern with issue 2 actually.
By varying lengths we observe that is input1.shape[1] = x and
input1.shape[1]-(16-input1.shape[1]%16) <= input2.shape[1] <= input1.shape[1]+(input1.shape[1]%16)
Then the embeddings are the same with no diff.
Further, for input2.shape[1] \in range (129, 160) or range(257, 336) the different in embeddings is super large and for others is the absolute mean difference is in 0.01 range.
Below is the code snippet and result.
THanks
print("-----------------------------------ISSUE 2-------------------------------------------------")
print("EMBEDDINGS FOR MODEL WITH TRANSFORMER KERNEL: 2 INPUTS DIFFERENT SIZE BATCH BUT SAME DATA")
seq_len = 9
ipid2 = torch.arange(128).view([1,128])
ipid2 += 1000
ipid2[:,seq_len:] = torch.randint(10,32000,(128-seq_len,))
ipid2 = ipid2.to("cuda")
for cnrt in range(9,512):
    ipid1 = torch.arange(cnrt).view([1,cnrt])
    ipid1 += 1000
    ipid1[:,seq_len:] = torch.randint(10,32000,(cnrt-seq_len,))
    ipid1 = ipid1.to("cuda")
    valid_len = torch.tensor([seq_len], dtype=torch.long)
    valid_len = valid_len.to("cuda")
    with torch.no_grad():
        s1,p1 = model_with_ds_tr_kernel.network(ipid1, valid_len, embedding_mode=True)
        s2,p2 = model_with_ds_tr_kernel.network(ipid2, valid_len, embedding_mode=True)
    s1 = s1.cpu().numpy()[:,:seq_len]
    s2 = s2.cpu().numpy()[:,:seq_len]
    print("EMBEDDINGS GENERATED SAME FOR PADDED SEQ LENS 128 and {}: {}".format(cnrt, np.allclose(s1,s2, atol=1e-4)))
    print("EMBEDDINGS ABSOLUTE MEAN ERROR: {}".format(np.abs(s1-s2).mean()))

OUTPUT:
.......

EMBEDDINGS GENERATED SAME FOR PADDED SEQ LENS 128 and 511: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 0.0116729736328125

Hi @sriramiscool
Thanks for checking this. And this behavior is for sure strange.
I have updated the branch more today. Could you give it another try?
Also, if you can let me know of the model configuration that you are testing and what sequence-length you are using and which part is padded, I can solve this faster.
Thanks,
Reza

Sure, I can do that. Thanks.
And other details:
Model max sequence length: 512
At the time of input in the code I pasted earlier:
seq_len = 9
ipid2 = torch.arange(128).view([1,128])
input 2 array length is 128 but the valid sequence length is 9, rest of the tokens are random tokens (as you suggested earlier).
input 1 array length I varried from 9 to 512.
The configurations used:
DEEPSPEED CONFIG:
{
"train_batch_size": 16384,
"train_micro_batch_size_per_gpu": 128,
"steps_per_print": 1000,
"prescale_gradients": false,
"zero_optimization": {
"stage": 2
},
"zero_allow_untested_optimizer": true,
"optimizer": {
"type": "LANS",
"params": {
"lr": 0.0001,
"weight_decay": 0.01,
"bias_correction": true,
"eps": 0.000001
}
},
"gradient_clipping": 1,
"wall_clock_breakdown": false,
"fp16": {
"enabled": true,
"loss_scale": 0,
"initial_scale_power": 20,
"loss_scale_window": 1000
}
}
model config:
{
"name": "bert_500M_1024",
"bert_token_file": "bert_asin.model",
"bert_model_config": {
"vocab_size_or_config_json_file": 32003,
"hidden_size": 1024,
"num_hidden_layers": 38,
"num_attention_heads": 16,
"intermediate_size": 4096,
"hidden_act": "gelu",
"hidden_dropout_prob": 0.1,
"attention_probs_dropout_prob": 0.1,
"max_position_embeddings": 512,
"initializer_range": 0.02
},
"data": {
"mixed_seq_datasets": {
"512": {
...
}
}
},
"mixed_seq_training": {
"128": {
"warmup_proportion": 0.05,
"const_proportion": 0.25,
"learning_rate": 0.003,
"num_workers": 2,
"total_training_steps": 112500
},
"512": {
"warmup_proportion": 0.025,
"const_proportion": 0.125,
"learning_rate": 0.003,
"num_workers": 2,
"total_training_steps": 12500
}
}
}
Other command line parameters:
CUDA_VISIBLE_DEVICES=0 deepspeed deepspeed_predict_test.py
--cf model_config.json
--max_seq_length 512
--output_dir $OUTPUT_DIR
--deepspeed
--deepspeed_transformer_kernel
--job_name $JOB_NAME
--deepspeed_config deepspeed_config.json
--load_training_checkpoint chkpt_folder
--load_checkpoint_id model1
--attention_dropout_checkpoint
--log_file ${JOB_NAME}.log

Hello,
I tried the latest commit and looks like the super large difference I was seeing like these:
"
EMBEDDINGS GENERATED SAME FOR PADDED SEQ LENS 128 and 262: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 7.5625
EMBEDDINGS GENERATED SAME FOR PADDED SEQ LENS 128 and 263: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 7.78515625
EMBEDDINGS GENERATED SAME FOR PADDED SEQ LENS 128 and 264: False
EMBEDDINGS ABSOLUTE MEAN ERROR: 7.13671875
"
are gone. And all differences are of order 0.01. And the issue 1 still persists.
THanks

Okay, cool. So, there is only one issue remaining. I will work on it and let you know.
Thanks
Reza

I guess. But, would be nice if in  issue 2 the difference is of the order
0.0001 which is what I see in model without deepspeed kernel.
And thanks a lot. :)
On Tue, Dec 8, 2020, 9:01 PM Reza Yazdani<EMAIL_ADDRESS>wrote:

Okay, cool. So, there is only one issue remaining. I will work on it and
let you know.
Thanks
Reza
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/DeepSpeed/issues/589#issuecomment-741528140,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADUIPRGGZMMSIRGONKZWBZ3ST4ABTANCNFSM4URJ5UBA
.

Hello @RezaYazdaniAminabadi ,
Wondering if there are any updates.
Thanks,
Sriram

Hey @sriramiscool
I got distracted by some other work. I will come back to this very soon.
Thanks,
Reza

thanks a lot 👍

Hi @sriramiscool
I still cannot repro your first issue! I am not sure what model checkpoint you are using to load the weights of transformer kernel. We have done this previously for the huggingface models, and it gave us the same result. You can find the way we convert the model here.
Can you please provide me some script for the way you are loading weights from the model into the transformer kernel?
Thanks.
Reza

Hi @RezaYazdaniAminabadi
Thanks for getting back, I have created a small standalone script to test just issue 1.
Strangely I am getting a different issue here.
I am attaching the files (changing type from py to txt) here and pasting the output here.
BertLayer: bert_layers.txt
TestRun code: deepspeed_transformer_test.txt
Quick logic of the code and summary:
We create 2 layers : one DeepspeedTransformerLayer (DST) and another BertLayer (BL)
Next, create an embedding layer to generate random embedding as input to these layers.
Next, pass random vector to the embedding layer to generate embedding
Finally, pass the embeddings and masks (zero tensor) to both the layers.
Strangely the output of DST has NANs and BL is fine.
Not sure why I am seeing NANs, will be great if you can help be fix this too. Thanks
Output:
DeepSpeed Transformer config is  {'layer_id': 0, 'batch_size': 128, 'hidden_size': 1024, 'intermediate_size': 4096, 'heads': 16, 'attn_dropout_ratio': 0.1, 'hidden_dropout_ratio': 0.1, 'num_hidden_layers': 1, 'initializer_range': 0.02, 'fp16': True, 'pre_layer_norm': True, 'local_rank': -1, 'seed': 42, 'normalize_invertible': False, 'gelu_checkpoint': False, 'adjust_init_range': True, 'test_gemm': False, 'training': True, 'is_grad_enabled': True, 'attn_dropout_checkpoint': False, 'stochastic_mode': False}
Using ~/.cache/torch_extensions as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file ~/.cache/torch_extensions/transformer/build.ninja...
Building extension module transformer...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module transformer...
Time to load transformer op: 0.1545085906982422 seconds
layer #0 is created with date type [half].
Embeddings generated using deepspeed model with transformer kernel:
tensor([[[nan, nan, nan,  ..., nan, nan, nan],
[nan, nan, nan,  ..., nan, nan, nan],
[nan, nan, nan,  ..., nan, nan, nan],
...,
[0., 0., 0.,  ..., 0., 0., 0.],
[0., 0., 0.,  ..., 0., 0., 0.],
[0., 0., 0.,  ..., 0., 0., 0.]]], device='cuda:0')
Embeddings generated using bert transformer:
tensor([[[-1.1806,  1.5912, -1.6555,  ..., -0.4264, -0.5958, -1.7613],
[-0.5481, -0.8406,  1.0613,  ..., -2.9294,  0.6937, -1.9304],
[-0.4797, -1.3884,  0.0496,  ..., -0.7242,  1.3464, -0.1275],
...,
[ 0.4223,  0.3089,  1.7952,  ...,  0.2685, -1.9444, -0.5416],
[-0.1111,  0.1745, -0.6499,  ..., -0.9291,  1.0662, -1.0272],
[-1.8118,  1.5704,  0.4855,  ..., -0.1584, -0.9928, -1.0782]]],
device='cuda:0')
MAE between two embeddings: nan
Thanks,
Sriram

Hi @sriramiscool
Thanks for providing the scripts.
I have tried this with one of our test units, trying to do the same thing you have done in your test-bench and I could repro the same issue. The important problem I see is that you want to send fp32 values to the kernel whereas configuring it as fp16. I think PyTorch does the conversion itself, however, we have not such checking when passing the inputs to the kernel. I assume the main problem of seeing NaNs is reading fp16 values of the fp32 input-matrix.
I have done some changes to the run_forward function in tests/unit/test_cuda_forward.py file at this branch. The error that I see between deepspeed-kernel (y) and pytorch (x) is about 0.001.

Please run this test on your side to verify this.
Thanks,
Reza

Thank you! That makes sense. I am able to get the same result as you for the code you provided also when I added .half I am able to get an error of 0.0001 in my test code too 👍
I still have issue with my earlier code I posted. I will try to debug more on that and see if we can get same difference there as well and let you know.
Thanks,
Sriram

Hi @RezaYazdaniAminabadi,
Ran the previous code I shared with you with output_all_encoded_layers=True and I think the reason for high discrepancy there is because of accumulation of errors.
Below is the output of difference at every encoded layer.:
EMBEDDINGS DIFFERENT FOR MODEL WITH AND WITHOUT TRANSFORMER KERNEL
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 0 : 0.0062408447265625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 1 : 0.0093231201171875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 2 : 0.01204681396484375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 3 : 0.015289306640625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 4 : 0.5078125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 5 : 0.96142578125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 6 : 1.298828125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 7 : 1.5703125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 8 : 1.8701171875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 9 : 2.052734375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 10 : 2.34765625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 11 : 2.658203125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 12 : 3.1171875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 13 : 3.599609375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 14 : 4.11328125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 15 : 4.41796875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 16 : 4.78125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 17 : 5.1640625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 18 : 5.39453125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 19 : 5.94140625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 20 : 6.23046875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 21 : 6.52734375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 22 : 6.8515625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 23 : 7.14453125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 24 : 7.33203125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 25 : 7.703125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 26 : 7.89453125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 27 : 8.2109375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 28 : 8.3515625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 29 : 8.4921875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 30 : 8.671875
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 31 : 8.8984375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 32 : 9.078125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 33 : 9.15625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 34 : 9.265625
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 35 : 9.328125
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 36 : 9.3359375
EMBEDDINGS GENERATED SAME: False
EMBEDDINGS ABSOLUTE MEAN ERROR encoder 37 : 9.390625

Hi @RezaYazdaniAminabadi, it seems there is still NAN issue if i use arbitrary hidden_size during the training. For example, in my previous run for a 1.5B model, I used the following model config:
    "bert_model_config": {
        "vocab_size_or_config_json_file": 32003,
        "hidden_size": 1536,
        "num_hidden_layers": 52,
        "num_attention_heads": 24,
        "intermediate_size": 6144,
        "hidden_act": "gelu",
        "hidden_dropout_prob": 0.1,
        "attention_probs_dropout_prob": 0.1,
        "max_position_embeddings": 512,
        "initializer_range": 0.02
    }

The training looks good. But after I incorporated #587 and changed it to
    "bert_model_config": {
        "vocab_size_or_config_json_file": 32003,
        "hidden_size": 1600,
        "num_hidden_layers": 48,
        "num_attention_heads": 25,
        "intermediate_size": 6400,
        "hidden_act": "gelu",
        "hidden_dropout_prob": 0.1,
        "attention_probs_dropout_prob": 0.1,
        "max_position_embeddings": 512,
        "initializer_range": 0.02
    }

I received NAN issue again. These two configs both lead to a 1.5B model. I also tried an experiment with the former config and #587, the training looks ok.

Hi @szhengac
Thanks for reporting this. That branch that you mentioned has already merged into the master branch. I have tried you configuration (hidden: 1600, heads: 25), using the test_cuda_backward.py, and the gradients and forward results seem to be healthy and match the baseline with PyTorch. Here is the difference I see for the difference between these two (which is in the order of 0.001):

Can you please check in the latest branch of deepspeed, and try this again?
Thanks,
Reza

Hi @RezaYazdaniAminabadi. I am using the deepspeed master, and the latest commit I have is 7435b2f10af773b0204e77c3549b2b7df9a7a65b on Dec 17. I think my branch is already up to date? I also successfully ran a larger model with the following config
        "hidden_size": 2560,
        "num_hidden_layers": 127,
        "num_attention_heads": 40,
        "intermediate_size": 10240,

What I saw is that 1536 and 2560 are both applicable hidden size before arbitrary hidden_size commit.

Hi,
Sorry, I am a bit confused here. Are you saying that the new commit is making the training with hidden dimensions 1536/2560 produce NaN or undesired results?
Thanks,
Reza

I think they both should still work now. I am surprised if they produce different results! Also, can you please verify the issue with hidden dimension 1600 is resolved?
Thanks,
Reza

@RezaYazdaniAminabadi, sorry for not making it clear. I already used the latest master branch that has https://github.com/microsoft/DeepSpeed/commit/7435b2f10af773b0204e77c3549b2b7df9a7a65b for the experiments. The training looks ok for both the 1536 and 2560 hidden sizes before and after https://github.com/microsoft/DeepSpeed/commit/7435b2f10af773b0204e77c3549b2b7df9a7a65b. But when I switched to 1600, I received NAN in the beginning.

I see. Is it possible to send me an script to reproduce this. Because I am not able to repro this with my test units.
Thanks,
Reza

I will see if I can make a minimal reproducible script. On the other hand, our training script is adapted from https://github.com/microsoft/DeepSpeedExamples/tree/78d69cb2f89a27b1e9b072df8c3e47d00c024fdc/bing_bert. Would it be possible for you to try bing_bert with 1600 hidden_size?

sure, will do.
Thanks,
Reza

Thanks. I will see if I can get NAN with other different configurations.

Hi @RezaYazdaniAminabadi, I have tried the following different configurations:
1. hidden_size 1600, num_layers=4, num_attention_heads=25, intermediate_size=6400: NAN
2. hidden_size 1600, num_layers=4, num_attention_heads=20, intermediate_size=6400: NAN
3. hidden_size 1600, num_layers=4, num_attention_heads=2, intermediate_size=6400: NAN
4. hidden_size 1536, num_layers=4, num_attention_heads=2, intermediate_size=6400: No issue
5. hidden_size 800, num_layers=4, num_attention_heads=2, intermediate_size=3200: No issue
6. hidden_size 800, num_layers=4, num_attention_heads=2, intermediate_size=6400: No issue
7. hidden_size 1600, num_layers=1, num_attention_heads=2, intermediate_size=6400: NAN
8. hidden_size 1600, num_layers=1, num_attention_heads=1, intermediate_size=6400: NAN
9. hidden_size 160, num_layers=1, num_attention_heads=1, intermediate_size=640: NAN
10. hidden_size 16, num_layers=1, num_attention_heads=1, intermediate_size=64: RuntimeError: CUDA error: invalid configuration argument

Both hidden_size=1536 and hidden_size=800 work fine, while hidden_size=1600 and hidden_size=160 result in NAN. hidden_size=16 leads to another issue.

Hi @szhengac,
I have tried running the bing-bert example with hidden size 1600 and batch-size 64. I have not seen the NaN issue thought. By the way, there are some overflows at the beginning that will reduce the loss scale to 4K.

Here is the loss curve I got without changing the training hyper-parameters:

I will also run with PyTorch kernels to verify the loss curve.
Could you please try running the same test and see if you get the same results?
Thanks,
Reza

@RezaYazdaniAminabadi, you mean your result was obtained without using the transformer cuda kernel? In my experiment, I use dynamic batch, where the sequence length is not fixed and it depends on the longest sequence in the batch. And, I pad the sequence such that final length is a multiple of 8.

I have run with the deepspeed transformer cuda kernel:

Please remove the padding and let the transformer API takes care of that. I am also not passing the sequence-length, so it can change dynamically. For the batch size, please specify the maximum batch size that your application can have. If you are running on V100 (32 GB), 64 is the maximum batch size you can have if the sequence-length is lower than 128.
Here is the sample of how I pass in the parameters for the kernel in my recent run with hidden size 1600: https://github.com/microsoft/DeepSpeedExamples/blob/master/bing_bert/nvidia/modelingpreln.py#L575-L592
Thanks,
Reza

I am using 128 A100 GPUs with micro-batch-size=64 and maximum sequence length 128. After I changed it to static batch size with fixed sequence length 128, I can still see the NAN for hidden_size=1600. I am using exactly the same code as in https://github.com/microsoft/DeepSpeedExamples/blob/master/bing_bert/nvidia/modelingpreln.py#L575-L592.

This is pretty strange. Can you please send me your script so that I can try it on my end!?
Thanks,
Reza

There is also another possibility that this issue only appears on A100! Unfortunately, I have limited access to this machine right now! So, it would be helpful if you can see whether the issue exists on V100 too. If that's the case, I need to debug this by the script that you provide me.
Thanks,
Reza

@RezaYazdaniAminabadi, thanks for your suggestions. I need to think about how to make some tweaks, as it is in the internal repo.

Sure, I understand it.
If you think this is helpful, we can have an offline chat and find the source the problem. The issue is that I cannot reproduce the same NaN problem on my side, otherwise I would be happy to help solve it.
By the way, I have run the same experiments you've mentioned earlier, and I have seen the same problem with hidden_size 16. However, I can run the rest of the configurations with no problem.
Thanks,
Reza

@RezaYazdaniAminabadi, sounds good to me. We can have an offline chat.
And, I just tried the experiment on P3, I still receive NAN issue. And, I found the loss of the first iteration is normal and its grad is NAN:
12/22/2020 06:52:41 - INFO - utils.logger -   bert_progress: step=1, loss=10.6953125, lr=0.0, sample_count=4096
12/22/2020 06:52:42 - INFO - utils.logger -   bert_progress: step=2, loss=nan, lr=0.0, sample_count=8192
I use Adam optimizer with zero lr. It seems it is also related to the input.

When I changed the input_ids, attention_mask, and mlm_labels to all zeros, I can receive:
12/22/2020 07:29:14 - INFO - utils.logger -   bert_progress: step=1, loss=11.265625, lr=0.0, sample_count=32
12/22/2020 07:29:15 - INFO - utils.logger -   bert_progress: step=2, loss=nan, lr=0.0, sample_count=64
12/22/2020 07:29:15 - INFO - utils.logger -   bert_progress: step=3, loss=nan, lr=0.0, sample_count=96
Here, I only use one GPU with global_batch_size=micro_batch_size=32.

Thanks @szhengac for the detailed information.
I will try more tests and let you know whether I can repro this on my side.

@RezaYazdaniAminabadi I also tried modifying your unittest cases in test_cuda_backward.py. I got the following:
(3,160,128,2,24,False,True, 0.2)

E           AssertionError:
E           Not equal to tolerance rtol=1e-07, atol=6.1788
E           Index: 3
E           Mismatched elements: 128 / 160 (80%)
E           Max absolute difference: 79.4
E           Max relative difference: 130.1
E            x: array([   1.992 ,  -37.6   ,   70.5   ,   95.1   ,   19.5   ,  -42.5   ,
E                    32.94  ,   39.84  ,  -30.89  ,  -29.73  ,   74.7   ,   -0.8247,
E                   -50.22  ,  -16.9   ,  103.75  ,   31.67  ,    4.566 ,  -17.62  ,...
E            y: array([  17.7   ,   41.78  ,   65.6   ,   44.22  ,    9.01  ,  -74.75  ,
E                    24.8   ,   12.945 ,  -52.03  ,    0.2544,   54.78  ,    6.74  ,
E                   -42.47  ,  -34.44  ,   32.9   ,    1.252 ,  -24.47  ,   -7.484 ,...

and
(64,160,128,2,24,False,True, 0.2)

E           AssertionError:
E           Not equal to tolerance rtol=1e-07, atol=28.9088
E           Index: 3
E           Mismatched elements: 153 / 160 (95.6%)
E           Max absolute difference: 1270.
E           Max relative difference: 51.97
E            x: array([ -63.8   , -165.5   ,   32.84  ,   63.47  ,   40.03  ,  146.    ,
E                   -86.    ,  188.2   , -123.44  ,   38.12  , -215.8   ,  -87.25  ,
E                  -102.7   ,  202.    ,   96.44  ,   76.56  ,  322.2   ,  111.56  ,...
E            y: array([ -298.8  ,   571.5  ,  -385.5  ,  -166.9  ,   -50.47 ,  -451.   ,
E                   -586.5  ,  -308.8  ,  -617.   ,   591.5  ,  -205.4  ,  -349.2  ,
E                   -181.4  ,  -343.   ,  -696.5  ,   -75.2  ,  -141.4  ,   -42.47 ,...

and
(64,1600,128,2,24,False,True, 0.2)

!!!! kernel execution error. (m: 1600, n: 1600, k: 8192, error: 13)
!!!! kernel execution error. (m: 1600, n: 8192, k: 1600, error: 13)
!!!! kernel execution error. (m: 1600, n: 1600, k: 8192, error: 13)
!!!! kernel execution error. (m: 1600, n: 8192, k: 1600, error: 13)
!!!! kernel execution error. (m: 1600, n: 1600, k: 8192, error: 13)
!!!! kernel execution error. (m: 1600, n: 8192, k: 1600, error: 13)
!!!! kernel execution error. (m: 800, n: 128, k: 128, error: 13)
!!!! kernel execution error. (m: 128, n: 128, k: 800, error: 13)
!!!! kernel execution error. (m: 800, n: 128, k: 128, error: 13)
!!!! kernel execution error. (m: 800, n: 128, k: 128, error: 13)
!!!! kernel execution error. (m: 1600, n: 4800, k: 8192, error: 13)
!!!! kernel execution error. (m: 1600, n: 8192, k: 4800, error: 13)
========================================================================================= short test summary info ==========================================================================================
FAILED test_cuda_backward.py::test_backward[64-1600-128-2-24-False-True-0.2] - RuntimeError: CUDA error: an illegal memory access was encountered

Hi @szhengac
Thanks for trying these configurations.
Yes, you are right. I see the difference too. However, for the last one, it does not crash in my case, but it produces nan!
I will check where the issue is and get back to you soon.
Thanks,
Reza

@szhengac, can you tell me for which parameter's gradient you see the difference when running the first test? It can be printed during the checking if you use -sv flags for the pytest.
Thanks,
Reza

Never mind, I found it! It is for the output_gemm bias.

Hi @szhengac
I have found the source of the problem, which is still related to the layer-norm kernels! I have pushed my changes to this PR. I am seeing that the backward_test passes with the first two tests that you've mentioned earlier.
It also has similar results in the last case with 1600 hidden-dimension (with 4 values having 0.04 difference):

One important thing about this backward_test script is that you need to reduce the batch size to fit 2 similar models, one using PyTorch kernel and one deepspeed-transformer, so that we can compare the results. Otherwise, it will crash the same way you reported. Thus, I had to reduce the batch size to 16, so that it can run for both.
One side not is that Pytorch runs OOM for batch 64 on my machine whereas deepspeed_kernels can run fine.
I have added a few test-cases here. Please run them on your side and let me know if they pass. If you see some tests crashes, please run them alone!
Hope this also fixes the problem of seeing the nans, too! :-)
Thanks,
Reza

@RezaYazdaniAminabadi Thanks for the quick fix. I will give it a try. As I am using V100 for the testing, I think I shouldn't have memory issue as I only use a few number of layers?

@RezaYazdaniAminabadi I still receive RuntimeError: CUDA error: an illegal memory access was encountered for the following two tests:
test_backward[32-800-128-2-4-True-True-0.2] and test_backward[64-1600-128-25-4-True-True-0.2]
I monitor the GPU memory during the testing, it is far away from hitting on the limit.

 I have seen also the same problem, but when I ran them alone they pass fine! I am not exactly sure what the problem is. Will investigate this further.
Thanks,
Reza

I just removed some tests that can pass, and I do not see that CUDA error anymore. It seems to me that the kernel may have some storage issue. But I get error for test_backward[64-1600-128-25-4-False-True-0.2]
E           AssertionError:
E           Not equal to tolerance rtol=1e-07, atol=1
E           Index: 1
E           Mismatched elements: 246 / 1600 (15.4%)
E           Max absolute difference: 32.
E           Max relative difference: 0.000966
E            x: array([40300., 19180., 15816., ..., 19340., 12216., 15420.], dtype=float16)
E            y: array([40300., 19180., 15816., ..., 19340., 12220., 15420.], dtype=float16)

I think this might be because we are registering the backward hooks to get the gradients from PyTorch and compare them, which causes memory bloat. However if we only run the kernels, this problem won't exist. I have verified this with the DeepSpeed-Examples and running the training tests.
Regarding the last test, I would not worry too much, as the relative difference is too low. However, if you think this is problematic, I can fix that too.

I wonder why getting gradients from pytorch hook and compare it to the kernel's could cause memory bloat?
"Regarding the last test, I would not worry too much, as the relative difference is too low. However, if you think this is problematic, I can fix that too." If that error could be further reduced, that would be great. As we need to convert the transformer cuda kernel model to model based on pytorch kernel for inference, smaller difference reduces the risk.

I was previously having this problem that the hooks registered won't be released immediately. However, I did check this again, but that was not the issue. So, I take my previous comment back! :-)
However, I think I have found the source of the issue for testing all the configurations together, which I suspect is due to the many allocations/deallocations happening because of changing the dimensions. This is the part that takes care of this, and since the context class is static, there is only one instance of it for all our tests and that's why the allocation and deallocations need to happen. In order to show if this really is the problem and solve the issue, I did reorder my tests as follows, so that less deallocations happen:
(64,1600,128,25,4,True,True, 0.2),
(32,1600,128,25,4,False,True, 0.2),
(32,1600,128,20,4,False,True, 0.2),
(32,800,128,2,4,False,True, 0.2),
(32,800,128,2,4,True,True, 0.2),
(3,160,128,2,24,False,True, 0.2),
(64,160,128,2,24,False,True, 0.2),
(3,160,128,2,24,True,True, 0.2),
(64,160,128,2,24,True,True, 0.2),
and, all the tests can pass together now!

But that part of the code only reallocates the memory when the current memory block is not large enough for holding the new input. And, in real case, we may need to use dynamic mini-batch, so it is very likely that the input batches have varying shape.

True, and that's why we use the max_batch_size to initialize the workspace with! However, I am thinking of moving this part to PyTorch, so that all memory management is handled through it.

That’s a good idea. But it is still strange as we generally get cuda
illegal access error when we are making an out-of-bound memory access (size
is wrong or too many launched blocks so that thread is out-of-bound?).
On Tue, 22 Dec 2020 at 23:15, Reza Yazdani<EMAIL_ADDRESS>wrote:

True, and that's why we use the max_batch_size to initialize the workspace
with! However, I am thinking of moving this part to PyTorch, so that all
memory management is handled through it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/DeepSpeed/issues/589#issuecomment-749980626,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6GZVGAN7LWLV6KTPFKYZTSWGKHRANCNFSM4URJ5UBA
.

Hi @szhengac
Sorry, I am not sure if I understand your question here! These are some reasons of seeing this error but not all of them. If you are still referring to the error happened during the test, I have checked that the workspace memory could not be allocated, and normally this error is silent and your program still continues on, and when reaching to some kernels that need to access this memory, it throws an illegal memory access, since there is no memory allocated!
Reza

Thanks for the explanation. It is very helpful. Do you have idea why the workspace memory is not allocated?

no problem! I have an idea why this happens, but I am not entirely sure. I am ganna run more tests and let you know once I am certain.
Thanks,
Reza

Hi @RezaYazdaniAminabadi, happy new year! I would like to know if we have any update on this issue? Thanks.

Hi @RezaYazdaniAminabadi, happy new year! I would like to know if we have any update on this issue? Thanks.

Hi @szhengac
Happy new year!
Sorry for the delay. I had involved in something else, but I will get back to this soon.
By the way, for some of these issues we can better discuss it if we can meet offline. Would it be okay if I schedule a team meeting for next week?
Thanks,
Reza

Hi @szhengac
Happy new year!
Sorry for the delay. I had involved in something else, but I will get back to this soon.
By the way, for some of these issues we can better discuss it if we can meet offline. Would it be okay if I schedule a team meeting for next week?
Thanks,
Reza

@RezaYazdaniAminabadi Sounds good to me.

@RezaYazdaniAminabadi Sounds good to me.

@RezaYazdaniAminabadi Sorry to bother you again. I wonder if there is any update on this issue or about the offline discussion? Thanks.

Hi @szhengac
No, no bother at all, sorry for the delay in getting back on this thread! I have found some sources of where the issue you were referring to may stem from. I will push some changes to the kernels and let you know to test it.
Apologies for not sending the meeting request, I got involved into some other projects. Let me first add these changes and then if it did not resolve your issue, we can discuss this further offline. How does that sound?
Thanks,
Reza

@RezaYazdaniAminabadi Thanks! Sounds good to me.

@RezaYazdaniAminabadi It seems that test_cuda_backward.py in the more recent master branch is broken. Without changing anything, I received: RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

Hi @szhengac
Thanks for reporting this. Is this some issue you see with PyTorch 1.7 and cuda 11!?
I am also fixing some bugs that I see the transformer kernel may have by some hidden dimensions. I have already made some changes to fix one issue with the bias-add. I am also keep searching other kernels for some other corner cases and will update you soon.
Thanks for your patience!
Reza

@RezaYazdaniAminabadi I used CUDA 11.0 and compiled pytorch with commit ae5c2febb912066f1a8dec8b54451c09195b2c6d from source

@RezaYazdaniAminabadi Hi Reza, do you have any progress regarding this issue? Thanks.

Hi @szhengac
Yes, I think I got it solved with some of the dimensions we discussed. I have pushed my changes here in this branch. Could you please give it a try?
Thanks,
Reza

@RezaYazdaniAminabadi I get the following error when I tried unittest test_cuda_backward.py:
    def __init__(self, config, initial_weights=None, initial_biases=None):
        super(DeepSpeedTransformerLayer, self).__init__()

        self.config = config
        self.config.layer_id = DeepSpeedTransformerLayer.layer_id
        DeepSpeedTransformerLayer.layer_id = DeepSpeedTransformerLayer.layer_id + 1

        print("DeepSpeed Transformer config is ", self.config.__dict__)

        if self.config.local_rank >= 0:
            torch.cuda.set_device(self.config.local_rank)

        if initial_weights is None and initial_biases is None:
            self.attn_qkvw = nn.Parameter(
                torch.Tensor(self.config.hidden_size * 3,
                             self.config.hidden_size))
            self.attn_qkvb = nn.Parameter(torch.Tensor(self.config.hidden_size * 3))
            self.attn_ow = nn.Parameter(
                torch.Tensor(self.config.hidden_size,
                             self.config.hidden_size))
            self.attn_ob = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.attn_nw = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.attn_nb = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.inter_w = nn.Parameter(
                torch.Tensor(self.config.intermediate_size,
                             self.config.hidden_size))
            self.inter_b = nn.Parameter(torch.Tensor(self.config.intermediate_size))
            self.output_w = nn.Parameter(
                torch.Tensor(self.config.hidden_size,
                             self.config.intermediate_size))
            self.output_b = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.norm_w = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.norm_b = nn.Parameter(torch.Tensor(self.config.hidden_size))
            self.init_transformer_weights(self.config.adjust_init_range)
        else:
            # For testing only.
            self.attn_qkvw = nn.Parameter(
                torch.Tensor(self.config.hidden_size * 3,
                             self.config.hidden_size))
            for i in range(3):
                self.attn_qkvw[i * self.config.hidden_size:(i + 1) * self.config.hidden_size] = \
>                   torch.empty_like(initial_weights[i]).copy_(initial_weights[i])
E               RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.

/usr/local/lib64/python3.7/site-packages/deepspeed/ops/transformer/transformer.py:526: RuntimeError

Interesting! So, the new version of PyTorch does not allow us to copy the weights like this anymore! This part is needed for the test script since we need to make sure both baseline and deepspeed gets the same weights.
Let me try to see if I can replace this with some other functions.
Thanks,
Reza

Meanwhile, can you try this on an older version of PyTorch?
Thanks

@RezaYazdaniAminabadi With Pytorch 1.7.0, I got:
E
E             #endif /* defined(__CUDACC__) */
E           #endif /* defined(__cplusplus) */
E           #undef __HALF_TO_US
E           #undef __HALF_TO_CUS
E
E           typedef __half half;
E
E           extern "C" __global__
E           void func_5(half* t0, half* t1, half* aten_mul_flat, half* aten_add_flat, half* aten_div_flat, half* aten_mul_flat_1) {
E           {
E             float v = __half2float(t0[(512 * blockIdx.x + threadIdx.x) % 1600]);
E             float v_1 = __half2float(t1[512 * blockIdx.x + threadIdx.x]);
E             aten_div_flat[512 * blockIdx.x + threadIdx.x] = __float2half((v + v_1) / 1.414209961891174f);
E             float aten_mul_flat_ = __half2float(aten_mul_flat_1[512 * blockIdx.x + threadIdx.x]);
E             aten_mul_flat__1 = __float2half((__half2float(t0[(512 * blockIdx.x + threadIdx.x) % 1600]) + __half2float(t1[512 * blockIdx.x + threadIdx.x])) * 0.5f);
E             aten_mul_flat_1[512 * blockIdx.x + threadIdx.x] = aten_mul_flat_;
E             float v_2 = __half2float(t0[(512 * blockIdx.x + threadIdx.x) % 1600]);
E             float v_3 = __half2float(t1[512 * blockIdx.x + threadIdx.x]);
E             aten_add_flat[512 * blockIdx.x + threadIdx.x] = __float2half((erff((v_2 + v_3) / 1.414209961891174f)) + 1.f);
E             float v_4 = __half2float(t0[(512 * blockIdx.x + threadIdx.x) % 1600]);
E             float v_5 = __half2float(t1[512 * blockIdx.x + threadIdx.x]);
E             float v_6 = __half2float(t0[(512 * blockIdx.x + threadIdx.x) % 1600]);
E             float v_7 = __half2float(t1[512 * blockIdx.x + threadIdx.x]);
E             aten_mul_flat[512 * blockIdx.x + threadIdx.x] = __float2half(((v_4 + v_5) * 0.5f) * ((erff((v_6 + v_7) / 1.414209961891174f)) + 1.f));
E           }
E           }

modelingpreln.py:235: RuntimeError

========================================================================================= short test summary info =========================================================================================
FAILED test_cuda_backward.py::test_backward[8-1600-128-25-3-True-True-0.05] - RuntimeError: default_program(57): error: identifier "aten_mul_flat__1" is undefined
FAILED test_cuda_backward.py::test_backward[8-160-128-2-3-True-True-0.1] - RuntimeError: default_program(57): error: identifier "aten_mul_flat__1" is undefined
FAILED test_cuda_backward.py::test_backward[8-1600-128-2-3-True-True-0.05] - RuntimeError: default_program(57): error: identifier "aten_mul_flat__1" is undefined
FAILED test_cuda_backward.py::test_backward[3-1024-119-16-24-True-False-0.05] - RuntimeError: default_program(57): error: identifier "aten_mul_flat__1" is undefined
FAILED test_cuda_backward.py::test_backward[3-1024-115-16-24-True-True-0.05] - RuntimeError: default_program(57): error: identifier "aten_mul_flat__1" is undefined

Hi @szhengac
Sorry, I got a bit busy with other matters, but I think this is still coming from PyTorch. I am not sure what bug it has with these newer version, but I have tried the unit tests with torch1.6 and all passed. I understand you will need to run this on newer GPU architecture, so you will need newer version of torch (such as 1.8) and I hope I can solve those issues soon. I wonder if you could see if this works on Torch 1.6 on your side.
Thanks,
Reza

@RezaYazdaniAminabadi Sorry for getting back to you late. Our platform currently does not have <=cu10.1, but I may not have time to recompile everything for cu10.1. I see that this issue is closed. It has been fixed with >=pytorch1.7.0?

Hi @szhengac,
There is still problem with higher versions of PyTorch. We can find a way to support this on Torch1.8. Please feel free to open a different issue with this regard, since this issue is getting quite old and I think this is not anymore an issue with DeepSpeed Transformer kernels.
Thanks,
Reza

@RezaYazdaniAminabadi Can you help with this issue: https://github.com/microsoft/DeepSpeed/issues/1967


DEVAPIS-000 experiment with higher batch size

to support celsius and other customers that want to stake many hundreds of validators in a one-time migration.
this will reduce the number of transactions they need to do by two thirds.

Superceded by


-updated kick-out page

-update content for you cannot use this service

CI: No security warnings found

CI: No security warnings found

CI: No security warnings found


Charter Icon in Services List not being updated

Description
When delete/update or create a schedule action in service the charter Icon in Services List (Datatable) not being updated
To Reproduce
delete/update or create a schedule action in service
Details

Affected Component: [Sunstone]
Hypervisor: [KVM]
Version: [6.4]

Progress Status

[ ] Code committed
[ ] Testing - QA
[ ] Documentation (Release notes - resolved issues, compatibility, known issues)

@tinova PRs to approve:
Code
Doc


48 | Rotate Image | C++

48 | Rotate Image | 90deg clockwise rotation of matrix in C++
Type of Issue -

[x] Adding New Code

Programming Language

[x] C++

Assign me to this problem.


WFLY-17968 Missing permissions in InfinispanCounterTestCase

Resolves
https://issues.redhat.com/browse/WFLY-17968

Please review @pferraro

Thanks @jamezp !


feature request: reactions to pages

Found this quite interesting feature on https://notes.fe-mm.com/fe/javascript/types

It's part of Giscus, do you mean to reimplement one?

I want to make a dedicated component and integrate it with Gitcus like protocol, and https://github.com/importantimport/kkna.


Update Relations.lagda

to be consistent with previous definition and the description below i.e.: "Base case: for all naurals n, ....."

Thank you!


IntelliJ IDEA Ultimate Edition (the doctor check crashed)

i am facing this issue

Closing as duplicate of https://github.com/flutter/flutter/issues/94060


New Property Testing

TODO:

[ ] Restructure
[ ] Clean up
[ ] Have it documented
[ ] Another property test for probably Proposal or Governor

Thanks @emiflake !
I really haven't had time to work on agora side of the testing as I was writing some examples and reading code on PCB. Once I am confident with PCB, most of code here will be replaced with PCB!

Since testing will be enormous, this PR will be broken down into each tests


Grouping Options

Hi, I want to group data. Example:

The data is group by resource_id and I want this:

Thanks for Helping !

Data grouping should be done prior to display in controller.loadData or on the server.
As for group rendering in the grid checkout the following issue: https://github.com/tabalinas/jsgrid/issues/223


React and React-native versions in peerDependencies.

@kmagiera Is there any specific version which should be kept in package.json?
With the latest upgrade of react-navigation some warnings appeared.
react-navigation/react-navigation#4982

Thanks @dsznajder for working on this


Add Selenium Integration Tests

I've added Selenium integration tests for the Fn UI meaning that we can automatically test basic Fn UI functionality. This should help with cases like #71 where we need to test the entire system.
I'm not sure if I've used async/await correctly - I've basically made the entire thing synchronous (apart from where I fill in form input fields). However, as I need Selenium to do things in order and the Selenium webdriver functions return promises, I'm being forced to do this. I either need to do this or use promise chaining and I feel promise chaining is a lot less maintainable. If you have any other suggestions, please let me know.
Testing
Apart from adding a few IDs/Name tags to HTML elements, the changes should be self contained and won't affect the Fn UI so testing should be straight forward.
To test, follow the instructions under Automated Testing on the readme test_integration/etc/config.yaml
Results
Here's a little gif of it working with headless set to false in the config

Hi Chad,
I haven't looked at getting it working in CircleCI yet - my aim was to get something up and running first which I could use to automate the tests that I run manually before pushing code.
However, I believe it can be incorporated into CircleCI. I would expect all we'd need to do is install Chrome and the Chrome Driver. However, I see there are also docs on CircleCI's website specifically for it which might be better to follow: https://circleci.com/docs/2.0/browser-testing/.
One concern I had about incorporating them into CircleCI is how long it can take to run - each test currently takes about half a second to run. As we add more tests this could add up. Would this cause issues with CircleCI costs?
The other concern is that it might end up being a pain to maintain. I'm hoping the benefit they provide will outweigh the maintenance costs but I'm not sure - time will tell with this one.

Ah I forgot, we'd also need to run a local fn server in the CircleCI image too


Version bump 0.1.2

Summary
Version bump 0.1.2

last time we missed couple of updates of version in helm too.
Another + to automate versioning :)

should also bump the docs: docs/deployment/local.rst

Agree

yes, and Aki's PR will be merged with updates to ray cluster creation


Total entries errors for queries with group by clause

test "can not be used with a group by clause" do
      create_posts()

      page =
        Comment
        |> group_by([c], c.post_id)
        |> Scrivener.Ecto.Repo.paginate

      assert page.total_entries == 1
end

This test fails with the following errors:
1) test paginate can not be used with a group by clause (Scrivener.Paginator.Ecto.QueryTest)
     test/scrivener/paginator/ecto/query_test.exs:208
     ** (Postgrex.Error) ERROR 42803 (grouping_error): column "c0.id" must appear in the GROUP BY clause or be used in an aggregate function
     stacktrace:
       (ecto) lib/ecto/adapters/sql.ex:436: Ecto.Adapters.SQL.execute_and_cache/7
       (ecto) lib/ecto/repo/queryable.ex:130: Ecto.Repo.Queryable.execute/5
       (ecto) lib/ecto/repo/queryable.ex:35: Ecto.Repo.Queryable.all/4
       (ecto) lib/ecto/repo/queryable.ex:68: Ecto.Repo.Queryable.one/4
       (scrivener_ecto) lib/scrivener/paginater/ecto/query.ex:38: Scrivener.Paginater.Ecto.Query.total_entries/3
       (scrivener_ecto) lib/scrivener/paginater/ecto/query.ex:10: Scrivener.Paginater.Ecto.Query.paginate/2
       test/scrivener/paginator/ecto/query_test.exs:214: (test)

The select statement is important in group by clauses, so excluding it causes these issues.  I realize that using Repo.all with the query that is passed in may not be the most performant option, but at least results would be accurate and it would work for all queries.

I think this can be solved with a subquery. I'll take a look.


Is there a way to define custom properties in task.json and consume those in the build task?

Is there a way to define a static custom property in task.json and consume that in the associated (PowerShell) build task? This would not be a user editable input.
We have multiple TFS environments and the property of the task only needs to vary between environments. My thought is that at the point the task is uploaded, our deployment process could set the property in task.json to the value specific to the environment. Or would you recommend an alternative approach?
Hope this makes sense. Thanks

Not sure I entirely understand what you're trying to do. The task is uploaded as an internal process to VSTS. If something needs to vary by environment can you use the build variables? Which task are you looking at? Some more context/details would help. Thanks!

@ppejovic any update on this? Did you find a solution to this/if not could you provide more details?

@damccorm I ended up writing a separate config.json file which is read by the build step, The values in the config.json are modified by our deployment process that packages up the build step and pushes it to our TFS instances.

Ah, gotcha, that makes sense. Glad you got it to work, I'm going to close here - feel free to reopen if you have any additional thoughts/questions.

Well I think having it defined in the task.json would be nicer, either in its own config/options section or as an input with a default value that isn’t editable / hidden


DR for arms does not appear in game aid

DR for arms shows up in the GCS character sheet on 3 lines. However, the imported character sheet only shows it on 2 but does have all 3 lines. Will attach screenshots and source character file

Source character file
Sizh'zaia II.zip
GCS screenshot

game aid screen shot

Looks like the body parts for snake men did not align with the names used in RAW. I'll look for either "arm" or "left/right arm".

Fixed! Next release (which is probably tonight).


As a user, I want to be able to  create a new class by providing the name, speciality, level and the college year of the class


Reorganize examples directory

As discussed a little in #482, we are all in agreement that the examples directory needs to be reorganized. Let’s figure out how to restructure it.
First point: does everyone agree with getting rid of the extra layer of examples/ipython and moving all the ipynb examples into the main examples directory? (All of our examples are ipynbs, the only other things in there will be initial information needed for startup, e.g., PDBs.)
Second: the structure within that directory. I gave a partial suggestion before, based on the idea that we would organize by system (toy, alanine dipeptide), then by method (TPS, TIS). @jchodera felt that it was better to organize by method, then by system.
I’ll put a proposal in the first comment below, and I’ll reserve this top post for description of the consensus.
Consensus / Summary

do we move everything to examples? (Agreed yes)
what overall structure do we decide on? (Basically as below)

What I’m suggesting now is a compromise between the method-first and system-first approaches. An important point is that there shouldn’t be too many examples that we distribute with the code: I’d like the web site to have links to lots of other examples (constantly growing as we develop new examples), but I think the core examples should be relatively few.
This means that we can use a combination of system_method as the directory names, and flatten the directory structure. So in that case, I suggest (and I think I included everything, but not entirely sure):
examples/
|- alanine_dipeptide_tps/
|- |- AD_flex_len_tps_setup.ipynb # alanine_dipeptide_tps_first_traj
|- |- AD_fixed_len_tps_setup.ipynb # alanine_dipeptide_fixed_tps_traj
|- |- AD_flex_len_tps_run.ipynb
|- |- AD_fixed_len_tps_run.ipynb
|- |- AD_tps_analysis.ipynb
|- alanine_dipeptide_mstis/
|- |- AD_mstis_setup.ipynb
|- |- AD_mstis_prepare.ipynb
|- |- AD_mstis_restart.ipynb
|- |- AD_mstis_run.ipynb
|- |- AD_mstis_analysis.ipynb
|- |- AD_srtis_run.ipynb # still to come
|- |- AD_srtis_analysis.ipynb # still to come
|- simple_tps/
|- |- simple_fixed_length_tps.ipynb
|- |- simple_flexible_length_tps.ipynb
|- toy_model_mstis/
|- |- toy_mstis_setup.ipynb # currently “bootstrap”
|- |- toy_mstis_run.ipynb
|- |- toy_mstis_analysis.ipynb
|- |- toy_srtis_run.ipynb # still to come
|- |- toy_srtis_analysis.ipynb # still to come
|- toy_model_mistis/
|- |- toy_mistis_setup_run.ipynb
|- |- toy_mistis_analysis.ipynb
|- tests/
|- |- test_cv.ipynb
|- |- test_netcdfplus.ipynb
|- |- test_openmm_integration.ipynb
|- |- test_pyemma.ipynb
|- |- test_snapshot.ipynb
|- |- test_snapshot_modifer.ipynb
|- |- test_storageview.ipynb
|- |- storage_mem_test.ipynb
|- |- langevin_integrator_check.ipynb
|- misc/
|- |- committors.ipynb
|- |- fun_with_pathmovers.ipynb #?
|- |- sequential_ensembles.ipynb
|- |- tutorial_storage.ipynb
|- |- tutorial_visualization.ipynb
|- helper_files/ # better name for that?
|- |- toy_plot_helpers.py
|- |- AD_initial_frame.pdb #Alanine_solvated.pdb
|- |- debug_logging.conf
|- |- logging.conf
|- |- localtests.sh

NOTE: All directories will have a README.md with a brief summary of the files contained.
This means that examples/README.md reads something like this:

alanine_dipeptide_tps: Transition path sampling (both fixed length and flexible length) on alanine dipeptide as a 2-state system. Compare with Bolhuis, Dellago, Chandler PNAS 2000.
alanine_dipeptide_mstis: Multiple state transition interface sampling (including both multiple replica and single replica approaches) for alanine dipeptide as a 6-state system. Compare with Du, Marino, Bolhuis JCP 2011.
toy_model_mstis: Multiple state transition interface sampling (including both multiple replica and single replica approaches) for a simple 3-state 2D model.
toy_model_mistis: Multiple interface set transition interface sampling for a simple 3-state model.
tests: Tests of various OpenPathSampling modules
misc: Miscellaneous example code
helper_files: Additional files used by the examples (initial PDB structures, logging configurations, etc)

Also, I think that most of the test_*ipynb notebooks that are run with Travis should be made into proper nose tests. Unit tests belong in nose, system tests can be variants on examples. Then there are tests that we can’t truly automate, because they are too stochastic to guarantee quick correctness results. For example, the langevin_integrator_check takes too long to be worth including in official tests, but needed to be run at some point to make sure that the integrator was giving reasonable distributions.
Move to docs/
move_strategies_and_schemes.ipynb
sliced_sequential_ensembles.ipynb #? really advanced topic; mainly docs on edge cases
troubleshooting_ops.ipynb

Not included above (remove?):
alanine.ipynb
alanine_dipeptide_old_example.ipynb
input.pdb
netinfo.dat
toy_bootstrapping.ipynb
toy_tis.ipynb
toy_analysis.ipynb

Features I like about this approach:

The names of the example notebooks are still unique, so if the user moves them, it is still clear what they were.
The names of the example notebooks are reasonably short (fit  2-wide in ls on an 80 character terminal)
Not too many things per directory (largest directory is tests, with 9 items (plus README) -- and some of those should move into being nose tests).
All examples that exist in sequences (with some notebooks depending on others) are in the same directory. This allows created files to be shared between examples.

Rename helper_files into resources

@jchodera's comment from the other day

add numbers to files in sequences so that they sort into the right order to run them

Anything else we talked about adding to this re-org that I left out? (Excluding things that are about adding things inside the examples, e.g., adding relevant links to docs -- those will come later)
Self-assigning this; I'll do it after we merge #386, #500, and #521 (all of which include new examples)

From looking through the current examples, we need you guys to do the following:

Organize examples into subdirectories with a few notebooks/subdirectory
Rename the examples so they are sorted in the order they should be run
Each directory should have a README.md briefly describing the example (and relevant reference, if applicable) and indexing the files
Ensure the current examples are being pulled into the online documentation are up to date
The examples should give you a brief idea of the system you're working with early on. This could involve, for example, an illustration of the alanine dipeptide at the top and/or in the README.md, etc.
Reduce the barriers to getting to illuminating examples about what you can do with the tool via http://openpathsampling.org so that it's easy for people to discover what they can do with the tool without a lot of reading.

This work might span multiple


Per-TaskRule setting of e.g. slurm config

Allow particular tasks to take longer, use different queue, or use more memory.

This is possible by using Remake(config=dict(slurm=slurm_config)), where slurm_config is a dict.


Error while running shadcn-ui@latest init for remix

Getting an error while running the init script on different node versions for Remix applications.
Node ver - 18.19.0

Node ver - 21.6.1

The install tailwind step comes after the init step in the docs. Also installing tailwind doesn't solve the issue with init.

I have encountered the same problem.

Same issue here.

oh I tried doing that too, no avail. I made a template based out of the next js default theme - https://github.com/avsssai/remix-shadcn-ui-template

Same problem for me, too

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.


Acessing state in plugin's setup function

What system are you running Yazi on?
Linux Wayland
What terminal are you running Yazi in?
wezterm 20240123-102946-02bdd7ed
Yazi version
yazi 0.2.3 (8efafd6 2024-02-19)
Did you try the latest main branch to see if the problem got fixed?
Tried, but the problem still
Describe the bug
Yazi panics when trying to access the state variable inside a setup function as demonstrated in https://yazi-rs.github.io/docs/plugins/overview#sync-vs-async.
Expected Behavior
It should be possible to set variables in the state so that the plugins can have custom configurations.
To Reproduce
I've created a new branch in bookmarks.yazi which has a simple setup function that tries to set a table in the state variable.
To recreate the issue, simply call the setup method in init.lua
Configuration
No response
Anything else?
I've looked a bit into the lua API and I think that the issue is because the require just runs the lua code without the context of YAZI_PLUGIN_NAME.
A possible solution might be to have a custom require that does something similar to entry.

This is a bit off topic, but in the example that I gave I'm using print in the setup function instead of ya.err because init.lua is executed before the logs are initialized.
Do you want me to make a PR to fix this?

Thank you for the report, this is a known issue, and I am still considering how to address it.
You can use state("your-plugin-name") as a workaround for now:
return {
  setup = function(self, opts)
    local st = state("your-plugin-name")
    st.foo = "bar"
  end
}

Needs 0cb572a

Cool, thanks for the quick workaround, I'll use it in the mean time.

Hi, I made a PR trying to fix it, https://github.com/sxyazi/yazi/pull/710, let me know if it works for you!


Code is being commented out without the use of symbols to comment out code.

Does this issue occur when all extensions are disabled?: Yes/No
Yes it does.

VS Code Version: 1.73.0
OS Version: macOS Monterey
Chip: Apple M1

Steps to Reproduce:

I uninstalled VS Code
I reinstalled VS Code
Start Extension Bisect
The Problem was there on the first bisect; "... This might be a problem with Code."

I included a couple screen shots of the issue.
Code looks like it is commented out without using the right symbols to comment it out. And only part of a line of code may be commented out. It's quite strange.

@bennettbe this is with C++? Can you share a copy/pastable example?

Also, when I open the code in another editor (Sublime, Notepad++) I do not get this issue.

I cannot post a pastable example since the code is for a company I am working for and it is not open source.
I will post two more photos though. The following are the end of a function.
Notice how in the first image the curly bracket on line 604 is commented out.

Description: Image 1
But in the second image if I add the curly bracket to the end of line 603 the colors register correctly.

Description: Image 2
Then in the third image if I remove the comment on line 602 and add the curly bracket back to line 604 the colors register correctly but only if I remove that comment otherwise its the same as in the first image.

Description: Image 3
Im not sure how much this gives you because you can't reproduce yourself but this is what I am working with.

If you create a new file can you reproduce the issue with the following snippet?
static test() {
#if 0 // line comment
	test( arg1 , arg2); // line comment
#endif
}

Usually for a bug like this there some line in the file that throws off all the following lines in some way, which is why I'm not able to reproduce it. The sample I pasted above doesn't repro the issue for me.

Thanks for checking. That means there's something on one of the lines above where you're seeing the problem that is causing the problem. Without being able to see more of the file I don't have any clues as to what might be causing the problem though. If you're able to share more or narrow down the problematic earlier line please do add it to this issue so I can keep investigating.

@alexr00, Thank your for your help! Unfortunately I wont be able to share more code. Ill try cleaning up more comments on my end.


Can we avoid being rate-limited by github?

We make heavy use of this project in our unit tests, and if we run them often enough, we'll see lots of failures due to being rate-limited by github, specifically this request:
Setting up the test database: ArchiveError(IoError(HTTP status client error (403 Forbidden) for url (https://api.github.com/repos/theseus-rs/postgresql-binaries/releases?page=2&per_page=100)))

We create a new db for each test, and they all seem to fail getting that list of releases. I know there is mention in the README of setting a github token to mitigate the chance of rate-limiting, but I'd prefer to avoid it altogether. Is there any configuration option I can use to do that? I've already tried setting a specific version (16.3.0), but it still seems to execute the above request.
Thanks!

There was a change starting in 0.13.0 with the way PostgreSQL version are handled.  Previously, if you were specifying "16.3.0" this would be treated as an exact match and logic would check to see if that version is already installed and skip making requests to GitHub.  Starting with version 0.13.0, you would need to specify the string "=16.3.0" to indicate and exact match to trigger the optimization that would skip the GitHub requests.
If your code does something like this and is caching the ~/.theseus directory between runs, I would not expect any requests to GitHub:
    let settings = Settings {
        version: VersionReq::parse("=16.3.0")?,
        ..Default::default()
    };
    let mut postgresql = PostgreSQL::new(settings);

Alternatively, starting with 0.13.0, there is a zonky feature which pulls the PostgreSQL binaries from Maven Central which may have different rate limits (though I believe your project ran into an issue with one of the dependencies for that).  The postgresql_archive crate was also updated to allow users to define their own custom repositories and archives (in preparation for supporting installing extensions); so you could also define your own custom location for the binary that is used.

Thanks, Brian! Works as described!


Example for RetryOperation

Struggling to find a way to generate RetryOperation?
let uploadOp = PostMachineUpdateOperation()

uploadOp.addObserver(DidFinishObserver { operation,errors in
   if let error = errors.first! as ErrorType? {
                let errOperation = AppAlertOperation(error: error, from: self)
                self.operationQueue.addOperation(errOperation)

                errOperation.alert.addAction(UIAlertAction(title: "Retry", style: UIAlertActionStyle.Default, handler: { action in

      //HOW TO CREATE A RETRY OPERATION?
      let operation = RetryOperation(anyGenerator(PostMachineUpdateOperation()), retry: retry)
                    self.operationQueue.addOperation(operation)

             })

    }
})

Hi @vamsiac - a great place to start looking for examples of RetryOperation is in the project itself. Both the unit tests, and also other classes, such as CloudKitOperation.
I hope that the header docs in the class will give a good indication of what to do.
But in essence the class works by using a GeneratorType of your Operation instances. If the operation finishes with error then the RetryOperation will get the next instance and try again.
There is also a "retry handler" which is a block the RetryOperation can also be initialized with, and this can be used to modify/refine the next operation using error information from the previous instance.
Additionally, there is the concept of a delay between attempts. The generator discussed above actually is for a Payload which is a tuple of (Delay?, T) where T is the operation type. Given this, there are other constructors which can create this Delay automatically, such as for exponential back off , random or fixed delays.
And lastly, there is a maxCount argument which can limit the number of attempts made before eventually failing the RetryOperation itself.
The other way to end the RetryOperation is to return nil from the generator.
Hope that makes sense - fully aware that I've not got full programming guide or docs on this yet, but if you read the source header docs it hopefully makes it clearer?

Cheers Dan, definitely unit tests helped me in lot of other cases but not this one.
    typealias Test = PostMachineUpdateOperation
    typealias Retry = RetryOperation<Test>
    typealias Handler = Retry.Handler

    let uploadOp = PostMachineUpdateOperation()

cannot create RetryOperation for the above operation
let operation = RetryOperation(anyGenerator(XXXXXXXX) , retry: retry)
XXXXXXX what should i enter here to create a new instance of PostMachineUpdateOperation()
I am sorry new to functional stuff, trying to get my head around it.

Yeah, that's no worries - so you'd do something like this:
let operation = RetryOperation(anyGenerator { PostMachineUpdateOperation() })

See docs here on anyGenerator.

I think in the unit tests, because I need to create operations, which fail N times and then succeed, I have that logic wrapped up into a produce function, which is just returning a closure. i.e.
func produce() -> () -> PostMachineUpdateOperation? {
    // Return a closure
    return {
        // Return an instance of PostMachineUpdateOperation or nil
    }
}

Given something like that you can then also do:
let operation = RetryOperation(anyGenerator(produce()), retry: retry)

haha, its one of those things I was struggling from this morning and you solved it with two lines.
This helped.
let operation = RetryOperation(anyGenerator { PostMachineUpdateOperation() }, retry: retry) See docs here on anyGenerator.

@vamsiac - great. I'm gonna close this for now. Please re-open if you have any more questions on RetryOperation. 😀


Remove the use of actions-rs/* actions

Codecov Report

Merging #397 (13c9b10) into main (3edbd48) will increase coverage by 0.05%.
 Consider uploading reports for the commit c7eef56 to get more accurate results

@@            Coverage Diff             @@
##             main     #397      +/-   ##
==========================================
+ Coverage   92.23%   92.28%   +0.05%
==========================================
  Files          51       51
  Lines       14860    14860
==========================================
+ Hits        13706    13714       +8
+ Misses       1154     1146       -8

Impacted Files
Coverage Δ

crates/crochet_parser/src/lib.rs
94.32% <0.00%> (-0.53%)
:arrow_down:

crates/crochet_infer/src/util.rs
79.61% <0.00%> (-0.16%)
:arrow_down:

crates/crochet_codegen/src/d_ts.rs
80.07% <0.00%> (+3.48%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


decide upon the appropriate formatting for footnotes

Footnotes are common in the text. See for example page 513.

See Amherst College. I and parenthetical text below
century sermon more than ten years ago, and is still able to
preach, has always been a warm friend of Amherst College. l
Entering College when he was only fourteen, he was a boy in
stature and a boy in his love of ease and pleasure till his Junior
year, wnen for the first time he began to awake to a genuine
love of those studies in which his strength has ever since lain,
viz., classics, rhetoric and belles-lettres. He graduated with
highly respectable standing in the Class of '39, of which Dr.
Huntington was the Valedictorian, and of which some of the
brightest ornaments, Bancroft, Miller, Palmer and others, died
within a few years after their graduation. He then taught with
marked success, for some years, in Monson Academy and Willis-
ton Seminary. He studied law also, before the great questions
of personal religion and his life-work were settled. At length
he went to Andover where these questions were made clear
to him, and where he completed the theological course in 1845.
The same year he took charge of the Congregational Church
at Brookline, Mass. In November, 1846, he accepted a call to
the Church of the Pilgrims in Brooklyn, of which he is still
pastor. Our readers need not be told that that church is the
center, and its pastor the head-center of Congregationalism in
" the City of Churches." Nor is his power confined to the pulpit
or within the pale of the church, still less of any denomination.
His influence is felt everywhere. The Brooklyn Historical So-
ciety, with its Library and Museum, is his foster-child. Litera-
ture, art, politics, morals, education and religion all feel his
guiding and inspiring touch.
(1 Mrs. Billings of Conway, who might well be called one of the founders, was of
the Storrs family.)

This issue will be closed when a plan for how to handle footnotes is proposed and agreed upon.

Our proposal is to extract the footnote sections from the pages and then paste them into the section they correspond to enclosed in square brackets. This would have to be done manually to some extent. The above page would look like:
century sermon more than ten years ago, and is still able to
preach, has always been a warm friend of Amherst College. [Mrs. Billings of Conway, who might well be called one of the founders, was of the Storrs family.]
Entering College when he was only fourteen, he was a boy in
stature and a boy in his love of ease and pleasure till his Junior
year, wnen for the first time he began to awake to a genuine
love of those studies in which his strength has ever since lain,
viz., classics, rhetoric and belles-lettres. He graduated with
highly respectable standing in the Class of '39, of which Dr.
Huntington was the Valedictorian, and of which some of the
brightest ornaments, Bancroft, Miller, Palmer and others, died
within a few years after their graduation. He then taught with
marked success, for some years, in Monson Academy and Willis-
ton Seminary. He studied law also, before the great questions
of personal religion and his life-work were settled. At length
he went to Andover where these questions were made clear
to him, and where he completed the theological course in 1845.
The same year he took charge of the Congregational Church
at Brookline, Mass. In November, 1846, he accepted a call to
the Church of the Pilgrims in Brooklyn, of which he is still
pastor. Our readers need not be told that that church is the
center, and its pastor the head-center of Congregationalism in
" the City of Churches." Nor is his power confined to the pulpit
or within the pale of the church, still less of any denomination.
His influence is felt everywhere. The Brooklyn Historical So-
ciety, with its Library and Museum, is his foster-child. Litera-
ture, art, politics, morals, education and religion all feel his
guiding and inspiring touch.

I wanted to thank you all for your thoughtful brainstorming: it was very helpful to be able to explore various ways of tagging the footnotes.
I'm happy with @jpapagelis24's proposal that footnotes be moved into bracketed sentences:
... has always been a warm friend of Amherst College. [Mrs. Billings of Conway, who might well be called one of the founders, was of the Storrs family.]

Implementation will involve some work, but that can be deferred until after break.

We need to start to document our process. This issue will be closed when our decision on the formatting of footnotes is documented in an appendix of a report. We should discuss where this might be located.

Perhaps in a Wiki? https://github.com/STAT325-S24/HistoryAmherstCollege/wiki

Closed in favor of #28


[ASK] How to get top 10 recommendations based on a userID and itemID

Hey all!
@gramhagen
I know the trained model has predict function that will give a score for that user-item pair.
What I want to know is, If i give a userID say 1001 and a itemID ( say 1234 it might be a new purchase made by the usedID ). How do I incorporate the new itemID (1234) with userID (1001) and give top 10 recommendations i.e the top 10 items for that particular UserID - 1001  considering the new itemID (1234) and also based on the previous interactions?
Thanks!

It looks like you are developing a collaborative filtering model for a user-to-item recommendation scenario? If so, in the first place a CF model cannot handle cold items, that is to say, your model can only generate top 10 items (ranked by the predicted scores) for the user (1001), which are existing in the previous interactions. Then if you want to incorporate the new item (1234) you need to put the interactions (if there are) between item-1234 and user-1001 to retrain the CF model.
Alternative to the above is to use a content-based model where features of users and items are used for predicting scores.

Thanks @yueguoguo . I am using NCF notebook and I wanted to implement/try the above mentioned feature. So there is no way I can do this without retraining the CF model?
Also thanks for suggesting content-based model, I will try DKN or VW model which was available in this repo

Another quick question? is there a way to recommend top 10 items for a particular user(1001) based on the users history? is there any inbuilt function available in this repo for this feature .
P.S.    I am using NCF deep dive notebook

@karthikraja95 this implementation of NCF doesn't have a method which will automatically return the top 10 items for a particular user, you will need to score all items for a user and sort them.
It would be possible to generate such a method, please feel free create a pull request if you end up developing some code that you think would be useful.


PP-2502 Apply custom branding in Frontend

WHAT
This PR, hooks a Service's custom barding in to our payment pages.
The wiring with backend in done using;

using the views.js to declare customBrandingCssPath and customBrandingImagePath template variables which is used to override GOV.UK Pay branding from our payment pages.
In order to resolve the Service for the Charge in context, I've introduced a middleware resolve_service.js and applied on the payment routes.

Regarding styling bit ( mimicing @jonheslop )

Had to override GOV.UK templates with a monkey patched version (see package.json govuk_template_mustache_inheritance) in order to override fonts options properly
(after discussing with GOV.UK template folks)

Other than that had to further extend charge_ft_test.js in order to make it work with the new middleware. This looks 😖 but leaving the refactoring for another day.

Think I'm happy for this to be merged - I have put a few questions in but none are blockers (but perhaps should be considered in subsequent PRs).
One other thing I wanted to check is the general approach, and who has been involved in kicking this off so far: the approach as it stands (and as I understand it) gives a service a lot of power in that they can specify a css url, and we will happily load that in? What thought are we giving to CSS injection etc? Or are we just trusting services to 'be good'?


Add a Dependency Analyzer in node-swc

Describe the feature
let deps = analyze(`import * from "./a.ts"`);
// [ "./a.ts" ]

Get local dependencies by reading the imports/dynamic import statements.
Example:
https://github.com/nestdotland/deno_swc/blob/master/swc_core/src/analyzer.rs
Babel plugin or link to the feature description
N/A
Additional context
Useful for hot reloading, dependencies analysis, tree shaking.
I'd be happy to open a PR!

Oh... I never thought about those usecases.
I can do it soon. There exists a dependency analysi in spack.

@kdy1 Will it be recursive? Analyzing sub dependencies would be very cool, just need to take care of circular deps!

Of course. Bundlers should fetch all dependencies.

Closing as wontfix. This is too opinionated.


Unsupported features in SSDT needs to be updated

It looks like the unsupported features in SSDT section on this page may need some updates. For example I believe identity columns are now supported from within SSDT.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 787b8c68-0836-9a76-47f3-12937e9034f3
Version Independent ID: 4347cab2-83a2-f09b-c7a5-153d0fc54de3
Content: Install Visual Studio 2019 - Azure Synapse Analytics
Content Source: articles/synapse-analytics/sql-data-warehouse/sql-data-warehouse-install-visual-studio.md
Service: synapse-analytics
Sub-service: sql-dw
GitHub Login: @kevinvngo
Microsoft Alias: kevin

@jasonhorner Thanks for the feedback !  I have assigned this issue to content author to investigate and update the document as appropriate.

#reassign:gaursa

@WilliamAntonRohm Please reassign this to gaursa. Thank you.

#reassign:gaursa

gaursa is not a valid GitHub ID, or is not a collaborator on this repo.

#reassign:gaursa

#assign:julieMSFT

@jasonhorner Thank you for contacting Microsoft and bringing this to our attention. You are correct, identity columns are now supported from within SSDT. I have updated the article and you should see the changes in the next day or so. I will go ahead with closing this issue. The @notification mechanism continues to work even after an item is closed, in case further communication is needed. We hope to hear from you again.

#https://github.com/MicrosoftDocs/azure-docs-pr/pull/153117

#please-close


Compilation instructions for languages that require it

Feature Request
Description
It would be a good idea to have compilation instructions available for languages that require them. For example, C / C++ could have a makefile and rust could have a cargo.toml file available. I had some trouble getting the #686 working, and this might help.
There are a large number of existing implementations that could benefit from this.

For Algorithm Archive Developers

[ ] This feature can be added to the Master Overview (if it cannot be, explain why in a comment below -- lack of technical expertise, not relevant to the scope of this project, too ambitious)
[ ] There is a timeline for when this feature can be implemented
[ ] The feature has been added to the Master Overview
[ ] The feature has been implemented (Please link the PR)

myusernamefrank from twitch chat said:

sometimes other books or tutorials have a general build environment for all examples, maybe might be a good idea if it is one big project for each language which can be just compiled and generates all binaries?

I'm with myusernamefrank, one example should do it, something trivial in the code itself (hello world) but non-trivial in the structure ( import a few libraries...).

So to be clear: You guys would prefer a single chapter on compilation where we explain the process for each language used in the AAA?
I think this is a good idea and would save some headache in the future. I don't know if we also need local build configuration as well?

Instructions globally are fine, but we need some part of it locally as well. For example, some of the code in the archive uses libraries (such as for Rust: generating random numbers). It's useful to have some instructions on which libraries to load locally. In the case of rust, this is a Cargo.toml file (and a small project structure). In the case of c, maybe a makefile? For Javascript maybe a package.json etc.

Including the Cargo.toml file, Julia files should probably all have a Project / Manifest available so users can load everything with just julia --project

Maybe for C/C++ we could have some script files? A combination of "build.sh" and "build.bat", to make it easily cross platform, and not have to depend on too many third parties? Makefiles don't work out of the box on Windows, and things like CMake are a pain to setup and manage.
This could apply to other languages as well, but in C/C++ it would be a good practice to encourage enabling more warnings, to detect more bugs.
For example in C++ I use all of these:
-Wall -Wextra -Wshadow -Wnon-virtual-dtor -Wold-style-cast -Wcast-align -Wunused -Woverloaded-virtual -Wpedantic -Wconversion -Wsign-conversion -Wnull-dereference -Wdouble-promotion -Wformat=2

Talking about Make and CMake, maybe we could use SCons. (I'll add the link once I'm on my PC again)
I'm not entirely sure it is completely cross-platform in all cases, but I think it's close enough in all the common cases, which is what the AAA mostly needs.
I agree with the additional warnings, though.

I would lean towards Make and CMake because it's what I know; however, I also agree that they might not be the right tools for this job. Why is CMake hard to set up? Isn't it just install and go?
If everyone prefers SCons, I am happy to use that as well!

I guess it depends at what level everything is used. CMake is kinda verbose, so for small projects it's a bit annoying to use. But in this case it's probably worth it, and would be easily cross-platform.
I use this repo as a reference for CMake https://github.com/lefticus/cpp_starter_project,
It's got a CMakeLists.txt and then inside the cmake folder, there are more files to help manage CMake. For a 50 line project, it feels very overkill :joy: But I also like being minimalistic (and perhaps too much). Not everything needs to be kept, but something simpler would be better I think.
Although now that I think about it, providing project templates could be a way to manage this.

I don't know CMake at all, so I'm not able to provide some perspective on it.
I know SCons was developed as an alternative to the Make/CMake combo, so that's something to take into account.
Of course, requiring a Makefile or CMake file could create a barrier of entry for beginner, which I think SCons could (maybe) attenuate, assuming we provide sufficient infrastructure to support this (and CMake has the same problem, of course).

Yeah, this is the problem with C++. There are no standards. Let me see if I can write up a MakeFile, CMakeLists.txt, and whatever Scons uses for a simple project to compare

For now, since we do not have a set plan, I will go ahead and merge C/C++ code. But we will come back to add it in later!

likewise for C# project files

Here is a table to keep track of all the languages that currently have implementations, what tool they'll use for the build system, what additional files need to be added, and what interpreter/compiler they'll be calling. Please modify it as you see fit.

language
tool?
file(s)
implementation(s)?

Assembly (x86)

Bash

C and C++

C#

Clojure

Coconut

Common Lisp

D

Elm

Emojicode

Factor

Fortran (90)

Go

Haskell

Java

JavaScript

Julia

Kotlin

LOLCODE

Lua

Matlab/Octave

Nim

OCaml

PHP

Piet

PowerShell

Python

CPython

R

Racket

Ruby

Rust
Cargo
Cargo.toml
rustc

Scala

Scheme

Swift

V

Whitespace

Yeah, this is the problem with C++. There are no standards. Let me see if I can write up a MakeFile, CMakeLists.txt, and whatever Scons uses for a simple project to compare

@leios Have you had the time to do what you said? I made PR #863 to show an example structure for SCons instructions.

Take a look at my draft #862 to see a lot of the implementations

Take a look at my draft #862 to see a lot of the implementations

By the way, if we go with SCons, you'd need to add pip install scons to your Dockerfile.

As apparently we are using SCons as a basis (see this message and the next one for those on the AAA Discord server), a few options come to mind.
First, there is the possibility of providing algorithm-by-algorithm SConstruct files, which provide compilation instructions for each language, and even a way to run programs which produce an output file (see PR #863 for that, but it's still a WIP). It's quite easy to do, but it asks for some copy-pasting, which is annoying, and the compilation instructions are not that straight-forward for some languages, so we would need to provide a library that defines Builder objects that are importable individually.
However, that gives the possibility of running the algorithm directly and redirect stdout to a file (say by running scons stdout.txt or something like that), which would help with automated verification efforts for issue #824 by providing a unified way to run the algorithms.
One other way would be to provide several language targets (not sure exactly how to do it for whole directories, but I'm sure it's possible), and build each algorithm (just copy files for languages like Python) in the correct folder.
This one demands more work, but seems easier to deal with, because we can just create Builder objects at the root and just propagate them down the hierarchy.
That would make things easier for contributors (simply provide an SConscript file which imports all the variables and uses the Builder objects we provide), compared to the approach cited above.
However, the main downside is that it would not help with the automated verification efforts, in the sense that it would compile all the algorithms in one language.
Of course, there is always the option of doing both (providing both SConscript and SConstruct files), which is… okay, I guess? I'm not going to lie, this is not my favourite way of handling the situation, but it gives users the choice.
So, what do you guys think?

I think having several language targets makes the most sense here and would be happy to see things move in that direction, but I think we need @stormofice to chime in here.

So the main point with that approach would be that we would have to have a build directory where all compiled binaries (or copied files for interpreted languages) would be inside their respective language folders, and algorithms subfolders (or maybe we could drop the subfolders).
To run them, users would need to go to the respective folders, and run the binaries (we should probably make sure that there are shebangs in the files for interpreted languages and make them executable) with ./filename.
With the Docker container, we can ensure Ubuntu 20.04 behaviour, so I don't think we have a problem with that.
It will take some time to implement, though, as I'm not really familiar enough with SCons Builder objects yet.


Add support for micro benchmarks to core with reporting

purpose

provide a performance hint on targeted system
provide performance feedback to detect regressions and improvements
provide a better comparison between multiple implementations (storage options etc..)

to do

[x] Add a BenchmarkExecutor that executes a benchmark suite.
[ ] Add a BenchmarkReporter that creates test reports
[x] Test reports should aggregate on type (network, storage) implementation (class) and test case (name).
[ ] Test results are to be printed to console if invoked directly (ie: in-IDE main)
[ ] May be started from the Launcher with CommandExecutor (--benchmark report=html or text as default)
[ ] If benchmark report type  is html check if it makes sense to open the output file after writing to it.
[ ] Include the benchmark html report in the published testreport.zip

requirements

create a test that generates an index value to provide performance over time and to decouple results from hardware capacity.
on startup check that the appropriate JVM arguments are provided for benchmarking.

[ ] - Add support for templates in the report using jade.
[ ] - Add support for dynamic atributes using json serialize

[ ] add support for annotations.. @Benchmark(name="")

[ ] add diagrams to HTML report.

[ ] groups and test groups are to be collapsed as default, for better overview and navigation.

Removed parallelization parameter to testing groups. It increased the implementations complexity and did not provide enough value. There has been some issues with the implementation regarding the serialization of benchmarks and orderly executions. The default stack was not large enough to work with Future::compose and queues did not work very well. The current solution is very aggressive and does not use any backpressure.

[ ] reduce top and bottom margin for diagrams.
[ ]  save/serialize results to file by date
[ ] convert ms to HH:mm:ss.zzz as some tests may be long running
[ ] add more benchmarks.
[ ] add tests

Added --benchmark to launcher and --iterations  parameter support.

Skipped including benchmark report in testreport. Closing now.


Project is Done! Set up Distributables and Final Project

Project is Done! Merge into Main, Final Checks for everything.

Love to see it, so proud of us!!

bro what


Qr from file

Please add qr from file

will add in the future

Hello!
Are you suggesting to add an option to scan a QR code from an image file (e.g. screenshot)?
I suppose, this would be a great addition for web/desktop users.
However, I'd suggest using manual creation for such cases, as it can be easier.
Cheers.


How to calculate "Video Detailed Description (Score)" as shown in blogs

Hi, thanks for the great works~!
I just want to ask that in this blog, it contains a "Video Detailed Description (Score)" metric. How to get or calculate it?
Thanks~!

Please refer there: https://github.com/LLaVA-VL/LLaVA-NeXT/blob/video_inference/llava/eval/evaluate_benchmark_video_detail_description.py