couldn't replicate  scores on en_conll2003

I'm sorry to bother you but I couldn't replicate same scores on en_conll2003 dataset.
I only reproduced to 92.12, 1.4 lower than yours.
I check my dataset and ensure the labels are the same as conll2003 annotate software. So I think my en_conll2003 is complete. This dataset is download from here .
Then I write a script named "data_process.py"(python3) to convert conllx format to your dict format. I use '-DOCSTART- -X- -X- O' to split it. I think this script is no problem because there is no error in extract_feature.sh and evaluate.py and I also check the processed dataset. so I think the difference between your dataset and mine is very small.
environment:
cuda 9.0 + cudnn 7.6.5 + python 2.7 + tensorflow-gpu 1.12
the Pre-trained models from your link "acl2020 best models"
bert_large_case
glove 6B.300d for embedding

the only difference is I replace fastText with glove 6B because I couldn't find fasttext/cc.en.300.vec.filtered, I just find a vector file named "cc.en.300.vec" from fastText. I met a error when I try "cc.en.300.vec", this is the first I try fastText and I don't know what happened. The error log ls as follow:
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=4 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../pretrained/cc.en.300.vec"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../pretrained/cc.en.300.vec...
Traceback (most recent call last):
  File "evaluate.py", line 15, in <module>
    model = biaffine_ner_model.BiaffineNERModel(config)
  File "/data1/zhaoxiaofeng/biaffine-ner/biaffine_ner_model.py", line 17, in __init__
    self.context_embeddings = util.EmbeddingDictionary(config["context_embeddings"])
  File "/data1/zhaoxiaofeng/biaffine-ner/util.py", line 234, in __init__
    self._embeddings = self.load_embedding_dict(self._path)
  File "/data1/zhaoxiaofeng/biaffine-ner/util.py", line 251, in load_embedding_dict
    assert len(embedding) == self.size
AssertionError

Then I continue to use glove 6B because I think embedding shouldn't have huge impact.
This is the log of the en_conll2003 Pre-trained models from your link "acl2020 best models":
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=6 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../glove/glove.6B.300d.txt"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../glove/glove.6B.300d.txt...
Done loading word embeddings.
/data0/zhaoxiaofeng/.conda/envs/biaffine-ner/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-24 00:13:43.106943: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-24 00:13:45.888032: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:88:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2020-10-24 00:13:45.888082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-10-24 00:13:46.248727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-24 00:13:46.248776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2020-10-24 00:13:46.248784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2020-10-24 00:13:46.249496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11378 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:88:00.0, compute capability: 6.1)
Restoring from logs/eng_conll03/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 21 second, 2207.99 w/s
Mention F1: 90.30%
Mention recall: 89.24%
Mention precision: 91.39%
****************SUB NER TYPES********************
ORG F1: 89.76%
ORG recall: 91.81%
ORG precision: 87.80%
MISC F1: 77.93%
MISC recall: 73.93%
MISC precision: 82.38%
PER F1: 95.99%
PER recall: 96.29%
PER precision: 95.70%
LOC F1: 90.25%
LOC recall: 86.27%
LOC precision: 94.61%

This is the log of the model I train again:
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=6 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../glove/glove.6B.300d.txt"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../glove/glove.6B.300d.txt...
Done loading word embeddings.
/data0/zhaoxiaofeng/.conda/envs/biaffine-ner/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-24 16:22:56.141437: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-24 16:22:59.562773: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:88:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2020-10-24 16:22:59.562834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-10-24 16:22:59.977785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-24 16:22:59.977846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2020-10-24 16:22:59.977854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2020-10-24 16:22:59.978513: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11378 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:88:00.0, compute capability: 6.1)
Restoring from logs/eng_conll03/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 21 second, 2133.06 w/s
Mention F1: 92.12%
Mention recall: 92.30%
Mention precision: 91.94%
****************SUB NER TYPES********************
ORG F1: 91.08%
ORG recall: 92.17%
ORG precision: 90.01%
MISC F1: 81.81%
MISC recall: 82.62%
MISC precision: 81.01%
PER F1: 96.64%
PER recall: 97.03%
PER precision: 96.26%
LOC F1: 93.16%
LOC recall: 91.91%
LOC precision: 94.45%

The raw_en_conll2003 and the en_conll2003 I processed and the data_process.py(python3) are in sharelink1 or sharelink2, and the extract_feature.sh and experiments.conf I used also in it. You can check it.
Do you know what's wrong with me?
The dataset, the fastText embedding, or the hyperparameter?
Can you help me? Thank you.

@zhaoxf4 I've checked your data, they are identical to mine and I also tested on my best model they give the same results I reported in the paper. The log is attached below. I got at least 93 over 5 runs, 92.1 is way too low:) I can find three differences:

the fasttext embedding, but I don't think this would be a large issue if you retrain the system yourself. The cc.en.300.vec.filtered is just a subset of the cc.en.300.vec that only contains words appeared in the dataset, so that it loads faster. The reason you got the error is you need to remove the first line of the cc.en.300.vec which is not an embedding.

I train the system using the concatenation of train and dev set and without early stopping.

Due to typos in the early version of the extract_bert_features.sh, the --window_size I actually used is not 128 but its default value 511 in **extract_features.py **, so you need change it to 511 in order to get the same results. I've just updated the extract_bert_features.sh to make the --window_size = 511 to avoid confusing.

WARNING:tensorflow:From` /homes/juntao/miniconda2/envs/tf15/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../fasttext/cc.en.300.vec.filtered"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "../cogsci/bert-model/bert_conll03_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "train_dev.conll03.jsonlines"
eval_path = ""
test_path = "test.yu.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../fasttext/cc.en.300.vec.filtered...
Done loading word embeddings.
use_lee_lstm
/homes/juntao/miniconda2/envs/tf15/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-26 11:15:59.377855: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-26 11:15:59.683691: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:09:00.0
totalMemory: 10.92GiB freeMemory: 10.77GiB
2020-10-26 11:15:59.683785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2020-10-26 11:16:00.103129: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-26 11:16:00.103192: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917]      0
2020-10-26 11:16:00.103212: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0:   N
2020-10-26 11:16:00.103399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10428 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:09:00.0, compute capability: 6.1)
Restoring from logs/eng_conll03/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 29 second, 1588.77 w/s
test Mention F1: 93.46%
test Mention recall: 93.31%
test Mention precision: 93.62%
93.6&93.3&93.5
****************SUB NER TYPES********************
ORG F1: 92.51%
ORG recall: 93.02%
ORG precision: 92.02%
MISC F1: 84.77%
MISC recall: 84.47%
MISC precision: 85.08%
PER F1: 97.55%
PER recall: 97.40%
PER precision: 97.70%
LOC F1: 94.11%
LOC recall: 93.35%
LOC precision: 94.88%

@juntaoy
Hi, do you have any results with BiLSTM-Softmax/CRF model in CoNLL 03 en? I'm trying to reproduce the results reported in the BERT paper with document contextual but failed. I wonder whether the embedding extracted in biaffine-ner can successfully reproduce it. Thank you.

@wangxinyu0922 I did 3 runs using BiLSMT+CRF model on CoNLL 02 English, the best I can get is 91.9 a bit lower than the BERT paper's 92.8. They might get lucky for that number:)

Thank you, could you give me some details for your CRF model so that I can do more experiments on the topic

Is it trained on train+dev set?
Is it trained following the configuration/hyperparameters in your paper?

Yes, it is the same as in my paper, apart from the window_size is not 128 but 512, so make sure you download the latest  extract_bert_features.sh
I train it on the train+dev for 80k steps which is about 80 epoch

Can you share your cc.en.300.vec.filtered? I cannot reproduce the score of your trained NER model, it seems that the fasttext file is distinct

Well, I successfully reproduce the F1 after I use the jsonlines file provided by @zhaoxf4 .
Running experiment: eng_conll03_new
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "best_models/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "./cc.en.300.vec"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "./bert_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "extract_bert_features/train_dev.eng.conll03.jsonlines"
eval_path = ""
test_path = "new_conll_03_test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03_new"
Loading word embeddings from ./cc.en.300.vec...
Done loading word embeddings.
/home/wangxy/anaconda3/envs/neuronlp/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-28 11:30:31.376861: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2020-10-28 11:30:31.654372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: TITAN V major: 7 minor: 0 memoryClockRate(GHz): 1.455
pciBusID: 0000:3b:00.0
totalMemory: 11.75GiB freeMemory: 6.33GiB
2020-10-28 11:30:31.654416: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-10-28 11:30:32.224585: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-28 11:30:32.224634: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2020-10-28 11:30:32.224646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2020-10-28 11:30:32.224883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6076 MB memory) -> physical GPU (device: 0, name: TITAN V, pci bus id: 0000:3b:00.0, compute capability: 7.0)
Restoring from logs/eng_conll03_new/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 22 second, 2070.13 w/s
Mention F1: 93.47%
Mention recall: 93.33%
Mention precision: 93.62%
****************SUB NER TYPES********************
ORG F1: 92.51%
ORG recall: 93.02%
ORG precision: 92.02%
MISC F1: 84.86%
MISC recall: 84.62%
MISC precision: 85.10%
PER F1: 97.55%
PER recall: 97.40%
PER precision: 97.70%
LOC F1: 94.11%
LOC recall: 93.35%
LOC precision: 94.88%

@wangxinyu0922 the use_lee_lstm will not affect the results:) the final version does use the custom LSTM from Lee et al 2018 system. It is just some experiment I did to see if we can get the same results by using the default TensorFlow LSTM, the answer is yes the improvement from custom LSTM is minimal (0.1-0.2).

@juntaoy OK, I think I have replicated it on eng_conll2003 approximately.
the six evaluating results on  six re-train models (windows_size=511, train+dev for train 80000 steps) are as follow:

index\time
1
2
3
4
5
6

F1
93.14
93.07
92.83
93.16
93.26
93.00

recall
93.43
92.30
92.30
93.22
93.48
92.81

precision
92.86
93.07
93.37
93.10
93.04
93.19

and evaluate your ACL2020 best model can gain:
Mention F1: 93.47%
Mention recall: 93.33%
Mention precision: 93.62%
****************SUB NER TYPES********************
ORG F1: 92.51%
ORG recall: 93.02%
ORG precision: 92.02%
MISC F1: 84.86%
MISC recall: 84.62%
MISC precision: 85.10%
PER F1: 97.55%
PER recall: 97.40%
PER precision: 97.70%
LOC F1: 94.11%
LOC recall: 93.35%
LOC precision: 94.88%

Are there any random items that didn't set seed? I want to know the reason of differences between six times and I'm checking the code.

I didn't set seed for the final version. I tried some times ago, and for an unknown reason, after I fixed both python seed and TensorFlow seed I still get different results, it somehow does not work for my code, not sure if this is because of the multiple threading I used.

很抱歉打扰您，但我无法在 en_conll2003 数据集上复制相同的分数。 我只复制到92.12，比你的低 1.4 。 我检查了我的数据集并确保标签与conll2003 annotate software相同。所以我认为我的 en_conll2003 是完整的。这个数据集是从这里下载的。 然后我编写了一个名为“ data_process.py”(python3)的脚本来将 conllx 格式转换为您的 dict 格式。我使用' -DOCSTART- -X- -X- O'来分割它。我认为这个脚本没有问题，因为extract_feature.sh和evaluate.py没有错误，我还检查了处理后的数据集。所以我认为您的数据集和我的数据集之间的差异非常小。
环境：
cuda 9.0 + cudnn 7.6.5 + python 2.7 + tensorflow-gpu 1.12
the Pre-trained models from your link "acl2020 best models"
bert_large_case
glove 6B.300d for embedding

唯一的区别是我用glove 6B替换了 fastText因为我找不到fasttext/cc.en.300.vec.filtered ，我只是从fastText中找到了一个名为**“cc.en.300.vec”的矢量文件。我在尝试"cc.en.300.vec"时遇到错误，这是我第一次尝试 fastText，我不知道发生了什么。错误日志如下：**
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=4 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../pretrained/cc.en.300.vec"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../pretrained/cc.en.300.vec...
Traceback (most recent call last):
  File "evaluate.py", line 15, in <module>
    model = biaffine_ner_model.BiaffineNERModel(config)
  File "/data1/zhaoxiaofeng/biaffine-ner/biaffine_ner_model.py", line 17, in __init__
    self.context_embeddings = util.EmbeddingDictionary(config["context_embeddings"])
  File "/data1/zhaoxiaofeng/biaffine-ner/util.py", line 234, in __init__
    self._embeddings = self.load_embedding_dict(self._path)
  File "/data1/zhaoxiaofeng/biaffine-ner/util.py", line 251, in load_embedding_dict
    assert len(embedding) == self.size
AssertionError

然后我继续使用glove 6B，因为我认为嵌入不应该有很大的影响。 这是来自链接“acl2020 最佳模型”的 en_conll2003 预训练模型的日志：
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=6 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../glove/glove.6B.300d.txt"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../glove/glove.6B.300d.txt...
Done loading word embeddings.
/data0/zhaoxiaofeng/.conda/envs/biaffine-ner/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-24 00:13:43.106943: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-24 00:13:45.888032: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:88:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2020-10-24 00:13:45.888082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-10-24 00:13:46.248727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-24 00:13:46.248776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2020-10-24 00:13:46.248784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2020-10-24 00:13:46.249496: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11378 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:88:00.0, compute capability: 6.1)
Restoring from logs/eng_conll03/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 21 second, 2207.99 w/s
Mention F1: 90.30%
Mention recall: 89.24%
Mention precision: 91.39%
****************SUB NER TYPES********************
ORG F1: 89.76%
ORG recall: 91.81%
ORG precision: 87.80%
MISC F1: 77.93%
MISC recall: 73.93%
MISC precision: 82.38%
PER F1: 95.99%
PER recall: 96.29%
PER precision: 95.70%
LOC F1: 90.25%
LOC recall: 86.27%
LOC precision: 94.61%

这是我再次训练的模型的日志：
(biaffine-ner) zhaoxiaofeng@omnisky:/data1/zhaoxiaofeng/biaffine-ner$ CUDA_VISIBLE_DEVICES=6 python evaluate.py eng_conll03
Running experiment: eng_conll03
ffnn_size = 150
ffnn_depth = 2
filter_widths = [
  3
  4
  5
]
filter_size = 50
char_embedding_size = 8
char_vocab_path = "logs/char_vocab.eng.conll03.txt"
context_embeddings {
  path = "../glove/glove.6B.300d.txt"
  size = 300
}
contextualization_size = 200
contextualization_layers = 3
lm_size = 1024
lm_layers = 4
lm_path = "bert-model/bert_conll2003_features.hdf5"
max_gradient_norm = 5.0
lstm_dropout_rate = 0.4
lexical_dropout_rate = 0.5
dropout_rate = 0.2
optimizer = "adam"
learning_rate = 0.001
decay_rate = 0.999
decay_frequency = 100
train_path = "corpus/en_conll2003/train.json"
eval_path = "corpus/en_conll2003/dev.json"
test_path = "corpus/en_conll2003/test.json"
ner_types = [
  "ORG"
  "MISC"
  "PER"
  "LOC"
]
eval_frequency = 500
report_frequency = 100
log_root = "logs"
max_step = 80000
flat_ner = true
log_dir = "logs/eng_conll03"
Loading word embeddings from ../glove/glove.6B.300d.txt...
Done loading word embeddings.
/data0/zhaoxiaofeng/.conda/envs/biaffine-ner/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:112: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2020-10-24 16:22:56.141437: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-24 16:22:59.562773: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: TITAN Xp major: 6 minor: 1 memoryClockRate(GHz): 1.582
pciBusID: 0000:88:00.0
totalMemory: 11.91GiB freeMemory: 11.76GiB
2020-10-24 16:22:59.562834: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2020-10-24 16:22:59.977785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-24 16:22:59.977846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2020-10-24 16:22:59.977854: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2020-10-24 16:22:59.978513: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11378 MB memory) -> physical GPU (device: 0, name: TITAN Xp, pci bus id: 0000:88:00.0, compute capability: 6.1)
Restoring from logs/eng_conll03/model.max.ckpt
Loaded 231 eval examples.
Evaluated 1/231 examples.
Evaluated 11/231 examples.
Evaluated 21/231 examples.
Evaluated 31/231 examples.
Evaluated 41/231 examples.
Evaluated 51/231 examples.
Evaluated 61/231 examples.
Evaluated 71/231 examples.
Evaluated 81/231 examples.
Evaluated 91/231 examples.
Evaluated 101/231 examples.
Evaluated 111/231 examples.
Evaluated 121/231 examples.
Evaluated 131/231 examples.
Evaluated 141/231 examples.
Evaluated 151/231 examples.
Evaluated 161/231 examples.
Evaluated 171/231 examples.
Evaluated 181/231 examples.
Evaluated 191/231 examples.
Evaluated 201/231 examples.
Evaluated 211/231 examples.
Evaluated 221/231 examples.
Evaluated 231/231 examples.
Time used: 21 second, 2133.06 w/s
Mention F1: 92.12%
Mention recall: 92.30%
Mention precision: 91.94%
****************SUB NER TYPES********************
ORG F1: 91.08%
ORG recall: 92.17%
ORG precision: 90.01%
MISC F1: 81.81%
MISC recall: 82.62%
MISC precision: 81.01%
PER F1: 96.64%
PER recall: 97.03%
PER precision: 96.26%
LOC F1: 93.16%
LOC recall: 91.91%
LOC precision: 94.45%

我处理的raw_en_conll2003和en_conll2003以及data_process.py(python3)在sharelink1或sharelink2中，我也在其中使用了extract_feature.sh和Experiments.conf。你可以检查一下。
你知道我怎么了吗？ 数据集、fastText 嵌入还是超参数？ 你能帮助我吗？谢谢你。

您好，我用了你提供的数据集和模型设置复现了eng_conll2003实验，但是我碰到了模型路径保存的问题，我不知道改如何修改模型保存路径。。。


Oracle baseline

There seems to be no code for reproducing the Oracle baseline. Only for Lead baseline :
https://github.com/nlpyang/PreSumm/blob/ba17e95de8cde9d5ddaeeba01df7cace584511b2/src/models/trainer.py#L292-L293
How can we reproduce the Oracle baseline ?

same question !

There seems to be no code for reproducing the Oracle baseline. Only for Lead baseline :
  PreSumm/src/models/trainer.py

    Lines 292 to 293
  in
  ba17e95

       if (cal_lead):

           selected_ids = [list(range(batch.clss.size(1)))] * batch.batch_size

How can we reproduce the Oracle baseline ?

they have the correct lead and oracle code at Trainer.test() in src/models/trainer_ext.py, but i have no idea why in the train.py they call baseline when args.task == 'abs' rather than construct another baseline function in train_extractive.py and use it when args.task == 'ext'.

they have the correct lead and oracle code at Trainer.test() in src/models/trainer_ext.py, but i have no idea why in the train.py they call baseline when args.task == 'abs' rather than construct another baseline function in train_extractive.py and use it when args.task == 'ext'.

I think it's because the authors reused their code from their first, extractive-only repository
@nlpyang is it the same code used to compute baseline in abstractive settings ?


Cancel the loading process

Discussed in https://github.com/google/model-viewer/discussions/2591

Originally posted by Godelaune July 19, 2021
Hi,
I'd like to know if there is a way to cancel the loading-progress?
I figured out that even destroying all the DOM of model-viewer didn't stop it from loading the 3D-file in background.
Thank you for the good work and best regards!

Any updates on this?

Alas no, hasn't made it far enough up the priority list. PRs welcome!


🎈 11.3.0

🎈 Release
Preparation for 11.3.0 release.
Comparison to 11.2.2 https://github.com/osrf/sdformat/compare/sdformat11_11.2.2...sdf11
Needed by https://github.com/ignitionrobotics/ign-sensors/pull/154
Checklist

[x] Asked team if this is a good time for a release
[x] There are no changes to be ported from the previous major version
[x] No PRs targeted at this major version are close to getting in
[x] Bumped minor for new features, patch for bug fixes
[x] Updated changelog
[ ] Updated migration guide (as needed)
[ ] Link to PR updating dependency versions in appropriate repository in ignition-release (as needed):

Note to maintainers: Remember to use Squash-Merge

Codecov Report

Merging #699 (edaab47) into sdf11 (3f11d07) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##            sdf11     #699   +/-   ##
=======================================
  Coverage   88.08%   88.08%
=======================================
  Files          73       73
  Lines       11023    11023
=======================================
  Hits         9710     9710
  Misses       1313     1313

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f11d07...edaab47. Read the comment docs.


use Distributed and MergeTree ENGINE with shard and relica,select the data error

- my metrika.xml file as follow:
<clickhouse_remote_servers>
    <devops_shard_one>
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>host1</host>
                <port>9000</port>
            </replica>
            <replica>
                <host>host3</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>host2</host>
                <port>9000</port>
            </replica>
            <replica>
                <host>host4</host>
                <port>9000</port>
            </replica>
        </shard>
    </devops_shard_one>
</clickhouse_remote_servers>

- then i create table on host1,host2,host3,host4,the command as follow:
CREATE TABLE ontime_local (FlightDate Date,Year UInt16) ENGINE = MergeTree(FlightDate, (Year, FlightDate), 8192);
CREATE TABLE ontime_all AS ontime_local ENGINE = Distributed(devops_shard_one, test, ontime_local, rand());
- then i insert two data on host1,the command as follow:
insert into ontime_all (FlightDate,Year)values('2001-10-12',2001);
insert into ontime_all (FlightDate,Year)values('2001-10-12',2002);
then i select data on host1,host2,host3,host4,the data i can see as follow:

i can not see all the data on any clickhouse server .What's wrong with me?
thank！！！

i fix my problem .the internal_replication must by set "false"


ccm json - Couldn't change permissions to use sstable2json

Getting this error while trying to see the json output of a sstable.
ccm node1 json -k test -c test "C:\Users\user1\Desktop\testjson.txt"

running
WARN: Couldn't change permissions to use sstable2json.
WARN: If it didn't work, you will have to do so manually.
['C:\Users\user1\.ccm\c2\node1\data0\test\test-71ff922070dc11e7864b5ff09c5396dc\mb-1-big-Data.db']
-- mb-1-big-Data.db -----
Traceback (most recent call last):
File "E:\Studies\Cassandra\ccm\ccm-master\ccm.py", line 74, in
cmd.run()
File "E:\Studies\Cassandra\ccm\ccm-master\ccmlib\cmds\node_cmds.py", line 487, in run
enumerate_keys=self.options.enumerate_keys)
File "E:\Studies\Cassandra\ccm\ccm-master\ccmlib\node.py", line 885, in run_sstable2json
subprocess.call(args, env=env, stdout=out_file)
File "C:\Program Files\Python27\lib\subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Program Files\Python27\lib\subprocess.py", line 710, in init
errread, errwrite)
File "C:\Program Files\Python27\lib\subprocess.py", line 958, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

Python 2.7, Windows 10
Thanks,
Dinesh

What version of Cassandra is your ccm cluster?

C* version 3.7

Cassandra 3.7 doesn't have sstable2json anymore, you'll want to use sstabledump. I'll change the command to print a warning to the user, when they try to run ccm node json on a cluster w/o the tool.

I see ccm command shows

json - Call sstable2json/sstabledump on the sstables of this node

I do see the sstabledump tool under C:\Users\user1\.ccm\repository\3.7\tools\bin, still not getting the json output of a sstable.
But, I don't see the sstabledump tool under C:\Users\user1\.ccm\c2\node1\bin
Am I missing something?

Okay, I see the issue now. ccm should be trying to run sstabledump out of .ccm\repository\3.7\tools\bin if sstable2json does not exist. The check if sstable2json exists, must be flawed in some way on Windows.

I don't see sstable2json under .ccm\repository\3.7\tools\bin since it's been deprecated. So, it should try running with sstabledump.
Are you planning for this fix?

I don't see sstable2json under .ccm\repository\3.7\tools\bin since it's been deprecated. So, it should try running with sstabledump.

Yes, I agree, sorry if I was unclear.

Are you planning fix this?

It's a valid bug that should be fixed, but since it seems to only happen on Windows, and I don't have a windows box at the moment, it's not something I'm going to personally fix anytime soon. I would review/merge a PR that fixed it though, and will be keeping this issue open.

Thanks Philip!
I copied the old C* version's (2.2.8) sstable2json files(sstable2json, sstable2json.bat) into .ccm\repository\3.7\tools\bin and tried running the ccm node json command, it worked.
I will consider this as a temporary workaround till you fix it.

I think the better workaround is just to run sstabledump directly. There's no guarantee that sstable2json will work on a post 3.0 cluster.

Ok. I know it's a silly question. How should I run sstabledump directly through ccm on windows?

As you said in your original comment, the tool can be found at C:\Users\user1\.ccm\repository\3.7\tools\bin\sstabledump


Avoid padding in converse message header due to alignment issues

Reordering members of converse message header to fix alignment padding

If you run git grep CMK_MSG_HEADER, there are more files that could benefit from this change.


Blazor auth: Remove parts referring to localhost Redirect URI port numbers

The part about providing the port number for the localhost redirect UI in the "Register an AAD B2C app:" section is not needed, as localhost addresses are handled differently. It might make this part of the docs a bit easier to consume.
On the Azure Portal's relevant section (B2C tenant, AAD B2C -> App Registrations -> Select one -> Authentication -> Single Page Application), there is a link: Learn more about Redirect URIs and their restrictions
On the linked page, there is a section regarding localhost port numbers:

Redirect URI (reply URL) restrictions and limitations
[...]
Localhost exceptions
Due to ephemeral port ranges often required by native applications, the port component (for example, :5001 or :443) is ignored for the purposes of matching a redirect URI. As a result, all of these URIs are considered equivalent:
http://localhost/MyApp
http://localhost:1234/MyApp
http://localhost:5000/MyApp
http://localhost:8080/MyApp
[...]

It could simply be stated that no port number needs to be set for localhost URIs, and the above could be linked for more information.
The port numbers are referred again later in the same docs as reminder to set it if it was randomized in the process.
This could simplify all pages that use client-side auth via AAD or AAD B2C.
Additionally, I think it should be noted that all public redirect URIs (like custom domains) should be added here, and it can be added later without issue.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ce89fc06-2c18-fbf8-0ac5-f50d41a67d8a
Version Independent ID: 490b7aec-0d31-9032-cb1d-ffcdfea5c976
Content: Secure an ASP.NET Core Blazor WebAssembly standalone app with Azure Active Directory B2C
Content Source: aspnetcore/blazor/security/webassembly/standalone-with-azure-active-directory-b2c.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @guardrex
Microsoft Alias: riande

Hello @yugabe ... Curious ... did you open this using the This page button and form from the bottom of the topic? Our automated labeling system seems to have failed to label it properly.

Hi @guardrex. Yes, I used the link on the bottom of the page.

Hum ... strange. No worries. I just hope it doesn't start constantly failing. It's been great having all of the labels automatically applied.
I'm out of sick leave today due to a dental procedure yesterday ... I'm getting a shiny new tooth to replace one that rotted out of my head! 🤕😆 All good tho ... Dr. Laggen and his staff in Pensacola did a great job! 100% painless! Still tho, he wants me chill'in for a day to recover. I'll take a look at this tomorrow (Thursday).

Hah, dentists... 😰
No worries, no hurries. It's just that the whole B2C/auth part is the toughest of all the stack imo, so improving this part even by a bit might go a long way.
It sucks this probably relates to multiple pages, as there are some redundancies, but all's for a good cause, I swear.

Hah, dentists... 😰

Yeah! I was a bit stressed out over it ... BUT ... within 15 minutes of arriving, I was knocked out. Immediately after awaking after what seems like mere seconds, I was being wheeled out in a wheelchair to the car. I didn't even feel the IV get placed because the assistant was so good at it. I have one more major procedure for this tooth ... the titanium post about seven months from now ... but have NO concerns or stress over it based on the smooth, no pain experience yesterday of getting that rotten tooth out of my head! 😆
There's a rumor that they'll look at the entire Blazor security design for an upcoming release (.NET 8, perhaps). They might be able to create a new API with more baked-in simplicity to get everything wired up.
We might also be dropping some or most of these Blazor AAD/B2C/Graph API docs in favor of the Azure/MS Identity and Security docs because we're duplicating a lot of coverage that they maintain. Also, they're better at security than I am in some ways. I prefer writing at a lower level. I prefer more gotchas 😈 and TIPS! in the docs. I prefer hybrid-style security docs that are both security topics that have both reference content (facts and API discussion) AND tutorial content (step-by-step to get a successful experience). For those reasons, I like these docs here. Over in Azure docs and MS Identity/Security docs, they don't have this type of coverage. However, we are duplicating a lot of coverage, and it's costly 💰 to maintain. I was going to perform one of my total overhauls ⛏🗻 on the security node, but it's obvious that it will take me a full month to do that. With the potential of a total PU Blazor security overhaul in the coming years, we decided not to invest the 💸 treasure on it. It's just too expensive. I'll perform a quick, light pass instead, and we might cut some/most of the coverage when I do.
These are easy/quick updates to make that you're suggesting, and I think we should work your suggestions in to the docs. For example in addition to what you're suggesting on the port number, it's trivial to add a sentence there for the redirect URI like ...

If you know the production redirect URI for the Azure default host (for example, azurewebsites.net) or the custom domain host (for example, contoso.com), you can also add the production redirect URI at the same time that you're providing the localhost redirect URI. Be sure to include the port number for a non-:443 ports in any production redirect URIs that you add.


concurrency adjustment should not happen when mean is 0

Handles division by zero in adjust_concurrency function
I'm not sure how to handle it though so I just removed adjustment

Looks good; thanks for the fix!

If possible can you please release new version?
My colleague often had this issue to break runs as it makes balter to crash due to trying impossible concurrency values?

If possible can you please release new version? My colleague often had this issue to break runs as it makes balter to crash due to trying impossible concurrency values

New version released :)


TCP connection drops with LB mode: DSR ON when trying to reach ingress endpoint from outside the cluster

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
Components:

VMware (vSphere)
Talos Linux (v1.6.1)
Client VM (Ubuntu 22.04.4 LTS)

Cilium installed with kubeProxyReplacement = true.
Also we have installed some pods with configured Ingress (HTTPS disable).
When we’re trying to initialize TCP connection from Client VM to Ingress with DSR ON - it breaks the connection. Our guess it’s because we receive a response from some Node and not from our ingress endpoint (Grafana LB).

Note.  We see that response packets coming from node IP are being rejected.

Command used: curl -k http://grafana.e2.powercom.dev
Control Plane IP: <IP_ADDRESS>
Talos-node-01 IP: <IP_ADDRESS>
Talos-node-02 IP: <IP_ADDRESS>
Talos-node-03 IP: <IP_ADDRESS>
Grafana LB IP: <IP_ADDRESS>
We use Cilium L2 LB CRD

CiliumL2AnnouncementPolicy
CiliumL2AnnouncementPolicy.txt
CiliumLoadBalancerIPPool
CiliumLoadBalancerIPPool.txt

Screenshot and dumpfile of tcpdump

tcpdump_dsr_on.zip
Important: Nginx Ingress Controller is on Talos-node-01, so the response is from <IP_ADDRESS>
Cilium Version
Client: 1.15.4 9b3f9a8c 2024-04-11T17:25:42-04:00 go version go1.21.9 linux/amd64
Daemon: 1.15.4 9b3f9a8c 2024-04-11T17:25:42-04:00 go version go1.21.9 linux/amd64
Cilium Helm chart: 1.15.4
Kernel Version
Linux EEOPE2_talos-01 6.1.69-talos #1 SMP PREEMPT_DYNAMIC Thu Dec 21 15:48:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Linux EEOPE2_talos-02 6.1.69-talos #1 SMP PREEMPT_DYNAMIC Thu Dec 21 15:48:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Linux EEOPE2_talos-03 6.1.69-talos #1 SMP PREEMPT_DYNAMIC Thu Dec 21 15:48:53 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Kubernetes Version
Client Version: v1.29.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.0
Regression
No response
Sysdump
cilium-sysdump-20240508-152222.zip
Relevant log output
No response
Anything else?
This archive includes additional cilium-config and cilium-pod-config files.
cilium-configs.zip
Cilium Users Document

[ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

This certainly looks like an issue.
Could you try and create a minimal reproducer? Ideally on kind, but other environments would be acceptable. Does it have to be a LoadBalancer service, or are other types of kubernetes services affected?

This might be https://github.com/cilium/cilium/issues/32189 - are you using DSR with Geneve dispatch, but BPF masquerading is disabled?

This might be #32189 - are you using DSR with Geneve dispatch, but BPF masquerading is disabled?

As you can see in configuration I attached:

DSR ON with Geneve
bpf.masquerading false by default - docs
Masquerading (ipv4/ipv6) ON
Masquerading-to-route-source OFF, but it's for iptables. Ref - docs

Cool, thanks for confirming! Then yes, let's dup against https://github.com/cilium/cilium/issues/32189.


react native 0.75.4 (iOS) fetch HTTP Local Network

Description
We can't fetch a local network via HTTP on iOS release mode
Steps to reproduce

Create a new App
Add a button to trigger the code bellow with IP:PORT from a device on your network

  const onPing = () => {
    fetch('http://<IP_ADDRESS>:9100')
      .then((response) => {
        if (response.status === 200) {
          console.log('success')
        } else {
          console.log('error')
        }
      })
      .catch((error) => {
        console.log('network error: ' + error)
      })
  }

Add permissions in Info.plist

	<key>NSAppTransportSecurity</key>
	<dict>
		<key>NSAllowsArbitraryLoads</key>
		<true/>
		<key>NSAllowsLocalNetworking</key>
		<true/>
		<key>NSExceptionDomains</key>
		<dict>
			<key><IP_ADDRESS>:9100</key>
			<dict>
				<key>NSExceptionAllowsInsecureHTTPLoads</key>
				<true/>
				<key>NSIncludesSubdomains</key>
				<true/>
			</dict>
		</dict>
	</dict>
	<key>NSLocalNetworkUsageDescription</key>
	<string>blabla</string>

Launch a release build on iOS

It should accept the request because it's in the NSExceptionDomains
React Native Version
0.75.4
Affected Platforms
Runtime - iOS
Output of npx react-native info
System:
  OS: macOS 14.6.1
  CPU: (11) arm64 Apple M3 Pro
  Memory: 98.17 MB / 18.00 GB
  Shell:
    version: "5.9"
    path: /bin/zsh
Binaries:
  Node:
    version: 20.16.0
    path: ~/.nvm/versions/node/v20.16.0/bin/node
  Yarn:
    version: 3.6.4
    path: ~/.nvm/versions/node/v20.16.0/bin/yarn
  npm:
    version: 10.8.1
    path: ~/.nvm/versions/node/v20.16.0/bin/npm
  Watchman:
    version: 2<IP_ADDRESS>
    path: /opt/homebrew/bin/watchman
Managers:
  CocoaPods:
    version: 1.15.2
    path: /opt/homebrew/bin/pod
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 23.5
      - iOS 17.5
      - macOS 14.5
      - tvOS 17.5
      - visionOS 1.2
      - watchOS 10.5
  Android SDK:
    API Levels:
      - "33"
      - "34"
      - "35"
    Build Tools:
      - 30.0.3
      - 33.0.0
      - 33.0.1
      - 34.0.0
      - 35.0.0
    System Images:
      - android-29 | Intel x86_64 Atom
      - android-35 | Google APIs ARM 64 v8a
      - android-35 | Google Play ARM 64 v8a
      - android-35 | Pre-Release 16 KB Page Size Google APIs ARM 64 v8a
    Android NDK: Not Found
IDEs:
  Android Studio: 2024.1 AI-241.18034.62.2411.12071903
  Xcode:
    version: 15.4/15F31d
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: 17.0.12
    path: /usr/bin/javac
  Ruby:
    version: 2.6.10
    path: /usr/bin/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react: Not Found
  react-native: Not Found
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: true
  newArchEnabled: false
iOS:
  hermesEnabled: true
  newArchEnabled: false

Stacktrace or Logs
Connection 4: received failure notification
Connection 4: failed to connect 1:50, reason -1
Connection 4: encountered error(1:50)
Task <970D7D16-313B-4CD4-BF29-895D4A6F2EE2>.<4> HTTP load failed, 0/0 bytes (error code: -1009 [1:50])
Task <970D7D16-313B-4CD4-BF29-895D4A6F2EE2>.<4> finished with error [-1009] Error Domain=NSURLErrorDomain Code=-1009 "The Internet connection appears to be offline." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x303a21350 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "(null)" UserInfo={_NSURLErrorNWPathKey=unsatisfied (Local network prohibited), interface: en0[802.11], ipv4, ipv6, uses wifi, _kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <970D7D16-313B-4CD4-BF29-895D4A6F2EE2>.<4>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <970D7D16-313B-4CD4-BF29-895D4A6F2EE2>.<4>"
), NSLocalizedDescription=The Internet connection appears to be offline., NSErrorFailingURLStringKey=http://<IP_ADDRESS>:9100/, NSErrorFailingURLKey=http://<IP_ADDRESS>:9100/, _kCFStreamErrorDomainKey=1}

network error: TypeError: Network request failed

Reproducer
https://github.com/Gueoff/Example
Screenshots and Videos
No response

My bad it's not related to RN, we just have to add "Access wifi information" to the certificate and rebuild


Mixer: invalid attributes: duplicate attribute

This is only seen when 1.1 proxy communicates with 1.0 Mixer.
Ostensibly, this is not supported, but It is very helpful for testing.
If this is easy to fix, we should fix it in 1.1
2018-11-16T18:17:43.920654Z    error    api    Report failed:1 error occurred:

* request could not be processed due to invalid attributes: duplicate attribute  (type string, was string)

This is because the lookup fails for a global attribute (the list grew once again).
Words 203 and 205 are not present in 1.0. The code leniently falls back to an empty string, which later leads to a collision.
What should mixer do in general with regards to unknown words from the global dictionary?
/cc @geeknoid @douglas-reid

Which version of the attribute manifest is in use in a 1.1 proxy -> 1.0 mixer scenario ?

I think it's the proxy error code and the query parameter attributes that @bianpengyuan added in between 1.0 and 1.1. We don't really version the manifest itself.

Yeah, I wondering though if the manifest contains those attributes... and what the meaning of the manifest is in such a scenario.
Maybe we should appropriately version the manifests?  That might help with any error responses and/or graceful degradation issues.

The expected protocol here is that Mixer must always be >= to the proxy's
version.
On Fri, Dec 7, 2018 at 2:02 PM Douglas Reid<EMAIL_ADDRESS>wrote:

Yeah, I wondering though if the manifest contains those attributes... and
what the meaning of the manifest is in such a scenario.
Maybe we should appropriately version the manifests? That might help with
any error responses and/or graceful degradation issues.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/issues/10014#issuecomment-445379459, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AVucHaBVmKcIVi9y5Z1TYAfQya2L2rWJks5u2uVfgaJpZM4Ymwb2
.

@mandarjog Per Martins' comment, closing the issue as the requested scenario is not supported.
There's nothing Mixer can do if it doesn't know extra global words.


Llama3 8B with QLoRA ends in: You can't train a model that has been loaded in 8-bit precision on a different device than the one you're training on

System Info
torch==2.1.2
transformers==4.37.2
accelerate==0.26.1
numpy==1.26.3
OS type and version: fedora, 7.9
Python version: 3.11.0

I am using 8*A100-40Gb.

Related bug: https://github.com/huggingface/accelerate/issues/1515

I also tried the same code with brand new env, with the latest versions of every packages. The same error happens there.

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[ ] One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
[ ] My own task or dataset (give details below)

Reproduction
I am calling an extension of QLoRA with meta-llama/Meta-Llama-3-8B-Instruct. I am using an extended version of https://github.com/jondurbin/qlora.git. The main parameters to set are:
--model_name_or_path  meta-llama/Meta-Llama-3-8B-Instruct   --do_train   --lora_modules all   --bf16   --bits 4   --double_quant   --quant_type nf4

Expected behavior
The model should start fine-tuning, but it looks like that accelerate reserves most memory of cuda:0 for some future operation and as a result the remaining memory is not enough for loading any part of the model. As a result, when in accelerate/accelerator.py:1329 it checks for
# if on the first device (GPU 0) we don't care
if (self.device.index is not None) or (current_device_index != 0)`

it founds that current_device_index != 0 and it throws the error:
raise ValueError(
                        "You can't train a model that has been loaded in 8-bit precision on a different device than the one "
                        "you're training on. Make sure you loaded the model on the correct device using for example `device_map={'':torch.cuda.current_device() or device_map={'':torch.xpu.current_device()}"
                    )

To add more details, I set verbose=True in accelerate/utils/modeling.py:infer_auto_device_map() where at while len(modules_to_treat) > 0: which designs the device_map. It returns the following parameters allocation:
Not enough space on 0 to put model (space available 167116948.79999995, module size<PHONE_NUMBER>).
Splitting model.

Treating module model.embed_tokens.
Not enough space on 0 to put model.embed_tokens (space available 167116948.79999995, module size 525336576).
This module cannot be split, going to the next device.

Treating module model.embed_tokens.
Putting model.embed_tokens (size=525336576) on 1 (available=1217790100.8).

Treating module model.layers.
Not enough space on 1 to put model.layers (space available 692453524.8, module size<PHONE_NUMBER>).
Splitting model.layers.

Treating module model.layers.0.
Putting model.layers.0 (size=220209216) on 1 (available=692453524.8).

Treating module model.layers.1.
Putting model.layers.1 (size=220209216) on 1 (available=472244308.79999995).

Treating module model.layers.2.
Putting model.layers.2 (size=220209216) on 1 (available=252035092.79999995).

Treating module model.layers.3.
Not enough space on 1 to put model.layers.3 (space available 31825876.799999952, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.3.
Putting model.layers.3 (size=220209216) on 2 (available=1217790100.8).

Treating module model.layers.4.
Putting model.layers.4 (size=220209216) on 2 (available=997580884.8).

Treating module model.layers.5.
Putting model.layers.5 (size=220209216) on 2 (available=777371668.8).

Treating module model.layers.6.
Putting model.layers.6 (size=220209216) on 2 (available=557162452.8).

Treating module model.layers.7.
Putting model.layers.7 (size=220209216) on 2 (available=336953236.79999995).

Treating module model.layers.8.
Not enough space on 2 to put model.layers.8 (space available 116744020.79999995, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.8.
Putting model.layers.8 (size=220209216) on 3 (available=1217790100.8).

Treating module model.layers.9.
Putting model.layers.9 (size=220209216) on 3 (available=997580884.8).

Treating module model.layers.10.
Putting model.layers.10 (size=220209216) on 3 (available=777371668.8).

Treating module model.layers.11.
Putting model.layers.11 (size=220209216) on 3 (available=557162452.8).

Treating module model.layers.12.
Putting model.layers.12 (size=220209216) on 3 (available=336953236.79999995).

Treating module model.layers.13.
Not enough space on 3 to put model.layers.13 (space available 116744020.79999995, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.13.
Putting model.layers.13 (size=220209216) on 4 (available=1217790100.8).

Treating module model.layers.14.
Putting model.layers.14 (size=220209216) on 4 (available=997580884.8).

Treating module model.layers.15.
Putting model.layers.15 (size=220209216) on 4 (available=777371668.8).

Treating module model.layers.16.
Putting model.layers.16 (size=220209216) on 4 (available=557162452.8).

Treating module model.layers.17.
Putting model.layers.17 (size=220209216) on 4 (available=336953236.79999995).

Treating module model.layers.18.
Not enough space on 4 to put model.layers.18 (space available 116744020.79999995, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.18.
Putting model.layers.18 (size=220209216) on 5 (available=1217790100.8).

Treating module model.layers.19.
Putting model.layers.19 (size=220209216) on 5 (available=997580884.8).

Treating module model.layers.20.
Putting model.layers.20 (size=220209216) on 5 (available=777371668.8).

Treating module model.layers.21.
Putting model.layers.21 (size=220209216) on 5 (available=557162452.8).

Treating module model.layers.22.
Putting model.layers.22 (size=220209216) on 5 (available=336953236.79999995).

Treating module model.layers.23.
Not enough space on 5 to put model.layers.23 (space available 116744020.79999995, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.23.
Putting model.layers.23 (size=220209216) on 6 (available=1217790100.8).

Treating module model.layers.24.
Putting model.layers.24 (size=220209216) on 6 (available=997580884.8).

Treating module model.layers.25.
Putting model.layers.25 (size=220209216) on 6 (available=777371668.8).

Treating module model.layers.26.
Putting model.layers.26 (size=220209216) on 6 (available=557162452.8).

Treating module model.layers.27.
Putting model.layers.27 (size=220209216) on 6 (available=336953236.79999995).

Treating module model.layers.28.
Not enough space on 6 to put model.layers.28 (space available 116744020.79999995, module size 220209216).
This module cannot be split, going to the next device.

Treating module model.layers.28.
Putting model.layers.28 (size=220209216) on 7 (available=72000000000.0).

Treating module model.layers.29.
Putting model.layers.29 (size=220209216) on 7 (available=71779790784.0).

Treating module model.layers.30.
Putting model.layers.30 (size=220209216) on 7 (available=71559581568.0).

Treating module model.layers.31.
Putting model.layers.31 (size=220209216) on 7 (available=71339372352.0).

Treating module model.norm.
Putting model.norm (size=4096) on 7 (available=71119163136.0).

Treating module lm_head.
Putting lm_head (size=1050673152) on 7 (available=71119159040.0).

After this step, I get some logs and the mentioned error:
Loading checkpoint shards: 100%|██████████████████| 4/4 [00:14<00:00,  3.61s/it]
adding LoRA modules...
loaded model
Filter: 100%|█████████████████████████| 427/427 [00:02<00:00, 168.65 examples/s]
Detected kernel version 5.4.17, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
trainable params: 167772160 || all params:<PHONE_NUMBER> || trainable: 2.046361887309209
torch.bfloat16<PHONE_NUMBER> 0.12824983600627698
torch.int8<PHONE_NUMBER> 0.851286545120631
torch.float32 167772160 0.02046361887309209
Traceback (most recent call last):
  File "/home/afshin/.pycharm_helpers/pydev/pydevd.py", line 1496, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afshin/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/afshin/codes/qlora/train.py", line 1490, in <module>
    train()
  File "/home/afshin/codes/qlora/train.py", line 1393, in train
    train_result = trainer.train()
                   ^^^^^^^^^^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/transformers/trainer.py", line 1539, in train
    return inner_training_loop(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/transformers/trainer.py", line 1687, in _inner_training_loop
    model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/accelerate/accelerator.py", line 1227, in prepare
    result = tuple(
             ^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/accelerate/accelerator.py", line 1228, in <genexpr>
    self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/accelerate/accelerator.py", line 1104, in _prepare_one
    return self.prepare_model(obj, device_placement=device_placement)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/afshin/miniconda3/envs/mixtral/lib/python3.11/site-packages/accelerate/accelerator.py", line 1330, in prepare_model
    raise ValueError(
ValueError: You can't train a model that has been loaded in 8-bit precision on a different device than the one you're training on. Make sure you loaded the model on the correct device using for example `device_map={'':torch.cuda.current_device() or device_map={'':torch.xpu.current_device()}

cc @SunMarc


Add origin-private-file-system feature

This is a Baseline 2023 feature.

For the pure OPFS, you need:

FileSystemHandle

FileSystemFileHandle
FileSystemDirectoryHandle

FileSystemSyncAccessHandle
FileSystemWritableFileStream
StorageManager.getDirectory()

These are all specified in the File System Living Standard.
If you want to look at the user-visible file system as well, which shares the interfaces of FileSystemHandle, FileSystemFileHandle, and FileSystemDirectoryHandle, you want to look additionally at:

Window.showOpenFilePicker()
Window.showSaveFilePicker()
Window.showDirectoryPicker()
DataTransferItem.getAsFileSystemHandle()

These are specified in the File System Access Draft Community Group Report.

@ddbeck if you want to leave a big comment and a TODO for this, that would be fine I think. I'm not sure if the file-and-directory-entries feature will survive a closer look, if you're also not sure maybe turn it into a draft?

Correct. File and Directory Entries has nothing to do with the OPFS. It's mostly its spiritual predecessor.

OK, I've prepared two big changes to this PR:

I've made file-and-directory-entries a draft. That feature got less scrutiny and I'm less confident about its suitability as a feature—it might be one of these odd "infrastructure" items that doesn't do anything in particular on its own. I'd like to revisit it later.
I've added most of the interfaces that @tomayac mentioned. Thank you! The one thing I left out was FileSystemWritableFileStream. This seems to be a later addition that Safari doesn't support yet and doesn't appear to be strictly required to use OPFS. I've left TODO comments until we can make concrete progress on https://github.com/web-platform-dx/web-features/issues/866.


Setup Drone CI for the project

In the Oct sprint we decided to use drone ci to be the infrastructure.

A related PR https://github.com/tai271828/attendees-analyzer/pull/29
@Lee-W  you may want to relate the issue number in your git commit messages in the future. Github will automatically relate the PR per the issue.

The very first drone CI run and it passes! https://cloud.drone.io/tai271828/attendees-analyzer/1

Ah yes, I forgot to do so. But, I personally like to mention the related issue or pr in the description of that pr instead of adding them to git commit directly.


Add "featured on" section

https://realpython.com/podcasts/rpp/158/#t=1588
There was also feedback that the exclusios docs were hard to find.

Fixed in https://github.com/basnijholt/rsync-time-machine.py/pull/18.
Thanks again @digiglean!


Suggestion for an Outreach Panel

Dmitri has been working on this area and would be a good candidate to kick start this conversation so I've added him as a reviewer.

Thank you @Mitzi-Laszlo, I think this panel would make a great addition to the Solid community and ecosystem.


docs(de): update contributing file

Checklist

[x] I ran npm test locally and it passed without errors.

See #1610

Thanks for the contribution :) I found one more typo, it would be great if you could fix that one and I'm happy to merge your PR :) thank you!

Sure, will do that asap :)

Done, with some other, minimal changes/optimizations.

@dkundel 👋

@dkundel Just curious: Shouldn't i be automatically added to the contributors list?


Canvas to Blob -> toBlob is not a function

Hey guys, this plugin is amazing though i am having problems with getting the base64 output to my server via ajax call. If i just try to send the getCroppedCanvas.toDataUrl() my browser almost crashes due to the huge base64 string so i am looking at the toBlob method from the jquery cropper.
I was trying to approach it like this:
this.refs.cropper.getCroppedCanvas().toBlob() ...
I get toBlob is not a function; from researching this happens because the whole canvas is not rendered yet so i would have to wrap it inside a "built" function which i don't know how to do in this example.
Can anybody help me with a working example on how to post the cropped image via ajax? Any form is fine.
Regards, Jasa

I tried this.refs.cropper.getCroppedCanvas().toBlob(), and it works.

Did you set ref correctly? if possible please paste your react component here might be helpful.
This react project is based on cropperjs, they have good example of uploading image. https://github.com/fengyuanchen/cropperjs#getcroppedcanvasoptions

Thanks for your response; i figured out that "toBlob" is actually only available on Chrome 50+ and my version vas 49; so after i updated my browser it worked normally. Though i have decided just to work with the dataUrl base64 since the blob function is so unsupported.


Outline colors

Hello,
I'm having difficulty changing the outline color of the inputs when they are active. Is there a way to achieve this?
Thank you in advance.

Hello,
To change the ring & border color you will have to update the Input.styles.ts file as shown on the theming section of the documentation: https://rewind-ui.dev/theming
Looks like though that this is highly requested from other developers that reached out as well, so I am about to add a new color property to all form elements to avoid having to use the theme provider just for that.
I will reply back to you on this issue once I push the update, so that you can test it out.

Hello again Farid. Please try installing version 0.9.2
I have added a color prop on all form components and have set the default to dark as I thought that purple was too opinionated and decided to choose a more neutral default.
Let me know if you have any further questions/considerations!

It's great! Thank you very much for your promptness. It's perfect now!


makenp: allows passing a float as np.array

When trying to summarize float numbers as np.array, this x2num gives a np.array of shape 0. Fixed this corner case.

Just found this not necessary -- I'ill close this PR. Nvmd


Redesign service badge in Lens

Service badge is these.

This design is ugly and this component is not extensible.
We must update this design as soon as possible before improving other components....
New service badge

Base

Clickable

Clickable & Deletable

This PR may fix #2504, too

Please review 🙇
I want this PR merged before continuing #2656.

sorry I looked at this earlier and thought I clicked approve! thanks for the ping


Bump to version 2.2.0

Update podspec to 2.2.0
Fixes #97

Hey there @ApplebaumIan, and thanks for finding out what was wrong with the #97 🎉. I have drafted a new release 2.2.0. As you can see in the description, we have some new features and bug fixes since version 2.1.1.  Therefore, we should have a minor version bump here.
Could you please bump the version in Podspec files to 2.2.0?

👍

👍


doc(FieldTheory/IsAlgClosed/Basic): add a TODO

... which is: prove that if K / k is algebraic, any monic irreducible polynomial over k has a root in K, then K is algebraically closed (in fact an algebraic closure of k).

Discussions: https://github.com/leanprover-community/mathlib4/pull/9488#discussion_r1443865997

By the way, from the reference it looks like that the proof of this result needs purely inseparable argument, so probably it can't be done in this file.

bors r+


Weird cvars issues

sv_anticheatkick (1.7.1 has Disable_AntiCheat_Kick)
sv_minimap (1.7.1 has Disable_Minimap)
sv_sniperline (1.7.1 has NoSniperLine)
As You see 1.7.1 has very weird counterintuitive names for those options...
The names has changed but the values work the same as in 1.7.1 i.e. setting sv_minimap "0" enables the minimap
I've set sv_gravity to 0.06(default value) in server.cfg but during launch it says sv_gravity is now set to: "0.05999999866"
Also this Unable to set sv_maxgrenades: Value must be between 0 and 1, IIRC Soldat always allowed for maximum 5 grenades

I have local fixes for all these, I'll push them soon. I chose to go with changing the behavior, so the name makes sense. Like setting sv_minimap "0" will disable the minimap. Also, the sv_gravity thing is expected. The GRAV variable is a Single, which corresponds to an IEEE 754 single precision float. Due to the way the IEEE 754 encoding works, 0.06 can't be exactly represented, so it uses the closest value it can come up with. The code is accurately printing the actual floating point value used for gravity, it just doesn't exactly match what the user entered as a string. IMO this is the right behavior.

I believe these are all fixed.


PDOSession Handler sess_lifetime vs cookie_lifetime

I've got Symfony setup to store sessions in a database.  I have my session cookies configured to last for 30 days.
framework:
    session:
        handler_id:  session.handler.pdo
        cookie_lifetime: 2592000 # 30 days
services:
    session.handler.pdo:
        class:     Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler
        public:    false
        arguments:
            - "mysql:host=%database_host%;port=3306;dbname=%database_name%"
            - { db_username: %database_user%, db_password: %database_password% , db_table: symfony_sessions, lock_mode: 1 }

The session cookie expiration is being set correctly, however, the sess_lifetime value stored in the database is always 1440.  How are these two values related? Shouldn't the sess_lifetime value be equal to the cookie_lifetime value?

These are different lifetimes. One is for the cookie and one for GC. Please have a look at php session documentation.

Is the cookie lifetime also extended on each request? Maybe it makes sense to adjust the gc lifetime default automatically by cookie lifetime since gc_lifetime < cookie_lifetime probably makes no sense.
But this issue is in the wrong repo anyway. Please reopen if needed on symfony/symfony.


Uncaught Error: Cannot call function 'getPosition' on missing remote object 1630Error: Cannot ca...

[Enter steps to reproduce:]

...
...

Atom: 1.30.0 x64
Electron: 2.0.5
OS: Mac OS X 10.13.6
Thrown From: Atom Core
Stack Trace
Uncaught Error: Cannot call function 'getPosition' on missing remote object 1630
Error: Cannot call function 'getPosition' on missing remote object 1630
at throwRPCError (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:143:17)
at EventEmitter. (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:354:7)
at emitMany (events.js:147:13)
at EventEmitter.emit (events.js:224:7)
at WebContents. (/Applications/Atom.app/Contents/Resources/electron.asar/browser/api/web-contents.js:295:13)
at emitTwo (events.js:126:13)
at WebContents.emit (events.js:214:7)
At /Applications/Atom.app/Contents/Resources/electron.asar/renderer/api/remote.js:221

Error: Cannot call function 'getPosition' on missing remote object 1630
Error: Cannot call function 'getPosition' on missing remote object 1630
    at throwRPCError (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:143:17)
    at EventEmitter.<anonymous> (/Applications/Atom.app/Contents/Resources/electron.asar/browser/rpc-server.js:354:7)
    at emitMany (events.js:147:13)
    at EventEmitter.emit (events.js:224:7)
    at WebContents.<anonymous> (/Applications/Atom.app/Contents/Resources/electron.asar/browser/api/web-contents.js:295:13)
    at emitTwo (events.js:126:13)
    at WebContents.emit (events.js:214:7)
    at Object.exception (/Applications/Atom.app/Contents/Resources/electron.asar/renderer/api/remote.js:221:30)
    at metaToValue (/Applications/Atom.app/Contents/Resources/electron.asar/renderer/api/remote.js:225:28)
    at Object.remoteMemberFunction (/Applications/Atom.app/Contents/Resources/electron.asar/renderer/api/remote.js:125:16)
    at AtomEnvironment.getWindowDimensions (/Applications/Atom.app/Contents/Resources/app/src/atom-environment.js:698:40)
    at AtomEnvironment.storeWindowDimensions (/Applications/Atom.app/Contents/Resources/app/src/atom-environment.js:735:40)
    at WindowEventHandler.handleWindowBlur (/Applications/Atom.app/Contents/Resources/app/src/window-event-handler.js:191:32)

Commands
     -2:30.7.0 core:move-left (input.hidden-input)
     -2:30 editor:newline (input.hidden-input)
  2x -2:18.8.0 core:move-right (input.hidden-input)
     -2:13.1.0 core:paste (input.hidden-input)
     -2:12.6.0 core:save (input.hidden-input)
     -2:12.5.0 linter:lint (atom-text-editor.editor.is-focused)
 10x -1:54.5.0 core:undo (input.hidden-input)
     -1:48.8.0 core:redo (input.hidden-input)
     -1:47.4.0 core:save (input.hidden-input)
     -1:15.6.0 editor:newline (input.hidden-input)
     -1:13.7.0 core:backspace (input.hidden-input)
     -1:11.4.0 core:move-right (input.hidden-input)
     -1:11 core:save (input.hidden-input)
     -0:35.4.0 core:move-right (input.hidden-input)
     -0:35 core:move-up (input.hidden-input)
     -0:32.5.0 core:save (input.hidden-input)

Non-Core Packages
busy-signal 1.4.3
intentions 1.1.5
language-ethereum 0.3.9
linter 2.2.0
linter-ui-default 1.7.1
prettier-atom 0.55.2

Thanks for contributing!
We noticed that this looks like a duplicate of https://github.com/atom/atom/issues/16630 so you can subscribe there for updates if you'd like (we believe that this will be fixed in a future Electron upgrade).

Because we treat our issues list as the Atom team's backlog, we close duplicates to focus our work and not have to touch the same chunk of code for the same reason multiple times. This is also why we may mark something as duplicate that isn't an exact duplicate but is closely related.
For information on how to use GitHub's search feature to find out if something is a duplicate before filing, see the How Can I Contribute? section of the Atom CONTRIBUTING guide.


Add sorting by Degraded on /nodes handler

Issue, related to backend, new issue is created in ydb repository: https://github.com/ydb-platform/ydb/issues/1060


proposed fix for #452

passing the element via this context in event handle binding seems broken. passing native event instead. fixes #452

It fixed the problem, however the statement "this.displayDayView=false" is not hidding the component. It is really strange because if I use "console.log(this.displayDayView)" it really outputs "false".


Where to change css for these items?

Hi there! First off, just wanted to say thank you for making this custom theme project - it's really nice and lets me colour coordinate with all my other things 👍
Not sure where to ask, just had a few questions on changing the colors for certain elements (I'm pretty bad at css):

is there any way I can change the colours of the usernames? I know that I can change them via /color @username <my hex>, but is there a css way of overriding them permanently / all the time?
is there anything to change the "add a topic" button for channels? Right now for me it's a white square sitting on top of my mostly dark theme
last thing is about the preferences page -> the text color seems to be using my background color https://puu.sh/y7eML/21238ee78b.png but I've added color: var(--text); to every single css property under whatever's tagged /* preferences page*'/.

Once again, many thanks for this, and sorry if I missed anything!

Thanks!

I didn't even know /color was a command on Slack! It looks like you can do it with .light_theme ts-message .message_content a[data-member-id="W4xxxxxxx"] { color: red !important; } as an example but you have to find people's member IDs instead of using their usernames. It looks like if you open someone's profile in the side panel, in the dropdown there's a "Copy member ID" option. Hopefully that's enough to get you started. Enabling dev mode as described in the readme can help too if that doesn't work.

I just fixed that today. Should be good now if you re-open it. Sometimes it takes awhile for CDNs to refresh the file though.

Good catch! I missed that. It looks like it at least highlights on hover for now so it's not completely unusable. My .css file is definitely unorganized though, which is why I have the eventual dream of completing #3. I'll see if I can fix that within the next few days.

I'll leave this issue open as a way to track the third bullet point.

Oh also I'm assuming it's not particularly obvious but you can add whatever CSS you want in the section where you define colors in the local Javascript modifications, so you won't have to fork for stuff like username colors.


The default value of gt_semantic_seg padding has potential problems

Checklist

I have searched related issues but cannot get the expected help.
I have read the FAQ documentation but cannot get the expected help.
The bug has not been fixed in the latest version.

Describe the bug
Since 255 in gt_semantic_seg means ignoring pixels and 0 means effective area, the default value of 0 will be used for padding when to stack a batch which will bring potential bugs.
Bug fix
DefaultFormatBundle supports custom padding values.

#6837

The result of panoptic_fpn_r50 is 40.1PQ，it is normal.

The bug was fixed by #6837, it was merged.


Bug. Scrollbar does NOT update when images are present

I just noticed an issue.
If you post an image the scrollbar position is not updating. Does not synch to the bottom.
This issue is not present with regular text messages.
What does it do that with images only? What would be the possible fix for this behavior? @lukejacksonn
Thanks!

Hey! Yes this is a known issue.. so the problem is how to know when to scroll (that is, not when the images has been appended to the DOM but when the image been rendered in the list, loaded and taken shape). I played around for a long time trying to find a solution that works for all cases and ina ll browsers but really struggled!
Perhaps adding a call to this action https://github.com/pusher/react-slack-clone/blob/master/src/index.js#L150 inside an onload function on these elements https://github.com/pusher/react-slack-clone/blob/master/src/components/Message/index.js#L25 might work out.. I'm sure I tried it but not can't remember now.


Drop support for Python 3.6

refs https://github.com/rm-hull/luma.core/issues/252

I can't seem to merge this because of a false python 3.6 test it's waiting for (which we remove with this pr). Can you merge this @rm-hull?


lua_getinfo reports incorrect linedefined

linedefined returned by lua_getinfo(L, -1, "s", &ar) (function on the stack) seems to be wrong. It's always 1 or 2 lines past the line that defines the function. Unfortunately our tooling breaks because of this so I'm trying to find a workaround. Simply subtracting one fixes some cases but not all.
EDIT: it looks like linedefined is the line number of the first statement inside the function. With standard Lua, it's the line where "function" keyword is.

That's correct - we don't track the line info for the function statement at all right now. Changing that requires changing debug info format and actually explicitly serializing that information.

We're going to look at fixing this internally because it requires bytecode versioning, but that's going to be next year.

Great news, thanks!

Note: for the fix to be active, LuauBytecodeV2Write must be set to true. This will happen automatically if the bytecode is produced by luau.exe --compile=binary, but in other scenarios the bytecode version is still 1 by default. To get the fix earlier you can set LuauBytecodeV2Write to true by patching BytecodeBuilder.cpp or setting it externally like this:
#include "Luau/Common.h"

LUAU_FASTFLAG(LuauBytecodeV2Write)

...
FFlag::LuauBytecodeV2Write.value = true;

(which needs to be done before compiling the bytecode)
Alternatively you can just wait a few weeks for us to enable/remove this flag :)

Awesome, thank you! I'll test it now.

Confirmed to work!


Handle Decoded TX failure

If Gnosis is unable to decode transactions, we should show the user a message.
Recommend keeping the "Show executable code" dropdown, but in the dropdown show a message of "Unable to decode proposal transaction data".
To test this, create a proposal with a junk transaction (call some function that doesn't exist on an random address).
Gnosis shouldn't be able to decode this TX.

I think keeping it simple with the message you suggested works great for this.


incorrect example in readme.rst

# Range of hours in a day:
 maya.interval(start=maya.now(), end=maya.now().add(days=1), interval=60*60)

no method named "interval" suppose to be intervals.

is mentioned before


Sending bundle as tx before new block is created

Synchronize with execution client on timing for sending bundles. This should probably be right before the next block.

@Vid201 Could you elaborate more on this one? What do you mean right before the next bloek?

@Vid201 Could you elaborate more on this one? What do you mean right before the next bloek?

TBH I am not quite sure what I meant here now. That was the idea to prevent front-running and if sent just before the next block, bundle is not in the public tx pool for the long time. But it's probably to use flashbots or conditional endpoint if available.

I think it is not possible to do that unless you are a block builder.

I think it is not possible to do that unless you are a block builder.
Maybe it makes more sense to change the issue to talk about "Integration with block builder"

Yeah, you're probably right. We can close this issue.


[Feature]:  Carousel section For enhancing user engagement #95

Describe the feature
Here the carousel section helps user to know more about the feature through latest tempting images regarding that topic. This will contribute in better user experience.
Add ScreenShots

Instead of one their should be more images that it will look more tempting.
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

/assign-me

/assign-me


Only require explicitly C++20 on Visual Studio

While technically speaking designated constructors are a C++20 only feature, apparently just requiring C++17 is sufficient on Linux/macOS, while we need to require C++20 on MSVC. As some downstream users prefer that we do not explicitly force C++20 (see https://github.com/ami-iit/meshcat-cpp/issues/7) and for us I guess it does not change much, I think we can only require C++20 on Windows.

Thanks a lot @traversaro. We are looking forward to using meshcat-cpp in our projects.


[Bug]: Crash at startup

Summary Description
Crash at startup on windows 10
Additional Details

Crash logs
MeerK40t crash log. Version: 0.7.6000 on Windows:AMD64 - 4.1.1 msw (phoenix) wxWidgets 3.1.5
Traceback (most recent call last):
  File "mk40t.py", line 10, in <module>
  File "meerk40t\main.py", line 422, in run
  File "meerk40t\kernel.py", line 983, in bootstrap
  File "meerk40t\gui\wxmeerk40t.py", line 122, in plugin
  File "meerk40t\kernel.py", line 1956, in console
  File "meerk40t\kernel.py", line 2042, in _console_parse
  File "meerk40t\kernel.py", line 1378, in inner
  File "meerk40t\gui\wxmeerk40t.py", line 464, in window_open
  File "meerk40t\gui\wxmeerk40t.py", line 179, in run_later
  File "meerk40t\gui\wxmeerk40t.py", line 455, in window_open
  File "meerk40t\kernel.py", line 603, in open_as
  File "meerk40t\gui\wxmmain.py", line 268, in __init__
  File "meerk40t\gui\wxmmain.py", line 780, in __set_panes
  File "meerk40t\gui\wxmmain.py", line 950, in on_config_panes
  File "meerk40t\gui\wxmmain.py", line 962, in on_pane_lock
TypeError: '_thread.lock' object is not callable

MeerK40t Version
0.7.6
MeerK40t Type
Executable
Your Operating System
Windows

Hm. Lemme check...

Well at least mine didn't autocrash so it's only some... lemme check that branch and the code in it.

                if hasattr(pane.window, "lock"):
                    pane.window.lock()

Well it does call a lock() function and checked if it existed. Somehow the code is being applied to a _thread object which has a lock() function? Hm. Lemme see if I turn on all the panes if I can get something to break there.

Seems like it's the console panel. It has a _thread.lock() object on it that tests true for hasattr but then can't be executed but tries anyway.

send me a version and i try it now. TY

I see the bug and I've taken care of it. legacy7 branch is fixed. Since it's a bug that prevents startup, I'll push out another version.

https://github.com/meerk40t/meerk40t/releases/tag/0.7.6001
Has the correction. The exe files should compile soon.

Start up preventing bugs are critical and should be fixed as soon as possible.

bug Fixed in 0.7.6001 ! TY

Ok. Thank you for the report.
If that's it, you can close the issue or I'll get to it in a few days.


Script doesn't wait long enough and takes actions before the game loaded

For now most of the timings are hard coded. I tried to find a balance between being fast, yet still working whenever the game decides to take a bit longer.
In the future I'd like to implement way more "state checks" for the script to wait and realise by itself when to take the next action.

Since this will be a core issue of the script forever, I'm going to close this issue. With the latest #44 update, users have way more control over the script and can enable or disable features on demand. If something does go wrong, simply disabling that feature and running the script again will most likely work.
As for timings, I'll just try and make them as accurate as possible. I've long decided that speed won't be top priority in this script, but I will still try and balance timings out.


C# Example: Mesh size convergence check

Part 1
INPUT:

File path to Model (.str/.struxml) with labelled sections [-]
List of mesh sizes to try [mm]
Max allowed deviation [%]

OUTPUT:

Table

Meshstorlek [mm]
<Some force> Txz
Avvikelse [%]

1.0
4.0
-

0.5
8.0
100%

0.25
10.0
25%

0.20
11.0
10%

0.15
11.1
1%

0.10
11.2
1%

[x] Script for part 1

Part 2
Like part 1 but also creating the result points / labelled sections for the user automatically

[ ] Script for part 2

Maybe also

Export to excel
Create a plot like below

Current output looks something like

Meshstorlek [mm]
Nx' total [Nmm/mm]
Avvikelse total [Nmm/mm]
Avvikelse [%]

1.0
4.0
-
-

0.5
20.0
20.0
100%

0.25
19.3
4.8
25%

0.20
11.0
1.2
10%

0.15
11.1
1.0
1%

0.10
11.2
1.0
1%

We consider this example done! In the future, when the results reader is properly implemented, we might want to change the example from the current method of reading data (index to read from an .csv file) to the new method.
Comments above, like the Part 2 task, writing to excel, and making a graph, have also yet to be implemented. I suggest whoever does the results reader changes does this as well.
For now, however, this issue is closed, and can be reopened once relevant again.


to_csv doesn't support file-like objects

Hi @Martmists-GH, do you have any detail on this?
I tried to recreate the issue and I guess this is the issue you're talking about:
import pandas as pd
import dask.dataframe as dd

df = pd.DataFrame({"a": [1, 2, 3]})

with open("eg.csv", "w") as file:
    df.to_csv(file)

ddf = dd.from_pandas(df, npartitions=2)

with open("eg.csv", "w") as file:
    ddf.to_csv(file)

pandas will write fine to an io stream object like that, but dask throws a "UnsupportedOperation".
I'm not really an authority at all on anything much, but it seems like if it was implemented, it would mean write operations couldn't be parallelized (dask will by default save to lots of different csvs which is really handy for big data).
Would a more expressive error message saying something along those lines been nicer? Or is there something specific you're trying to achieve that involves writing to a file-like object>

Hmm, I'm not getting an UnsupportedOperation but an AttributeError:
>>> import pandas as pd
>>> import dask.dataframe as dd
>>> import io
>>> df = pd.DataFrame({"a":[1,2,3],"b":[4,5,6]})
>>> ddf = dd.from_pandas(df, npartitions=1)
>>> out = io.StringIO()
>>> ddf.to_csv(out)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\dask\dataframe\core.py", line 1897, in to_csv
    return to_csv(self, filename, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\dask\dataframe\io\csv.py", line 955, in to_csv
    files = open_files(
            ^^^^^^^^^^^
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\fsspec\core.py", line 280, in open_files
    fs, fs_token, paths = get_fs_token_paths(
                          ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\fsspec\core.py", line 610, in get_fs_token_paths
    chain = _un_chain(urlpath0, storage_options or {})
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\fsspec\core.py", line 331, in _un_chain
    bit = cls._strip_protocol(bit)
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M66G898\.pyenv-win-venv\envs\nn_monitoring\Lib\site-packages\fsspec\implementations\local.py", line 214, in _strip_protocol
    if path.startswith("file://"):
       ^^^^^^^^^^^^^^^
AttributeError: '_io.StringIO' object has no attribute 'startswith'

It's fine if it's not fully parallelized for writing as (at least for our use-case) it will only write aggregated results. Perhaps it would be an idea to support file-like objects without parallelizing, but warn the user about the writes not being parallel?

That makes sense, the dask documentation calls that first argument "filename" and says it has to be a string or a list. Someone closer to the project could probably comment better, but it doesn't look like writing to IO objects is part of the designed behaviour.
If you did need to do that for some reason, you could compute first (you'd need everything stored in memory to have it in a single IO object anyway)
ddf.compute().to_csv(out)


Update to 1.17

The 1.17 module gets compiled on Java 16 due to it being needed.

Up

Thanks I’ll accept it when I add the 1.17 stuff to my Jenkins

@HexedHero It doesn't work even if I use your build.
https://nekobin.com/padavuroku

test


[genesis] Record the sequencer public key

We need to record the sequencer public key in the state. Others need to verify the sequencer info in the transaction.

The TX sequence info includes the public key.