CUDA out of memory in decoding

Hi, I am newer to learn icefall,I finished the training of tdnn_lstm_ctc, when run the decode steps, I meet the following error, I change the --max-duration, there are still errors:

we set  --max-duration=100 and use Tesla V100-SXM, the GPU info follow:

would you give me some advice？thanks

The 100 is still big for max-duration. Maybe you can reduce the max-duration to 50, 30 or even less.

You could also mess with the decoding parameters, e.g. reduce the
max-active and/or the beam.
On Fri, Oct 8, 2021 at 6:41 PM Lzhang-hub @.***> wrote:

@csukuangfj https://github.com/csukuangfj We have use you advices (1)
and (3) ,but the problem is not solved. If you can give some other advices,
thank you very much!
[image: 企业微信截图_16336894139504]
https://user-images.githubusercontent.com/57925599/136542593-f8c0f1e1-2bc4-44a7-9fa8-bdeb4077d9ea.png
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/icefall/issues/70#issuecomment-938538727, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO6XVD6EXTWOT4X2TSDUF3DDZANCNFSM5FTIMTGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

What do you mean by very poor?  Is this your own data, or Librispeech?
The model quality and data quality can affect the memory used in decoding.
On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.***> wrote:

https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135
You can reduce search_beam, output_beam, or max_active_states.
By the way, does CUDA out of memory abort your decoding process? Does it
continue to decode after pruning?
Thanks! I will attempt to decode with your advices.
CUDA out of memory do not abort my decoding process, the decode can be
done, but the results are very pool.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939201372, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Hm, can you show the last part of one of the training logs or point to the
tensorboard log (tensorfboard dev upload --logdir blah/log)?
I wonder whether the model is OK.
On Sat, Oct 9, 2021 at 11:21 AM cdxie @.***> wrote:

What do you mean by very poor? Is this your own data, or Librispeech? The
model quality and data quality can affect the memory used in decoding.
… <#m_4937422915890188941_>
On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.***> wrote:
https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135
You can reduce search_beam, output_beam, or max_active_states. By the way,
does CUDA out of memory abort your decoding process? Does it continue to
decode after pruning? Thanks! I will attempt to decode with your advices.
CUDA out of memory do not abort my decoding process, the decode can be
done, but the results are very pool. — You are receiving this because you
commented. Reply to this email directly, view it on GitHub <#70 (comment)
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939201372>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ
. Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.
@danpovey https://github.com/danpovey @csukuangfj
https://github.com/csukuangfj ,Thanks for your reply, we are newer to
learn icefall, we just run the recipes of Librispeech, we finished the
training steps, the above errors occured in the decoding steps. The
decoding process can finised, but the wer of test-other is 59.41%.
The device we used is V100 NVIDIA GPU-32G, and we follow the csukuangfj
advices (1) and (3), the above errors still occurs：
##############
2021-10-09 10:38:49,103 INFO [decode.py:387] Decoding started
2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir':
PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'),
'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3,
'search_beam': 15, 'output_beam': 5, 'min_active_states': 30,
'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg':
5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale':
0.5, 'export': False, 'full_libri': True, 'feature_dir':
PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True,
'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0,
'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts':
True, 'num_workers': 2}
2021-10-09 10:38:50,467 INFO [lexicon.py:113] Loading pre-compiled
data/lang_phone/Linv.pt
2021-10-09 10:38:52,312 INFO [decode.py:397] device: cuda
2021-10-09 10:40:48,429 INFO [decode.py:428] Loading pre-compiled
G_4_gram.pt
2021-10-09 10:43:25,546 INFO [decode.py:458] averaging ['tdnn_lstm_ctc/exp/
epoch-15.pt', 'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/
epoch-17.pt', 'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/
epoch-19.pt']
2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total
capacity; 27.41 GiB already allocated; 365.75 MiB free; 30.23 GiB reserved
in total by PyTorch)
2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before pruning:
2061527
2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs after pruning: 113145
2021-10-09 10:44:16,184 INFO [decode.py:336] batch 0/?, cuts processed
until now is 18
2021-10-09 10:44:16,944 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total
capacity; 21.89 GiB already allocated; 4.36 GiB free; 26.23 GiB reserved in
total by PyTorch)
2021-10-09 10:44:16,944 INFO [decode.py:732] num_arcs before pruning:
2814753
2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning: 120129
2021-10-09 10:44:18,624 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total
capacity; 21.80 GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in
total by PyTorch)
#########################
we reduce search_beam(20->15), max_active_states(10000->7000) a moment
ago, the error is same. We suspect the error could be casued by processing
G, and we may follow the https://github.com/kaldi-asr/kaldi/pull/4594
http://url to prune your G.
We now can't pinpoint the cause of the error, so we need help,thanks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939215235, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Hm, can you show the last part of one of the training logs or point to the tensorboard log (tensorfboard dev upload --logdir blah/log)? I wonder whether the model is OK.
…
On Sat, Oct 9, 2021 at 11:21 AM cdxie @.> wrote: What do you mean by very poor? Is this your own data, or Librispeech? The model quality and data quality can affect the memory used in decoding. … <#m_4937422915890188941_> On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.> wrote: https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135 You can reduce search_beam, output_beam, or max_active_states. By the way, does CUDA out of memory abort your decoding process? Does it continue to decode after pruning? Thanks! I will attempt to decode with your advices. CUDA out of memory do not abort my decoding process, the decode can be done, but the results are very pool. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 (comment) <#70 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . @danpovey https://github.com/danpovey @csukuangfj https://github.com/csukuangfj ,Thanks for your reply, we are newer to learn icefall, we just run the recipes of Librispeech, we finished the training steps, the above errors occured in the decoding steps. The decoding process can finised, but the wer of test-other is 59.41%. The device we used is V100 NVIDIA GPU-32G, and we follow the csukuangfj advices (1) and (3), the above errors still occurs： ############## 2021-10-09 10:38:49,103 INFO [decode.py:387] Decoding started 2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir': PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'), 'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3, 'search_beam': 15, 'output_beam': 5, 'min_active_states': 30, 'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg': 5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'full_libri': True, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2} 2021-10-09 10:38:50,467 INFO [lexicon.py:113] Loading pre-compiled data/lang_phone/Linv.pt 2021-10-09 10:38:52,312 INFO [decode.py:397] device: cuda 2021-10-09 10:40:48,429 INFO [decode.py:428] Loading pre-compiled G_4_gram.pt 2021-10-09 10:43:25,546 INFO [decode.py:458] averaging ['tdnn_lstm_ctc/exp/ epoch-15.pt', 'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/ epoch-17.pt', 'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/ epoch-19.pt'] 2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total capacity; 27.41 GiB already allocated; 365.75 MiB free; 30.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before pruning: 2061527 2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:16,184 INFO [decode.py:336] batch 0/?, cuts processed until now is 18 2021-10-09 10:44:16,944 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.89 GiB already allocated; 4.36 GiB free; 26.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:16,944 INFO [decode.py:732] num_arcs before pruning: 2814753 2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:18,624 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.80 GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in total by PyTorch) ######################### we reduce search_beam(20->15), max_active_states(10000->7000) a moment ago, the error is same. We suspect the error could be casued by processing G, and we may follow the kaldi-asr/kaldi#4594 http://url to prune your G. We now can't pinpoint the cause of the error, so we need help,thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

OK, this is the training log of tdnn_lstm_ctc :
tdnn-lstm-ctc-log-train.txt

Your model did not converge; loss should be something like 0.005, not 0.5.
I believe when we ran it, we used --bucketing-sampler=True, that could
possibly be the reason.
Also we used several GPUs, but that should not really affect convergence I
think.
(Normally this script converges OK).
On Sat, Oct 9, 2021 at 2:17 PM cdxie @.***> wrote:

Hm, can you show the last part of one of the training logs or point to the
tensorboard log (tensorfboard dev upload --logdir blah/log)? I wonder
whether the model is OK.
… <#m_3115122914114341433_>
On Sat, Oct 9, 2021 at 11:21 AM cdxie @.> wrote: What do you mean by
very poor? Is this your own data, or Librispeech? The model quality and
data quality can affect the memory used in decoding. …
<#m_4937422915890188941_> On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.>
wrote:
https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135
You can reduce search_beam, output_beam, or max_active_states. By the way,
does CUDA out of memory abort your decoding process? Does it continue to
decode after pruning? Thanks! I will attempt to decode with your advices.
CUDA out of memory do not abort my decoding process, the decode can be
done, but the results are very pool. — You are receiving this because you
commented. Reply to this email directly, view it on GitHub <#70
https://github.com/k2-fsa/icefall/issues/70 (comment) <#70 (comment)
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939201372>>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ
. Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
. @danpovey https://github.com/danpovey https://github.com/danpovey
@csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj
,Thanks for your reply, we are newer to learn icefall, we just run the
recipes of Librispeech, we finished the training steps, the above errors
occured in the decoding steps. The decoding process can finised, but the
wer of test-other is 59.41%. The device we used is V100 NVIDIA GPU-32G, and
we follow the csukuangfj advices (1) and (3), the above errors still
occurs： ############## 2021-10-09 10:38:49,103 INFO [decode.py:387]
Decoding started 2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir':
PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'),
'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3,
'search_beam': 15, 'output_beam': 5, 'min_active_states': 30,
'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg':
5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale':
0.5, 'export': False, 'full_libri': True, 'feature_dir':
PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True,
'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0,
'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts':
True, 'num_workers': 2} 2021-10-09 10:38:50,467 INFO [lexicon.py:113]
Loading pre-compiled data/lang_phone/Linv.pt 2021-10-09 10:38:52,312 INFO
[decode.py:397] device: cuda 2021-10-09 10:40:48,429 INFO [decode.py:428]
Loading pre-compiled G_4_gram.pt 2021-10-09 10:43:25,546 INFO
[decode.py:458] averaging ['tdnn_lstm_ctc/exp/ epoch-15.pt',
'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/ epoch-17.pt',
'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/ epoch-19.pt']
2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception: CUDA out of
memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total capacity; 27.41
GiB already allocated; 365.75 MiB free; 30.23 GiB reserved in total by
PyTorch) 2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before
pruning: 2061527 2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs
after pruning:<PHONE_NUMBER>-10-09 10:44:16,184 INFO [decode.py:336] batch
0/?, cuts processed until now is 18 2021-10-09 10:44:16,944 INFO
[decode.py:731] Caught exception: CUDA out of memory. Tried to allocate
8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.89 GiB already allocated;
4.36 GiB free; 26.23 GiB reserved in total by PyTorch) 2021-10-09
10:44:16,944 INFO [decode.py:732] num_arcs before pruning: 2814753
2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning: 120129
2021-10-09 10:44:18,624 INFO [decode.py:731] Caught exception: CUDA out of
memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.80
GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in total by
PyTorch) ######################### we reduce search_beam(20->15),
max_active_states(10000->7000) a moment ago, the error is same. We suspect
the error could be casued by processing G, and we may follow the
kaldi-asr/kaldi#4594 https://github.com/kaldi-asr/kaldi/pull/4594
http://url to prune your G. We now can't pinpoint the cause of the error,
so we need help,thanks — You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#70 (comment)
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939215235>, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ
. Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.
OK, this is the training log of tdnn_lstm_ctc :
tdnn-lstm-ctc-log-train.txt
https://github.com/k2-fsa/icefall/files/7315071/tdnn-lstm-ctc-log-train.txt
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/icefall/issues/70#issuecomment-939237157, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO5Y56FEMGRNZ4QATXDUF7M73ANCNFSM5FTIMTGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Your model did not converge; loss should be something like 0.005, not 0.5. I believe when we ran it, we used --bucketing-sampler=True, that could possibly be the reason. Also we used several GPUs, but that should not really affect convergence I think. (Normally this script converges OK).
…
On Sat, Oct 9, 2021 at 2:17 PM cdxie @.*> wrote: Hm, can you show the last part of one of the training logs or point to the tensorboard log (tensorfboard dev upload --logdir blah/log)? I wonder whether the model is OK. … <#m_3115122914114341433_> On Sat, Oct 9, 2021 at 11:21 AM cdxie @.> wrote: What do you mean by very poor? Is this your own data, or Librispeech? The model quality and data quality can affect the memory used in decoding. … <#m_4937422915890188941_> On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.> wrote: https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135 You can reduce search_beam, output_beam, or max_active_states. By the way, does CUDA out of memory abort your decoding process? Does it continue to decode after pruning? Thanks! I will attempt to decode with your advices. CUDA out of memory do not abort my decoding process, the decode can be done, but the results are very pool. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 <#70> (comment) <#70 (comment) <#70 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . @danpovey https://github.com/danpovey https://github.com/danpovey @csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj ,Thanks for your reply, we are newer to learn icefall, we just run the recipes of Librispeech, we finished the training steps, the above errors occured in the decoding steps. The decoding process can finised, but the wer of test-other is 59.41%. The device we used is V100 NVIDIA GPU-32G, and we follow the csukuangfj advices (1) and (3), the above errors still occurs： ############## 2021-10-09 10:38:49,103 INFO [decode.py:387] Decoding started 2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir': PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'), 'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3, 'search_beam': 15, 'output_beam': 5, 'min_active_states': 30, 'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg': 5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'full_libri': True, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2} 2021-10-09 10:38:50,467 INFO [lexicon.py:113] Loading pre-compiled data/lang_phone/Linv.pt 2021-10-09 10:38:52,312 INFO [decode.py:397] device: cuda 2021-10-09 10:40:48,429 INFO [decode.py:428] Loading pre-compiled G_4_gram.pt 2021-10-09 10:43:25,546 INFO [decode.py:458] averaging ['tdnn_lstm_ctc/exp/ epoch-15.pt', 'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/ epoch-17.pt', 'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/ epoch-19.pt'] 2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total capacity; 27.41 GiB already allocated; 365.75 MiB free; 30.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before pruning: 2061527 2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:16,184 INFO [decode.py:336] batch 0/?, cuts processed until now is 18 2021-10-09 10:44:16,944 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.89 GiB already allocated; 4.36 GiB free; 26.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:16,944 INFO [decode.py:732] num_arcs before pruning: 2814753 2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:18,624 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.80 GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in total by PyTorch) ######################### we reduce search_beam(20->15), max_active_states(10000->7000) a moment ago, the error is same. We suspect the error could be casued by processing G, and we may follow the kaldi-asr/kaldi#4594 <kaldi-asr/kaldi#4594> http://url to prune your G. We now can't pinpoint the cause of the error, so we need help,thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment) <#70 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . OK, this is the training log of tdnn_lstm_ctc : tdnn-lstm-ctc-log-train.txt https://github.com/k2-fsa/icefall/files/7315071/tdnn-lstm-ctc-log-train.txt — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO5Y56FEMGRNZ4QATXDUF7M73ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Thanks, I will modify the parameters and run again.
The GPU device I used is a100 NVIDIA GPU-40G, and single-GPU single-machine. The parameters of the script "./tdnn_lstm_ctc/train.py" are not modified

And please show us some sample decoding output, it is written to somewhere (aligned output vs. the ref text). I want to see how the model failed. To get 59% WER is unusual; would normally be either 100% or close to 0, I'd expect.
…
On Sat, Oct 9, 2021 at 4:36 PM cdxie @.> wrote: Your model did not converge; loss should be something like 0.005, not 0.5. I believe when we ran it, we used --bucketing-sampler=True, that could possibly be the reason. Also we used several GPUs, but that should not really affect convergence I think. (Normally this script converges OK). … <#m_7389038400197059205_> On Sat, Oct 9, 2021 at 2:17 PM cdxie @.> wrote: Hm, can you show the last part of one of the training logs or point to the tensorboard log (tensorfboard dev upload --logdir blah/log)? I wonder whether the model is OK. … <#m_3115122914114341433_> On Sat, Oct 9, 2021 at 11:21 AM cdxie @.> wrote: What do you mean by very poor? Is this your own data, or Librispeech? The model quality and data quality can affect the memory used in decoding. … <#m_4937422915890188941_> On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.> wrote: https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135 You can reduce search_beam, output_beam, or max_active_states. By the way, does CUDA out of memory abort your decoding process? Does it continue to decode after pruning? Thanks! I will attempt to decode with your advices. CUDA out of memory do not abort my decoding process, the decode can be done, but the results are very pool. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 <#70> <#70 <#70>> (comment) <#70 <#70> (comment) <#70 (comment) <#70 (comment)>>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . @danpovey https://github.com/danpovey https://github.com/danpovey https://github.com/danpovey @csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj ,Thanks for your reply, we are newer to learn icefall, we just run the recipes of Librispeech, we finished the training steps, the above errors occured in the decoding steps. The decoding process can finised, but the wer of test-other is 59.41%. The device we used is V100 NVIDIA GPU-32G, and we follow the csukuangfj advices (1) and (3), the above errors still occurs： ############## 2021-10-09 10:38:49,103 INFO [decode.py:387] Decoding started 2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir': PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'), 'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3, 'search_beam': 15, 'output_beam': 5, 'min_active_states': 30, 'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg': 5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'full_libri': True, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2} 2021-10-09 10:38:50,467 INFO [lexicon.py:113] Loading pre-compiled data/lang_phone/Linv.pt 2021-10-09 10:38:52,312 INFO [decode.py:397] device: cuda 2021-10-09 10:40:48,429 INFO [decode.py:428] Loading pre-compiled G_4_gram.pt 2021-10-09 10:43:25,546 INFO [decode.py:458] averaging ['tdnn_lstm_ctc/exp/ epoch-15.pt', 'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/ epoch-17.pt', 'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/ epoch-19.pt'] 2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total capacity; 27.41 GiB already allocated; 365.75 MiB free; 30.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before pruning: 2061527 2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:16,184 INFO [decode.py:336] batch 0/?, cuts processed until now is 18 2021-10-09 10:44:16,944 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.89 GiB already allocated; 4.36 GiB free; 26.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:16,944 INFO [decode.py:732] num_arcs before pruning: 2814753 2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:18,624 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.80 GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in total by PyTorch) ######################### we reduce search_beam(20->15), max_active_states(10000->7000) a moment ago, the error is same. We suspect the error could be casued by processing G, and we may follow the kaldi-asr/kaldi#4594 <kaldi-asr/kaldi#4594> < kaldi-asr/kaldi#4594 <kaldi-asr/kaldi#4594>> http://url to prune your G. We now can't pinpoint the cause of the error, so we need help,thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 <#70> (comment) <#70 (comment) <#70 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . OK, this is the training log of tdnn_lstm_ctc : tdnn-lstm-ctc-log-train.txt https://github.com/k2-fsa/icefall/files/7315071/tdnn-lstm-ctc-log-train.txt — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment) <#70 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO5Y56FEMGRNZ4QATXDUF7M73ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . Thanks, I will modify the parameters and run again. The GPU device I used is a100 NVIDIA GPU-40G, and single-GPU single-machine. The parameters of the script "./tdnn_lstm_ctc/train.py" are not modified — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO7PDH2BU52NMFTXSL3UF75JNANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

OK, I choose the best results file(lm_scale_0.7) of tdnn-lstm-ctc model, the decoding parameters { 'search_beam': 15, 'output_beam': 5, 'min_active_states': 30, 'max_active_states': 7000 } :
errs-test-clean-lm_scale_0.7.txt
errs-test-other-lm_scale_0.7.txt
recogs-test-clean-lm_scale_0.7.txt
recogs-test-other-lm_scale_0.7.txt
wer-summary-test-clean.txt
wer-summary-test-other.txt

And please show us some sample decoding output, it is written to somewhere (aligned output vs. the ref text). I want to see how the model failed. To get 59% WER is unusual; would normally be either 100% or close to 0, I'd expect.
…
On Sat, Oct 9, 2021 at 4:36 PM cdxie @.> wrote: Your model did not converge; loss should be something like 0.005, not 0.5. I believe when we ran it, we used --bucketing-sampler=True, that could possibly be the reason. Also we used several GPUs, but that should not really affect convergence I think. (Normally this script converges OK). … <#m_7389038400197059205_> On Sat, Oct 9, 2021 at 2:17 PM cdxie @.> wrote: Hm, can you show the last part of one of the training logs or point to the tensorboard log (tensorfboard dev upload --logdir blah/log)? I wonder whether the model is OK. … <#m_3115122914114341433_> On Sat, Oct 9, 2021 at 11:21 AM cdxie @.> wrote: What do you mean by very poor? Is this your own data, or Librispeech? The model quality and data quality can affect the memory used in decoding. … <#m_4937422915890188941_> On Sat, Oct 9, 2021 at 9:39 AM Lzhang-hub @.> wrote: https://github.com/k2-fsa/icefall/blob/adb068eb8242fe79dafce5a100c3fdfad934c7a5/egs/librispeech/ASR/tdnn_lstm_ctc/decode.py#L132-L135 You can reduce search_beam, output_beam, or max_active_states. By the way, does CUDA out of memory abort your decoding process? Does it continue to decode after pruning? Thanks! I will attempt to decode with your advices. CUDA out of memory do not abort my decoding process, the decode can be done, but the results are very pool. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#70 <#70> <#70 <#70>> (comment) <#70 <#70> (comment) <#70 (comment) <#70 (comment)>>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO66KEUIGGBQEQCGRCLUF6MN3ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . @danpovey https://github.com/danpovey https://github.com/danpovey https://github.com/danpovey @csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj https://github.com/csukuangfj ,Thanks for your reply, we are newer to learn icefall, we just run the recipes of Librispeech, we finished the training steps, the above errors occured in the decoding steps. The decoding process can finised, but the wer of test-other is 59.41%. The device we used is V100 NVIDIA GPU-32G, and we follow the csukuangfj advices (1) and (3), the above errors still occurs： ############## 2021-10-09 10:38:49,103 INFO [decode.py:387] Decoding started 2021-10-09 10:38:49,241 INFO [decode.py:388] {'exp_dir': PosixPath('tdnn_lstm_ctc/exp'), 'lang_dir': PosixPath('data/lang_phone'), 'lm_dir': PosixPath('data/lm'), 'feature_dim': 80, 'subsampling_factor': 3, 'search_beam': 15, 'output_beam': 5, 'min_active_states': 30, 'max_active_states': 7000, 'use_double_scores': True, 'epoch': 19, 'avg': 5, 'method': 'whole-lattice-rescoring', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'full_libri': True, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 150, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2} 2021-10-09 10:38:50,467 INFO [lexicon.py:113] Loading pre-compiled data/lang_phone/Linv.pt 2021-10-09 10:38:52,312 INFO [decode.py:397] device: cuda 2021-10-09 10:40:48,429 INFO [decode.py:428] Loading pre-compiled G_4_gram.pt 2021-10-09 10:43:25,546 INFO [decode.py:458] averaging ['tdnn_lstm_ctc/exp/ epoch-15.pt', 'tdnn_lstm_ctc/exp/epoch-16.pt', 'tdnn_lstm_ctc/exp/ epoch-17.pt', 'tdnn_lstm_ctc/exp/epoch-18.pt', 'tdnn_lstm_ctc/exp/ epoch-19.pt'] 2021-10-09 10:44:14,941 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 4.38 GiB (GPU 0; 31.75 GiB total capacity; 27.41 GiB already allocated; 365.75 MiB free; 30.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:14,942 INFO [decode.py:732] num_arcs before pruning: 2061527 2021-10-09 10:44:14,977 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:16,184 INFO [decode.py:336] batch 0/?, cuts processed until now is 18 2021-10-09 10:44:16,944 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.89 GiB already allocated; 4.36 GiB free; 26.23 GiB reserved in total by PyTorch) 2021-10-09 10:44:16,944 INFO [decode.py:732] num_arcs before pruning: 2814753 2021-10-09 10:44:16,982 INFO [decode.py:739] num_arcs after pruning:<PHONE_NUMBER>-10-09 10:44:18,624 INFO [decode.py:731] Caught exception: CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 21.80 GiB already allocated; 1.54 GiB free; 29.05 GiB reserved in total by PyTorch) ######################### we reduce search_beam(20->15), max_active_states(10000->7000) a moment ago, the error is same. We suspect the error could be casued by processing G, and we may follow the kaldi-asr/kaldi#4594 <kaldi-asr/kaldi#4594> < kaldi-asr/kaldi#4594 <kaldi-asr/kaldi#4594>> http://url to prune your G. We now can't pinpoint the cause of the error, so we need help,thanks — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 <#70> (comment) <#70 (comment) <#70 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO74EHY6MLXZINC73RDUF6YLBANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . OK, this is the training log of tdnn_lstm_ctc : tdnn-lstm-ctc-log-train.txt https://github.com/k2-fsa/icefall/files/7315071/tdnn-lstm-ctc-log-train.txt — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment) <#70 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO5Y56FEMGRNZ4QATXDUF7M73ANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . Thanks, I will modify the parameters and run again. The GPU device I used is a100 NVIDIA GPU-40G, and single-GPU single-machine. The parameters of the script "./tdnn_lstm_ctc/train.py" are not modified — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#70 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO7PDH2BU52NMFTXSL3UF75JNANCNFSM5FTIMTGQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

@danpovey , I also run the recipes of Librispeech of Confromer CTC, using a100 NVIDIA GPU-40G, and single-GPU single-machine, and  no parameters are modified during the training, my model may be not converge according your opinions.
#############

The conformer model logs look normal.
Likely the memory usage in decoding is related to the convergence problems of the model.
We will rerun the TDNN+LSTM+CTC script locally to make sure there is no problem.    @luomingshuang can you do this?

The conformer model logs look normal. Likely the memory usage in decoding is related to the convergence problems of the model. We will rerun the TDNN+LSTM+CTC script locally to make sure there is no problem. @luomingshuang can you do this?

@danpovey @csukuangfj Sorry to truble again, I just run the decode steps of conformer-ctc, the same mistaked occured again(reduced the search_beam, max_active_states). Is this the same reason as TDNN+LSTM+CTC? or some wrong with our machine（we use docker environment）?  :
################
2021-10-09 20:50:49,123 INFO [decode.py:538] Decoding started
2021-10-09 20:50:49,123 INFO [decode.py:539] {'lm_dir': PosixPath('data/lm'), 'subsampling_factor': 4, 'vgg_frontend': False, 'use_feat_batchnorm': True, 'feature_dim': 80, 'nhead': 8, 'attention_dim': 512, 'num_decoder_layers': 6, 'search_beam': 13, 'output_beam': 8, 'min_active_states': 30, 'max_active_states': 7000, 'use_double_scores': True, 'epoch': 34, 'avg': 20, 'method': 'attention-decoder', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'exp_dir': PosixPath('conformer_ctc/exp'), 'lang_dir': PosixPath('data/lang_bpe'), 'full_libri': True, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 30, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2}
2021-10-09 20:50:49,620 INFO [lexicon.py:113] Loading pre-compiled data/lang_bpe/Linv.pt
2021-10-09 20:50:49,795 INFO [decode.py:549] device: cuda
2021-10-09 20:50:57,672 INFO [decode.py:604] Loading pre-compiled G_4_gram.pt
2021-10-09 20:51:08,755 INFO [decode.py:640] averaging ['conformer_ctc/exp/epoch-15.pt', 'conformer_ctc/exp/epoch-16.pt', 'conformer_ctc/exp/epoch-17.pt', 'conformer_ctc/exp/epoch-18.pt', 'conformer_ctc/exp/epoch-19.pt', 'conformer_ctc/exp/epoch-20.pt', 'conformer_ctc/exp/epoch-21.pt', 'conformer_ctc/exp/epoch-22.pt', 'conformer_ctc/exp/epoch-23.pt', 'conformer_ctc/exp/epoch-24.pt', 'conformer_ctc/exp/epoch-25.pt', 'conformer_ctc/exp/epoch-26.pt', 'conformer_ctc/exp/epoch-27.pt', 'conformer_ctc/exp/epoch-28.pt', 'conformer_ctc/exp/epoch-29.pt', 'conformer_ctc/exp/epoch-30.pt', 'conformer_ctc/exp/epoch-31.pt', 'conformer_ctc/exp/epoch-32.pt', 'conformer_ctc/exp/epoch-33.pt', 'conformer_ctc/exp/epoch-34.pt']
2021-10-09 20:51:27,902 INFO [decode.py:653] Number of model parameters: 116147120
2021-10-09 20:51:30,958 INFO [decode.py:474] batch 0/?, cuts processed until now is 2
2021-10-09 20:51:44,208 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 2.00 GiB (GPU 0; 31.75 GiB total capacity; 26.26 GiB already allocated; 1.46 GiB free; 29.13 GiB reserved in total by PyTorch)
2021-10-09 20:51:44,274 INFO [decode.py:732] num_arcs before pruning: 103742
2021-10-09 20:51:44,288 INFO [decode.py:739] num_arcs after pruning: 45225
2021-10-09 20:51:46,104 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.47 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,104 INFO [decode.py:732] num_arcs before pruning: 233253
2021-10-09 20:51:46,116 INFO [decode.py:739] num_arcs after pruning: 90555
2021-10-09 20:51:46,235 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,235 INFO [decode.py:732] num_arcs before pruning: 90555
2021-10-09 20:51:46,247 INFO [decode.py:739] num_arcs after pruning: 90414
2021-10-09 20:51:46,360 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,360 INFO [decode.py:732] num_arcs before pruning: 90414
2021-10-09 20:51:46,370 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:46,482 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,483 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:46,492 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:46,605 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,605 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:46,615 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:46,728 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,728 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:46,739 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:46,853 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,853 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:46,864 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:46,978 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:46,978 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:46,989 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:47,101 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:47,101 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:47,112 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:47,226 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:47,226 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:47,237 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:47,351 INFO [decode.py:731] Caught exception:
CUDA out of memory. Tried to allocate 8.00 GiB (GPU 0; 31.75 GiB total capacity; 19.32 GiB already allocated; 2.63 GiB free; 27.96 GiB reserved in total by PyTorch)
2021-10-09 20:51:47,351 INFO [decode.py:732] num_arcs before pruning: 90366
2021-10-09 20:51:47,361 INFO [decode.py:739] num_arcs after pruning: 90366
2021-10-09 20:51:47,361 INFO [decode.py:743] Return None as the resulting lattice is too large
Traceback (most recent call last):
File "./conformer_ctc/decode.py", line 688, in
main()
File "/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context
return func(*args, **kwargs)
File "./conformer_ctc/decode.py", line 664, in main
results_dict = decode_dataset(
File "./conformer_ctc/decode.py", line 447, in decode_dataset
hyps_dict = decode_one_batch(
File "./conformer_ctc/decode.py", line 365, in decode_one_batch
best_path_dict = rescore_with_attention_decoder(
File "/workspace/icefall/icefall/decode.py", line 812, in rescore_with_attention_decoder
nbest = Nbest.from_lattice(
File "/workspace/icefall/icefall/decode.py", line 209, in from_lattice
saved_scores = lattice.scores.clone()
AttributeError: 'NoneType' object has no attribute 'scores'
##################

I suggest that you can use ctc decoding to verify your model according to #71. If the results based on ctc decoding are normal, maybe the problem happens to your language model.

According your advice, we use ctc decoding,but the decoding progress  is stuck. The logs are as follows and has been in this state for more than 30 hours.
Beside, we tested it on both Tesla V100 and A100-SXM4-40GB, We are not sure whether it is related to the machine configuration, could you please provide your machine configuration?

2021-10-11 10:57:41,892 INFO [decode.py:538] Decoding started
2021-10-11 10:57:41,892 INFO [decode.py:539] {'lm_dir': PosixPath('data/lm'), 'subsampling_factor': 4, 'vgg_frontend': False, 'use_feat_batchnorm': True, 'feature_dim': 80, 'nhead': 8, 'attention_dim': 512, 'num_decoder_layers': 6, 'search_beam': 20, 'output_beam': 8, 'min_active_states': 30, 'max_active_states': 10000, 'use_double_scores': True, 'epoch': 25, 'avg': 1, 'method': 'ctc-decoding', 'num_paths': 100, 'nbest_scale': 0.5, 'export': False, 'exp_dir': PosixPath('conformer_ctc/exp'), 'lang_dir': PosixPath('data/lang_bpe_5000'), 'full_libri': False, 'feature_dir': PosixPath('data/fbank'), 'max_duration': 300, 'bucketing_sampler': False, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2}
2021-10-11 10:57:42,222 INFO [lexicon.py:113] Loading pre-compiled data/lang_bpe_5000/Linv.pt
2021-10-11 10:57:43,125 INFO [decode.py:549] device: cuda
2021-10-11 10:58:17,325 INFO [checkpoint.py:92] Loading checkpoint from conformer_ctc/exp/epoch-25.pt
2021-10-11 10:58:18,410 INFO [decode.py:653] Number of model parameters: 116147120

Could you follow https://k2-fsa.github.io/k2/installation/for_developers.html
to build a debug version of k2?

Yes, typically you'd want to scale the learning rate by the number of GPUs. The intuition is that the larger the batch size, the better the gradient estimate so you can take larger steps in optimization.

Can we focus on the decoding issue first, as it is not related to the number of GPUs?
The issue is that they cannot decode with a pre-trained model provided by us, even using the same type of GPU, i.e. Tesla V100 with 32 GB RAM.

The learning rate is computed by the following formula:
https://github.com/k2-fsa/icefall/blob/39bc8cae94cb3b5824a93b5033136fba546322b9/egs/librispeech/ASR/conformer_ctc/transformer.py#L767-L772

Thanks, I will run the training steps again.

For the decoder OOM, you may be able to debug it using code similar to the following, used after a try..except.
The main thing I am wondering is: is most of the memory use in linear tensors (1 axis) which would be generated by
k2 algorithms, or in tensors with more axes which might come from PyTorch stuff?  You could perhaps modify it to compute totals of bytes owned in 1-d vs more-than-1-d tensors.
import torch
import gc
for obj in gc.get_objects():
    try:
        if torch.is_tensor(obj) or (hasattr(obj, 'data') and torch.is_tensor(obj.data)):
            print(type(obj), obj.size())
    except:
        pass

I notice that your k2 is
k2 version: 1.9
Build type: Release
Git SHA1: 8694fee66f564cf750792cb30c639d3cc404c18b
Git date: Thu Sep 30 15:35:28 2021
Cuda used to build k2: 11.0
CMAKE_CUDA_FLAGS: --expt-extended-lambda -gencode arch=compute_35,code=sm_35 --expt-extended-lambda -gencode arch=compute_50,code=sm_50 --expt-extended-lambda -gencode arch=compute_60,code=sm_60 --expt-extended-lambda -gencode arch=compute_61,code=sm_61 --expt-extended-lambda -gencode arch=compute_70,code=sm_70 --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-unknown-pragmas --compiler-options -Wno-strict-overflow

Looks like you compiled k2 from source by yourself. If everything goes normal, your CMAKE_CUDA_FLAGS should contain only one type of compute arch, i.e, arch=compute_70,code=sm_70, as you are using Tesla V100. Not sure what's wrong with your configuration.
#70 (comment) shows that you have two machines with different types of GPUs and CUDA versions:

Tesla V100, 32 GB, CUDA 11.0
A100-SXM4, 40GB, CUDA 11.2

Have you tried to use the pre-trained model to decode some sound file, not the whole test-clean or test-other dataset, in the above two machines? (Please see https://icefall.readthedocs.io/en/latest/recipes/librispeech/conformer_ctc.html#pre-trained-model)

we use the pre-trained model to decode some sound file according you advice,the result are following:

v100:

CTC decoding:  the result is right
HLG decoding:  the result is right
HLG decoding + LM rescoring:  the result is right
HLG decoding + LM rescoring + attention decoder rescoring:   error:
Traceback (most recent call last): File "./conformer_ctc/pretrained.py", line 418, in <module> main() File "./conformer_ctc/pretrained.py", line 409, in main raise ValueError("Please use a supported decoding method.") ValueError: Please use a supported decoding method.

a100

CTC decoding:  the result is right
HLG decoding:   error: Segmentation fault (core dumped)
HLG decoding + LM rescoring:  error: Segmentation fault (core dumped)
HLG decoding + LM rescoring + attention decoder rescoring:   error: Segmentation fault (core dumped)

HLG decoding + LM rescoring + attention decoder rescoring: error:
Traceback (most recent call last): File "./conformer_ctc/pretrained.py", line 418, in  main() File "./conformer_ctc/pretrained.py", line 409, in main raise ValueError("Please use a supported decoding method.") ValueError: Please use a supported decoding method.

What is the value you gave to --method?

I test all 4 decoding methods:
CTC decoding HLG decoding HLG + n-gram LM rescoring HLG + n-gram LM rescoring + attention decoder rescoring

I test all 4 decoding methods:

I mean, you passed --method ctc-decoding to pretrained.py when you used ctc decoding.
What's the value you gave to --method when using HLG + n-gram LM rescoring + attention decoder rescoring?
It should not raise an exception.

Could you try the latest icefall? I just made a change to pretrained.py (#77)
The original implementation swallowed the exception raised from decoding code and threw an incorrect new exception.

 E.g., you can print the shape of self.pe and x and find which line of code is problematic.

IndexError: too many indices for tensor of dimension 2

That error is just fixed in #79

Thaks, the decoding can be done rightly now.

As for A100-SXM4-40GB, could you compile k2 from source by changing
https://github.com/k2-fsa/k2/blob/master/CMakeLists.txt#L209
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75)

to
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75 80)

and run pretrained.py again with the newly compiled k2?

As for A100-SXM4-40GB, could you compile k2 from source by changing https://github.com/k2-fsa/k2/blob/master/CMakeLists.txt#L209
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75)

to
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75 80)

and run pretrained.py again with the newly compiled k2?
The decoding can be done wiht the newly compiled k2 on  A100-SXM4-40GB.

Does it still cause OOM when running decode.py with the pre-trained model?

As for A100-SXM4-40GB, could you compile k2 from source by changing https://github.com/k2-fsa/k2/blob/master/CMakeLists.txt#L209
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75)

to
  set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75 80)

and run pretrained.py again with the newly compiled k2?

yes, we run pretrained.py again with the newly compiled k2

Does it still cause OOM when running decode.py (not pretrained.py) with the pre-trained model?

@danpovey
Yes, we didn't change any pruning parameters in the decode.py, the parameters:

we will change the max_active_states  or beam to run the decoding again. we use the same GPU device as you, we can not finish the decoding, could you give us more advices ?
@csukuangfj
Another thing: we following your advices: 1、set(K2_COMPUTE_ARCH_CANDIDATES 35 50 60 61 70 75 80) in k2 and compiled
2、compile a Debug version of k2. But, we can not finish the decoding using --epoch 99 --avg 1 even the --method ctc-decoding，so, did some change cause the decoding failed?

But, we can not finish the decoding using --epoch 99 --avg 1 even the --method ctc-decoding

Please share with us the error logs for the failure.

Yes, we didn't change any pruning parameters in the decode.py, the parameters:

Dan meant that you can tune the value of max_active_states, output_beam, and search_beam.
But it's odd that the same configuration used by us causes problems for you.

Can you also add a print statement in the main program body so that it
prints out some general PyTorch CUDA memory stats after each minibatch or
each N minibatches?
I want to see if there might be a leak of some kind.
On Thu, Oct 21, 2021 at 10:50 AM Fangjun Kuang @.***>
wrote:

But, we can not finish the decoding using --epoch 99 --avg 1 even the
--method ctc-decoding
Please share with us the error logs for the failure.
Yes, we didn't change any pruning parameters in the decode.py, the
parameters:
Dan meant that you can tune the value of max_active_states, output_beam,
and search_beam.
But it's odd that the same configuration used by us causes problems for
you.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/k2-fsa/icefall/issues/70#issuecomment-948206419, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAZFLO64KAI6PWQ25F5LS33UH55V7ANCNFSM5FTIMTGQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

arc_end_scores.RowSplits(1).Data()  is a data pointer, should it be arc_end_scores.RowSplits(1)?


Calling .first on an empty List causes a StateError

By far the most common error getting reported for my app is a StateError. This overwhelmingly happens on iOS, and very rarely happens on Android (at a ratio of approximately 250 errors on iOS to 1 on Android). This error gets reported so frequently that bugsnag has rate limited errors from my app. This has been happening for months across multiple versions of Flutter.
Here's an example of the error:

   StateError Bad state: No element

  #0 _GrowableList.first (dart:core-patch/growable_array.dart:343)
  #1 ChannelClient._createEvent (package:bugsnag_flutter/src/client.dart:526)
  #2 ChannelClient._notifyInternal (package:bugsnag_flutter/src/client.dart:479)
  #3 ChannelClient._onFlutterError (package:bugsnag_flutter/src/client.dart:466)
  #4 FlutterError.reportError (package:flutter/src/foundation/assertions.dart:1184)
  #5 ImageStreamCompleter.reportError (package:flutter/src/painting/image_stream.dart:733)
  #6 MultiFrameImageStreamCompleter. (package:flutter/src/painting/image_stream.dart:862)
  #7 _RootZone.runBinary (dart:async/zone.dart:1658)
  #8 _FutureListener.handleError (dart:async/future_impl.dart:162)
  #9 Future._propagateToListeners (dart:async/future_impl.dart:778)
  #10 Future._propagateToListeners (dart:async/future_impl.dart:799)
  #11 Future._completeError (dart:async/future_impl.dart:574)
  #12 _completeOnAsyncError (dart:async-patch/async_patch.dart:318)
  #13 _RootZone.runBinary (dart:async/zone.dart:1658)
  #14 _FutureListener.handleError (dart:async/future_impl.dart:162)
  #15 Future._propagateToListeners (dart:async/future_impl.dart:778)
  #16 Future._propagateToListeners (dart:async/future_impl.dart:799)
  #17 Future._completeError (dart:async/future_impl.dart:574)
  #18 Future.wait (dart:async/future.dart:507)
  #19 _RootZone.runBinary (dart:async/zone.dart:1658)
  #20 _FutureListener.handleError (dart:async/future_impl.dart:162)
  #21 Future._propagateToListeners (dart:async/future_impl.dart:778)
  #22 Future._propagateToListeners (dart:async/future_impl.dart:799)
  #23 Future._completeError (dart:async/future_impl.dart:574)
  #24 Future._chainForeignFuture (dart:async/future_impl.dart:519)
  #25 _microtaskLoop (dart:async/schedule_microtask.dart:40)
  #26 _startMicrotaskLoop (dart:async/schedule_microtask.dart:49)

This error comes from the following line in the bugsnag_flutter library:
https://github.com/bugsnag/bugsnag-flutter/blob/f2a34037cd284506bd58d24c033ea60cdb25a30e/packages/bugsnag_flutter/lib/src/client.dart#L526
error.stacktrace happens to be empty, so calling .first will throw a StateError.
I forked the bugsnag_flutter repo and fixed that line. I don't know why error.stacktrace is empty, but I have an idea of the order of events that leads to the StateError.
Since releasing a new version of my app with the forked repo yesterday, it has always happened because of an HttpException. Nearly all of those HttpExceptions were from a request for an image via network either timing out (HttpException Operation timed out) or having its connection closed (HttpException Connection closed before full header was received). The HttpException then gets sent to bugsnag for error reporting, and then bugsnag throws a StateError because of the line I mentioned.
I'm going to make a pull request with the fix I made in my forked repo.

Here's the PR I made https://github.com/bugsnag/bugsnag-flutter/pull/187

Here's an example of one of the errors after using my forked repo. It looks like not having a stacktrace is an acceptable outcome that gets handled a special way in bugsnag error reports. That points more to the line I mentioned above being an incorrect use of .first.

Hi @msisinni. Thanks for reporting this and the related PR. We're going to do some investigation around why the stacktrace is empty. We will update here as soon as we can.

Hi @msisinni, a fix for this was released in v2.5.0.


<EMAIL_ADDRESS>download via aria2 failed

When I try install Cudatext, I got this message:
WARN  Scoop uses 'aria2c' for multi-connection downloads.
WARN  Should it cause issues, run 'scoop config aria2-enabled false' to disable it.
Installing 'cudatext' (<IP_ADDRESS>) [64bit]
Starting download with aria2 ...
Download: Download Results:
Download: gid   |stat|avg speed  |path/URI
Download: ======+====+===========+=======================================================
Download: 09712b|ERR |        n/a|C:/Users/t480s/scoop/cache/cudatext#<IP_ADDRESS>#https_www.fosshub.com_CudaText.html_cudatext-win-x64-<IP_ADDRESS>.zip
Download: Status Legend:
Download: (ERR):error occurred.
Download: aria2 will resume download if the transfer is restarted.
Download: If there are any errors, then see the log file. See '-l' option in help/man page for details.
ERROR Download failed! (Error 1) An unknown error occurred
ERROR
    referer=https://www.fosshub.com/CudaText.html/
    dir=C:\Users\t480s\scoop\cache
    out=cudatext#<IP_ADDRESS>#https_www.fosshub.com_CudaText.html_cudatext-win-x64-<IP_ADDRESS>.zip

ERROR & 'C:\Users\t480s\scoop\apps\aria2\current\aria2c.exe' --input-file='C:\Users\t480s\scoop\cache\cudatext.txt' --user-agent='Scoop/1.0 (+http://scoop.sh/) PowerShell/5.1 (Windows NT 10.0; Win64; x64; Desktop)' --allow-overwrite=true --auto-file-renaming=false --retry-wait=2 --split=5 --max-connection-per-server=5 --min-split-size=5M --console-log-level=warn --enable-color=false --no-conf=true --follow-metalink=true --metalink-preferred-protocol=https --min-tls-version=TLSv1.2 --stop-with-process=1996 --continue

Please try again or create a new issue by using the following link and paste your console output:
https://github.com/lukesampson/scoop-extras/issues/new?title=cudatext%4<IP_ADDRESS>%3a+download+via+aria2+failed

lukesampson/scoop#4051
Duplicate of #4152


[GSoC 2022] Port CRIT functionalities

Overview
This PR intends to provide a fully functional Go library that enables users to directly operate on CRIU image files natively within their Go programs. It also provides a CLI app that does the same thing as the Python crit.
Implementation
The library provides a CriuImage struct with the entries generalised as a slice of proto.Message interfaces. This lets us use the same struct to hold all kinds of images, with minimal type assertion while converting between formats. Once the struct object itself is created, no type assertion is necessary as Go provides the concrete proto type for each member of the slice. The extra data for all entries is stored as a string, as almost all of it is base64 encoded data. Custom JSON marshalers and unmarshalers are used to separate the extra data from the protobuf entries during encoding and decoding.
Features

[x] Decoder (load binary into Go struct)
[x] Encoder (write Go struct into binary file)
[x] JSON Parser (load JSON file into Go struct)
[ ] Dir explorer (load {ps,fds,mems,rss} data into Go struct)

CLI

[x] crit decode
[x] crit encode
[x] crit show
[x] crit info
[ ] crit x

Codecov Report

Merging #66 (e523519) into master (6ba3429) will decrease coverage by 0.14%.
The diff coverage is n/a.

:exclamation: Current head e523519 differs from pull request most recent head 19e9d60. Consider uploading reports for the commit 19e9d60 to get more accurate results

@@            Coverage Diff             @@
##           master      #66      +/-   ##
==========================================
- Coverage   26.30%   26.15%   -0.15%
==========================================
  Files          11       11
  Lines        2015     2015
==========================================
- Hits          530      527       -3
- Misses       1388     1390       +2
- Partials       97       98       +1

Impacted Files
Coverage Δ

main.go
55.74% <0.00%> (-1.73%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6ba3429...19e9d60. Read the comment docs.

Great work, Prajwal!
It would be great to have some ultra-simple integration test that will show that go-crit works equivalently to the original CRIT. I believe that we can start from some bash script and some image set generated from the ZDTM test suite run.

@snprajwal Would it be possible to include crit in the main Makefile as well?
Example:
diff --git a/Makefile b/Makefile
index 347062b..8ef29d0 100644
--- a/Makefile
+++ b/Makefile
@@ -10,6 +10,7 @@ lint:
 build:
        $(MAKE) -C scripts
        $(GO) build -v ./...
+       $(MAKE) -C crit

It would be great to have some ultra-simple integration test that will show that go-crit works equivalently to the original CRIT. I believe that we can start from some bash script and some image set generated from the ZDTM test suite run.

@mihalicyn incoming in the next set of commits. I thought it would be better to complete crit x and be done with all the features before adding tests in.


Switch to producing portable PDBs instead of classic PDBs.

Embed all source files directly into the PDBs.

@tmat @gafter @jaredpar folks, is this the right usage for embedding sources into portable PDBs? Does the debugger and tooling support this? When I debug using these pdbs, will the source open automatically?
+==========================+
| Old .pdb size: | 3,096,064 |
| New .pdb size (including all embedded source): | 1,938,484 |
| New .pdb size (without source): | 716,556 |
+==========================+

Turns out this can't go in because the debugger doesn't yet support sources embedded in pdbs. Was worth a try though :P


Bring the legal texts up to date

We need to find out if this is needed for Milestone v1.0

Discuss with CISPA.


MINOR: Log encountered exception during rebalance

Some other minor changes:

Do not throw the exception form callback as it would only be swallowed by consumer coordinator; remembering it and re-throw in the next loop is good enough.
Change Creating to Defining in Stores to avoid confusions that the stores have already been successfully created at that time.
Do not need unAssignChangeLogPartitions as the restore consumer will be unassigned already inside changelog reader.

ping @bbejeck @dguy @mjsax for reviews. This is the similar logging improvements for trunk as #3771 for 0.11.0.

Merged to trunk.


Replaced switch with if-else in ShareContentProvider.java

Describe the pull request
Describe the pull request
"switch" statements should have at least 3 "case" clauses. And I replaced the switch with if-else.
Link to the the issue
https://github.com/anurag444/opentracksFall2024/issues/10
Screenshot of updated code

Fixed this Issue in this PR


use POST method for rest transports

related issue #65

We should also remove the skip test case if transport is rest in ToolkitFunctional.js

Other than that it LGTM tests are passing :+1:

resolves #65


update entry

Is there a way to update entry in the list view?
I have the custom button that update state for my entry and I want to update it in the list view.

I found this by searching. Not a complete answer but could get you on the right path : https://github.com/marmelab/ng-admin/issues/287

@beatelite I found this https://github.com/marmelab/ng-admin-demo/blob/9b9c9e15311cd72edddde7c2b54081791f7ab7d4/js/reviews/approveReview.js#L20 and updating $state now. Not a good solution, but it works.

Not supported by ng-admin, use your own directives for that.


Improvements to wrapper API

Currently, the MockMvcRestDocumentationWrapper provides various overloaded document methods that allow you to pass in details needed for building the resource model like description, summary, privateResource, deprecated, etc.  This works fine but it can be difficult to read when using all the parameters.
For instance:
  document(
    "get-book-by-id",
    "Get book by id",
    true,
    false,
    responseFields(
      fieldWithPath("title").description("the title of the book"),
      fieldWithPath("author").description("the author of the book")
    )
  )

In that example, it's not clear that true refers to the privateResource flag and false refers to the deprecated flag.
Another issue is that some parameters on ResourceSnippetParameters are not available to be set, like tags.  We could add them to the overloaded methods but that would just make the above problem worse.
This pull requests adds the ability to just provide a ResourceSnippetDetails object which is a subset of ResourceSnippetParametersBuilder. This allows us to change the above example to the following:
  document(
    "get-book-by-id",
    resource()
      .description("Get book by id")
      .privateResource(true)
      .deprecated(false)
      .tag("book"),
    responseFields(
      fieldWithPath("title").description("the title of the book"),
      fieldWithPath("author").description("the author of the book")
    )
  )

Coverage decreased (-0.1%) to 87.561% when pulling 522fe81c51bb09d6b0aa1eea160d91fa7bd8b2eb on klieber:feature/api-improvements into 0cac5d6d1bbb592ee04835854f29b2b7c1b90b98 on ePages-de:master.

@mduesterhoeft, this should be ready to go with the change you requested.


Normalize logging accross services

The default configuration of the three bacula services have some logging inconsistencies regarding where the log files are written and whether messages are send to syslog or not:

Service
Append to file
Log to syslog

bacula-dir
/var/log/bacula/log
No

bacula-fd
/var/log/bacula/bacula-fd.log
No

bacula-sd
/var/log/bacula/bacula-sd.log
Yes

This pull-request attempts to make things more consistent by always including the service name is the log filename, and not logging to syslog by default (logging to syslog is disabled by default in bacula and is not customizable, so outputing a log of informational messages at priority 'err').

LGTM :+1:

Thanks @smortex.


Monospaced font is not used for code and literal roles

🔍 Before submitting the issue

[X] I have searched among the existing issues
[X] I am using a Python virtual environment

🐞 Description of the bug
With the following RST file:
- double backquotes: ``mmmm + iiii``
- code role: :code:`mmmm + iiii`
- literal role: :literal:`mmmm + iiii`

I would expect that a monospaced font is used, but this is not the case:

Note that the default alabaster theme uses a monospaced font:

📝 Steps to reproduce

unzip ansys-theme-monospaced.zip
run sphinx-build -M html . _build

💻 Which operating system are you using?
Windows
📀 Which ANSYS version are you using?
ansys-sphinx-theme==1.1.4
🐍 Which Python version are you using?
3.11
📦 Installed packages
accessible-pygments==0.0.5
alabaster==1.0.0
ansys-sphinx-theme==1.1.4
babel==2.16.0
beautifulsoup4==4.12.3
certifi==2024.8.30
charset-normalizer==3.4.0
colorama==0.4.6
docutils==0.21.2
idna==3.10
imagesize==1.4.1
importlib_metadata==8.5.0
Jinja2==3.1.4
MarkupSafe==3.0.1
packaging==24.1
pdf2image==1.17.0
pillow==10.4.0
pydata-sphinx-theme==0.15.4
Pygments==2.18.0
requests==2.32.3
snowballstemmer==2.2.0
soupsieve==2.6
Sphinx==8.1.3
sphinxcontrib-applehelp==2.0.0
sphinxcontrib-devhelp==2.0.0
sphinxcontrib-htmlhelp==2.1.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==2.0.0
sphinxcontrib-serializinghtml==2.0.0
typing_extensions==4.12.2
urllib3==2.2.3
zipp==3.20.2

Thanks, @ansbmartin. This issue is solved in https://github.com/ansys/ansys-sphinx-theme/pull/537. It is not the original intend but using the variables fix this.


Platform: add D3D12 to modulemap

Add the DirectX 12 header to the modulemap for the Windows SDK.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

CC: @egorzhdan

@swift-ci please test

@swift-ci please test

Can we unlock these too? To make sure all 3 will be compatible with the same version of Swift.

Can we unlock these too? To make sure they are all compatible with the same version of Swift.
XInput XINPUT1_4.DLL
XAudio2 XAUDIO2_9.DLL

Sure, that seems reasonable; got a pointer to the docs for the covered API?

Yup
Xinput is here and the library in is here
DirectSound2 is here and library info here

Discussion: Can we place this module at DirectX.Direct3D.V12, for a clearer classification and versioning?

Consideration: There’re many APIs in Windows which has many patches (for example, Direct2D), which can be “subversion”ed in the module map. Therefore, I propose to use such structure for Windows SDK:
module DirectX {
  module Direct3D {
    explicit module V12 {
      header "d3d12.h"
      export *

      link "d3d12.lib"
    }
    explicit module V11 {
      // ...
    }
  }

  module Direct2D {
    module V1 {
      header "d2d1.h"
      export *

      link "d2d1.lib"

      explicit module 1 {
        header "d2d1_1.h"
        export *
      }
      explicit module 2 {
        header "d2d1_2.h"
        export *
      }
      // ...
    }
  }
  // ...
}

Yes, if we are adding the remainder of the DirectX APIs, it makes sense to organize them as @stevapple was pointing out.  Although, I plan on only doing the minimum for the requested set.  The organization of the modules has been somewhat ... difficult, at least in the past, so I tend to do the most minimal evolution of the module.

I think the bigger problem is about versioning the same API set (as for the case of Direct2D and many others). Direct3D 12 is newer so it doesn’t have such variety yet. Direct3D 11, for example, has extra headers ranging from d3d11_1.h to d3d11_4.h.

@stevapple you cannot - the user is responsible for handling the version compatibility checking.  That is one of the problems with not having control over the platform.  The best you can do is augment the information with APINotes.

To be a bit more clear, we already have the ability to have conditional versioning on Windows:
if #available(Windows 10.0.18362) {
  ...
} else {
  ...
}

I haven't made much use of the deployment target handling as of yet, but I don't see why that would not afford us the necessary support for the version checking.  We can continue to bump the minimum supported SDK, but as long as the runtime can be backdeployed to 17763, I think that we are okay.
The reason for the 17763 support to continue is that it is a LTSC release with support until 1/9/2029, and was the first version that I had targeted (due to missing APIs for UWP support in the earlier releases needed to support Dispatch and backtracing).  This allows us to continue to deploy to various targets - the current implementation was carefully considered in order to be able to support Windows desktop (laptops/tablets/desktops), XBox, HoloLens, and Windows Phone.

@STREGAsGate - I've added the DXCore bindings, but note that they are not usable.  It requires the C++ interop to be functional, which is currently not fully working on Windows.  There are problems with the modules (which is also why I prefer keeping the changes to the modules minimal) which prevent the C++ interop tests from even running on Windows with an updated environment.  Once the module is well formed enough to allow the C++ interop to be used with a modern SDK, you should be able to use it with -enable-cxx-interop flag.

@swift-ci please test

@swift-ci please test Windows platform

Build failed
Swift Test Linux Platform
Git Sha - d2b8462effe54c2c92a463a096cfd1a6509e7038

Okay, we don't actually need DXCore. I found it yesterday and it seems like a new thing Microsoft is doing, but it only works with D3d12.
The functionality I would need is also found in DXGI. Which works with D3d10, D3d11, and D3d12.
Only place I could find library info here.

The best you can do is augment the information with APINotes.

I mean we may introduce such headers with explicit module and an API note. Then everything depends on the user itself. @compnerd

I don't think that we need the explicit module, simply adding the APINotes for the interfaces should be sufficient.  I think that we can assume, at least for the time being, anything at 17763 to always be available and annotate the newer interfaces?  If so, if you want to create such an APINotes, I think that would be fine to add in.

@swift-ci please test Linux platform

Okay, we don't actually need DXCore. The functionality I would need is also found in DXGI so we could add that instead.
We do also need to add D3d12SDKLayers though, which is where all the debug and validation tools appear to be.

Sry I edited my comment instead of posting again.
We can ditch DXCore and use DXGI which is older and shouldn't need build flags or API notes.
Also will need SDKLayers to be added. And I think that's it I think 🤞

Sorry for the silly request, while I could try to fish out the docs, I want to make sure that I get the correct header.  Would you mind sharing a pointer to MSDN for the API surface so I can cross-check and validate the module setup?

No prob. I think these are what you want.
DXGI and D3d12/D3d12SDKLayers

It seems that we should be okay with additional link directives to pick up the import libraries.  The dxd312.h includes d3d12sdklayers.h and d3d12sdklayers.h will include dxgi.h.  The problem with the explicit include is that dxgi.h is a shared header, and the module map is in the um path, which makes it slight more challenging to get it explicitly handled.

It seems that we should be okay with additional link directives to pick up the import libraries.  The dxd312.h includes d3d12sdklayers.h and d3d12sdklayers.h will include dxgi.h.  The problem with the explicit include is that dxgi.h is a shared header, and the module map is in the um path, which makes it slight more challenging to get it explicitly handled.

@swift-ci please test

@swift-ci please test

@swift-ci please smoke test macOS platform

@swift-ci please smoke test macOS platform

@swift-ci please test Windows platform

@swift-ci please test Windows platform

@swift-ci please test Windows platform

@swift-ci please test Windows platform

Does this need any change in the VS2017 CI machines? Seems that at least xaudio2.h is not being parsed correctly.

Does this need any change in the VS2017 CI machines? Seems that at least xaudio2.h is not being parsed correctly.


Dropdowns not working with angular hydration

Is this a regression?

[ ] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
NA
Description
Dropdowns will be misplaced, and not clickable.
Reproduction
StackBlitz link:
Steps to reproduce:

Add mat select to application https://material.angular.io/components/select/overview
Integrate angular universal
Add hydration to angular universal

Expected Behavior
Dropdown should work as expected
Actual Behavior
Select dropdown floats to further most left of page. Select option items are non clickable
Environment

Angular: 16.1.1
CDK/Material: 16.1.1
Browser(s): Chrome
Operating System (e.g. Windows, macOS, Ubuntu): macOS

Not sure if hydration related, I am looking into further

closing as couldn't determine if was actually bug and was solved


Not supported for apex

I've tried to train model with DCNv2 via apex mix-precision method, but occurred a dtype bug:
"deform_conv_forward_cuda" not implemented for 'Half'

Does it mean that I cannot train model with half precision?
thx

I encounter the same problem


Idea: remove StableSeen, write offset in buffer to object headers

Currently when serializing data we update mutable object headers to StableSeen in the first pass where we compute the buffer size, and update again to the location of object's serialization in the heap when we see the object for the second time during serialization. When we see the object for the third, fourth, ... times we use the address in the header to compute the location of the object in serialization buffer and use that value.
I think we could avoid updating to StableSeen (and remove StableSeen tag) by writing the offset of the object in the buffer to the object's header in the pass that computes buffer size (we already know the answer as we maintain a counter for the buffer size). This simplifies serialization code because we don't need to compute offset of the data in the buffer from the heap location, and we can remove StableSeen.
The tricky part is how to distinguish an offset in serialization buffer from a tag, in the first pass (in the second pass we can assume mutable object headers will have the offset). With the current scheme where we store addresses in heap, rather than offsets in serialization buffer, we can use a value like 0 (not a valid address, points to Rust stack) or -1 (we can't have a mutable object at this location as we don't have space for the fields). With offsets, 0 is now a valid offset, but we can use -1 (0xFFFFFFFF) for the same reason (no space in the buffer for the fields).
Thoughts? @nomeata @crusso @ggreif

So let’s see:

First visit by size calculation: Change tag to expected position in output buffer.
Further visit by size calculation: Tag is not an object tag, so ignore.
First visit by serialization: Tag matches current position in output buffer: Serialize it
Subsequent visit by serizliation: Tag does not match current position, skip serialization and store an alias.

That seems to hold water.

The tricky part is how to distinguish an offset in serialization buffer from a tag

Right now, all tags are very small and definitely not heap addresses into a dynamic buffer. So it should work?

we can use a value like 0 or -1

You mean for the object tag, before serialization?

Right now, all tags are very small and definitely not heap addresses into a dynamic buffer. So it should work?

In my suggestion we store offsets in serialization buffer, instead of addresses in heap. So for example 12 will be a valid offset and a valid tag (Bits32).

we can use a value like 0 or -1
You mean for the object tag, before serialization?

No, during serialization, for StableSeen. In my suggestion we will need to distinguish an offset to the serialization buffer from tags.

If we store offsets as offset + max_tag, then we can distinguish a tag from an offset by checking if it's larger than the max. tag (currently 15).

The benefit is just to remove a single subtraction in the serialization code?

No, I want to remove StableSeen.

I thought you want to get rid of it?

Yes, I was just explaining why the value -1 currently works for StableSeen.
The goal is to remove StableSeen.

Okay - and why change the stored value from a heap address to an offset? It seems you can remove StableSeen with what we have right now as well?
Ah, but you are saying it's simply simpler with just offsets, as that's what the algorithms deal with anyways. I think I get it now.
Yes, some “encoding” (fixed offset, or maybe storing a negative number, or making the object tags negative) seems to be needed to distinguish offsets from object tags.


"pf2e-dorako-ux.hiding.no-effects-panel" is not a registered game setting

FoundryVTT: 12.328
PF2e: 6.0.4
Dorako UI: 3.5.8

Marked into wrong Module


ice40: tolerate unconnected SB_LUT4 inputs

Fixes #2472.

Could be superseded by #2390 if that PR is changed to use default values.

Has been fixed in #2669.


[csrng/dv] Verify INT_STATE_READ_ENABLE CSR

Description
The INT_STATE_READ_ENABLE CSR including REGWEN has been added to lock read access to the internal states via INT_STATE_VAL CSR on a per instance basis.
We should add some DV for this.

[ ] Verify that INT_STATE_VAL reads as 0 whenever INT_STATE_READ_ENABLE of the instance selected with INT_STATE_NUM is 0.
[ ] Verify that once locked, read access can no longer be enabled.

Note, when implementing the RESEED_COUNTER_X CSRs and the INT_STATE_READ_ENABLE in https://github.com/lowRISC/opentitan/pull/23539, I've briefly verified that things are working , so the risk that this is broken should be low. Thus P3.

The PR resolving this issue is here: #24000 .


title of the website must be in  different font from all other content

the title must be in different font and color to enhance the website look

@vishanurag  plz assign me this

You can assign that task to me...

@vishanurag  please tell me


How can I add image from excel?

Im not sure what you mean exactly. You mean to import an image from excel to a model field?
Can you specify what you have tried so far? And what did and did not work for you?


DFA: Handle inline entities, lists and tuple fields properly.

DFA was not drilling into the inline entity, lists, and tuple fields to determine the access paths of a handle. This PR fixes that issue.
Note that this PR is dependent on the changes in https://github.com/PolymerLabs/arcs/pull/5842. Ignore any change related to schema registration in this PR.

/gcbrun

I pushed a few more changes and cleanups. Please take a look. It would be most useful if you can take a look at the test examples and see if there is anything that is not covered. @csilvestrini @piotrswigon

@piotrswigon I will merge this in so as to send out the type variable support PR.


Chromedriver being installed is not up-to-date.

I am getting the following error:
selenium.common.exceptions.WebDriverException: Message: unknown error: result.webdriverValue.value list is missing or empty in Runtime.callFunctionOn response
  (Session info: chrome=119.0.6045.160)

I read on StackOverflow this error occurs when the Chromedriver doesn't match Chrome. So I checked the version of webdriver and Chrome. The Chrome version listed in the error message is the version of my browser, 19.0.6045.160. I checked and saw that webdriver_auto_update did in fact overwrite the driver today, but the version of the driver installed is only  v118.0.5993.70, not 119 . I am concerned that webdriver_auto_update is not up to date for some reason.
I have no control over my Chrome updates at work. I am on Windows.

I'm sorry, the Chrome Driver was not being updated. The last update was 10/20. Hmm. I must be doing something wrong.

@SteveScott Automatic version updates by Chrome may be different than the stable release versions of ChromeDriver. For example, Chrome and ChromeDriver's patch versions may differ slightly such as below, but their major, minor and build numbers for compatibility are the same:
Example as of 2023-11-24T22:08:52
Chrome: 119.0.6045.160
ChromeDriver (stable release): 119.0.6045.105

Thanks for the help. I have a new issue.
As I understand it webdriver_auto_update pulls the latest version of the ChromeDriver. This assumes that the version of Chrome is up-to-date. But what if my workplace holds-off on updating Chrome? Then my ChromeDriver will be ahead of the local installation of Chrome and breaks? I wonder if there is a way to look up the currently-installed version of Chrome, and pull the driver that matches it?

@SteveScott To answer your questions, I would separate it into two sections
1. Checking currently installed version of Chrome program.

Installation of Chrome locally should have a dedicated directory with a version number as the directory name. In the example image below (Windows machine), there is a directory "120.0.6099.130" corresponding to the current locally installed version of Chrome:

As such, the local Chrome version number can be extracted via libraries such as os or pathlib
Since this a rather important feature, it could be included in the next release this project.

2.  Pulling chromedriver that matches the locally installed Chrome version

This can be performed simply by setting the attribute online_driver_version. The download_url variable contains the string from online_driver_version.

As an example below using windows, when the online_driver_version is set to 116.0.5793.0, the download_url variable would be a complete link to download chromedriver of version 116.0.5793.0:  https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5793.0/win64/chromedriver-win64.zip
Perhaps a setter or alternative function could be introduced in the next release of this project to allow the option of downloading a particular chromedriver version (e.g. version 116.0.5793.0) instead of the default latest stable release.

There is an example of how to check the Chrome version with and without the web driver here:
https://sqa.stackexchange.com/questions/41354/programmatically-determine-chrome-browser-version-regardless-of-os

I will submit a pull request shortly

@SteveScott Currently work in progress. However, I have created a feature branch with a possible implementation to solve the issue of "Checking currently installed version of Google Chrome program".
A new release should be expected soon with a new utility class called ChromeAppUtils. Meanwhile, here is the code snippet:
https://github.com/competencytestlvl/webdriver_auto_update/blob/feature/chrome-app-version-extraction/src/webdriver_auto_update/chrome_app_utils.py


mxnet tensorrt bind does not support dynamic batch?

Now mxnet simple bind support dynamic batch size when inference, but I try the tensorrt bind, it does not support dynamic batch. Any suggestions?

@Edwardmark  are you requesting tensorrt support for simple bind operation in cpp-package (C++ API)?
@mxnet-label-bot add [Feature Request, C++]

@leleamol I try to use mxnet tensorrt python package to convert mxnet graph to tensorRT,  following the link below:
https://cwiki.apache.org/confluence/display/MXNET/How+to+use+MXNet-TensorRT+integration
But I found that if I bind the model with batch_size 1, I cannot forward the model with batch_size 8; but using the original simple bind of mxnet or the original tensorRT, the model can accept dynamic batch. So I want to request dynamic batch support for mxnet tensorRT forward. Thanks, @mxnet-label-bot @leleamol.


[task] It shouldn't be possible to assign status from outside

Which package(s) are affected?
Task (@lit/task)
Description
I noticed that it's possible to set the status of a task manually, from outside, not via args, but just by assigning a value to task.status.
As long as the value is valid TaskStatus (let's say: TaskStatus.ERROR), it will result in rendering using expected renderer, but

only on a following update (unless this.requestUpdate() is called as well: this.task.status = TaskStatus.ERROR
 this.requestUpdate();

onError of the task won't run
taskComplete promise won't be rejected

I think this is a bug and that status should be reimplemented as a getter so it's not possible to set state from outside.
It is potentially a breaking change - users might have already started exploiting that behaviour in case there's neither onError nor one uses taskComplete rejections
Reproduction
repro
Demo shows that it's possible to make task render with error renderer 2 ways

one way that's arguably "correct", one that will make onError run and will make taskComplete promise get rejected,
one way that's IMO wrong and shouldn't be allowed, one which doesn't make onError or taskComplete promise rejected

Workaround
To prepare a task in such way that there's a way to fail it if it's needed (repro has such way)
Is this a regression?
No or unsure. This never worked, or I haven't tried before.
Affected versions
all
Browser/OS/Node environment
all

That makes sense to me. It would be weird for a user to set the task status themselves. I can't think of a valid reason for that to happen. So making it a getter instead sounds reasonable.


boards: adi: Add AD-APARD32690-SL

Add board support for the ADI AD-APARD32690-SL. The platform is based on a MAX32690 microcontroller.
Tested with the following samples:

blinky
button
sockets/echo

The board's product page may be accessed here: https://www.analog.com/en/resources/evaluation-hardware-and-software/evaluation-boards-kits/ad-apard32690-sl.html

other then the image it LGTM

V3:

Converted the images to the webp format.

@CiprianRegus Can you have a look at the build errors in CI to hopefully get this in before freeze this Friday? Thanks!

V5:

Cherry-picked https://github.com/zephyrproject-rtos/zephyr/pull/73910/commits/80d5e2e7b8a8bfca3eb3a04d8effdef1d5f91de6 in order to fix CI errors, so https://github.com/zephyrproject-rtos/zephyr/pull/74038 has to be merged first.
Removed the zephyr,bt-uart property from the chosen node.
Used DT_FREQ_M() in order to set spi-max-frequency.


cupsEnumDests(): Add printer-is-temporary attribute to the destination options

I want to perform an operation only on the permanent queues (the ones manually set up) of the local CUPS daemon, not on temporary queues (the ones CUPS auto-creates when one accesses a driverless remote printer advertised by DNS-SD).
I can call cupsEnumDests() with flags so that only queues of the local CUPS daemon and not remote printers without local CUPS queues are shown), but the queues of the local CUPS daemon which I get listed then are both the permanent queues and temporary queues. Problem is that one cannot distinguish which of these are permanent and which are temporary. There is also no way to call cupsEnumDests() in a way that on gets only permanent or only temporary queues.
Adding a printer-is-temporary attribute to the options of the destination records would easily solve this problem.
See also the thread "cupsEnumDests(): Suppress listing temporary CUPS queues or distinguish temporary queues in resulting queue list" on the cups-devel mailing list.

[master 12b798a07] Report the value of the printer-is-temporary attribute (Issue #5028)