ASR recipe for Tedlium2 (code from Samsung AI Cambridge)

What does this PR do?
This PR provides a SOTA WER on the Tedlium2 dataset (matching ESPnet Branchformer). It was developed by Shucong Zhang from Samsung AI Center Cambridge.

Before submitting

[x] Did you read the contributor guideline?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[x] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[x] Did you list all the breaking changes introduced by this pull request?
[x] Does your code adhere to project-specific code style and conventions?

PR review

Reviewer checklist

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified
[x] Confirm that the changes adhere to compatibility requirements (e.g., Python version, platform)
[x] Review the self-review checklist to ensure the code is ready for review

I fixed most issues and added recipe testing, @s1zhang could you have a look at the comments?

Thanks for the comments and fixed issues! I've modified the code to address the issues (data preparation of csv files for ASR training, tqdm, assertion of positive time_mask_ratios, and typos and formatting), and made a pr to @TParcollet

@Adel-Moumen fixed!

Thank you @TParcollet and the Samsung team for this PR!
I've made some changes such as fixes to ensure that recipe tests can run smoothly and some overall improvements (please double check). Before merging, I believe there are a few steps we need to take:

Upload the output folder to Dropbox
Upload the model on Hugging Face: Sharing the model on Hugging Face
Add notes about training time in the README

@mravanelli Hi Mirco, thank you very much for the editing. For the ckpt, Samsung does not allow the team to release it, but we can release the train log. As suggested by @TParcollet, if possible, could the speechbrain team run the script to get the ckpt? Thank you again.

In particular @Adel-Moumen could run it :p

In particular @Adel-Moumen could run it :p

Will do.

Could you please @shucongzhang have a look at the comments? I think @TParcollet assigned you on some tasks :p Thanks!

Could you please @shucongzhang have a look at the comments? I think @TParcollet assigned you on some tasks :p Thanks!

@Adel-Moumen I assume you are talking about evaluate_batch and fit_batch. However, they are exact the same as "recipes/LibriSpeech/ASR/transformer/train.py". :P Thus, I'm not so sure about how to edit it, and will it affect the performance.

Could you please @shucongzhang have a look at the comments? I think @TParcollet assigned you on some tasks :p Thanks!

@Adel-Moumen I assume you are talking about evaluate_batch and fit_batch. However, they are exact the same as "recipes/LibriSpeech/ASR/transformer/train.py". :P Thus, I'm not so sure about how to edit it, and will it affect the performance.

I am talking about my latest comments on multiprocessing etc. I will do it myself for evaluate_batch and fit_batch after having merged your PR. :-)

Hello,
I trained multiple models on the Tedlium3 recipe, unfortunately I'm not reaching your claimed numbers.
My latest (and best) run gave me the following results:
speechbrain.utils.train_logger - Epoch loaded: 120 - test loss: 10.88, test ACC: 9.60e-01, test WER: 12.83
# ############################################################################
# Model: E2E ASR with Transformer
# Encoder: Branchformer Encoder
# Decoder: Transformer Decoder + (CTC/ATT joint) beamsearch
# Tokens: unigram
# losses: CTC + KLdiv (Label Smoothing loss)
# Training: Tedlium2
# Authors:  Titouan Parcollet, Shucong Zhang
# ############################################################################
# Seed needs to be set at top of yaml, before objects with parameters are made

seed: 3407
__set_seed: !apply:torch.manual_seed [!ref <seed>]
output_folder: !ref results/branchformer_large/<seed>
output_wer_folder: !ref <output_folder>/
save_folder: !ref <output_folder>/save
train_log: !ref <output_folder>/train_log.txt

# Data files
# IMPORTANT: before running this script, you need to train the tokenizer (refer to ../../Tokenizer/README.md for details).
# When training the tokenizer, we will create the csv files by default in
# ../../Tokenizer/results/tokenizer/train/train.csv
# ../../Tokenizer/results/tokenizer/dev/dev.csv
# ../../Tokenizer/results/tokenizer/test/test.csv
#
# The tokenizer is stored in ../../Tokenizer/results/tokenizer/tokenizer.ckpt
#
# The script also creates in the specified clipped_utt_folder the version of the dataset that will be used here for training the ASR model.

# Please ensure that the tokenizer has been trained before (refer to ../../Tokenizer/README.md for details).
pretrained_tokenizer_file: ../../Tokenizer/results/tokenizer/tokenizer.ckpt

train_csv: !ref ../../Tokenizer/results/tokenizer/train/train.csv
valid_csv: !ref ../../Tokenizer/results/tokenizer/dev/dev.csv
test_csv:
    - !ref ../../Tokenizer/results/tokenizer/test/test.csv

# Training parameters
# To make Transformers converge, the global bath size should be large enough.
# The global batch size is computed as batch_size * n_gpus * gradient_accumulation.
# Empirically, we found that this value should be >= 128.
# Please, set your parameters accordingly.
number_of_epochs: 120
batch_size: 16 # This works for 2x GPUs with 32GB
ctc_weight: 0.3
grad_accumulation_factor: 2
max_grad_norm: 5.0
loss_reduction: 'batchmean'
sorting: random
num_workers: 4

# stages related parameters
# stage_one_epochs: 90
lr_adam: 0.0005
weight_decay: 0.05

# Feature parameters
sample_rate: 16000
n_fft: 400
n_mels: 80
win_length: 25
n_time_mask: 7
# This setup works well for A100 80GB GPU, adapts it to your needs.
# Or turn it off (but training speed will decrease)
dynamic_batching: True
max_batch_len: 800
max_batch_len_val: 100 # we reduce it as the beam is much wider (VRAM)
num_bucket: 200

dynamic_batch_sampler:
    max_batch_len: !ref <max_batch_len>
    max_batch_len_val: !ref <max_batch_len_val>
    num_buckets: !ref <num_bucket>
    shuffle_ex: True # if true re-creates batches at each epoch shuffling examples.
    batch_ordering: random
    max_batch_ex: 128

# Dataloader options
train_dataloader_opts:
    batch_size: !ref <batch_size>
    shuffle: True
    num_workers: !ref <num_workers>

valid_dataloader_opts:
    batch_size: 1

test_dataloader_opts:
    batch_size: 1

####################### Model parameters ###########################
# Transformer
d_model: 512
nhead: 8
num_encoder_layers: 18
num_decoder_layers: 6
csgu_linear_units: 3072
csgu_kernel_size: 31
transformer_dropout: 0.1
activation: !name:torch.nn.GELU
output_neurons: 500

# Outputs
blank_index: 0
label_smoothing: 0.1
pad_index: 0
bos_index: 1
eos_index: 2

# Decoding parameters
min_decode_ratio: 0.0
max_decode_ratio: 1.0
valid_search_interval: 10
valid_beam_size: 20
test_beam_size: 20
ctc_weight_decode: 0.3

############################## models ################################

CNN: !new:speechbrain.lobes.models.convolution.ConvolutionFrontEnd
    input_shape: (8, 10, 80)
    num_blocks: 2
    num_layers_per_block: 1
    out_channels: (64, 32)
    kernel_sizes: (3, 3)
    strides: (2, 2)
    residuals: (False, False)

Transformer: !new:speechbrain.lobes.models.transformer.TransformerASR.TransformerASR # yamllint disable-line rule:line-length
    input_size: 640
    tgt_vocab: !ref <output_neurons>
    d_model: !ref <d_model>
    nhead: !ref <nhead>
    num_encoder_layers: !ref <num_encoder_layers>
    num_decoder_layers: !ref <num_decoder_layers>
    dropout: !ref <transformer_dropout>
    activation: !ref <activation>
    branchformer_activation: !ref <activation>
    encoder_module: branchformer
    csgu_linear_units: !ref <csgu_linear_units>
    kernel_size: !ref <csgu_kernel_size>
    attention_type: RelPosMHAXL
    normalize_before: True
    causal: False

tokenizer: !new:sentencepiece.SentencePieceProcessor

ctc_lin: !new:speechbrain.nnet.linear.Linear
    input_size: !ref <d_model>
    n_neurons: !ref <output_neurons>

seq_lin: !new:speechbrain.nnet.linear.Linear
    input_size: !ref <d_model>
    n_neurons: !ref <output_neurons>

normalize: !new:speechbrain.processing.features.InputNormalization
    norm_type: global
    update_until_epoch: 4

modules:
    CNN: !ref <CNN>
    Transformer: !ref <Transformer>
    seq_lin: !ref <seq_lin>
    ctc_lin: !ref <ctc_lin>
    normalize: !ref <normalize>

model: !new:torch.nn.ModuleList
    - [!ref <CNN>, !ref <Transformer>, !ref <seq_lin>, !ref <ctc_lin>]

Adam: !name:torch.optim.AdamW
    lr: !ref <lr_adam>
    betas: (0.9, 0.98)
    eps: 0.000000001
    weight_decay: !ref <weight_decay>

valid_search: !new:speechbrain.decoders.S2STransformerBeamSearch
    modules: [!ref <Transformer>, !ref <seq_lin>, !ref <ctc_lin>]
    bos_index: !ref <bos_index>
    eos_index: !ref <eos_index>
    blank_index: !ref <blank_index>
    min_decode_ratio: !ref <min_decode_ratio>
    max_decode_ratio: !ref <max_decode_ratio>
    beam_size: !ref <valid_beam_size>
    ctc_weight: !ref <ctc_weight_decode>
    using_eos_threshold: False
    length_normalization: False

test_search: !new:speechbrain.decoders.S2STransformerBeamSearch
    modules: [!ref <Transformer>, !ref <seq_lin>, !ref <ctc_lin>]
    bos_index: !ref <bos_index>
    eos_index: !ref <eos_index>
    blank_index: !ref <blank_index>
    min_decode_ratio: !ref <min_decode_ratio>
    max_decode_ratio: !ref <max_decode_ratio>
    beam_size: !ref <test_beam_size>
    ctc_weight: !ref <ctc_weight_decode>
    temperature: 1.15
    temperature_lm: 1.15
    using_eos_threshold: False
    length_normalization: True

log_softmax: !new:torch.nn.LogSoftmax
    dim: -1

ctc_cost: !name:speechbrain.nnet.losses.ctc_loss
    blank_index: !ref <blank_index>
    reduction: !ref <loss_reduction>

seq_cost: !name:speechbrain.nnet.losses.kldiv_loss
    label_smoothing: !ref <label_smoothing>
    reduction: !ref <loss_reduction>

noam_annealing: !new:speechbrain.nnet.schedulers.NoamScheduler
    lr_initial: !ref <lr_adam>
    n_warmup_steps: 30000

checkpointer: !new:speechbrain.utils.checkpoints.Checkpointer
    checkpoints_dir: !ref <save_folder>
    recoverables:
        model: !ref <model>
        noam_scheduler: !ref <noam_annealing>
        normalizer: !ref <normalize>
        counter: !ref <epoch_counter>

epoch_counter: !new:speechbrain.utils.epoch_loop.EpochCounter
    limit: !ref <number_of_epochs>

augmentation: !new:speechbrain.lobes.augment.SpecAugment
    time_warp: True
    time_warp_window: 5
    time_warp_mode: bicubic
    freq_mask: True
    n_freq_mask: 2
    time_mask: True
    n_time_mask: !ref <n_time_mask>
    replace_with_zero: False
    freq_mask_width: 30
    time_mask_width: 25
    time_mask_ratio: 0.05

speed_perturb: !new:speechbrain.processing.speech_augmentation.SpeedPerturb
    orig_freq: !ref <sample_rate>
    speeds: [95, 100, 105]

compute_features: !new:speechbrain.lobes.features.Fbank
    sample_rate: !ref <sample_rate>
    n_fft: !ref <n_fft>
    win_length: !ref <win_length>
    n_mels: !ref <n_mels>

train_logger: !new:speechbrain.utils.train_logger.FileTrainLogger
    save_file: !ref <train_log>

error_rate_computer: !name:speechbrain.utils.metric_stats.ErrorRateStats
acc_computer: !name:speechbrain.utils.Accuracy.AccuracyStats

# The pretrainer allows a mapping between pretrained files and instances that
# are declared in the yaml.
My latest (and best) run gave me the following results: speechbrain.utils.train_logger - Epoch loaded: 120 - test loss: 10.88, test ACC: 9.60e-01, test WER: 12.83

@Adel-Moumen Thank you very much for running the script. I've checked my log. "lr" and "steps" are exact the same with your train_log.txt, and "train/vaild/test loss" all look similar... However, valid and test WERs are indeed to high. I'll use the current script in this PR to do a data preparation and a decoding with my ckpt to see what numbers I can get.

Hello @shucongzhang,
Titouan fixed a bug in a recent PR. I have updated this PR with the latest commits and I am running the training of a new model. I'll get back to you with the new results.
best,
Adel

Hello @shucongzhang,
Titouan fixed a bug in a recent PR. I have updated this PR with the latest commits and I am running the training of a new model. I'll get back to you with the new results.
best, Adel

Hi @Adel-Moumen Thank you for the new experiment. I'm looking forward to the results.

Hello @shucongzhang,
Titouan fixed a bug in a recent PR. I have updated this PR with the latest commits and I am running the training of a new model. I'll get back to you with the new results.
best, Adel

Hi @Adel-Moumen Thank you for the new experiment. I'm looking forward to the results.

Hello @shucongzhang,
The results are out: 8.11% WER. Much better than before, slightly worse (by 0.2) than your claimed results but I think this is fine like that.
I changed several things in the PR:

data preparation part is now relying on our tool parallel_map, I also added some statistics (total duration, number of samples) and I cleaned a bit some things.
Output:

speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: results/tokenizer_v3
tedlium2_prepare - Preparing results/tokenizer_v3/train/train.csv...
tedlium2_prepare - JillSobuleMANHATTANINJANUARY_2006.sph is skipped
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1495/1495 [05:45<00:00,  4.33it/s]
tedlium2_prepare -      results/tokenizer_v3/train/train.csv successfully created!
tedlium2_prepare - Number of samples: 90659
tedlium2_prepare - Total duration: 206.2 Hours
tedlium2_prepare - Preparing results/tokenizer_v3/dev/dev.csv...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:02<00:00,  3.65it/s]
tedlium2_prepare -      results/tokenizer_v3/dev/dev.csv successfully created!
tedlium2_prepare - Number of samples: 502
tedlium2_prepare - Total duration: 1.6 Hours
tedlium2_prepare - Preparing results/tokenizer_v3/test/test.csv...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 11/11 [00:03<00:00,  3.13it/s]
tedlium2_prepare -      results/tokenizer_v3/test/test.csv successfully created!
tedlium2_prepare - Number of samples: 1143
tedlium2_prepare - Total duration: 2.61 Hours
speechbrain.tokenizers.SentencePiece - Train tokenizer with type:bpe
speechbrain.tokenizers.SentencePiece - Extract wrd sequences from:results/tokenizer_v3/train/train.csv
speechbrain.tokenizers.SentencePiece - Text file created at: results/tokenizer_v3/train.txt

Now, we are using the data prep also inside of the transformer folder in train.py. The reason is that when you are using a pretrained model (e.g. the model that I shared inside of the dropbox), you will end up with the tokenizer but the train.csv, dev.csv etc will contain invalid links. To tackle down this issue, the recipe will automatically run the data prep step (as usual in SpeechBrain).
I uploaded the model that I trained and updated the results.

Could you please @shucongzhang and @TParcollet have another look to this PR? Thanks :-)

Recipe tests for Tedlium2: ✅

Thank you @TParcollet  (and the Samsung team) for this PR. Thank you @Adel-Moumen  for the review. I've tested this PR, and everything appears to be functioning as expected. Before we merge it, there are a few final comments:

Please upload the model on the Hugging Face (HF) model and ensure this is mentioned in the README.

Include information about the training time in the README for users' reference.

In the README, specify how the users should change dynamic batch parameters to allow the recipe to work with a 32 GB GPU

I uploaded the model that I trained and updated the results.

Could you please @shucongzhang and @TParcollet have another look to this PR? Thanks :-)
Note: I'm running the recipe test.

Hi @Adel-Moumen , Thank you so much for all the training/code reviewing! I have read the PR again and I think I do not have any additional comment now.

Thank you @TParcollet (and the Samsung team) for this PR. Thank you @Adel-Moumen for the review. I've tested this PR, and everything appears to be functioning as expected. Before we merge it, there are a few final comments:

Please upload the model on the Hugging Face (HF) model and ensure this is mentioned in the README.
Include information about the training time in the README for users' reference.
In the README, specify how the users should change dynamic batch parameters to allow the recipe to work with a 32 GB GPU

Hello Mirco,

Done. See README / https://huggingface.co/speechbrain/asr-branchformer-large-tedlium2
Done.
I won't do it as I think no matters what you do with the dynamic batch parameters it wont fit in 32GB (due to the model). It would be better to add a new yaml (e.g. branchformer_small) instead which could fit in a 40GB GPU. But I propose to do it latter and solely if the community wants to use our Tedlium recipe.

Thank you for the changes! I think it is ready to be merged now.

Thanks again @shucongzhang and @TParcollet (and all the Samsung AI participants) for providing the training recipe. I hope you will continue contributing to SpeechBrain :-) (well done for your first contribution @shucongzhang ! 🎉)

Thank you so much @Adel-Moumen ! I'll definitely continue to work with SpeechBrain :-)


Make Github Landing Page to One Stop Go

Tasks :

[ ] #2
[ ] #1
[ ] Add all meeting info , joining info for new memebers and Discord Join link for communication and Github for maintaining github.io , meeting docs , discussions and todo

Repeat !


haddock-library on hackage with base < 4.8 bound breaks with ghc 7.10

Hi,
is there a chance that somebody could upload a haddock-library version on hackage that does not have the base < 4.8 version bound?  I am currently trying to install pandoc but it depends on haddock-library which fails to build because of this.
Cheers,
Markus

Does haddock-library work with base-4.8? What's the error you're seeing, pandoc doesn't constrain base directly.

The new git version of haddock-library 1.2 does work with ghc 7.10.  Pandoc currently constrains haddock-library to ==1.1, which is not haddock-library's fault, but as there is currently no version of haddock-library on hackage that works with 7.10 there is no point in changing it just now.
I don't know if 1.2 is ready for release but maybe a quick update for 1.1.* to work with ghc 7.10 would be nice :)

GHC 7.10 is not out and I can not release 1.2 early because it is due to ship when 7.10 does as is rest of Haddock which will depend on it. I could maybe to maintenance 1.1 release but I now remembered that we never did make the constraint on base in the release. I think @hvr added it with the Hackage feature after it was uploaded for a reason I can no longer remeber but maybe he can remind us.

I think @hvr added it with the Hackage feature after it was uploaded for a reason I can no longer remeber but maybe he can remind us.

...it was simply because it fails to compile with the current GHC 7.10.1
it's a bit messy (since we now have multiple packages in a single Git repo and can't cleanly branch off with the current versioning scheme), but we can backport the changes needed into a new haddock-library-1.1.x Git branch... (or maybe it rather belongs into a haddock-2.15.x branch? that's why I don't like multiple packages in a single Git repo with diverging package versions...)

Any ETA for this fix? Many tools, such as, hsdev, hdocs, etc. cannot be installed due to this problem which breaks people's dev. env.

Forgot about this but it has been fixed 53 minutes ago when I uploaded haddock-library-1.2 to hackage. Libraries which use it may need updating.

Thanks!
Yes, hdocs complains about:
Expected type: InstalledInterface -> Map Name (Doc Name)
     Actual type: InstalledInterface -> DocMap Name

But that's an easy fix.

@mantkiew  sadly hdocs disregards the PVP by omitting all upper bounds on its build-deps, thus causing deliberate install-plan bitrotting :disappointed:


Feature request: Global shortucts to create or find a note

It would so great to implement global shortcuts in Tusk. It is very convenient to quickly search for a note with a global hot-key like Win + Shift + F, rathen then click on Tusk app and then using Ctrl+F.
Original Evertnote client has following global shortcuts (Windows)
Shortcut | Action
Ctrl + Alt + N | New note
Win + Shift + F | Find in Evernote
Win + PrintScreen | Capture screen
Win + A | Copy selection
Ctrl + Alt + V | Paste clipboard
I think the most used are first two ones - create a note and find a note.
I don't have experience in developing with Electron, but here's what I've quickly found:
https://github.com/electron/electron/blob/master/docs/api/global-shortcut.md

Thank you for taking the time to suggest it : )


TE-1.2: add deviation for /system/mac-address/state

Adds -deviation_mac_address_missing for devices that do not support /system/mac-address/state

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 56.84%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1234

Relevant Lines:
2171

💛  - Coveralls

/gcbrun


Obtain a complete list of all Parms for the England, Wales The Channel Isles

@richardofsussex has provided a complete set of parms for all years in England and Wales and Islands. Suggest this be closed

Agreed!  @Captainkirkdawson so closing


octant: 0.25.0 -> 0.25.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for octant is: "Highly extensible platform for developers to better understand the complexity of Kubernetes clusters"
meta.homepage for octant is: "https://octant.dev/"
meta.changelog for octant is: "https://github.com/vmware-tanzu/octant/blob/v0.25.1/CHANGELOG.md"
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
Warning: no invocation of /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1/bin/octant had a zero exit code or showed the expected version
0 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 0.25.1 with grep in /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1
found 0.25.1 in filename of file in /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
octant

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A octant https://github.com/r-ryantm/nixpkgs/archive/9a15329e3f19ceb810a45e0b494a6a81d87cf6ff.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1
ls -la /nix/store/pv3dd6mmvqks4zsmrvg75crwkvy54i3s-octant-0.25.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

octant

Maintainer pings
cc @06kellyjac for testing.

Result of nixpkgs-review pr 161850 run on x86_64-linux 1

1 package built:

octant


tests(model): add test coverage for model

Current coverage is 54.66% (diff: 100%)

Merging #276 into master will increase coverage by 8.25%

@@             master       #276   diff @@
==========================================
  Files             3          4     +1
  Lines           306        311     +5
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits            142        170    +28
+ Misses          160        131    -29
- Partials          4         10     +6

Powered by Codecov. Last update 9a0d3c8...427fb4c

Before:
ok  	github.com/deis/router/model	0.056s	coverage: 5.9% of statements
?   	github.com/deis/router/utils	[no test files]

After:
ok  	github.com/deis/router/model	0.070s	coverage: 29.4% of statements
ok  	github.com/deis/router/utils	0.006s	coverage: 100.0% of statements

Awesome work on this!


Feature: Add shell tab completion to rstuf command

What do you want to share with us?
Adding tab shell completion is a nice touch to a CLI program and is something we can add to rstuf.
To enable completion for Click applications there is a manual process a user should follow according to Click's documentation.
While investigating how to do this seamlessly for the user I didn't find any solution that was flexible in how to add autocompletion, small, and with no 3rd party dependencies (except Click), so I published this small Python library, auto-click-auto.
With the provided functions from auto-click-auto we could either:

Run the check if autocomplete is enabled on every run of the rstuf command (it will enable it the first time and then just check the shell if it is enabled) - See https://github.com/repository-service-tuf/repository-service-tuf-cli/pull/322
Add it as an option, e.g., --autocomplete which will enable it and exit - See https://github.com/repository-service-tuf/repository-service-tuf-cli/pull/323
Add it as a new command or sub-command

I have tested it on:
Ubuntu 20.04.5 LTS
bash, version 5.0.17(1)-release
References

https://click.palletsprojects.com/en/8.1.x/shell-completion/
https://github.com/KAUTH/auto-click-auto

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Closed by #322


Bugfix

Hi @Malik166 - kindly provide the context for this update so I can understand the ratonale.

Closing as there's been no response to @mswaweru We can reopen as needed once more info is provided.


docs(readme): remove 0.x.x disclaimer

As this library is on 1.0.0, congrats!
Just remove 0.x.x disclaimer in readme. ☺

Hi @hongbo-miao 👋
Good catch, thanks for your contribution 😄 We've updated this disclaimer in the registry, and we will also update it in the README soon.

Hey @hongbo-miao. I updated the disclaimer in https://github.com/Snowflake-Labs/terraform-provider-snowflake/pull/3288. We can close this one :)

Got it, thanks!


For students, researchers, and everyone else - Search operators

It would be such a leap in functionality to include the option to construct custom search operators such as site: and "", allowing the user to filter the web search to just websites within domains such as .edu and .org, (or specific URLs) as well as only web results from pages that contain the "DOI:" phrase. For the time being, it does not even work well with custom prompts. It would be useful to be able to include anything like:
site:.edu AND site:.org AND "DOI:"
Optimally, it would be just two more fields in the footer bar: one for multiple site: operators separated by commas, and one for the "must contain" operators <""> separated by commas.

Hi, thanks for the feedback.
These filters are supported, but you need to type them into your query, not the prompt template.

Hi, thanks for the feedback. These filters are supported, but you need to type them into your query, not the prompt template.

I was kinda OK with that reply then... but now I use more complex prompt templates with injections of {query} into the instruction for the bot, and including things like site:.edu ruins the whole instruction.


upgrade fury to 0.9.0 and enable native build

Closes #7
Native build works:

Thanks a lot @chaokunyang !


Make auto-completion more user-friendly.

JIRA: PDC-1161

reference of pdc repo: lao605/product-definition-center#213
I think of a third solution where we can put python-argcomplete.sh file in bash completion scripts directory during package installation rather than put completion file of pdc specifically in bash completion scripts directory. So I think there are totally three plans we have now:

put pdc completion file (the result of eval "$(register-python-argcomplete pdc)") in bash completion scripts directory (which is always /etc/bash_completion.d/ for CentOS/RHEL)
let user run eval "$(register-python-argcomplete pdc)" when they need it
put python-argcomplete.sh (which is what activate-global-python-argcomplete puts in bash completion scripts directory); and developers should leave a PYTHON_ARGCOMPLETE_OK at the head of executables ready for auto-completion.

I have taken three factors in account: user-friendliness, extensibility and support for shells. So here is my analysis:
user-friendliness
by saying user-friendliness, I mean the steps users needed to do to make auto-completion work for pdc.

for plan 1, there will be two steps users (rather than one): install bash-completion using yum and install python-argcomplete using yum or pip. bash-completion functionality is not automatically enabled for CentOS/RHEL and must be enabled manually.
for plan 2, there will be two steps for users: install python-argcomplete using yum or pip and put the eval command argcomplete suggests in ~/.bashrc
for plan 3, there will be two steps for users which is the same with the first plan.

Although they all require 2 steps to make it work, plan 1 and plan 3 outweight plan 2 because in the plan 2 editing of ~/.bashrc file is needed. plan 1 and plan 3 wins.
extensibility

plan 1: for developers, when a new executable is to be added to pdc (if any), he will have to add a completion file for that exec and edit the spec file. This produces completion files scattered in pdc_client; for users, there is nothing to do when pdc upgrades.
plan 2: for developers, there is nothing to do other than write the new exec; for users, they have to manually add that to ~/.bashrc
plan 3: for developers ,there is nothing needed to do other than add a PYTHON_ARGCOMPLETE_OK at the top of the exec; for dev, there is nothing needed to do.

So, as for extensibility, plan 3 wins.
support for shells

plan 1 only supports bash
plan 2 support bash and zsh although there seems to be a different mechanism for zsh to do auto-complete
plan 3 only support bash

so, as for support for shells, plan 2 wins. But shells support is not of primary concern.
plan 3 is the same with plan 1 in user-friendliness and beat plan 1 and plan 2 in extensibility. I think it is a workable plan.

Any better suggestions, if no for now, how about merge this first?

Personally I would go with option 1. I'm not sure the extra magic of having it work with every python script is worth it and making the completion explicit seems like a good idea.
It would be a good idea to look at how other projects handle this.
Fedpkg ships with fedpkg.bash, but this is hand-written, package specific script. So does dnf. Finding examples that use argcomplete would be better, though.

Update this to use the plan 1.
Add argcomplete as optional(devel) requirement,
install pdc.bash to /etc/bash_completion.d/,
and update README to include guide about  bash completion.

:+1:


use this library with firebase sdk 10.2.4

Is it possible?

Any update about this?

yes. you can . please let me know if you see issue.


Support latest react-native version

Please add support for latest react-native version, i.e. 0.12.0

Thanks for reminding me. It supports React Native version after 0.10.0 from NPM now.
If having any problem, please tell me.


Update Aura to V2023.9.1

To fix network status detection....just so I don't forget to do it before stabe tomorrow.

It was updated already https://github.com/NickvisionApps/Tagger/pull/303/commits/c494d510bdaa8ce63e1162b2566e2efa6fb3d951

Ah ok my bad :)


Pushing bala with invalid access token still shows the progress bar before unauthorized message

Description:
When pushing a bala to central with an invalid access token, the progress bar still shows up before the error.
~/Desktop/TestProject ❯ bal push

ibaqu/TestProject:1.0.0 [project repo -> central] 100% [===============================================================] 1516/1516 B (0:00:00 / 0:00:00)
ballerina: unable to push package ibaqu/TestProject:1.0.0 as there are dependencies of the package missing in the registry. missing dependencies: [ballerina/io:1.0.1]

Steps to reproduce:
Affected Versions:
OS, DB, other environment details and versions:
Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

This happens since we send only one request to do both of the tasks:

validate token
upload the bala
We might have to decouple if we want to validate the token before uploading bala.

A new method to validate token before pushing package added.


From Carson: A UI that shows all answer to one SAQ question in annoymous way

This is done for SAQ.


Build /api/employee/{employee_id}/category/list

Follow /api/employee/{employee_id}/category/list endpoint documentation.

Adding endpoint to get all the categories associated to a specific employee.

Merged


Feature/history utils

This PR introduces 2 utility functions to interact with the History API. The push and replace functions allow you to replace parts of the current URL with the History API by merging the given options with the current URL, avoiding the need for complex merging strategies.
Parameters

options.path (String): the new path, defaults to location.pathname
options.search (Object): the new search, defaults to the current URL params
options.hash (String): the new hash, defaults to the current hash
data (Object): The data attached to the new history state
title (String): The title attached to the new history state

Usage
import { push, replace } from '@studiometa/js-toolkit/utils/history';

// Push a new state
push({ path: '/foo', search: { query: 'hello world' }, hash: 'results' });
// Old URL: /
// New URL: /foo?query=hello world#results

// Replace the current state
replace({ path: '/foo', search: { query: 'hello world' }, hash: 'results' });
// Old URL: /
// New URL: /foo?query=hello world#results

// Replace only the hash
replace({ hash: 'hello-world' });
// Old URL: /
// New URL: /#hello-world

// For the search and hash options, falsy values can be used to delete things
replace({ hash: '' });
// Old URL: /#hello-world
// New URL: /
replace({ search: { query: false } });
// Old URL: /?query=foo&page=10#hello-world
// New URL: /?page=10#hello-world

// Complex values for the search are converted into URL params according to the way PHP parses theme into the `$_GET` variable.
replace({
  search: {
    array: [1, 2],
    object: {
      foo: 'foo',
      bar: { baz: 'bar' }
    }
  }
});
// Replaced value: ?array[0]=1&array[1]=2&object[foo]=foo&object[bar][baz]=bar

Nice addition 👍 It seems history link miss from navbar.

Nice catch, I'll add it 🙂

Nice addition 👍 It seems history link miss from navbar.

Nice catch, I'll add it 🙂

👍

👍


Feature - (temp) bans WIP

basically a first draft to simply ban people, temp ban them etc. without needing an admin to kick them from a channel, including ban reason, moderator that banned etc.;
To-do list

[ ] automatically unban people
[ ] send message on unban
[ ] ban history
[ ] warn system
[ ] add warns and ban logs to the canvas
[ ] idk what else

Interesting work.
One possible feature:
Open source the log by editing canvass

it is still kinda basic since I am learning the stuff that is happening there rn but anyway - I'll add that to the todo

On a Quick Look I am not finding documentation on the proposed canvas feature, do you have any resources @V205Arduino ?

sounds like a good plan. please make sure you're caching bans, not making a database call on every message.

https://api.slack.com/reference/functions/canvas_update_content
Would likely have helpful content.

Due to https://github.com/mrhappyma/slack-ups-and-downs/issues/7 I will close this and re-open a new one WIP one on the new repo

for reference https://github.com/mrhappyma/ups-and-downs-rtm/pull/2


how to remove the red numbers

i extract the images.mcf and got the output images with res numbers

During extraction,  you get a question:
Do you want to print the image number on each image(y/n)
Answer this with "n" and you will not get any numbers.

During extraction, you get a question:
Do you want to print the image number on each image(y/n)


update dependencies

Suggest to close this PR, some of the changes are covered in other PRs and I don't think it addresses any known issues.


Getting line numbers/file name when errors occurs with mrb_load_file

I'm embedding mruby scripts from c/cpp side using mrb_load_file (and the mruby-require mgem).
However, whenever an error occurs in the said file, I get only the error description, but no line number/file name.
I saw that it may related to the fact that the file is compiled to mrb during this process, but I found no way to generate debug information from load_file.
Am I missing something or is it actually not supported ?

Probably you didn't use -g (debug option) when compiling your ruby files by mrbc.

I'm fully aware of the -g option.
I'm not using mrbc, I'm embedding mruby scripts (not compiled .rb files) in a c/c++ application.
My question being, when an error occurs in one file I load from my c/c++ application, is there any way to get the line numbers/file name.
Knowing that mruby turn .rb files into bytecode when I load them using mrb_load_file, is there any way to reproduce -g option with mrb_load_file ?

I found that using a combination of mrb_generate_code (with a call to mrb_filename to the context) and mrb_run was necessary for me to get context information.

@fourmond Thanks, will try that !

Hmm, mrb_load_file compiles a Ruby file into internal representation with debug information, so that you don't have to specify anything. Do you mean that you use something like the following code, and you don't get backtrace?
    v = mrb_load_file(mrb, path);
    if (mrb->exc) {
        mrb_print_error(mrb);
    }

Note that mrb_p(mrb, mrb_obj_value(mrb->exc)); does not print backtrace.

@matz Your first exemple is exactly what I use.
This is what I get with a file containing errors:
NameError: uninitialized constant Foo
Press [ENTER] to quit...

I tried different methods to get a more explicit backtrace: mrb_print_backtrace, using the mruby-backtrace mgem, I always get the same result. I tried digging a bit in the mrbc_context to see if I was missing something, but no luck so far.

I am sorry but I have no clue, if you are doing as you explained. Can I read your code?

Thanks to @fourmond I found what was missing from the context.
I get the proper backtrace when I do this:
int e;
FILE* fp;
if ((e = fopen_s(&fp, "./Main.rb", "r")) != 0)
{
	puts("Failed to open File");
	puts("Press [ENTER] to quit...");
	getchar();
	return e;
}

mrb_state* mrb = mrb_open();
mrbc_context* cxt = mrbc_context_new(mrb);

mrbc_filename(mrb, cxt, "Main.rb");
mrb_value obj = mrb_load_file_cxt(mrb, fp, cxt);

fclose(fp);

if ((mrb)->exc)
{
	mrb_print_error(mrb);
}

Getting this result:
trace (most recent call last):
        [0] Main.rb:3
Main.rb:3: uninitialized constant Foo (NameError)
Press [ENTER] to quit...

So I'm thinking that maybe the filename isn't properly retrieved/passed to the context somewhere ?
@matz Just saw your reply, my former code was as follow:
int e;
FILE* fp;
if ((e = fopen_s(&fp, "./Main.rb", "r")) != 0)
{
	puts("Failed to open File");
	puts("Press [ENTER] to quit...");
	getchar();
	return e;
}

mrb_state* mrb = mrb_open();
mrb_value obj = mrb_load_file(mrb, fp);

fclose(fp);

if ((mrb)->exc)
{
	mrb_print_error(mrb);
}

Ah, sorry for misleading comments. And I am glad that you have solved the issue.
May I close this issue?

Sure !


Output has degraded significantly!

I think you have disabled the "large" or "1.5B" (mega xl, I believe).
I "used" to use this tool to save me like an hour while creating content, but the output is less coherent and sometimes just repeats the same text I entered.
I am all up for pro version like $9 or $15 per month to access the larger models, but please reenable the bigger model.

@designgrande hey - yeah, i disabled the large because i didn't feel like the output was that good (tradeoff was speed) ... i sort of got frustrated at it and turned it off.
i'm hoping to learn a bit more about model compression and redo large (and medium) to run faster tbh


Bug on Influence

Hello, I have a bug that i thinks is due to upgrading foundry to latest about Influence. The dice roll doesn't appear like so :

here is what my console is warning when I try to Influence :
commons.js:1703 Error: You are accessing the ChatMessage#data object which is no longer used. Since V10 the Document class and its contained DataModel are merged into a combined data structure. You should now reference keys which were previously contained within the data object directly.
at logCompatibilityWarning (commons.js:1692)
at Function._logV10CompatibilityWarning (commons.js:6642)
at ChatMessage.get data [as data] (commons.js:6542)
at HTMLAnchorElement. (init.js:318)
at HTMLElement.dispatch (jquery.min.js:2)
at HTMLElement.y.handle (jquery.min.js:2)
logCompatibilityWarning @ commons.js:1703
_logV10CompatibilityWarning @ commons.js:6642
get data @ commons.js:6542
(anonymous) @ init.js:318
dispatch @ jquery.min.js:2
y.handle @ jquery.min.js:2
commons.js:1703 Error: You are accessing the ChatMessage#data object which is no longer used. Since V10 the Document class and its contained DataModel are merged into a combined data structure. You should now reference keys which were previously contained within the data object directly.
at logCompatibilityWarning (commons.js:1692)
at Function._logV10CompatibilityWarning (commons.js:6642)
at ChatMessage.get data [as data] (commons.js:6542)
at HTMLAnchorElement. (init.js:329)
at HTMLElement.dispatch (jquery.min.js:2)
at HTMLElement.y.handle (jquery.min.js:2)
logCompatibilityWarning @ commons.js:1703
_logV10CompatibilityWarning @ commons.js:6642
get data @ commons.js:6542
(anonymous) @ init.js:329
dispatch @ jquery.min.js:2
y.handle @ jquery.min.js:2
Could you try to fix that please ? Thanks you

Fixed


Ignore order json

Problem to solve
Currently, the list items of an actual response must be in the same order as list items in the expected response.
This is a problem when a consumer expects a specific set (or subset) of values in a list, but does not
care about order of elements (or potentially any extra values in the set).
Proposed changes

Adds an EqualsIgnoreOrderMatcher which will uniquely match each item of an actual array against the
expected array, regardless of the order of the actual array items.

For example, [2,3,1] would match [1,2,3].
The EqualsIgnoreOrderMatcher defaults to Equality matching for non-arrays, such as the items of
the array.
Matching rules can be defined for child items of the array using array indexes in
the path (corresponding to the expected item indexes), or for all child items of the array
using a wildcard in the path.
EqualsIgnoreOrderMatcher expects the actual array to be the same size as the expected array.

Adds Min/Max/MinMaxEqualsIgnoreOrderMatcher for cases where array sizes can differ.

Any extra items in the array will only be matched if a wildcard matcher is defined for the array
items.

When ignore-order array matching is being performed, a '*' is used instead of numeric index
values when reporting wildcard mismatches.

This is intended to show the cause of the mismatch more clearly, but is inconsistent with other
cases of reporting wildcard mismatches.
Because ignore-order matching must consider all permutations of the actual values, the numeric
index value wouldn't be meaningful.
As this functionality is a little inconsistent with the existing mismatch reporting behavior,
it was separated into its own commit so it could be easily dropped/reverted.

Fixed some small detekt issues in core:matchers

we triggered a TooManyFunctions warning in Matchers and ComplexMethod in
MatcherExecutor.domatch and there wasn't a clear way to correct these
problems without a significant refactor of code outside of our changes.

NOTICE
This software was produced for the U. S. Government under Contract No. FA8702-20-C-0001, and is subject to the Rights in Noncommercial Computer Software and Noncommercial Computer Software Documentation Clause DFARS<PHONE_NUMBER> (FEB 2014)
Approved for Public Release; Distribution Unlimited. Case Number 19-3203.
(c) 2020 The MITRE Corporation. All Rights Reserved.

This looks useful!
I've added an item to the Pact V4 RFC. The main problem with merging this, pacts using this new matcher will not be compatible with the other language implements. Also, probably better to create a 4.2.x branch for this to be merged into to

I have merged this into the v4.2.x branch. I'll also add a check in the mock server to verify that the current pact spec version is V4 when this feature is used.


Feat: support custom schema by Documentation and Intergration

Based on #1470  and thanks to #1730

THIS is getting so neat!


When creating insecure client, the provider information is being lost

https://github.com/DiUS/pact-jvm/blob/1ef2659b8dd7a70a59c265283af6f5582c87f31c/pact-jvm-provider/src/main/groovy/au/com/dius/pact/provider/HttpClientFactory.groovy#L36

In general, it seems that the information provided inside the provider (protocol, host, port, path) are being ignored and the information from the pact is being used to create the client.
I've set all the information via @TestTarget new HttpTarget(protocol, host, port, path, insecure)
I ended up doing
  @TargetRequestFilter
  public void filterRequest(HttpRequest request) throws MalformedURLException, URISyntaxException {
    HttpRequestBase getRequest = (HttpRequestBase) request;
    URI newUri = new URL(protocol, host, port, path).toURI();
    getRequest.setURI(newUri);
  }

imho, this should not be necessary

We will have to dig a bit deeper to get to the issue here.
createInsecure() is just creating the HTTP client instance, which will be used to execute the HTTP method. The HTTP method is what has all the host and port information. This should be created by the verifier based on the information from the test target and the pact file.
Which runner are you using for your test?

Thank you for your help.
I've got an annotation on the class
@RunWith(PactRunner.class)


Drop unnecessary imports from _pb2_grpc.py files

This is incremental progress towards issue 8622.

Pre-merge test assessment:

Basic Tests Mac: 10138, 10267.
Bazel Full Build & Tests: Not yet in effect.
Sanity Tests (internal CI, ignore for now): Ignoring for now.
gRPC_portability_pull_requests_linux: 10266.
gRPC_pull_requests_asan_c: 10266.


MINIFI-455: Updating C2 readme to include S3. Simplifying minifi-c2-c…

MINIFI-455: Updating C2 readme to include S3. Simplifying minifi-c2-context.xml.
Thank you for submitting a contribution to Apache NiFi - MiNiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[X] Does your PR title start with MINIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[X] Has your PR been rebased against the latest commit within the target branch (typically master)?

[X] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi-minifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under minifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under minifi-assembly?

For documentation related changes:

[X] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

Hi @Paul-Verardi, nothing comes to mind right off but I will walkthrough it again to see. Thanks!

@jzonthemtn Thanks for looking into this, I will try to get it working under the same implementation you described.

Sorry for overlooking this PR.  These changes look good.  Will merge them in.  Thank you!


Does this driver work for aggregating only multiple dbus-serialbattery BMS or of any type?

Hi,
In my setup I have two JK BMS using dbus-serialbattery driver and a CAN-BUS BMS for which I do not need dbus-serialbattery as it is directly recognized in Cerbo GX. Does dbus-aggregate-batteries support aggregating any type of BMS or does it only resolve the issue with multiple dbus-serialbattery instances?
Thanks a lot for this clarification.
Best regards,
Peter

Hi Peter, my tool aggregates the battery data available on the dBus. If your GX device publishes the data of the CAN battery, it could work. It should be the case because the ESS communicates with BMS via dBus. You can check it with dbus-spy command.

Thanks a lot. As you may see from #54 and #55, I am already working on the integration of the CAN-Bus BMS via dbus-aggregate-batteries.


11.1.0-beta1 Exception occurred while editing DataGrid

Describe the bug
Exception occurred while editing DataGrid

To Reproduce
DataGridEditingIssue.zip
Expected behavior
no exception
Avalonia version
11.1.0-beta1
OS
No response
Additional context
No response

consider using DataGridTemplateColumn


Update issue templates

Codecov Report

Merging #38 (84a46b6) into master (4f0b2c6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #38   +/-   ##
=======================================
  Coverage   96.00%   96.00%
=======================================
  Files          16       16
  Lines        1176     1176
=======================================
  Hits         1129     1129
  Misses         47       47

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f0b2c6...84a46b6. Read the comment docs.


Retrofit2 incompatibility on Android 5.x

When I add compile 'com.jmedeisis:draglinearlayout:1.1.0' to Gradle and I run the App on Android 5 device, the app crash when try to load Retrofit2 (compile 'com.squareup.retrofit2:retrofit:2.3.0' AND compile 'com.squareup.retrofit2:converter-gson:2.3.0'). With Android 6 o + the Apps aren't crashing.

Sorry, but it is not possible that this library is not compatible with Retrofit as it is purely a UI component and does not attempt in any way to integrate with networking libraries. When reporting crashes please always share the stack trace.


Udacity CarND-Capstone, not receiving traffic from the simulation window

I tried to put rospy. logwarn to see if the traffic information is import from the simulation window.  I put that in the traffic_cb node but there is nothing popped out. Would you please help me where should I put? My car doesn't stop at red light and I try to troubleshoot

Hi @liupaladin,
Though we are a little late to respond, we encourage such user-specific issues to be raised on the Knowledge hub, so that our experienced mentors can chime in and new students can also learn from the discussion thread.
Thank you


Modifications to Page Navigation

We want to encourage users of the site to sign up, but we do not want to discourage potential users from browsing existing data. So, make the following changes to navigation behavoir of the site:

[x] Remove the "My Sites" link for users that are not signed in to the site. It should appear when they have signed in.
[x] Change the "Browse Sites" link and page so that it is public and available to anyone regardless of whether they are signed in or not. Viewing individual site pages by clicking on links on the "Browse Sites" page should also be possible for unauthenticated users.

@jcaraballo17 - feel free to reassign to @mauriel133 if this is better addressed by him, but hopefully you can get to this before next week.

This is done on production. Closing.


fix(query-core): allow function value of notifyOnChangeProps query option to return undefined

undefined is a valid return value for the function value of notifyOnChangeProps. See linked issue for more details.
Fixes #7426.

can you amend the signature in the docs as well please?

also the jsdoc please:
https://github.com/TanStack/query/blob/3a8c2b10a1c2a3657d22b0304aab640683a4db0f/packages/query-core/src/types.ts#L319-L326

@winghouchan will you get this PR over the finish line or should I do it?

@TkDodo: sorry, missed your messages.
I've updated the signature in documentation in 43f0722. As far as I can tell it only affects React (and Vue, although they share the same docs). Please let me know if this is incorrect.
How would you like the JSDoc updated? The current description isn't wrong, although I suppose it could be improved. Maybe something like below:

If set to a function, the function will be executed to compute the list of properties.
If the value is set or computed to undefined, access to properties will be tracked, and the component will only re-render when one of the tracked properties change. This is the default behaviour.
If the value is set or computed to an array, the component will only re-render if any of the listed properties change. For example, if the value is set to or computed to ['data', 'error'], the component will only re-render when the data or error properties change.
If the value is set or computed to "all", the component will opt-out of smart tracking and re-render whenever a query is updated.


Illegal character when testing workflows and tools in Rabix Composer

I am new to Rabix and CWL in general. I am just trying to understand how to run my tools from the Rabix Composer on Windows 10. However, everytime I go to test a tool or workflow it gives me the following error
Exception in thread "main" java.lang.IllegalArgumentException: Illegal character in opaque part at index 2: C:\Users\pkara\RabixComposer\Executions\local\hello_world\2019-08-19-13-30-30\app.cwl
This is the tools I am trying to run
class: CommandLineTool
cwlVersion: v1.0
$namespaces:
  sbg: 'https://www.sevenbridges.com/'
id: hello_world
baseCommand:
  - echo
inputs:
  - id: input
    type: string
    inputBinding:
      position: 0
outputs: []
label: hello_world

And this is the full Error Log Output
Exception in thread "main" java.lang.IllegalArgumentException: Illegal character in opaque part at index 2: C:\Users\pkara\RabixComposer\Executions\local\hello_world\2019-08-19-13-30-30\app.cwl
	at java.net.URI.create(Unknown Source)
	at org.rabix.cli.BackendCommandLine.main(BackendCommandLine.java:122)
Caused by: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\Users\pkara\RabixComposer\Executions\local\hello_world\2019-08-19-13-30-30\app.cwl
	at java.net.URI$Parser.fail(Unknown Source)
	at java.net.URI$Parser.checkChars(Unknown Source)
	at java.net.URI$Parser.parse(Unknown Source)
	at java.net.URI.<init>(Unknown Source)
	... 2 more

It seems to me, that the path, that is generated is not being read correctly by Rabix
For reference I am running Rabix Composer 1.0.1 on Windows 10

Hi @Silberbear thanks for reporting this issue. Unfortunately, this is a problem with Rabix Executor on Windows. We've investigated this issue previously and verified that Composer is passing the command line correctly to the executor. (If you discover otherwise, please comment).

Did anyone find a way to fix this issue? Or does it just not work for now? Thank you for the reply.

It happen in my side too. I don't know how to do it. Any one can help me? Thanks.

Exception in thread "main" java.lang.IllegalArgumentException: Illegal character in opaque part at index 2: C:\Users\ThinkPad\RabixComposer\Executions\local\two_samples_ttest\2021-03-12-18-49-18\app.cwl
at java.net.URI.create(URI.java:852)
at org.rabix.cli.BackendCommandLine.main(BackendCommandLine.java:122)
Caused by: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\Users\ThinkPad\RabixComposer\Executions\local\two_samples_ttest\2021-03-12-18-49-18\app.cwl
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parse(URI.java:3058)
at java.net.URI.(URI.java:588)
at java.net.URI.create(URI.java:850)
... 1 more


WIP: Adding Roboto- quick

Resolves needing a font
Description

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not
work as expected)
This change requires a documentation update
Documentation update

How Has This Been Tested?

Screenshots

Hey @UnicodeRogue - do you know what was still remaining on this, and would you have some time to work on wrapping this up? let me know how I can help!

@UnicodeRogue - do you have time to wrap up the conflicts for this? If not, i'm happy to close this and put the issue back out for others to work on. Thank you!


Redis stream not working at all in 3.10.2

Hello
I am trying to make the stream example work with new redisson version 3.10.2 .
I get ArrayIndexOutOfBounds in class StreamIdDecider.
While it assume having 3 parameters. First one number-number key value.
This is not true input to this class is not these arguments.
Can you take a look?
Thanks

Could you provide code example to reproduce this?

I am trying exactly the example that in the documentation.
I noticed StreamId was changed to StreamMessageId. I am using the StringDecoder on getStream and getting an exception on StreamIdDecoder when trying to parse the id.
It parses by mistake the key of the stream.
I will try to send you the example later today.
Thanks

Guys I am sorry I was using non GA version of redis.
When I upgraded to latest GA everything is working as expected.
Thanks
Ziv


Fatal error when running SF Docker container on Ubuntu 18.04

Hello
I know that native 18.04 support is coming up, but for now I'm trying to get SF to run on my 18.04 machine using the Docker process described for macOS (https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-get-started-mac). The container builds and sets up correctly, but when trying to run it, I get the following error:
Failed to retrieve node last boot up time exception System.FormatException: String was not recognized as a valid DateTime.
   at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
   at System.Fabric.FabricDeployer.Utility.GetNodeLastBootUpTimeFromSystem() in /var/build/workspace/ServiceFabric/Release_6.5_Ubuntu1604_Nightly/src/prod/src/managed/Api/src/System/Fabric/Management/fabricdeployer/Utility.cs:line 1289
Failed to retrieve node last boot up time exception System.FormatException: String was not recognized as a valid DateTime.
   at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
   at System.Fabric.FabricDeployer.Utility.GetNodeLastBootUpTimeFromSystem() in /var/build/workspace/ServiceFabric/Release_6.5_Ubuntu1604_Nightly/src/prod/src/managed/Api/src/System/Fabric/Management/fabricdeployer/Utility.cs:line 1289
Last boot up time from registry:01/01/0001 00:00:00 from system:01/01/0001 00:00:00
Skipping container network iptable set up because node has not been rebooted.
DockerDnsHelper current node IPAddressOrFQDN <>
Starting DockerDnsHelper.CleanupAsync
Executing DockerDnsHelper.CleanupAsync, attempt 1 of 3
Persisted network details cleaned up
Removing firewall rule WindowsFabric.Dns.v1 if it exists...
iptables v1.6.0: can't initialize iptables table `filter': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
Error removing firewall rule WindowsFabric.Dns.v1. Ignoring and continuing. This must be cleaned up manually. Exception: System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at NetFwTypeLib.NetFwRules.Remove(String name) in /var/build/workspace/ServiceFabric/Release_6.5_Ubuntu1604_Nightly/src/prod/src/managed/Api/CoreCLRDlls/System.Fabric.Management/src/NetFwRules.cs:line 153
   at System.Fabric.FabricDeployer.DockerDnsHelper.RemoveFirewallRule() in /var/build/workspace/ServiceFabric/Release_6.5_Ubuntu1604_Nightly/src/prod/src/managed/Api/src/System/Fabric/Management/fabricdeployer/DockerDnsHelper.cs:line 765
Successfully completed DockerDnsHelper.CleanupAsync
Completed DockerDnsHelper.CleanupAsync
Released writer lock on /home/ClusterDeployer/ClusterData/Data/lock
Starting FabricHost
Starting Fabric Host as console application.
IPC server open failed with 0x80070063
Failed to set certificate acls. Error: E_FAIL

Sorry if this is a known issue. I figured that since the process that's failing is inside the Docker container, it would fail regardless of which OS is the container host.
Ubuntu 18.04 LTS
Docker 18.06

I had a similar issue to this. Make sure that you've applied the following settings to the Docker Daemon and restarted:
{
    "ipv6": true,
    "fixed-cidr-v6": "fd00::/64"
}

Ref: https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-get-started-mac

@ihildebrandt Thank you! I had actually made that change, but your comment made me think it didn't take. Turns out I had installed Docker using snap instead of apt. Because of this, the config file in question wasn't read.
Reinstalled docker using apt, and it works fine now!


dynamically hide table columns

Similarly, to how a table multi select can be invoked dynamically for an existing table by table.multiple=True it would be great if we were able to dynamically hide (maybe even add) columns, something like table.columns[-1].visible=False
Motivation : When we have experiments table in HT, there are some actions to be done for single row (like rename experiment), which are done using action column in the table, and some actions that are invoked by button and allow multi select (like delete multiple experiments).
When invoking multi select, we would like to hide the the action table column, this can be done by manually recreating the table again without the action column, however, all filters/sorts previously done on the table are lost.
If it was possible, it would be great to have the ability dynamically to hide some columns of current filtered/sorted table

Hey @MartinBarus, thanks for suggesting this feature!
What about allowing filter and sort parameters to be set when creating the table? Maybe this could solve your issue and be used in other scenarios too.
I know it's easier for you to just set table.columns[-1].visible=False but from a performance point of view the component will be recreated either way behind the scenes, right @mturoci?

from a performance point of view the component will be recreated either way

Not necessarily, if hidden, the col cells can still remain in the DOM and only CSS repaint would be needed. Not sure if Fluent provides such a feature though.

@aalencar allowing filter and sort parameters could also solve it, but we would have to be able to take updated filters from an existing table, something like new_table = ui.table(..., filters=old_table.filters, sort_filters=old_table.sort_filters) or something similar


WIP - Rewrite of parts of the breakpoint and stepping implementation.

This is a fairly large PR, because some 'missing' / 'unreliable' functionalities required new infrastructure before they could be reimplemented, and required some 'experimentation', so I was not able to break it into smaller PR's.
In summary, the limitations I focussed on were as follows:

The existing implementation for setting breakpoints at a specific address worked as expected.
The existing implementation for setting breakpoints at a specific file/line/column source location (mapping to a suitable instruction address) was limited to working for source code lines that were explicitly mentioned in the DWARF line program sequence rows, and would not work for cases such as described in #2180. This had implications for stepping also.
The existing stepping code had huge 'blindspots', as a result of limited DWARF information in the line program sequence rows, and no call site information for non-inlined functions.
The stepping would often just give errors ("can't do that"), or sometimes, and arguably worse, would try to step to unreachable code.

The approach to addressing these, was as follows:

Centralize all debug halting code (breakpoints and stepping) to a new module called debug::halting.
Implement the concept of Block's, as an intermediate between Sequence's and Instruction's.  There are some limitations where the implementation errs on the side of caution (e.g. unless we disassemble per platform, we don't know which instructions are branching, so don't assume sequential instructions are in the same block).
Adapt the current breakpoint setting implementation to use the new Sequence->Block->Instruction structures.
Ditto for the current stepping implementation. This allowed the deletion of a lot of unreliable logic.

Status after the rewrite:

There are opportunities to improve further, but these require additional infrastructure, like the support of software breakpoints. I've marked some of these with TODO comments in the code.
All the current breakpoint tests pass without modification.
TODO: We need new tests for breakpoints at previously "can't do that" locations, (e.g. #2180).
TODO: We need tests for the stepping code. Currently I have done manual testing only.  Tests for stepping will have some limitations, because we can't 'single step' the target processor in a test, but we should be able to get good test coverage of the 'target location' calculations required during stepping.

List of tests required for stepping code:
... wip ...

@SergioGasquez If you have the time/inclination to do so, I'd appreciate if you can do some testing with this branch. Hint: You can install it with cargo install probe-rs --features cli --git https://github.com/probe-rs/probe-rs --branch breakpoint_and_stepping --force
I am confident that it fixes your #2180, even though there are still a few stepping scenarios I know need work.
Feel free to document any issues you run into :)

@SergioGasquez If you have the time/inclination to do so, I'd appreciate if you can do some testing with this branch. Hint: You can install it with cargo install probe-rs --features cli --git https://github.com/probe-rs/probe-rs --branch breakpoint_and_stepping --force
I am confident that it fixes your #2180, even though there are still a few stepping scenarios I know need work. Feel free to document any issues you run into :)

Just tried your branch, and I was able to step into the method that was previously ignored! Thanks for working on this fix!

@noppej just FYI this PR has merge conflicts now that should be resolved. Other than that, are the TODOs in the PR description still unresolved?

@bugadani Thanks. I'm still actively working on #1935, which will contribute to being able to finalize this PR.

Is this branch functional?  I just installed the plugin, and I dunno if I'm just encountering a particular set of edge cases, but it's nearly unusable.  Only like 1 in 10 lines accept a breakpoint, and stepping forward (or even stepping out) almost always lands me e.g. at some __release call 6 levels into system code.  If this branch works well enough, I'm inclined to compile it and use it instead of the main branch.

Oh, phew.  I installed this branch, and immediately things work properly.  (So far as my 5 seconds of testing indicate.)

@Erhannis This is a bit of an 'experimental' branch. Doing breakpoints and stepping is rather tricky with the available debuginfo, and I really appreciate your positive feedback. I'm in the middle of finalizing another PR, and hopefully in the next day or so I will bring this branch up to date with the latest improvements on 'master'. That should give you something worthwhile to work with.
Any specific use cases where things are still wonky will be much appreciated :)

@noppej Nothing too specific that I've noticed - some lines don't accept breakpoints where I might expect them to, and a time or two stepping has landed me in a weird place, but usually I can work around it without too much trouble.  While I'm not sure the following is relevant to this branch, in the "variables" list I've been getting some "unsupported memory implementation" messages, as well as "Location Value not supported for referenced variables" which at least has pretty clear intent, and also, with a samples: &mut [u8], I see samples > data_ptr > *data_ptr > Error: This is a bug! Attempted to evaluate a Variable with no type or no memory location".  Oh - looks like I also get that when expanding a fixed array, like for both values of a [u8; 2]`.

in the "variables" list I've been getting some "unsupported memory

Hmmm ... I can't reproduce this. I get

Are you able to log a reproduce-able test case as a new issue?


[BUG] missing MPP (openslide.mpp-x). Skipping

Description
I am extracting the tiles from .ndpi files using Project by running the following code.
full_dataset.extract_tiles(save_tfrecords=True, enable_downsample=False)

However, I am encountering the following error.
Error loading slide
                   *.ndpi: Slide * missing MPP (openslide.mpp-x). Skipping

I read about issue #258 and exported the environmental variables as following
export SF_SLIDE_BACKEND=libvips

and restarted the kernel, the issue persisted.

Environment:

Slideflow Version (e.g., 1.0): 2.0.3-post1
OS (e.g., Ubuntu): Ubuntu
How you installed Slideflow (pip, source): pip
Python version: 3.10.9
CUDA/cuDNN version: cuda_11.7.r11.7/compiler.31442593_0
GPU models and configuration: NVIDIA T4
Any other relevant information: tensorflow backend, using Libvips

Thanks for raising this issue. Can you run the following code at the top of your script, confirm that the slide backend has been set appropriately?
sf.about()
Does this happen with all of your slides, or only some? How did you install Libvips?

If you're running from a Jupyter notebook, you'd need to run that command from the terminal just before starting the notebook:
export SF_SLIDE_BACKEND=libvips
jupyter lab

Alternatively, you can use this syntax;
SF_SLIDE_BACKEND=libvips jupyter lab

Finally, you can try adding the following line at the top of your script, before importing slideflow:
import os
os.environ['SF_SLIDE_BACKEND'] = 'libvips'


Windows Admin Center Update - An error is shown while checking for updates

Gateway Version: 1.3.2105.24004
Extension Version that's been used if it applies
To Reproduce
Steps to reproduce the behavior:

Go to 'Settings --> Updates'
Click on 'nothing. Just let it check for updates for the Windows Admin Center product'
Scroll down to '....'
See error: Couldn't get inventory data of gateway (see image)

Expected behavior
This error should not show? My Windows Admin Center gateway shows up to date though. I have already set web proxy in the application menu
Screenshots & Additional context

It seems i solved it via workaround. The proxy setting you have in the proxy is not honored when checking for updates for the product itself.

I set proxy for the network service profile (IE settings) that runs the WAC service and now error/event disappeared.
So the issue here is that the proxy setting in the product is not used when checking updates for the product


checkov: 3.2.239 -> 3.2.241

Diff: https://github.com/bridgecrewio/checkov/compare/refs/tags/3.2.239...3.2.241
Changelog: https://github.com/bridgecrewio/checkov/releases/tag/3.2.241
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 338939 run on x86_64-linux 1

2 packages built:

checkov
checkov.dist


[pilot] Show train version and build version while wating for build to re-appear

Unified how the numbers are shown

          1 Error

:no_entry_sign:
Tests have failed, see below for more information.

Tests:

Name
File

FastlaneCore::BuildWatcher.wait_for_build_processing_to_be_complete waits when the build disappears
build_watcher_spec.rb

  Generated by :no_entry_sign: Danger

Never mind, this doesn't work


Package request: gforth-unstable

Project description
the bleeding-edge version of gforth has a ton of features not found in the latest release.  most package managers with gforth have a seperate package for the unstable/dev version.
Metadata

homepage URL: gforth.org
source URL: https://www.complang.tuwien.ac.at/forth/gforth/Snapshots/current/
license: gpl
platforms: unix, linux, darwin, windows

Add a :+1: reaction to issues you find important.

There are release tags here https://github.com/forthy42/gforth/tags which seem reasonable up to date, the package just needs to be updated. We generally don't want to package prerelease software as it requires way more maintenance.
You also created a PR to update already here #292671


PLT-9051: Fixed price fetch

Summary by CodeRabbit

Enhancements

Improved price fetching reliability with an automatic retry mechanism for better user experience during network issues or service interruptions.

@RSoulatIOHK can you have another look at this one


#is helper is adding the value 'false' to generated pages

I have the following is helper:
{{#is pagename "===" "intel_introduction"}}class="active"{{/is}}
For false value its actually adding the text "false" in the generated html pages.
var rename = require('gulp-rename');
var assemble = require('assemble');
var helpers = require('handlebars-helpers');
var app = assemble();
var prettify = require('gulp-prettify');
var config = require('../config');
//var isHelper = require(config.paths.path_src + 'templates/helpers/ishelper');

gulp.task('init', function(cb) {
  //app.helper(config.paths.path_src + 'partials/_helpers/*.js');
  //app.helper(config.paths.path_src + 'templates/helpers/ishelper.js');
  app.helper('is', helpers.is);
  //app.helper('contains', helpers.comparison());
  app.partials(config.paths.path_src + 'templates/includes/**/*.hbs');
  app.layouts(config.paths.path_src + 'templates/layouts/**/*.hbs');
  app.pages(config.paths.path_src + 'templates/content/**/*.hbs');
  cb();
});

gulp.task('assemble', ['init'], function() {
  return app.toStream('pages')
    .pipe(app.renderFile())
    .pipe(prettify({
        indent_inner_html: false,
        preserve_newlines: true,
        end_with_newline: true,
        extra_liners: ['head', 'body']
    }))
    .pipe(rename({
        extname: '.html'
    }))
    .pipe(rename({dirname: ''}))
    .pipe(app.dest('site'));
});

gulp.task('test', ['assemble']);

@myorangeca please simplify your code down to the minimum that is necessary to reproduce the issue.
From looking at this, you're using the is helper incorrectly. See the usage docs for how to get the helpers you need and the {{is}} helper docs for how to use the helper in a template.

Hi @doowb
I am really a newbie at this, the documentation looks unclear to me. Would love to see an example of how it works or suppose to work. Not sure if I missed any example codes from the documentation.

I am really a newbie at this

Everyone starts somewhere :) I know from our perspective it can be hard to know what needs explaining and what is assumed to be known. I hope the following example helps.
This example is using the is helper from handlebars-helpers with assemble:
assemblefile.js
var assemble = require('assemble');
var helpers = require('handlebars-helpers');
var app = assemble();

app.task('init', function(cb) {
  /**
   * Get the `is` helper directly from the `comparison` collection.
   */

  var isHelper = helpers.comparison().is;

  /**
   * Register the `is` helper with assemble so that it can be used.
   */

  app.helper('is', isHelper);

  app.partials('src/templates/includes/**/*.hbs');
  app.layouts('src/templates/layouts/**/*.hbs');
  app.pages('src/templates/content/**/*.hbs');

  // add data to use in the example
  app.data({A: 'foo', B: 'foo'});
  cb();
});

app.task('assemble', ['init'], function() {
  return app.toStream('pages')
    .pipe(app.renderFile())
    .pipe(app.dest('site'));
});

app.task('default', ['assemble']);

module.exports = app;

src/templates/content/index.hbs
{{#is A B}}
  A is equal to B
{{else}}
  A is not equal to B
{{/is}}

You can run assemble from the command line to run the default task in the assemblefile.js that will run the code and you'll see this in:
site/index.hbs
  A is equal to B

I removed all of the extra code and I'm using assemble directly to remove the dependency on gulp just to show the simple use case.


Updated Ukrainian translation

Translation of the new word to ukrainian
Description

#236
Translated word "Excluding" to Ukrainian.
Type of change

[x] Bug fix (added a non-breaking change which fixes an issue)
[x] Updated documentation (updated the readme, templates, or other repo files)

How Has This Been Tested?

[x] Tested locally with a valid username
[x] Tested locally with an invalid username
[x] Ran tests with composer test
[x] Added or updated test cases to test new features

Checklist:

[x] I have checked to make sure no other pull requests are open for this issue
[x] The code is properly formatted and is consistent with the existing code style
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings

Screenshots

Looks great! Thanks for the contribution :tada:


[TOPIC-BLE-LLCP] WIP: New test concept

WIP: New test concept, based on PR 23093 with an extra commit

Rename tx_alloc_peek to tx_alloc_is_available

After LP_COMMON_STATE_WAIT_TX, there has to be a state to wait for Tx Ack, this is the state that shall be used to detect collisions (local initiated and acked, and reception of collision).

https://github.com/zephyrproject-rtos/zephyr/blob/d3cc09dec2c5717e8802b877a7640abc08b0a2ad/subsys/bluetooth/controller/ll_sw/ull_llcp.c#L1745
The check has to be with a list of received opcode's. Example, enc_req can be responded with a reject_ind.

Rename tx_alloc_peek to tx_alloc_is_available , similarly other _peek if they are not returning the buffer.
May be just use mem_tx.free as the peek. NULL or not NULL.

Renaming _peek is fine with me. Looking a .free seems a bit like a leaky abstraction but i'm fine with it for optimizing the performance.

After LP_COMMON_STATE_WAIT_TX, there has to be a state to wait for Tx Ack, this is the state that shall be used to detect collisions (local initiated and acked, and reception of collision).

Havn't gotten around to the collisions resolving yet, but will make a note to myself regarding this.

https://github.com/zephyrproject-rtos/zephyr/blob/d3cc09dec2c5717e8802b877a7640abc08b0a2ad/subsys/bluetooth/controller/ll_sw/ull_llcp.c#L1745
The check has to be with a list of received opcode's. Example, enc_req can be responded with a reject_ind.

Working on a code path that handles the LL_UNKNOW_RSP, LL_REJECT_IND and LL_REJECT_EXT_IND.


Fix error: ctmacros.hpp file not found with  include

Cherry-picking from release branch to prevent us from having to cherry-pick explicitly to multiple releases.

This should also update the version.hpp.template file, so that any subsequent change regenerates the correct file.
In addition, where is this failing? Missing the #include when using <>'s indicates that something's wrong with your INCLUDE paths.

@mkoscumb On Edge, there's stricter checking for using bracketed includes on standard headers and parenthesis for project headers. There was a bit of cleanup related to this in 2018, but any new files added that don't follow that pattern will bubble up as an issue for us.

Updated the template file with the change.


JS to TS inverter co-file for those who want to TS without the overhead while coding.

Search Terms
JavaScript to TypeScript from Typescript to JavaScript
Suggestion
As a very long time user of JavaScript I find a great deal of Typescript to be less useful during development but as  I get closer to production code it becomes a great means for ensuring the code works properly and can be maintained.
At the same time I look at the JavaScript code generated by the TypeScript compiler and while compatible with JavaScript it is code that is boring to look at (said politely). It feels like to the TS universe JS  is an afterthought. As a consequence many users are turned off by the amount of boiler plate and other overhead in coding with TS.  I think there is a great way to address that problem and further the acceptance of TS.
Here is my request/ idea.  There should be a way to take any typescript generated .js file and reverse it back to typescript.  If not i suggest that a typescript co file be generated along with the JavaScript file that makes this possible. The co-generated file could be in JSON or .js format and should be both complete and useful for others who want to emulate it for their own purposes.
This might seem counterintuitively useful but if there was a reasonable example set of co-generated typescript hinting that is produced during a typescript compile Typescript the  language would be more palatable to those who find  it to be (  as Mr Putin has so eloquently stated about a former NSA worker) .
"Like shearing a pig. A lot of squealing and not a lot of fur".
Use Cases
Along with the JS output of the TS compiler another co-generated file that can be combined with the js file to revert back to the TS definitions and files
Examples
Users look at the co-generated file to figure out how to create a ts file set out of their js file without having to to go through the formal TSificiation of their own code... If they follow the examples in the other code that is generated they can build TS without coding in TS.
Checklist
My suggestion meets these guidelines:

[ x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x ] This wouldn't change the runtime behavior of existing JavaScript code
[ x] This could be implemented without emitting different JS based on the types of the expressions
[ x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[ x? ] This feature would agree with the rest of TypeScript's Design Goals.

So you have an input file, you create an output file from it, and you want a separate and additional way to... get a second copy of the input file? What?
Source maps already exist for the purpose of correlating lines of code in the output to lines of code in the input.

I think I was unclear about the goal.
The goal is to afford someone who codes in JS the ability to migrate existing code to TS without the laborious effort of line by line annotation and type creation...
In other words the ability to construct all ts files associated with a typical typescript layout
Specifically i want to start with a well formed module.js file
and then i want to construct one and only one additional file that will in turn be able to construct

module.js.map
module.d.ts
module.tsx
module.types.ts

and any other ts files that i might be missing in this list.
However i recognize that today that is not possible without the laborious process of creating a tsx or ts file directly out of the initial module.js
This becomes a really big barrier to acceptance among the most experienced JavaScript coders.
While the value to an organization of having typescript files might be high.  For someone fluent in JavaScript it is not very much.   To be candid... I just delete all of it because the annotation it delivers relative to the page clutter it creates is a bad tradeoff.
Most of the types of errors that Typescript catches (the ones that have the most ts value)  are analogous to memory leaks in C and C++
Coders who have written a great deal of c/c++ know to code everything on the stack if possible and therefore can live with the risk.
And like memory leaks... when there are problems... having tools to lock down ones code becomes really useful as long as the bar is not too high to implement.
Having a co-generation file that can be combined with a js file to create all the TS files in a normal repo for a given module would be a boon for those who are on the fence about TS.
It also would be a great way to demo the two way relationship between js and ts... If you can take a ts file turn it into a js file and from that file (and one other file) revert back to the original ts it would be much more credible as a JS alternative.
Most of the time I would not use TS because I value my time :) but if I started having some strange issues it might be worth the time and effort to TS annotate all of my JavaScript files... but that is usually in the latter stages of an application development effort.
From a TS perspective this seems like a waste of time. But i promise you two way painless compatibility will really help reduce friction in converting the really experienced JavaScript coders who do not need training wheels.


Label image default to colorful LUT (glasbey) and create ROIs

Suggested in this post

Duplicate see https://github.com/BIOP/ijl-utilities-wrappers/issues/44


TCP fixes

2 important fixes:

Fix local TCP connections (currently leads to non functioning connection and memory leak)
Fix TCP disconnection initiated on Zephyr side (currently leads to memory leak)

Fix local TCP connections (currently leads to non functioning connection and memory leak)

This is pretty weird. Local TCP connections didn't work a few years ago, but since then, they were fixed and we even have a regression tests for them. Did you consider to add a testcase showing the current problem and that it's being fixed by these changes?

we even have a regression tests for them

This could still be a regression if the test sets CONFIG_NET_LOOPBACK and verifies things when only that is set. Just pondering here what might be the culprit, did not check the code.

@pfalcon The commit message says all about the test case: net tcp connect

The test one test that is failing in CI (for both qemu_x86 and mps2_an385) passes for me locally on this branch. I've heard about timing issues in qemu recently.
twister -i -p mps2_an385 -T tests/net/tcp2

This is pretty weird. Local TCP connections didn't work a few years ago, but since then, they were fixed and we even have a regression tests for them.

Actually, our tests are quite fragile against TCP scheduling change (as CI shows). They were really crafted against a certain scheduling behavior, with knowledge about things like "here we sleep for the packet to be processed" etc... Theoretically this should not happen, at least not in the socket based tests (the lower level ones are trickier).

Many unrelated tests are failing, I guess due to CI being overloaded.  Like DAD test on ipv6, that does not involve TCP nor net_context etc...

The test one test that is failing in CI (for both qemu_x86 and mps2_an385) passes for me locally on this branch. I've heard about timing issues in qemu recently.
twister -i -p mps2_an385 -T tests/net/tcp2

It's test related for some. At least I have a fix for tcp2. Still having issues on tls though

I just added DNM because there is something that is a bit weird: why socket has to ref + 1 the net_context itself in case of TCP  and not net_context direct users? (like net_shell). I noted as well that socket never unref by itself net_context... so is net_context under-referenced by design or by miss? Imo, socket should not have to ref it. Let's see.
I need to verify that my net_context patch did not mess up with this logic.

DNM label removed. It does not seem to pose any problem. Let's see what CI has to say then.


autoreconf gives warnings on 1.14.7-dev (with autoconf 2.71)

in src/secp256k1:
configure.ac:28: warning: The macro `AC_PROG_CC_C89' is obsolete.
configure.ac:28: You should run autoupdate.
configure.ac:28: the top level

to solve this we'd need to update the subtree, which is anyway a good idea. I'll try to do this with priority because that will need to see lots of exposure - this is consensus critical code.
in the root:
configure.ac:454: warning: $as_echo is obsolete; use AS_ECHO(["message"]) instead
lib/m4sugar/m4sh.m4:692: _AS_IF_ELSE is expanded from...
lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
./lib/autoconf/general.m4:2249: AC_CACHE_VAL is expanded from...
./lib/autoconf/general.m4:2270: AC_CACHE_CHECK is expanded from...
build-aux/m4/ax_pthread.m4:88: AX_PTHREAD is expanded from...
configure.ac:454: the top level

Should be fixable with an update of build-aux/m4/ax_pthread.m4

Note: these warnings occur on 1.21-dev too.

The first part gets addressed for 1.14.7 as part of #3082

Second part is addressed in #3414

This has been fixed


Azure Deployment Environment Support

Docs
https://azure.microsoft.com/en-us/services/deployment-environments
### Tasks
- [ ] #2528
- [ ] #2529
- [ ] #2531
- [x] New ADE Provision Provider
- [x] DevCenter Mode (via azd config)

Resolved by #2767


Update the candidate commitish value with the last green commit

Make it consistent with the rest of the release steps.

A possible fix for https://github.com/iree-org/iree/issues/10182