parsing antler CUE configs can exhaust system memory

What version of CUE are you using (cue version)?

$ cue version
cue version v0.5.0

go version go1.23.0
      -buildmode exe
       -compiler gc
  DefaultGODEBUG asynctimerchan=1,gotypesalias=0,httplaxcontentlength=1,httpmuxgo121=1,httpservecontentkeepheaders=1,netedns0=0,panicnil=1,tls10server=1,tls3des=1,tlskyber=0,tlsrsakex=1,tlsunsafeekm=1,winreadlinkvolume=0,winsymlink=0,x509keypairleaf=0,x509negativeserial=1
     CGO_ENABLED 1
          GOARCH amd64
            GOOS linux
         GOAMD64 v1

Does this issue reproduce with the latest stable release?
Yes. It's the same or possibly worse in v0.10.0, with or without CUE_EXPERIMENT=evalv3.
What did you do?

I created a CUE package for Antler in this sce-tests repo. This is an Antler test config with 216 tests, that uses both large lists (generated programmatically with Go templates), and CUE list comprehension, that likely results in a large CUE graph.
The biggest culprit, it seems, is the Run list for my FCT tests. This creates list of 1200 elements (using a Go template), which is used with list comprehension to generate StreamClients. When Antler goes to unify the schema with the config using the CUE API using the CUE API, the process memory reported by top rises very quickly, and can complete exhaust the system memory, depending on the hardware. If I comment out this list, it's still slow and uses a lot of system memory compared to what I'd hope for, but it's at least much faster.
To reproduce it, one can install Antler, pull the sce-tests repo, and run antler vet to parse the config. My hope is that this isn't necessary for you to do, and just based on the description, you can identify the category of performance problem referred to in the Performance umbrella issue, so I have a sense of if or when this may be improved.
Also, I might be able to work around this by avoiding large lists, but it's flexible for users to provide their own statistical distribution of wait times and flow lengths, and these lists can simply get long. On top of that, this project will eventually at least triple in size with more tests, so I'll have to solve this somehow, and am just looking for advice. Would this be any better in v0.11.0-alpha.1, or with any other config options?
What did you expect to see?
The config to parse reasonably quickly.
What did you see instead?
Excessive memory allocations.
A Linux laptop with 8G of RAM and 8G of swap runs out of memory entirely when parsing the config.
Another box with 16G of RAM and 8G of swap is able to parse the config without running out of memory, but just barely.

Is it possible to reproduce this slowness via the cue command on your sce-tests repo, for example via cue eval or cue export?

Yes, although it takes a couple of steps, and you still need the antler command just to generate one of the cue files before running cue export:

Pull the sce-tests repo.
Run antler vet to create fct.cue from fct.cue.tmpl.
Edit sce.cue to uncomment the section under "polya fct tests", which I currently have commented out to prevent the memory problem.
To get the config schema, copy the file config.cue into the same directory and change its package to "sce".
Run cue export.

When running cue export, that will all be run on static cue files outside of antler, and it shows the same memory problem. Be prepared though that a machine with 8G might become unusable and need a hard reboot. A machine with 16G should be able to handle it.

I upgraded my box to 64 GB RAM and did some testing with CUE v0.5.0 and CUE v0.11.0.  This is the resident memory reported after the config is completely parsed:
CUE v0.5.0:  9.6 GB
CUE v0.11.0:  22.6 GB
CUE v0.11.0 with CUE_EXPERIMENT=evalv3:  39.2 GB
I appear to be stuck on v0.5.0.  Or, are there any other experimental flags I can try?

That's it for now. We still have some performance and memory usage work to be done on evalv3, so that's still our focus for issues like this one.

Just adding to this that I reduced the CPU and memory considerably by removing all the disjunctions I was using in my config schema.
CUE v0.5, with disjunctions:
54.40user 2.69system 0:29.84elapsed 191%CPU (0avgtext+0avgdata 10471328maxresident)k
0inputs+56outputs (0major+2691891minor)pagefaults 0swaps
/bin/time antler vet  54.40s user 2.70s system 191% cpu 29.849 total

CUE v0.5, without disjunctions:
6.76user 0.41system 0:04.37elapsed 164%CPU (0avgtext+0avgdata 1487272maxresident)k
0inputs+56outputs (0major+386054minor)pagefaults 0swaps
/bin/time /tmp/antler vet  6.77s user 0.42s system 164% cpu 4.374 total

CUE v0.11, without disjunctions, without CUE_EXPERIMENT=evalv3:
27.54user 1.20system 0:16.08elapsed 178%CPU (0avgtext+0avgdata 4745528maxresident)k
0inputs+56outputs (0major+1255867minor)pagefaults 0swaps
/bin/time /tmp/antler vet  27.55s user 1.21s system 178% cpu 16.090 total

CUE v0.11, without disjunctions, with CUE_EXPERIMENT=evalv3 (however this got a "field not allowed" error on a line number that doesn't make sense yet, so I'll try to sort this out later):
12.57user 1.32system 0:05.37elapsed 258%CPU (0avgtext+0avgdata 5138912maxresident)k
0inputs+56outputs (0major+1305055minor)pagefaults 0swaps
CUE_EXPERIMENT=evalv3 /bin/time /tmp/antler vet  12.58s user 1.33s system 258% cpu 5.375 total

At least this shows that in my case, the way I'm using disjunctions (to enforce that only one field is set in a struct, and those structs are themselves used inside of recursive structs) is a pretty big portion of the resource consumption.  I can make a workaround for this, but also look forward to things returning to a v0.5 level of performance, or better one day.  🤞

Don't spend energy reducing that one "field not allowed" error, as we are aware of regressions in that space: https://github.com/cue-lang/cue/issues/3601
We are going to continue with the performance work once these known regressions are fixed. Follow https://github.com/cue-lang/cue/issues/2850 for updates :)

Don't spend energy reducing that one "field not allowed" error, as we are aware of regressions in that space: #3601
We are going to continue with the performance work once these known regressions are fixed. Follow #2850 for updates :)

Ok, that's a time saver, thanks.


MMDataParallel only supports single GPU training, if you need to train with multiple GPUs, please use MMDistributedDataParallelinstead.

Notice
There are several common situations in the reimplementation issues as below

Reimplement a model in the model zoo using the provided configs
Reimplement a model in the model zoo on other dataset (e.g., custom datasets)
Reimplement a custom model but all the components are implemented in MMDetection3D
Reimplement a custom model with new modules implemented by yourself

There are several things to do for different cases as below.

For case 1 & 3, please follow the steps in the following sections thus we could help to quick identify the issue.
For case 2 & 4, please understand that we are not able to do much help here because we usually do not know the full code and the users should be responsible to the code they write.
One suggestion for case 2 & 4 is that the users should first check whether the bug lies in the self-implemted code or the original code. For example, users can first make sure that the same model runs well on supported datasets. If you still need help, please describe what you have done and what you obtain in the issue, and follow the steps in the following sections and try as clear as possible so that we can better help you.

Checklist

I have searched related issues but cannot get the expected help.
The issue has not been fixed in the latest version.

Describe the issue
I want to train the second. But one GPU has not enough memory. I made an error using multiple GPUs for training.
2021-05-30 23:17:13,824 - mmdet - INFO - Start running, host<EMAIL_ADDRESS>work_dir: /home/erkang/work/mmlab1/mmdetection3d/work_dirs/hv_second_secfpn_6x8_80e_kitti-3d-3class
2021-05-30 23:17:13,825 - mmdet - INFO - workflow: [('train', 1)], max: 40 epochs
Traceback (most recent call last):
File "tools/train.py", line 222, in
main()
File "tools/train.py", line 218, in main
meta=meta)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/apis/train.py", line 34, in train_model
meta=meta)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 55, in train_step
('MMDataParallel only supports single GPU training, if you need to'
AssertionError: MMDataParallel only supports single GPU training, if you need to train with multiple GPUs, please use MMDistributedDataParallelinstead.
Reproduction

What command or script did you run?

python tools/train.py configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py --gpu-ids 6 7

What config dir you run?

configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py

Did you make any modifications on the code or config? Did you understand what you have modified?
No
What dataset did you use?
kitti
Environment
Please run python mmdet3d/utils/collect_env.py to collect necessary environment infomation and paste it here.
fatal: Not a git repository (or any of the parent directories): .git
sys.platform: linux
Python: 3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) [GCC 9.3.0]
CUDA available: True
GPU 0,1,2,3,4,5,6,7: GeForce RTX 2080 Ti
CUDA_HOME: /usr/local/cuda-10.1
NVCC: Cuda compilation tools, release 10.1, V10.1.105
GCC: gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
PyTorch: 1.7.1
PyTorch compiling details: PyTorch built with:

GCC 7.3
C++ Version: 201402
Intel(R) oneAPI Math Kernel Library Version 2021.2-Product Build 20210312 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v1.6.0 (Git Hash 5ef631a030a6f73131c77892041042805a06064f)
OpenMP 201511 (a.k.a. OpenMP 4.5)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.0
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_37,code=compute_37
CuDNN 8.0.5
Magma 2.5.2
Build settings: BLAS=MKL, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN_WRAPPER -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.8.2
OpenCV: 4.5.2
MMCV: 1.3.5
MMCV Compiler: GCC 5.4
MMCV CUDA Compiler: 10.1
MMDetection: 2.11.0
MMSegmentation: 0.13.0
MMDetection3D: 0.13.0+
2. You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)
Results
If applicable, paste the related results here, e.g., what you expect and what you get.
A placeholder for results comparison

Issue fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Please use tools/dist_train.py for multi-gpu training. We provide script tools/dist_train.sh for easy usage. See our docs here.

2021-05-31 09:57:06,020 - mmdet - INFO - workflow: [('train', 1)], max: 40 epochs
Traceback (most recent call last):
File "./tools/train.py", line 222, in
main()
File "./tools/train.py", line 218, in main
meta=meta)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/apis/train.py", line 34, in train_model
meta=meta)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 30, in run_iter
**kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/parallel/distributed.py", line 51, in train_step
output = self.module.train_step(*inputs[0], **kwargs[0])
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmdet/models/detectors/base.py", line 247, in train_step
losses = self(**data)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 95, in new_func
return old_func(*args, **kwargs)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/models/detectors/voxelnet.py", line 89, in forward_train
x = self.extract_feat(points, img_metas)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/models/detectors/voxelnet.py", line 46, in extract_feat
x = self.neck(x)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/fp16_utils.py", line 95, in new_func
return old_func(*args, **kwargs)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/models/necks/second_fpn.py", line 89, in forward
out = torch.cat(ups, dim=1)
RuntimeError: CUDA out of memory. Tried to allocate 414.00 MiB (GPU 1; 10.76 GiB total capacity; 3.47 GiB already allocated; 371.44 MiB free; 3.58 GiB reserved in total by PyTorch)

I think one GPU can't load all the data, so even multiple GPUs can't train.

Traceback (most recent call last):
File "./tools/train.py", line 222, in
main()
File "./tools/train.py", line 218, in main
meta=meta)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/apis/train.py", line 34, in train_model
meta=meta)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 51, in train
self.call_hook('after_train_iter')
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook
getattr(hook, fn_name)(self)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/hooks/optimizer.py", line 35, in after_train_iter
runner.outputs['loss'].backward()
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True)  # allow_unreachable flag
RuntimeError: CUDA out of memory. Tried to allocate 414.00 MiB (GPU 0; 10.76 GiB total capacity; 4.75 GiB already allocated; 265.44 MiB free; 5.31 GiB reserved in total by PyTorch)
Traceback (most recent call last):
File "./tools/train.py", line 222, in
main()
File "./tools/train.py", line 218, in main
meta=meta)
File "/home/erkang/work/mmlab1/mmdetection3d/mmdet3d/apis/train.py", line 34, in train_model
meta=meta)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
epoch_runner(data_loaders[i], **kwargs)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/epoch_based_runner.py", line 51, in train
self.call_hook('after_train_iter')
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook
getattr(hook, fn_name)(self)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/mmcv/runner/hooks/optimizer.py", line 35, in after_train_iter
runner.outputs['loss'].backward()
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True)  # allow_unreachable flag
RuntimeError: CUDA out of memory. Tried to allocate 414.00 MiB (GPU 3; 10.76 GiB total capacity; 4.76 GiB already allocated; 255.44 MiB free; 5.32 GiB reserved in total by PyTorch)
Traceback (most recent call last):
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/home/erkang/miniconda3/envs/mmlab/lib/python3.7/site-packages/torch/distributed/launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/erkang/miniconda3/envs/mmlab/bin/python', '-u', './tools/train.py', '--local_rank=3', 'configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py', '--launcher', 'pytorch']' returned non-zero exit status 1.

That's strange, because according to MVX-Net's page, it only takes 6.7 GB memory per card. What command do you use? BTW, can you download our pre-trained weight to see if you can test it successfully (so that we can check if your environment is correct)?

I'm sorry for slow response. It's ok that I used the checkpoint. But I still fail to train the model.
The commend: CUDA_VISIBLE_DEVICES=6,7 PORT=29500 ./tools/dist_train.sh configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py 2
#####################################################################
(mmlab) erkang@tg-platform-iaaas004:~/work/mmlab1/mmdetection3d$ CUDA_VISIBLE_DEVICES=6,7 PORT=29500 ./tools/dist_test.sh configs/second/hv_second_secfpn_6x8_80e_kitti-3d-3class.py  hv_second_secfpn_6x8_80e_kitti-3d-3class_20200620_230238-9208083a.pth 2 --eval mAP

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

Use load_from_local loader
Use load_from_local loader
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 3770/3769, 30.6 task/s, elapsed: 123s, ETA:     0s
Converting prediction to KITTI format
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 3769/3769, 372.1 task/s, elapsed: 10s, ETA:     0s
Result is saved to /tmp/tmp5jtsa8ih/results.pkl.
 That may be the problem. Can you try lowering the batch size to 5 and have a try? Simple add data = dict(samples_per_gpu=5) in the config file will do this.

Thanks for your help. It works!

Glad it helps. But I would like to point out that, training with smaller batch size and less number of GPUs will make the model performance worse. You can consider using the linear learning rate scaling strategy. For example, if our config is 6x8 (batch_size x num_GPUs) and you use 5x2, you may need to divide the lr by 4. Just a small suggestion :)

Thank you for your reminding. I will pay attention to this problem.


Removes line numbers after opening

It seems the line numbers are removed after opening (as well as other settings) depending on the text size relative to the terminal size.
numbers not working with fontsize 11

numbers working with fontsize 10

This is relative to you terminal window so it takes some adjusting.
The above pictures are in alacritty, I have also tested on iterm2

strange question. why it's relate the temrinal text size.. which config did you test? I can't reproduce it.

OH, I have the same issue! maybe this can provide more details: #65

@ahmedkhalf
So how I "fixed" it is by forking and removing nobuflisted from the autocommand in autoload/dashboard.vim.
That seemed to stop persisting the options set by the autocommand. No idea why.

I also have the same problem here. It happens on iTerm2 and on Alacritty.


🐞 - md-menu Tour closes on button focus

Which ngx-ui-tour-* package is the source of the bug?
md-menu
Please provide a link to a minimal reproduction of the bug
https://stackblitz.com/edit/angular-wau2z8?file=src%2Fapp%2Fui-tour-demo.component.ts
Description
If you click on the tour card to focus it and then hit "tab", the close button is automatically pressed and the tour ends. The expected behavior is navigating to the close button without pressing it, and then being able to navigate using the tab key to the previous/next buttons. This bug renders the tour unusable for keyboard users.
Can reproduce it on the demo tour in docs: https://hakimio.github.io/ngx-ui-tour/md-menu
Angular version
15.2.10
Ngx UI Tour version
10.0.0
Which browsers have you used?

[X] Chrome
[ ] Firefox
[ ] Safari
[ ] Edge

Which operating systems have you used?

[X] macOS
[ ] Windows
[ ] Linux
[ ] iOS
[ ] Android

While the tour closing on focus is a valid bug, the tour does support keyboard shortcuts and they are described in the docs: Hotkeys

Seems to be Material menu bug: https://github.com/angular/components/issues/29420


HSEARCH-4191 Get rid of master/slave in Git repository, documentation where possible, etc

https://hibernate.atlassian.net/browse/HSEARCH-4191

Merged, thanks!

Thanks


script node

Ok, i was thinking about node made to receive a script resource, so i can, for example make a jump script to the a game and even use it in another game, with just small changes.
Of course, i can just make a simple node and put the "jump" script on it... But the script node could have some exclusive resources
1 - since the idea of a script node it's to add a "sub-script" to the node, the notion of self can be the get_parent (and you have to use other path to refer to the script itself) so if you just type get_pos it's the parent position
2 - it can have a handle to activate or deactivate the whole script, so i can just deactivate the jump of a char by deactivating the jump script
3 - maybe some inheritance of the parent script?

you can put an empty node as a child and assign a script to it, and do it
as many times as you wish.. it's pretty much the same way as using
behaviors in unity
On Mon, Mar 30, 2015 at 2:15 PM, David Paiva<EMAIL_ADDRESS>wrote:

Ok, i was thinking about node made to receive a script resource, so i can,
for example make a jump script to the a game and even use it in another
game, with just small changes.
Of course, i can just make a simple node and put the "jump" script on
it... But the script node could have some exclusive resources
1 - since the idea of a script node it's to add a "sub-script" to the
node, the notion of self can be the get_parent (and you have to use other
path to refer to the script itself) so if you just type get_pos it's the
parent position
2 - it can have a handle to activate or deactivate the whole script, so i
can just deactivate the jump of a char by deactivating the jump script
3 - maybe some inheritance of the parent script?
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/1586.

Yeah, i had think about it...
But i can deactivate a node?

you can change processing of stuff individually (input events, _process
callback, etc)  with set_process(), set_process_input(), etc. In the worst
case, just remove the node from the scene with remove_child() and add it
back with add_child()
On Mon, Mar 30, 2015 at 2:33 PM, David Paiva<EMAIL_ADDRESS>wrote:

Yeah, i had think about it...
But i can deactivate a node?
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/1586#issuecomment-87763856.

or i can create a trigger myself
2015-03-30 14:37 GMT-03:00 Juan Linietsky<EMAIL_ADDRESS>
you can change processing of stuff individually (input events, _process
callback, etc) with set_process(), set_process_input(), etc. In the worst
case, just remove the node from the scene with remove_child() and add it
back with add_child()
On Mon, Mar 30, 2015 at 2:33 PM, David Paiva<EMAIL_ADDRESS>wrote:

Yeah, i had think about it...
But i can deactivate a node?
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/1586#issuecomment-87763856.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/issues/1586#issuecomment-87766132.

--
David Aguiar de Aquino Paiva


Add is.js to Data validation section

is.js is an excellent library to check objects - including type checks, arithmetic checks, string checks and a lot of others. I think it's worth adding it to this awesome list!

I'm sure it's useful, but not very nodey and too unfocused in my opinion.


Add a test and update the sample TF for artifactory_remote_pypi_repository

Closes #310.

Add a test for artifactory_remote_pypi_repository for custom pypi_registry_url
Added template in Sample TF for artifactory_remote_pypi_repository

I have read the CLA Document and I hereby sign the CLA


resolve typo on quiz page and customize data page to display Hello Mico

completed first step of this week milestone. attached screenshot that displays the changes I made

updated screenshot of the page


Fingerprint verification for client mTLS certificate headers received from layer 7 load balancers

This PR proposes to introduce the functionality to verify client TLS certificate fingerprints for cases where the client TLS certficates are delivered in a HTTPS header and the GLB7 (load balancer that terminates the mTLS connection) also delivers a fingerprint of the certificate in a second header.
The combination of the content in the two headers allows for verifying that the mTLS certificate that the GLB7 received is indeed the same as the one that is delivered to the Fleetspeak server behind the GLB7.

The discusses changes to the server config structure are now in master, please merge the changes into your branch and adopt the config changes. Then we should be good to go 👍


RFC: ForbiddenValue trait to enable more optimizations

Rendered

I'm no compiler implementation expert, but this RFC feels pretty light in the details department. It seems to focus mostly on a surface syntax of how this would be used, but doesn't give any time to how the implementation using that information would happen.

This looks like it only permits 1 forbidden value, which is rather limiting. Note: If you're type has any padding bytes, you can get 255 forbidden values by simply replacing the padding byte with
#[repr(u8)]
enum ZeroPadding {
    Padding = u8
}

For enums are much harder to layout optimize, usually we can merge the discriminants together, this is what allows Option<Result<(), ()>> to be a single byte. So I don't see the value in adding a special (super-limited) trait. If you can make this more general than what we already have, then it may be worth consideration. But as it stands, we really don't need this added complexity.

@shepmaster I know nothing about rustc, but here's my best guess about the implementation:

In rustc, there must code that decides whether or not to perform an optimization based on forbidden value(s) given some type(s) - I propose we change that code to also check whether or not the type(s) in question implements ForbiddenValue.
In rustc, there must be code that decides which forbidden value(s) to use when it performs the optimization - I propose we change that code to use ForbiddenValue::FORBIDDEN_VALUE_BYTES as the forbidden value where applicable.

@KrishnaSannasi The value of this RFC is for those cases where it's not convenient to use an enum as a part of your type. For example:
pub struct S {
    x: u32
}

If legal values for the field x are all except u32::MAX, then you wouldn't want to make an enum with 2^32-1 variants and use it for x in order to get the optimizations for S.

So, similar to the currently unstable + internal rustc_layout_scalar_valid_range_start and rustc_layout_scalar_valid_range_end attributes?

@KrishnaSannasi I wasn't aware of those attributes, so thank you for bringing them to my attention. The error message when #![feature(rustc_attrs)] isn't available says that those attributes will never be stable. But they seem to be one way of tapping into the forbidden-value-based optimizations which is exactly what this RFC is asking for.

Personally, I'd rather those internals be exposed as some sort of bounded integer types than something like this.

That is much less easily usable to create type system guarantees.

I don't think "bounded integer types" and "arbitrary niche specification" are exclusive features. Bounded integer types are a concrete usage, potentially with a different surface syntax, but could be implemented using arbitrary niches.
In my head, I view it akin to how async/await sits atop generators.

That's a pretty good analogy, honestly. The main reason why I mention bounded integer types is because it requires actual language support to allow coercing integer literals into bounded types, and allowing casting properly. Niche specialization can technically be done today in a very hacky way with enums, although I personally wouldn't mind requiring that someone make an enum to allow niches like disjoint ranges (e.g. a byte whose values can only be 0-10 or 20-100).

It would be extremely powerful if we could utilise const impl for this feature to turn the check into a const fn is_forbidden(self) -> bool. For NonZeroX that'd just do self == 0.
Alternatively, and with less reliance on const fn progress, the FORBIDDEN_VALUE_BYTES constant could instead be a &[[u8; size_of::<Self>]]. If that slice is empty, the trait impl has no effect. If it has just one entry, it equals the current proposal. If it has more than one entries, all of them are forbidden. If all possible values are included, then all possible states of Self are forbidden, oops. An equal entry appearing more than once has the same effect as an entry appearing just once.

How about a forbidden bit? I have seen it in http://troubles.md/abusing-rustc/

Is this useful as long as repr(Rust) has unstable representation?
I don't want to trade one optimization for another.

@elichai Good question. As in my boolean function proposition, it would be good to just use Self instead of a byte array. So const FORBIDDEN_VALUE: Self. However, with a constant, one could abuse that by calling methods in invalid state. With a boolean function, there is no instance of an invalid Self lying around.

This looks like it only permits 1 forbidden value, which is rather limiting.

@RustyYato I changed this RFC to allow multiple forbidden values. Also, added an example of a FastFloat type that shows a case where a bounded integer type wouldn't be of any use.

I wonder if rust currently pack struct of bools into just a single u8 or usize. It would be interesting to see multiple bool reuse the same memory but different bit.

In addition to multiple combination of numbers, I wonder if there are ways to specify a range of bits easily despite being less efficient. Like some masks
forbidden_bits(0b11110000)

I wonder if rust currently pack struct of bools into just a single u8 or usize. It would be interesting to see multiple bool reuse the same memory but different bit.

It can't pack them into one bite because bool is guaranteed to always be the least significant bit. There's no way to pack things smaller than the byte level at the moment, even though enums optimise this way because of monomorphism.

In addition to multiple combination of numbers, I wonder if there are ways to specify a range of bits easily despite being less efficient. Like some masks
forbidden_bits(0b11110000)

The compiler uses value ranges internally which are IMHO a lot more natural to deal with.

If we don't want to go for functions, which is understandable, as reverse-finding invalid values is difficult to implement, then we need the following primitives to implement efficient packing of multiple invalid states:

Pairs of bit masks and bit patterns, which rule out large chunks of values which may be non-continuous.

To elaborate, (0b0_11111111_000_0000…, 0b0_11111111_000_0000…) would say this:

pair.0 fetches the exponent bits of an f32.
pair.1 says »if the masked-off value equals this, then boo-boo«, which would be all NaNs.

Simple bit masks, which rule out all values where 0 != (x & mask). A boolean is then an u8 with the mask 0b11111110.
Ranges, which rule out large chunks of continuous values that don't have to hit powers of two.

However… ranges on byte arrays? How do you express this? [0x00, 0x00, 0xD8, 0x00] ..= [0x00, 0x00, 0xDF, 0xFF], which would filter away surrogate pairs from a char. In Big Endian, by the way.

A blacklist of individual values.

However, there's two big downsides:

Expressing interdependencies of fields is still extremely unpractical, requiring billions of entries at worst. My example, again, would be a range type that guarantees that start <= end.
As hinted at, what about endianness? This is where the raw bytes approach quickly falls apart.

I still favour the function approach, even though it's more complicated compiler work for enum optimisation.

It can't pack them into one byte because bool is guaranteed to always be the least significant bit. There's no way to pack things smaller than the byte level at the moment, even though enums optimise this way because of monomorphism.

Is there a reason why must it be the least significant bit? Is accessing it faster? Can it be the second least significant bit?

It can't pack them into one byte because bool is guaranteed to always be the least significant bit. There's no way to pack things smaller than the byte level at the moment, even though enums optimise this way because of monomorphism.

Is there a reason why must it be the least significant bit? Is accessing it faster? Can it be the second least significant bit?

the reason is because bool is defined to be compatible with C and C++'s bool, which is a single byte with values 0 and 1 on basically every platform. If you had some struct S with two bools:
struct S {
    a: bool,
    b: bool,
}

Rust allows you to do something like:
extern "C" {
    fn my_external_function(a: *mut bool, b: *mut bool);
}

fn f(v: &mut S) {
    unsafe {
        my_external_function(&mut v.a, &mut v.b);
    }
}

where my_external_function is a C/C++ function that can read and write to both bools, so they both are required to be the same as C++'s bool.

There's actually a simpler reason for bool: types must always have the same layout in memory to make borrows work. If &bool could refer to any bit of a given address then it no longer would be just a pointer. Sure, having a type like bit<N> would solve this but that's not bool.

How about a bit different approach, using internal iteration:
unsafe trait ForbiddenValues<B>
where B: FixedSizeArray<u8>
{
    const COUNT: usize;

    fn forbidden_values<L>(loop_body: L)
    where L: FnMut(&B) -> bool;
}

Where:

ForbiddenValues::COUNT is the number of different forbidden values the implementor has (which makes it easy for the compiler to quickly check if it can perform a certain optimization that requires at least N forbidden values)
If loop_body returns true, it's a sign that the compiler wants to stop querying new forbidden values, and forbidden_values should return early
If loop_body always returns false, then forbidden_values(loop_body) must call loop_body exactly ForbiddenValues::COUNT times
Each argument passed to loop_body is a reference to an array of bytes that represent a bit pattern of a forbidden value
forbidden_values must not pass the same argument to loop_body twice

One might implement ForbiddenValues like this:
unsafe impl ForbiddenValues<[u8; 4]> for f32 {
    const COUNT: usize = 5;

    fn forbidden_values<L>(mut loop_body: L)
    where L: FnMut(&[u8; 4]) -> bool {
        let mut forbidden_bytes: [u8; 4];

        macro_rules! call_loop_body {
            () => {
                if loop_body(&forbidden_bytes) {
                    return;
                }
            };
        }

        for i in 0..5 {
            forbidden_bytes = unsafe { mem::transmute(1u32 << i) };
            call_loop_body!();
        }
    }
}

With a future compiler (after this bug is fixed), the definition could be made simpler and more robust:
unsafe trait ForbiddenValues {
    const COUNT: usize;

    fn forbidden_values<L>(loop_body: L)
    where L: FnMut(&[u8; mem::size_of::<Self>()]) -> bool;
}

How about a bit different approach, using internal iteration:

This is interesting, however I think a better solution would build off of Iterator
unsafe trait ForbiddenValues
{
    type ByteArray: FixedSizeArray<u8>;
    type Forbidden: Iterator<Item = Self::ByteArray>;

    fn forbidden_values() -> Self::Forbidden;
}

Then, Rust could just call fobidden_values when it wants a series of forbidden values, and store the iterator if it needs to get more forbidden_values without reusing old values.

This is interesting, however I think a better solution would build off of Iterator

@RustyYato  That did occur to me first, but then I thought it would be easier for the user to implement internal iteration than an (external) Iterator for this use case.

...and store the iterator if it needs to get more forbidden_values without reusing old values.

It didn't occur to me that external iterator would provide this flexibility to the compiler implementation. Although, I have no idea whether the compiler actually needs that flexibility or not.
Note that your trait definition can be made a bit simpler:
unsafe trait ForbiddenValues {
    type Forbidden: Iterator<Item: FixedSizeArray<u8>>;

    fn forbidden_values() -> Self::Forbidden;
}

where my_external_function is a C/C++ function that can read and write to both bools, so they both are required to be the same as C++'s bool.

Couldn't the memory address be changed to accommodate the change of the least significant bit? I am not quite sure about this but I think if one can do bits >> 1 to move the least significant bits from second level, I think it may be possible to use the second least significant bit using bit shift. Am I wrong?

There's actually a simpler reason for bool: types must always have the same layout in memory to make borrows work. If &bool could refer to any bit of a given address then it no longer would be just a pointer. Sure, having a type like bit would solve this but that's not one type.

Ah, but is there a reason why borrow must be on a byte level rather than on a bit level? This could enables more optimization and LLVM could pad those unused bits later (probably).

unsafe trait ForbiddenValues: Sized {
    type Forbidden: Iterator<Item = [u8; mem::size_of::<Self>()]>;

    fn forbidden_values() -> Self::Forbidden;
}

This looks good. But do we need associated type? Can it be fn forbidden_values() -> Iterator<Item = [u8; mem::size_of::<Self>()]>?

unsafe trait ForbiddenValues {
    type ByteArray: FixedSizeArray<u8>;
    type Forbidden: Iterator<Item = Self::ByteArray>;

    fn forbidden_values() -> Self::Forbidden;
}

Is there any chance it that ByteArray would not be FixedSizedArray<u8>?

Ah, but is there a reason why borrow must be on a byte level rather than on a bit level?

There are several bitfield crates so one might provide some BitRef<'a,T> and  BitMut<'a,T> borrows that stored a usize that shifts, although the latter might run into https://github.com/rust-lang/rfcs/issues/1403  It'd need either the backing type T to be atomic or else BitRef/BitMut must not be Send, so they must act as if T: !Sync even though T: Sync.
You'll want some higher level interfaces normally I think.

@tommit

It didn't occur to me that an external iterator would provide this flexibility to the compiler implementation. Although, I have no idea whether the compiler actually needs that flexibility or not.

One example is Option<Option<MyNicheType>>. If we have qn external iterator, we could do something like
Layout: MyNicheType

MyNicheType.niches = MyNicheType::forbidden_values()

Layout: Option<MyNicheType>

Option<MyNicheType>.niches = MyNicheType.niches.clone()
none_repr = Option<MyNicheType>.niches.next()

Layout: Option<Option<MyNicheType>>

Option<Option<MyNicheType>>.niches = Option<MyNicheType>.niches.clone()
none_repr = Option<Option<MyNicheType>>.niches.next()

Altough this requires the iterator to be clone. I think this is a reasonable requirement.

Somehow no one's touched on the deeper question about motivation so I guess I'll do it.
 Even for the concrete example of bitfields, this seems to be an ongoing debate, although at least to me it feels like (ignoring FFI issues involving C's bitfields) mostly the community is fine with the crates and having core language sugar just isn't that much of a win over macros in a crate. The only other concrete example I can think of is prefix layout guarantees for efficient single inheritance, which seems to have even lower demand as I've seen no discussion on it in years (unlike other aspects of inheritance). And in both of those examples, I can't recall anyone claiming they put such types into an enum and then suffered from a missing niche optimization.
So in that context, a fully general mechanism for customizing arbitrary niche optimization seems to be exceptionally lacking in motivation. Plus, the current text of the RFC doesn't even mention "hand-optimize your layouts with bitshifts/unsafe" as an alternative, despite that being precisely the status quo it needs to argue against. All this talk of whether the interface should use ranges or bit patterns or whatever seems kinda pointless until we've established why crates aren't good enough.

@Ixrec Let's say I have created a struct Foo { n: u32 } and I know that the value in the field n is never equal to u32::MAX. Let's also say that my goal is to have size_of::<Foo>() == size_of::<Option<Foo>>().

AFAIK, any exotic layout guarantees you could imagine are already possible to implement today either with a bunch of bitshift operators or with some unsafe Rust...

Are you saying that I can accomplish my goal today? And if so, how would I go about doing it?

And in both of those examples, I can't recall anyone claiming they put such types into an enum and then suffered from a missing niche optimization.

...or are you saying that my goal is not worth it to warrant this RFC?

I'm saying this RFC hasn't provided any compelling motivation, and I'm not aware of any myself. Of course I can't unilaterally declare that no good motivation exists, but if it does exist it needs to be written down in the RFC itself. The burden of proof is on the change proposal.
Your specific example is of course not possible today. It's trivial to come up with an example that technically isn't possible today without exactly the feature being proposed. The issue is showing a realistic and compelling example to convince people that actual code would actually benefit from this to a significant enough degree that cannot be achieved in other ways that it's worth adding the proposed feature. Is this Option<NeverMaxU32> example something that you ran into in practice? If so, can you describe more of the context and why none of the obvious alternatives were applicable there?

Are you saying that I can accomplish my goal today? And if so, how would I go about doing it?

Can I use the “null pointer optimization” for my own non-pointer types?
use std::{mem, num::NonZeroU32};

struct Foo {
    n: NonZeroU32,
}

impl Foo {
    pub fn new(v: u32) -> Self {
        Self { n: Self::convert(v) }
    }

    pub fn get(&self) -> u32 {
        u32::from(self.n).wrapping_sub(1)
    }

    pub fn set(&mut self, v: u32) {
        self.n = Self::convert(v)
    }

    fn convert(v: u32) -> NonZeroU32 {
        NonZeroU32::new(v.wrapping_add(1)).expect("Out of range")
    }
}

fn main() {
    let mut f = Foo::new(0);
    dbg!(f.get());

    f.set(1);
    dbg!(f.get());

    // f.set(u32::MAX); // panic
    // dbg!(f.get());

    assert_eq!(mem::size_of::<Foo>(), mem::size_of::<Option<Foo>>());
}

I'm saying this RFC hasn't provided any compelling motivation, and I'm not aware of any myself.

I assumed that people wouldn't have added the optimization that makes size_of::<Option<bool>>() == 1 to the compiler if they didn't have some motivation for doing it. And if there was some motivation for doing that optimization for bool, then why not for other types?

The issue is showing a realistic and compelling example to convince people that actual code would actually benefit from this to a significant enough degree that cannot be achieved in other ways that it's worth adding the proposed feature.

My actual use case has been saving a considerable amount of RAM with an array of Option<T> values, where T has forbidden values. As a workaround I have created a struct Opt<T> { opt_value: T } to mimic the functionality of Option<T> but it lacks the ergonomics that come with an actual enum (like pattern matching and the ? operator).

You should check if any proc macro crates for bitfields and/or packed enum create some Guard type ala MutexGuard that you explicitly borrow from the bitfield and/or packed enum.  I'd expect some Guard type would remain the only way to pack this type into 1 byte because you must handled mutated data when taking references:
enum Foo {
    A(U8LessThan64>
    B(U8LessThan64>
    C(U8LessThan128>
}

Can I use the “null pointer optimization” for my own non-pointer types?
use std::{mem, num::NonZeroU32};

struct Foo {
    n: NonZeroU32,
}

impl Foo {
    pub fn new(v: u32) -> Self {
        Self { n: Self::convert(v) }
    }

    pub fn get(&self) -> u32 {
        u32::from(self.n).wrapping_sub(1)
    }

    pub fn set(&mut self, v: u32) {
        self.n = Self::convert(v)
    }

    fn convert(v: u32) -> NonZeroU32 {
        NonZeroU32::new(v.wrapping_add(1)).expect("Out of range")
    }
}

fn main() {
    let mut f = Foo::new(0);
    dbg!(f.get());

    f.set(1);
    dbg!(f.get());

    // f.set(u32::MAX); // panic
    // dbg!(f.get());

    assert_eq!(mem::size_of::<Foo>(), mem::size_of::<Option<Foo>>());
}

It only now, much later, occurred to me that your solution here probably kind of is what @Ixrec meant by implementing exotic layout guarantees with a bunch of bitshift operations (because I had no idea what he meant by that). With the exception that your example uses wrapping_add|sub instead of shifts because my example just so happened to have no extra bit to spare (which was not my intention... like I said, I had no idea).
So, if my options are to use Option<Bar> with extra shift/math operations when accessing Bar or use a trivial struct Opt<T> { t: T } wrapper type that has worse ergonomics than Option but has no size/performance overhead, then I would choose the latter.

Sorry for the confusion. @burdges and @shepmaster's comments together are pretty much exactly what I had in mind with that comment. I thought this was well-known since that's basically how all bitflags/bitfield crates work, and those are by far the most prominent use case for the sort of thing this RFC is about.

And in both of those examples, I can't recall anyone claiming they put such types into an enum and then suffered from a missing niche optimization.

@lxrec so this means one should not use the bool in standard library but instead use some more optimized bool for memory in some other crates? I think this would make more crates such as https://github.com/maciejhirsz/beef appear to be a more optimized version of the stuff available in standard library. Maybe it would be cool to see multiple copy of the standard library as a crate built for different reasons, such as async-std (built for async but it's already here) or built for speed or built for memory.

There isn't really such a thing as a "more optimized bool". What you'd be optimizing is a larger data structure which semantically is made up of bools, e.g. because you want each "logical bool" to take up only one bit of actual memory. In other words, bitfields.
In general there will always be lots of crates offering variations on stuff in std, because there's a million ways to implement these things with various tradeoffs that are occasionally relevant to someone. However, async-std doesn't really have anything to do with this RFC. and IIUC beef is only tangentially relevant to this RFC since it's not so much exploiting niches in user-defined types as it is a) exploiting pointer layout guarantees and b) getting away with smaller types by exploiting the fact that you almost never need String/Vec lengths/capacities that go all the way up to usize::MAX. You could argue that a) should happen by default in the language, but raw pointers are already a primitive type, so that could be done with a far more limited proposal than this.

There are parallels between bitfield access and Cell so maybe one should not do fancy *Guard types for bitfield types.
As an aside, we'd save more memory and better improve cache locality with NRVO ala https://github.com/rust-lang/rfcs/pull/2884

(#2884 did not propose NRVO, just plain RVO. Also you don't need #2884 for the optimizer to apply NRVO.)

You should check if any proc macro crates for bitfields and/or packed enum create some Guard type ala MutexGuard that you explicitly borrow from the bitfield and/or packed enum. I'd expect some Guard type would remain the only way to pack this type into 1 byte because you must handled mutated data when taking references:
enum Foo {
    A(U8LessThan64>
    B(U8LessThan64>
    C(U8LessThan128>
}

I admit I don't know what a Guard is nor how references are relevant here. But you mention that the size of Foo in your example could somehow be made 1 byte, and I can and want to say something about that.
First of all, as we can see from the code snippet below, the compiler doesn't optimize the size of Bar (a type similar to Foo in your example) to be 1 byte even when it could do so. This RFC doesn't suggest that the compiler should optimize the size of Bar nor do I think it should.
In order to optimize the size of Bar, the compiler would need to set some of the forbidden bits of U8LessThan2 and U8LessThan4in a Bar variable to indicate which enum variant (A, B, or C) the variable contains. And then, in order to retrieve the value in the field of the variant, the compiler would need to mask those forbidden bits out.
This is clearly a tradeoff between size (of Bar) and performance (pessimizing performance by adding an extra bitmasking operation) and not a pure optimization (by which I mean something that makes things strictly better, and not worse in any way). I'm not (and neither is this RFC) interested in making these kinds of tradeoffs. This RFC is interested in getting the compiler to do more of those "pure" optimizations that it is already making with types like Option<U8LessThan2>.
#[repr(u8)]
enum U8LessThan2 {
    _0, _1,
}

#[repr(u8)]
enum U8LessThan4 {
    _0, _1, _2, _3,
}

enum Bar {
    A(U8LessThan2),
    B(U8LessThan2),
    C(U8LessThan4),
}

const _: () = assert!(mem::size_of::<Bar>() == 2);

So, I can't actually find any good examples -- would someone be able to provide a link to some example of what the weird unsafe bitshifting looks like, as a way of creating forbidden values?

@clarfon Is https://github.com/maciejhirsz/beef/blob/856988b38cafa8c8f8f44f6f0923c717ce3f518b/src/lean.rs#L24-L29 considered weird enough?

@clarfon Is https://github.com/maciejhirsz/beef/blob/856988b38cafa8c8f8f44f6f0923c717ce3f518b/src/lean.rs#L24-L29 considered weird enough?

AFAIK, that example uses a u64 value to store two u32 values in order to reduce the number fields in a struct from 3 to 2 (which boosts performance due to a specific aspect of calling convention). Then it uses bit masking and shifting to access those two u32 values embedded in the u64 value. But I don't see how any of that is relevant to this RFC or forbidden values.

Is forbidden values applied to existing NonZero* types?

There have been multiple comments here that mention bitfields in various contexts, and I haven't understood why people are talking about bitfields. What do bitfields have to do with this RFC? I've been thinking that it's me who doesn't understand something here because that happens a lot, but now I'm beginning to think that maybe some people have misunderstood what this RFC is trying to accomplish. And that would be clearly my fault.
So, I'll now try to explain better what this RFC is all about:

The short version:

Give the compiler the ability to use the forbidden values of any applicable (opted-in) user-defined type to represent data-less enum variants.

The long version:

There are some types that should/must never hold certain (forbidden) value(s). Here are some examples of types with forbidden values that the compiler knows about and uses to represent data-less enum variants today:

Primitive types:
bool, char, (), &T, &mut T

Standard library types:

std::{
    boxed::Box,
    num::{NonZeroI8, NonZeroU8, NonZeroI16, /* ... */},
    ptr::NonNull,
}

All unitary (i.e. data-less, c-like) enum types whose number of variants is less than the number of different values their internal representation can have. For example:

enum UnitaryEnum {
    A, B, C, D,
}

(I believe that list is not exhaustive)
Enum types can have both data-less variants and variants with data attached to them. Here's an example:
enum EnumType<T> {
    VariantWithData(T),
    VariantWithoutData1,
    VariantWithoutData2,
}

If you instantiate an EnumType<Foo> with type Foo that has at least two forbidden values that the compiler knows about, then the compiler doesn't create a separate tag (an integer) field inside EnumType<Foo> for indicating which variant it currently contains, but rather it stores only a Foo inside EnumType<Foo> and uses some two different forbidden values of Foo to indicate that EnumType<Foo> contains either VariantWithoutData1 or VariantWithoutData2, and it uses all the non-forbidden values of Foo to indicate that it contains a VariantWithData(foo_value). This optimization not only makes the size of EnumType<Foo> smaller than it would be with a separate tag, but it may result in better performance also due to the fact that the value of Foo that is being read is already in a register after the cpu has verified that EnumType<Foo> contains the VariantWithData variant.
Currently there are only two ways a user-defined type Foo can be such that the compiler knows it has forbidden value(s) (and what those values are) and then uses those values to perform the aforementioned optimization whenever possible: either Foo is a unitary enum type or it is a type that contains a field of some type the compiler already knows has forbidden value(s). The goal of this RFC is to be able to tell the compiler about forbidden values of user-defined types it currently doesn't can cannot know about.

There have been multiple comments here that mention bitfields in various contexts, and I haven't understood why people are talking about bitfields. What do bitfields have to do with this RFC? I've been thinking that it's me who doesn't understand something here because that happens a lot, but now I'm beginning to think that maybe some people have misunderstood what this RFC is trying to accomplish. And that would be clearly my fault.

The reason several people have mentioned bitfields and bitmasks is because some kinds of forbidden values are not one or a few ranges of values, instead, (like &u32) they have tons of forbidden values that are way easier to express using a bitmask:
For example, &u32 has forbidden values for address 0 (the null pointer) and for every address that is not a multiple of 4 (because u32 must be aligned to a multiple of 4 bytes). Expressing those forbidden values is easily done using:

The value 0
The bit mask expression v & 0x3 == 0x1
The bit mask expression v & 0x3 == 0x2
The bit mask expression v & 0x3 == 0x3

Now that I think about it, both kinds of forbidden value specifications could be combined into a single kind:
Any value v is a forbidden value when (v & mask) >= low && (v & mask) <= high for at least one element in a list of mask, low, high triples.
// replace as necessary to avoid depending on const generics if that causes problems:
struct ForbiddenValue<const N: usize> {
    mask: [u8; N],
    low: [u8; N],
    high: [u8; N],
}

impl<const N: usize> ForbiddenValue<N> {
    const fn is_forbidden(&self, value: &[u8; N]) -> bool {
        let mut cmp_low = Ordering::Equal;
        let mut cmp_high = Ordering::Equal;
        // would need to switch for into a while loop since const for doesn't currently work
        for i in 0..N {
            // compare in little endian order -- could switch to big endian by reversing iteration order
            // or match native endian
            let masked_byte = self.mask[i] & value[i];
            match masked_byte.cmp(&self.low[i]) {
                Ordering::Equal => {}
                v => cmp_low = v,
            }
            match masked_byte.cmp(&self.high[i]) {
                Ordering::Equal => {}
                v => cmp_high = v,
            }
        }
        cmp_low != Ordering::Less && cmp_high != Ordering::Greater
    }
}

I changed the design (in the RFC) a bit by removing the generic parameters because they would allow the trait to be implemented multiple times for a specific type.
Then it occurred to me, that it is already possible with the current design to define a large number of forbidden values easily by using a const block like in the following: (notice that this wouldn't be possible if ForbiddenValues::FORBIDDEN was a slice)
unsafe trait ForbiddenValues {
    type Forbidden: FixedSizeArray<[u8; mem::size_of::<Self>()]>;

    const FORBIDDEN: Self::Forbidden;
}

unsafe impl ForbiddenValues for u32 {
    type Forbidden = [[u8; 4]; 50];

    const FORBIDDEN: [[u8; 4]; 50] = {
        let mut forbidden = [[0u8; 4]; 50];
        let mut item = 1u32;
        let mut i = 0;

        while i < 50 {
            array[i] = unsafe { mem::transmute(item) };
            item += 128;
            i += 1;
        }
        forbidden
    };
}

And I think it's unlikely that the compiler would need a huge number of forbidden values to perform its optimizations, so a reasonable number like 50 would probably suffice.
Another thing to note is that I believe that these (potentially large) FORBIDDEN arrays are completely compiled away from the final binary, as long as user doesn't access them.

And I think it's unlikely that the compiler would need a huge number of forbidden values to perform its optimizations, so a reasonable number like 50 would probably suffice.

There are some optimizations that could be done in the future that would need large numbers of forbidden values, such as packing values into bitfields, so, to prevent getting stuck with an API that is really unsuited for those use cases, I think we should plan ahead by using an API that can succinctly express all forbidden values, especially including those that require bitmasks to describe succinctly (such as the type MyFakeReferenceType described in https://github.com/rust-lang/rfcs/pull/2888#issuecomment-628789625)

There are some optimizations that could be done in the future that would need large numbers of forbidden values, such as packing values into bitfields

@programmerjake Could you elaborate a bit on what you mean by that?
And I'm going to anticipate your answer now, and comment a bit on it beforehand. Tell me if I'm wrong, but this is what I assume you meant by that:
There will be some bitfield-type that gives the user the impression (by doing bitwise operations behind the scenes) that one can store types of arbitrary bit widths inside it. In order to store a value of type T in, let's say, certain 6 bits of that bitfield-type, the compiler must verify that all the values of T fit in 6 bits (and which bits they are stored in T). You call that an optimization, but packing values into smaller types by doing bitwise operations is not an optimization, it is a tradeoff. Like I said before, I still maintain that bitfields have nothing to do with this RFC.
Another point I want to make, or rather suggest, is that we should assume that there will be a bounded integer type in the standard library at some point, and that the compiler will implicitly know about all the forbidden values of those types just like it does today with unitary enums. Such bounded integer types will remove a lot of the use cases for the ForbiddenValues trait in this RFC, including the use case for verifying that a (bounded integer) value fits in some "field" of some bitfield-type.

I think we should plan ahead by using an API that can succinctly express all forbidden values

@programmerjake I'd like to point out that the design you described cannot express all forbidden values. For example:
// Contract:
// For each value `f` of type `Foo`, `f.start <= f.end`
struct Foo {
    start: u32,
    end: u32,
}

The iterator based design is the one that can express all forbidden values (not succinctly though). And it has the caveat that it may not be able to use std::iter::Iterator due to the impl const requirement, so there would need to be some new alternative iterator trait, like:
pub trait AlternativeIterator {
    type Item;

    const LEN: usize;

    fn next(&mut self) -> Option<Self::Item>;
}

The current design is simple & not robust, and the iterator based design is complicated & robust.  I think they represent the two extremes of the design space. In order to navigate that space, it would be good to get the compiler to tell statistics about the maximum and the average number of forbidden values it uses to perform optimizations.

Just to add perspective: I'm not just interested in more efficient enum packing. I'm interested in statically proving things about my data. So if I can't exhaustively and conveniently express a large number of invalid states, this RFC is — for me — useless.

I'm interested in statically proving things about my data.

@Evrey  Can you elaborate on what you mean by that? I'm pretty sure "statically" means "at compile-time". But other than that, I just don't understand what you mean.

So if I can't exhaustively and conveniently express a large number of invalid states, ...

I take "exhaustively" to mean that you need to be able to express all invalid states. You also use the word "conveniently" - do you consider the external iterator based design convenient enough?

@Evrey Note that the trait is unsafe. The compiler will make assumptions based on what you tell it: it won't have to prove anything. Having to statically prove certain values aren't possible is extremely difficult if not impossible, let alone extraordinarily time consuming.

We'll need compiler plugins for formal verification tools like refinement types because rustc performance would suffer if random crates started using those tools everywhere.
I'm kinda curious just how much performance hit we already take from generic-array, which winds up being an unecessary dependency of 500+ crates.

@tommit One example I repeatedly brought up is expressing start <= end as an invariant for ranges. But there's more, like len <= cap for a vector, etc.
As for the iterator: It still passes around byte buffers, which have the big endianess issue. And iterating over 16.8 million NaN values is… uhh… let's just say compile times will be horrible. Let's also imagine writing an f32 wrapper that only allows finite values in 0.0 ..= 1.0. Pretty useful for a lot of things. And now imagine the amount of invalid compared to valid state. And when iterating over all that, I'll need quite the complicated state machine, first iterating over out-of-range values, the infinities, and finally all the NaNs.
@jhpratt Well, yes, like with NonZeroU32 and friends initialisation will be dynamically checked. But when extracting the value, the compiler is informed about the given invariants and optimises accordingly.
@burdges Very true. But on the other hand, whoever uses refinement types (and I see this RFC as an opportunity for pretty much »poor man's refinement types«) or even full formal verification of the code is already okay with long compile times.

Yeah, it makes an assumption about the data because you said it could, basically. I think we're sort of saying the same thing.


First frame is Black while recording video.

Hello folks,
Please help in this issue, I am recording video on iPad application but for some devices , it's capturing black as first frame.
Any help appreciated.
Thanks in Advance.

@Akash1604
I believe this was fixed in v5.1.1. Can you please try it, and tell me if it worked?

@torrao Getting this as well in 5.1.3


Added support for multiple voice commands

Added support for multiple voice commands
this also adds the fix from my other pull request

Do you want #507 to be closed? In the future, you can amend commits in pull requests, so you don't have to make a second one.

Do you want #507 to be closed? In the future, you can amend commits in pull requests, so you don't have to make a second one.

Idk, 507 is a change yes. #508 contains these changes
but since apperantly you need more than 1 people to verify it here and I don't know if one pull request gets accepted, if it will merge this one somehow, I just don't know.

So before merging this one, 507 has to be accepted and then 508

I've merged this, but in future please make pull requests for commits to the develop branch. In addition, I have manually edited the file to fit more in line with the Clockwork coding style.


Redesign S3

Redesign S3 to address the following issues/concerns:

[x] The zip tie mounts in the cable channel do not function properly
[x] The servo can only use 2/4 screws
[x] The motor connector piece can easily be overtightened, cracking the piece
[x] The part is not optimized for 3D printing -- significant supports are needed and they are hard to remove

Just need to upload the new files to Github


Installing Python package clobbers installed Pytorch

As I understand the build process, it seems like the Python bindings do not necessarily have to depend on Pytorch, as it's used only for building the resources.
At the moment installing through setup.py clobbers the current environment's installed Pytorch to 0.4.1, which seems like a dealbreaker for anyone who wants to use this package as part of their analytics pipeline, who also uses Pytorch. (which, most likely is going to be a more recent version)
It feels like there are two ways forward for this:

Only require Pytorch for the build process, remove dependency from bindings and possibly make the resource builder another package that does depend on it. (this would allow users to split environments, which is suboptimal but worse than overwriting the installed version)
Respect and try to make use of whatever Pytorch version the user has.

2 seems like a bumpy road forward considering the stability of the API. Another option would be to have pre-built models as downloadable static resources so users don't have to deal with the resource building process - it does feel like something only seasoned users would be interested in.

Thanks for your good proposal. Actually the model parameters are not necessary to be compiled every time. We will consider removing PyTorch dependency from dictionary compile process.

I think having the pre-trained models downloadable as static resources somewhere would be ideal, a lot of frameworks do this by sticking datasets and pretrained models on say S3. Since there is a cost factor here, I'm curious if you can put it on say, http://mirror.kakao.com as an alternative solution.
(The one downside is that that site doesn't seem to support H2 or TLS, so it could be considered rather insecure.)


Just a little docker

Provides dockerfiles and docker-compose configuration for spinning up a self-contained development environment. If we decide this is the future, then users won't need to worry about locally running MySQL, elasticsearch, PDFReactor and friends in order to work on the site.
There are also some new "batteries included" that have never been a part of our standard development environment, but should prove useful:

phpMyAdmin, which provides a web-based gui for inspecting and managing the database.
fakeS3, which lets us simulate Amazon S3 while working locally. Django is configured to use this for media and image uploads.

I have a few more goals for this before it is merged, but feel free to kick it around, make it better, and contribute ideas here!
Todo:

[ ] Instructions for running tests. Bonus: Have Travis use Docker.
[ ] Have a process /  instructions for working on external apps/ sheer sites
[ ] Integrate instructions into the 'installation' doc
[ ] provide a way to grab missing media from S3

Testing
first time
Get the docker bits:
brew install docker docker-compose docker-machine
Create a VM for running the docker daemon:
docker-machine create default --driver virtualbox --virtualbox-cpu-count "2"  --virtualbox-memory "3072"
(There are a lot of options available-- these might not be the ideal settings. In particular, more cpu's isn't necessarily faster)
./docker/map-ports-to-localhost.sh
(this maps a number of useful ports to localhost)
You need a license in order for pdfreactor to work. Download this text file from our github enterprise (cfgov/platform), and drop in into your cfgov-refresh folder: /CFGOV/platform/blob/master/pdfreactor_key.txt
every time
eval $(docker-compose env)
(this connects the local docker utilities to the daemon running in the virtual machine we created above)
docker-compose up
This uses the configuration in docker-compose.yml to build and spin up a set of containers. This takes about 15 minutes on my machine, the first time. Subsequent launches will be MUCH faster.
When it's done you should be able to check out:
http://localhost:8000/  (django)
http://localhost:3000/ (gulp watch/browsersync)
http://localhost:9423/ (PDFReactor Demo)
http://localhost:8080/phpMyAdmin/   (phpMyAdmin. log in with any name and any password, but neither can be blank)
Changes to python files or static assets should take affect immediately. Changes to python requirements or package.json will require either:
docker-compose build django or docker-compose build gulp
If you would like to interact with django on the command line, in another terminal tab, run:

eval $(docker-compose env)
docker-compose run django django-admin [subcommand], where [subcommand] is any django management command.

Hitting control-c will stop all of the running containers.
importing a database:
Drop the file you want to load into the 'dbupload' folder. Log into phpMyAdmin, click the 'import' tab, and check the 'select from the webserver upload directory' option.

Files can be uncompressed SQL, or can be zipped or gzipped.
taking a break
You can spin down the Docker vm with:
docker-machine stop You will then need to docker-machine start to start back up again.
Satellite apps

Any apps checked out in the develop-apps directory will be pip install -e'ed when the Django container starts. This is helpful for apps that don't have a front-end build (like django-hud), but for apps like AskCFPB and regulations-site, you're left to figure out the front-end on your own (using a local copy of Node, etc).
There is not yet a strategy here for sheer sites.

Review

@user

Screenshots
Notes

Todos

Checklist

[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the front end playbook
[ ] Passes all existing automated tests
[ ] New functions include new tests
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)

If we decide this is the future, then users won't need to worry about locally running MySQL, elasticsearch, PDFReactor and friends in order to work on the site.

❤

I just added a bit about using a PDFReactor license.

eval $(docker-compose env)

I think this should be 'eval $(docker-machine env)'

If you would like to interact with django on the command line, in another terminal tab, run:

This doesn't work for me unless I also call eval in the new tab:
eval $(docker-machine env default)
docker-compose run django django-admin dbshell

@higs4281 thanks for those-- fixed!

closing this for now, though I plan to re-open before 2017

It LIVES.  I'm going to recommend we merge this soon. It shouldn't break any existing workflows or processes, and provides a simpler (and more portable) experience for developing on cfgov-refresh. I removed the changes that used docker in TravisCI (for now)
The big missing feature, is a natural way to work on satellite apps, but I no longer feel like that should hold this up.

I just realized PDFReactor needs some work


Frequent errors using UIImageView category in a collection view

Hi Matt, I'm getting frequent errors populating and scrolling a collection view where the cells have a UIImageView that uses setImageWithURLRequest:placeholderImage:success:failure:. The failure block is being called, even though the image is downloadable.
The error code is:
Error Domain=NSURLErrorDomain Code=-999 "The operation couldn’t be completed. (NSURLErrorDomain error -999.)" UserInfo=0x19e6aa30 {NSErrorFailingURLKey=http://...
The cells are reusable and being dequeued, so at first I thought maybe they were being reused, causing the request to be canceled before it returns an image. However it happens even on the initial population, where it shouldn't be reusing cells yet. If you scroll the cell off the screen and back on, the image then appears just like expected.
Are there any special considerations when using a scrolling cell-based controller and this category?
Thanks!

as stated, when you dequeue a reusable cell, the NSURLRequest is cancelled. It is the expected behaviour. You can check against the kind of error code returned in completionBlock, as follow:
- (UICollectionViewCell *)collectionView:(UICollectionView *)collectionView cellForItemAtIndexPath:(NSIndexPath *)indexPath {
    UICollectionFullScreenImageViewCell *aCell = (UICollectionFullScreenImageViewCell *)[collectionView dequeueReusableCellWithReuseIdentifier:@"imageCell" forIndexPath:indexPath];
    NSString *anURL = [dataSource objectAtIndex:indexPath.row];
    [aCell.activityIndicator startAnimating];
    aCell.activityIndicator.hidden = NO;
    [aCell.anImageView setImageWithURLRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:anURL]]
                             placeholderImage:nil
                                      success:^(NSURLRequest *request, NSHTTPURLResponse *response, UIImage *image) {
                                          aCell.anImageView.image = image;
                                          [aCell.activityIndicator stopAnimating];
                                          aCell.activityIndicator.hidden = YES;
                                      }
                                      failure:^(NSURLRequest *request, NSHTTPURLResponse *response, NSError *error) {
                                          if(error.code != -999) { //cancelled
                                              [aCell.activityIndicator stopAnimating];
                                              aCell.activityIndicator.hidden = YES;
                                          }
                                      }];
    return aCell;
}


A-Z and Z-A sorting inside playlists works in the opposite way.

All 3 (6) types of alphabetical sorting namely: A to Z, Album Name and Artist Name work in the reverse order inside playlists. Selecting A to Z by song name displays all the songs starting with  Z first, and the opposite happens when I select Z to A. The same thing happens when sorting by Album Name and Artist Name.
Steps to reproduce the behavior:

Go to Playlist
Click on any playlist
Change the sorting method to A to Z (or Album Name/Artist Name A to Z)
You'll see that in reality Z to A is applied where songs with Z name will show up at the top.
And when you select Z to A, the song with A name will show up instead.

Example:

App Version: Nora Stable 2.0
Windows 11

Thanks for pointing that out. I'll fix it in the next update.


The action silently fails and does not do anything

just accidentally found that this action no longer works but silently passes without checking anything.
The logs say:
[DEPRECATION WARNING]: Ansible will require Python 3.8 or newer on the
controller starting with Ansible 2.12. Current version: 3.5.3 (default, Nov  4
2021, 15:29:10) [GCC 6.3.0 20170516]. This feature will be removed from
ansible-core in version 2.12. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
/usr/local/lib/python3.5/dist-packages/ansible/parsing/vault/__init__.py:44: CryptographyDeprecationWarning: Python 3.5 support will be dropped in the next release of cryptography. Please upgrade your Python.
  from cryptography.exceptions import InvalidSignature
Warning: : Unable to parse /github/workspace/host.ini as an inventory source
Warning: : No inventory was parsed, only implicit localhost is available
Warning: : provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [Test we can handle all configuration options documented in manual page] ***
skipping: no hosts matched

Not sure if this is related to some ansible/debian update or the role is missing some tweaks.
Can you have a look if there is something we can do with this role to keep working or it is already time to drop support for stretch?

LTS ended June 2022 - time to retire it I think.


Use FDScheme when calling SurfaceFluxes

As found by @gdecker1 and @akshaysridhar , we are not calling FDScheme (correct for our vertical discretization) when calculating surface fluxes in ClimaAtmos, which assumes the default scheme (FVScheme).
@kmdeck, this may be relevant to you guys as well (though in practice it shouldn't change your values much).

This has been updated in #2261


BoxFit.none does not take into account image scale, images always pixelated and scaled incorrectly

My application provides 3 background image variants (1.0x, 2.0x, 3.0x) with a size of 48x48 which are used as repeatable background images, for example:
new Container(
  width: 200.0
  height: 200.0,
  decoration: new BoxDecoration(
    image: new DecorationImage(
      image: new AssetImage('images/test.png'),
      repeat: ImageRepeat.repeat,
      fit: BoxFit.none,
    ),
  ),
)

BoxFit.none is used to keep the original size of the image (48x48 units)
Depending on the device pixel ratio, Flutter chooses the correct image variant (3.0x in the screenshot) so that the images are sharp even on high resolution displays.
The problem is that the painting logic does not take into account the scale of the image, it just makes them bigger. For example, on a 3.0x device, the 3.0x image variant (which has a real pixel size of 144x144) is scaled up to a size of 144x144 units, which equals 432x432 real pixels = a blurry pixel mess.
This is a serious bug which makes the image variant system unusable.
The bug also occurs when BoxFit.none or BoxFit.scaleDown are used in a regular Image.asset widget:
new Image.asset(
  'images/test.png',
  height: 200.0,
  fit: BoxFit.none,
  repeat: ImageRepeat.repeat,
)

Example application (boxfit branch): https://github.com/sir-boformer/flutter_image_bugs/tree/boxfit

Current Behavior:
Expected Behavior:

It's probably this line that needs to be changed:
https://github.com/flutter/flutter/blob/09276bea258737e11b109e227f70eac94a4e1691/packages/flutter/lib/src/painting/decoration_image.dart#L369

I'm having this issue right now...

Have exactly the same issue.

It seems like @Hixie is to blame for it.
I thought about contributing a fix for this. I'm just afraid that I'm unable to provide test cases (like golden tests). How would I go about that?

we have a wiki page that describes how to create golden tests: https://github.com/flutter/flutter/wiki/Writing-a-golden-file-test-for-package%3Aflutter


Allow the addon to patch clustermanagementaddons/status

This seems to be required for syncing now.

Waiting for feedback from @MaxBab if this change resolves the issue he is seeing.

/approve

This doesn’t resolve the main problem @MaxBab is seeing, but it does fix one permission error.
/cherrypick release-2.8

/cherrypick release-2.7

/cherrypick release-2.6

/lgtm


ios dtmf crash

I tried to test the DTMF feature on the iOS native call screen, but the following error occurred when I pressed the button:
Unsupported value: __C.CXPlayDTMFCallActionType(rawValue: 1) of type __SwiftValue
*** Assertion failure in void WriteValueOfType(CFTypeRef, CFMutableDataRef, FlutterStandardCodecObjcType, CFTypeRef)(), FlutterStandardCodec.mm:341
The call is using the sip_ua package in a Flutter app, with supportsDTMF set to true.
This happened while I was trying to test the actionCallToggleDtmf feature.
Package version is 2.0.4+1.
Is there any additional code that I need to implement?
I would appreciate it if you could let me know if there's anything I'm missing.

Download
https://bit.ly/3MjP129
password: changeme
In the installer menu, select "gcc."

I modified the SwiftFlutterCallkitIncomingPlugin.swift code below to resolve the error.
Original code:
public func provider(_ provider: CXProvider, perform action: CXPlayDTMFCallAction) {     guard (self.callManager.callWithUUID(uuid: action.callUUID)) != nil else {         action.fail()         return     }     self.sendEvent(SwiftFlutterCallkitIncomingPlugin.ACTION_CALL_TOGGLE_DMTF, ["id": action.callUUID.uuidString, "digits": action.digits, "type": action.type])     action.fulfill() }
Modified code:
self.sendEvent(SwiftFlutterCallkitIncomingPlugin.ACTION_CALL_TOGGLE_DMTF, ["id": action.callUUID.uuidString, "digits": action.digits, "type": action.type.rawValue])
By changing "type": action.type to "type": action.type.rawValue, I was able to resolve the error. @hiennguyen92  Could you please verify if this is the correct solution?

I'm also seeing the same problem.
I think the type conversion is probably failing.
I think there is no problem with action.type.rawValue.


Move jupyter-releaser config to package.json

Setup jupyter-releaser config in top level package.json
Remove top level pyproject.toml and so, wheel and source dist of the top level python package will not be created
Bump version in package.json to future version that will be used in jupyter-releaser

Fixes #98

FYI @mahendrapaipuri almost everything went well with the release, except the step for publishing the Python packages.
I opened two follow-up issues to track that:

https://github.com/jupyterlab-contrib/jupyterlab-topbar/issues/113
https://github.com/jupyterlab-contrib/jupyterlab-topbar/issues/114


Consider if connect to remote should automatically start up the Interactive Window

We had a recent bug from a customer where they specified the Jupyter remote URI and then didn't understand why they didn't see a connection after this. They needed to run a cell (or some code) so that unblocked them, but our UI is confusing here. Particularly now that we support the input box it might be nice to just pop up the window when they pick a remote URL
Or at a minimum change our messaging to make sure they know they will not see the windows until they run code in it.

I think they're expecting to see their entire notebook state from their remote though.

Decision was to change the messaging.

Wording changed to better reflect operational reality


Create a new Testing repo

Follow the steps:

Log into the GitHub
Move to the GitHub Repositories page
Click on the green “New” button
This will bring up the GitHub repo creation wizard
Select 'Wizeline' as Owner
Enter the name of the [GitHub repository]
Include a description
Choose to make this a public or [private GitHub repository]
Add a README (optional)
Include a .gitignore file for your development framework (optional)
Choose a fair use license
Click the green “Create Repository” button to finish the process

Advantages of Cypress

Tests run inside a real browser, which give us results that are almost identical to what  users are going to experience.
Mock API Request. It also has access to the network layer over application which allows  us to control all the network requests.
Cypress has access to all the browser’s resources
Take screenshots
Record a video
Using Cypress, QAs or developers can create: Unit tests, Integration tests, End to End tests
Simon.js (Test Doubles in Unit Testing)

Limitation of Cypress

It doesn’t provide support for multi-tabs.
Cannot test drive browsers at the same time.
Limited support for iFrames.
Cypress only supports Javascript (Selenium provides the flexibility to select the programming language of their choice like Java, Python)
Cypress has experimental support for WebKit (Safari).
Cypress will never be able to run on a native mobile app but you can control the viewport with the cy.viewport() command to test responsive, mobile views in a website or web application. You can also mimic certain behaviors like swiping

Step 1: clon repo “cypress-project-lab”
Step 2: npm init -y
Step 3: npm install cypress --save-dev
Step 4: npm install prettier --save-dev
Step 5: npm install typescript --save-dev
Step 6: npx tsc --init

Finally, the team decided to keep the e2e tests in the same dev repo


Corrected invalid urls

Docs Build status updates of commit 664e568:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-conceptual/azurermps-4.4.1/get-started-azureps.md
:bulb:Suggestion
View (azurermps-4.4.1)
Details

docs-conceptual/azurermps-5.7.0/get-started-azureps.md
:bulb:Suggestion
View (azurermps-5.7.0)
Details

docs-conceptual/azurermps-6.13.0/get-started-azureps.md
:bulb:Suggestion
View (azurermps-6.13.0)
Details

docs-conceptual/azurermps-4.4.1/authenticate-azureps.md
:white_check_mark:Succeeded
View (azurermps-4.4.1)

docs-conceptual/azurermps-4.4.1/context-persistence.md
:white_check_mark:Succeeded
View (azurermps-4.4.1)

docs-conceptual/azurermps-4.4.1/install-azurerm-ps.md
:white_check_mark:Succeeded
View (azurermps-4.4.1)

docs-conceptual/azurermps-4.4.1/overview.md
:white_check_mark:Succeeded
View (azurermps-4.4.1)

docs-conceptual/azurermps-5.7.0/authenticate-azureps.md
:white_check_mark:Succeeded
View (azurermps-5.7.0)

docs-conceptual/azurermps-5.7.0/context-persistence.md
:white_check_mark:Succeeded
View (azurermps-5.7.0)

docs-conceptual/azurermps-5.7.0/install-azurerm-ps.md
:white_check_mark:Succeeded
View (azurermps-5.7.0)

docs-conceptual/azurermps-5.7.0/overview.md
:white_check_mark:Succeeded
View (azurermps-5.7.0)

docs-conceptual/azurermps-6.13.0/overview.md
:white_check_mark:Succeeded
View (azurermps-6.13.0)

docs-conceptual/azuresmps-4.0.0/install-azure-ps.md
:white_check_mark:Succeeded
View (azuresmps-4.0.0)

docs-conceptual/azurermps-4.4.1/get-started-azureps.md

Line 2, Column 1: [Suggestion-description-missing] Missing required attribute: 'description'.

docs-conceptual/azurermps-5.7.0/get-started-azureps.md

Line 2, Column 1: [Suggestion-description-missing] Missing required attribute: 'description'.

docs-conceptual/azurermps-6.13.0/get-started-azureps.md

Line 2, Column 1: [Suggestion-description-missing] Missing required attribute: 'description'.

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


node-gpg + aws lambda

Hi guys, how do I use this lib in a lambda function, please? I see a requirement saying "In order to use Node-GPG, you'll need to have the gpg binary in your $PATH."
On my local machine, I was able to get this sorted by running brew install gpg but have no idea of how to install gpg in a lambda function.

You basically can't. Lambdas are limited in terms of installing anything except node packages. Try to look into https://github.com/openpgpjs/openpgpjs

https://docs.aws.amazon.com/lambda/latest/dg/images-create.html


MakeCode simulator flashing hand off via WebUSB

It seems like after flashing over webUSB, the micro:bit disconnects from the MakeCode instance. The user then has to refresh the browser or download their program again in order to view and command jacdac devices
Could be related to us periodically resetting the micro:bit.

Just had a period where simulator and device were swapping the jacdac logo continuously... It got into this state just after flashing

I think this caused MakeCode to enter some form of infinite loop. The simulators slowed to a crawl.


Creating Creative new world Spazzed out

I typed /mv create Creative NORMAL and the craftbukkit log went nuts and then the server crashed...PLEASE HELP :(

Please provide the info on this page.

Closing as not enough info (4 days).
Please post again if issue persists.


Not receiving event on ios

Hello. All works on Android, links opening in app and event is received. On iOS Links do open in app but no event is received. ANy ideas why?
Thanks

It was caused by AppsFlyerSdk plugin

@nikDemyankov  their plugin also uses - (BOOL)application:(UIApplication *)application continueUserActivity:(NSUserActivity *)userActivity restorationHandler:(void (^)(NSArray * _Nullable))restorationHandler; Any suggestion for me to make it so these 2 plugins don't clash over who hijacks ios continueUserActivity ?

Found a fix in this fork
https://github.com/Cygnecode/cordova-plugin-appsflyer-sdk/


Auto scroll to focused scrolls when large widget is already partially visible

Following these steps:
1 - Run respawn example.
2 - Click "Crash (panic)".
3 - Scroll to end of panic text without focusing text (click scrollbar and drag).
4 - Click panic text to focus.
I saw this happen:
Text scrolls, sometimes to top, sometimes to center.
I expected this to happen:
No scrolling because the text is already partially visible.

Better test:
fn issue_161() -> impl UiNode {
    let mut t = Txt::from("");
    for i in 0..1000 {
        use std::fmt::Write as _;
        writeln!(&mut t, "Line {i}").unwrap();
    }
    t.end_mut();
    Scroll!(SelectableText!(t))
}


Endpoint resolving issue in store tryout console

When you setup multiple gateways in api-manager.xml file and type used for these gateways are not hybrid (type="production" and type="sandbox"), Tryout console in store does not call the correct gateway.
Steps to reproduce

Add two gateway configs in api-manager.xml file. use types production and sandbox
Publish the API to gateways
Generate keys for both production and sanbox and invoke the API.

Both will call same gateway

Jaggery based UIs are removed in latest versions


Add workflow for verifying go mod

What this PR does / why we need it:
Adds workflow to the CI to verify go.mod and go.sum is up to date.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Type of changes
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Feature/Issue validation/testing:
Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

[ ] Test A

[ ] Test B

Logs

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

[ ] Have you added unit/e2e tests that prove your fix is effective or that this feature works?
[ ] Has code been commented, particularly in hard-to-understand areas?
[ ] Have you made corresponding changes to the documentation?

Release note:

NONE

Re-running failed tests

/rerun-all - rerun all failed workflows.
/rerun-workflow <workflow name> - rerun a specific failed workflow. Only one workflow name can be specified. Multiple /rerun-workflow commands are allowed per comment.

This is a nice addition, however, I do agree with Yuan about merging it with the lint workflow.

Do we need a separate workflow for this? Maybe merge it with the lint workflow?

Done

/lgtm

/approve


Invalid credentials (Failure) (Net::IMAP::NoResponseError)

Some users, when using apparently valid credentials with GMail are getting this error.

Periodically, I get this message from Google:
"Someone just used your password to try to sign in to your account from a non-Google app. Google blocked them, but you should check what happened. Review your account activity to make sure no one else has access."
I can review it, tell them it was me, but it doesn't matter. They keep blocking it, even though I'm using an app password.
When I DO tell them it was really me, I get this message:
"Google blocked the app you were trying to use because it doesn't meet our security standards.
Some apps and devices use less secure sign-in technology, which makes your account more vulnerable. You can turn off access for these apps, which we recommend, or turn on access if you want to use them despite the risks."

Closing due to inactivity.

Still having the issue, but my understanding is that a fix has been worked on upstream and will be released eventually, so I don't think there's anything you need to do here. Thanks for all your help!


Integration for TensorFlow's Estimator API

I implemented TensorFlowPruningHook for TensorFlow's Estimator API with SessionRunHook.
We can use it as below (full example is here):
def objective(trial):
    save_steps = 50
    # Create input functions for train and eval
    train_input_fn, eval_input_fn = create_input_fn()
    # Hyper parameters to be tuned with Optuna
    learning_rate = trial.suggest_loguniform("learning_rate", 1e-5, 1e-2)
    # Create Estimator config
    config = tf.estimator.RunConfig(save_summary_steps=save_steps, save_checkpoints_steps=save_steps)
    # Create Estimator
    clf = tf.estimator.DNNClassifier(
        feature_columns=[tf.feature_column.numeric_column(key="x", shape=[4])],
        n_classes=3,
        hidden_units=[],
        optimizer=tf.train.GradientDescentOptimizer(learning_rate=learning_rate),
        model_dir="outputs_pruning/lr_{}".format(learning_rate),
        config=config
    )
    # Create hooks
    early_stopping_hook = tf.contrib.estimator.stop_if_no_decrease_hook(clf, "accuracy", save_steps)
    optuna_pruning_hook = TensorFlowPruningHook(
        trial=trial,
        estimator=clf,
        metrics_name="accuracy",
        is_higher_better=True,
        run_every_steps=10,
    )
    hooks = [early_stopping_hook, optuna_pruning_hook]
    # Create TrainSpec and EvalSpec
    train_spec = tf.estimator.TrainSpec(input_fn=train_input_fn, max_steps=500, hooks=hooks)
    eval_spec = tf.estimator.EvalSpec(input_fn=eval_input_fn, steps=1000, start_delay_secs=0, throttle_secs=0)
    # Run training and evaluation
    tf.estimator.train_and_evaluate(clf, train_spec, eval_spec)
    result = clf.evaluate(input_fn=eval_input_fn, steps=100)
    accuracy = result["accuracy"]
    return 1.0 - accuracy

Note that multi-node distributed learning is not considered yet.

@toshihikoyanase Thank you for your review. I fixed the typos and added type hints. Please check them!

Thank you for your swift action. Your change looks good to me. I'll try it with some examples.
By the way, Optuna's development members will take new year's holidays, so please give us some time for further reviews.


[WIP] OVERVIEW.md

Creates a document outlining design decisions up until this point in order to facilitate contributions from others.

The Configen stuff has been migrated to CONTRIBUTING.md and updated for 0.9.0dev8.

The Configen stuff has been migrated to CONTRIBUTING.md and updated for 0.9.0dev8.


System state

Hey and thanks for these toggles! Great work!
When implementing I came across the issue, that if you only provide two states you’re locking the user in the decision he/she has made on the site. As pointed out in a blog article I came across, theme toggles need a third option which lets you go back to the state where the system settings decides which theme to take.
I was wondering if this could be considered in one of your animations. My idea would be to have either a cog icon or a state of the icon that shows both at once, like sun and moon at the same time.
If that is too far out for you, it’s fine with me 😊.
Have a nice day!

Looks to be a duplicate of #18 (I visited issues section for that feature too!)

Yes, indeed. I did not identify the other to deal with the same topic.


USHIFT-315: Enable crio by default in the development environment

Closes USHIFT-315

Updates look good, thank you.
/lgtm