Green image on 1660 Ti even with full precision

Describe your environment

GPU: 1660 Ti
VRAM: 6 GB
CPU arch: x86
OS: Windows
Python: miniconda
Branch: main

Describe the bug
Image is flat green even with --full_precision

Loading model from models/ldm/stable-diffusion-v1/model.ckpt
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Using slower but more accurate full-precision math (--full_precision)
Model loaded in 30.19s
Max VRAM used to load the model: 4.28G
Current VRAM usage:4.28G
Setting Sampler to k_lms

Expected behavior
Image is not green

This is a recurring problem involving use of mixed precision, and I'm sorry it's come back.
We'll get this figured out. In the meantime is there another fork of Stable Diffusion that works reliably in your hands?

This is a recurring problem involving use of mixed precision, and I'm sorry it's come back.
We'll get this figured out. In the meantime is there another fork of Stable Diffusion that works reliably in your hands?

i have the 1660 super and i can only run the version from basujindal that actualy can create images that are not green in full precision mode hope that helps

That's a good data point. Thanks!

The problem here is there's no good way to configure both autocast and precision correctly at the same time. Currently autocast is picked only depending on device type (cuda/cpu/mps) in ldm/dream/devices.py. Precision is only "--full-precision" (i.e. the default torch.float32) or nothing.
@Pheenx and @ArDiouscuros try checking out the latest development branch, opening ldm/dream/devices.py and changing return device_type,autocast to return device_type,nullcontext.
Finally, I propose removing the full-precision flag and replace it with one "precision" flag with following options and behavior:
-default/unspecified: automatically pick float32/float16 as per device type (cuda/cpu/mps)
-float32: no autocast, no model.half()
-autocast: mixed precision allowed, no model.half(), as per the intentions of https://pytorch.org/docs/stable/amp.html
-float16: no autocast, model.to(torch.float16)
-bfloat16: no autocast, model.to(torch.bfloat16)

The problem here is there's no good way to configure both autocast and precision correctly at the same time. Currently autocast is picked only depending on device type (cuda/cpu/mps) in ldm/dream/devices.py. Precision is only "--full-precision" (i.e. the default torch.float32) or nothing. @Pheenx and @ArDiouscuros try checking out the latest development branch, opening ldm/dream/devices.py and changing return device_type,autocast to return device_type,nullcontext.
Finally, I propose removing the full-precision flag and replace it with one "precision" flag with following options and behavior: -default/unspecified: automatically pick float32/float16 as per device type (cuda/cpu/mps) -float32: no autocast, no model.half() call -autocast: mixed precision allowed, no model.half() call (trying to match the intended usecase of https://pytorch.org/docs/stable/amp.html here) -float16: no autocast, model.to(torch.float16) -bfloat16: no autocast, model.to(torch.bfloat16)

omg thank you so much!!! That actualy worked with the --full_precision tag im so happy thank you

Try the latest development branch. I believe that the embiggen support wasn't fully merged, but just tested it and it seems fine now.

The problem here is there's no good way to configure both autocast and precision correctly at the same time. Currently autocast is picked only depending on device type (cuda/cpu/mps) in ldm/dream/devices.py. Precision is only "--full-precision" (i.e. the default torch.float32) or nothing. @Pheenx and @ArDiouscuros try checking out the latest development branch, opening ldm/dream/devices.py and changing return device_type,autocast to return device_type,nullcontext.
Finally, I propose removing the full-precision flag and replace it with one "precision" flag with following options and behavior: -default/unspecified: automatically pick float32/float16 as per device type (cuda/cpu/mps) -float32: no autocast, no model.half() call -autocast: mixed precision allowed, no model.half() call (trying to match the intended usecase of https://pytorch.org/docs/stable/amp.html here) -float16: no autocast, model.to(torch.float16) -bfloat16: no autocast, model.to(torch.bfloat16)

This proposal looks very good to me. Can we set the precision and context in just one place where the model loads, or will the casing logic need to be scattered around various modules?
By the way, earlier this evening I tried to generate some images on a CPU-only Intel Xeon HPC node using --full_precision. It was slow compared to CUDA (8s/iteration), but it did work. Nice job on catching and fixing all the places where the precision was messed up. Half precision, by the way, gave me a runtime error of "LayerNormKernelImpl" not implemented for 'Half', which seems like a reasonable error message.

This is a recurring problem involving use of mixed precision, and I'm sorry it's come back.
We'll get this figured out. In the meantime is there another fork of Stable Diffusion that works reliably in your hands?

Yes, I am using basujindal optimized repo, which with the latest changes allows me to generate 512x512 images in full precision mode, and it is also very quick in my opinion. I can literally hear that my card is working hard,  by the increased fan noise. =)

The problem here is there's no good way to configure both autocast and precision correctly at the same time. Currently autocast is picked only depending on device type (cuda/cpu/mps) in ldm/dream/devices.py. Precision is only "--full-precision" (i.e. the default torch.float32) or nothing.
@Pheenx and @ArDiouscuros try checking out the latest development branch, opening ldm/dream/devices.py and changing return device_type,autocast to return device_type,nullcontext.
Thank you, change allowed me to generate image up to 384x384 and it is not green!

The problem here is there's no good way to configure both autocast and precision correctly at the same time. Currently autocast is picked only depending on device type (cuda/cpu/mps) in ldm/dream/devices.py. Precision is only "--full-precision" (i.e. the default torch.float32) or nothing. @Pheenx and @ArDiouscuros try checking out the latest development branch, opening ldm/dream/devices.py and changing return device_type,autocast to return device_type,nullcontext.
Finally, I propose removing the full-precision flag and replace it with one "precision" flag with following options and behavior: -default/unspecified: automatically pick float32/float16 as per device type (cuda/cpu/mps) -float32: no autocast, no model.half() call -autocast: mixed precision allowed, no model.half() call (trying to match the intended usecase of https://pytorch.org/docs/stable/amp.html here) -float16: no autocast, model.to(torch.float16) -bfloat16: no autocast, model.to(torch.bfloat16)

@mh-dm, just to confirm, are you working on implementing this solution?

I haven't started yet (I will make a note here if I do). I'm looking into #508 first esp. since I think the training flow may be relevant to what precision options we should have.

Could it be related to the that issue?
https://github.com/pytorch/pytorch/issues/77955

I have started on switching from full_precision to precision

@wzgdesigntech on GTX 1650 is experiencing the same issue. However, since they have 4GB they can't use full precision. But more importantly @wzgdesigntech demonstrated that it's possible to use a float16 model and still get good output. The slow but working version of stable-diffusion used was:
https://github.com/basujindal/stable-diffusion/blob/main/optimizedSD/optimized_txt2img.py
This means that there's a change somewhere in lstein/stable-diffusion in that results in green images. Likely candidates are changes to
https://github.com/lstein/stable-diffusion/blob/development/ldm/modules/diffusionmodules/model.py
https://github.com/lstein/stable-diffusion/blob/development/ldm/modules/attention.py
@ArDiouscuros Pulling you into this as well since if the issue is found and you can use a float16 model then you'll very like be able to generate larger than 384x384 images. Btw, what's the max size images you can currently generate in the development branch?
As for next steps I'll soon post a branch with some changes where you can test if you still get green images.

I've created a new github branch for trying out potential fixes / tracking down the cause. You can obtain it through:
git fetch origin green-images-fix:green-images-fix
git checkout green-images-fix

To test out:
scripts/dream.py --precision=float16
[...initialization text..]
dream> brown corgi, clean background, flat design -s20 -Addim -S1

You should get an image like this:

@ArDiouscuros since you have more VRAM you can also try full precision/float32 but with autocast enabled:
scripts/dream.py --precision=autocast

@mh-dm Thanks for taking the lead on this!

@wzgdesigntech on GTX 1650 is experiencing the same issue. However, since they have 4GB they can't use full precision. But more importantly @wzgdesigntech demonstrated in discussion #584 that it's possible to use a float16 model and still get good output. The slow but working version of stable-diffusion used was: https://github.com/basujindal/stable-diffusion/blob/main/optimizedSD/optimized_txt2img.py

@mh-dm
I have my doubts. I am using the same repo basujindal and optimized_txt2img.py
If I call it like this:
python optimizedSD/optimized_txt2img.py --prompt "brown corgi, clean background, flat design" --H 512 --W 512 --seed 42 --n_iter 1 --n_samples 1 --ddim_steps 20 --precision full --turbo --unet_bs 1
I will get a picture:

If I omit full precision:

@ArDiouscuros Pulling you into this as well since if the issue is found and you can use a float16 model then you'll very like be able to generate larger than 384x384 images. Btw, what's the max size images you can currently generate in the development branch? (asking cause there's been recent optimizations)?
Was not able to go higher than 384, if it is 64 pixels more I get exception after the sampling is complete.

As for next steps I'll soon post a branch with some changes where you can test if you still get green images.
I am interested very much to test it...

Have you tried the green-images-fix branch? My last post had instructions.

Also now I notice that @wzgdesigntech was passing --precision full to optimized_txt2img.py. https://github.com/lstein/stable-diffusion/discussions/584#discussioncomment-3678914
Finally took a closer look into the script. I thought the way it still worked on 4GB was that they called model.half() but it turns out they split the model and computation into parts so only part of the model has to be in VRAM at a time. That's a much more involved change to do esp. since we don't want to slow anyone else down.
I also took a quick at pytorch/pytorch#77955 . Could be related but doesn't really give me any further leads.

scripts/dream.py --precision=float16

Could not get it working...
(ldm) G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion>python scripts/dream.py --precision=float16

Initializing, be patient...

Traceback (most recent call last):
File "G:\Programs\SDAI\StableDiffusionManual\conda\envs\ldm\lib\site-packages\skimage_init_.py", line 151, in
from ._shared import geometry
ImportError: cannot import name 'geometry' from 'skimage.shared' (G:\Programs\SDAI\StableDiffusionManual\conda\envs\ldm\lib\site-packages\skimage_shared_init.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion\scripts\dream.py", line 356, in
main()
File "G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion\scripts\dream.py", line 38, in main
from ldm.generate import Generate
File "g:\programs\sdai\stablediffusionmanual\repo\lstein\stable-diffusion\ldm\generate.py", line 19, in
import skimage
File "G:\Programs\SDAI\StableDiffusionManual\conda\envs\ldm\lib\site-packages\skimage_init_.py", line 154, in
raise_build_error(e)
File "G:\Programs\SDAI\StableDiffusionManual\conda\envs\ldm\lib\site-packages\skimage_init.py", line 131, in _raise_build_error
raise ImportError("""%s
ImportError: cannot import name 'geometry' from 'skimage.shared' (G:\Programs\SDAI\StableDiffusionManual\conda\envs\ldm
\lib\site-packages\skimage_shared_init.py)
It seems that scikit-image has not been built correctly.
Your install of scikit-image appears to be broken.
Try re-installing the package following the instructions at:
https://scikit-image.org/docs/stable/install.html

That's from a new dependency for development which you can satisfy with pip install scikit-image

@mh-dm

I've created a new github branch for trying out potential fixes / tracking down the cause.

Testing new branch, I like how it works without any additional tuning. But lower precision modes still not work for me.
So if I just run python scripts\dream.py I am able to get correct image:
(ldm) G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion>python scripts\dream.py
* Initializing, be patient...
Using more accurate float32 precision
>> Model loaded in 29.74s
>> Max VRAM used to load the model: 4.28G
>> Current VRAM usage:4.28G

dream> brown corgi, clean background, flat design -s20 -W512 -H512 -C7.5 -Addim -S1
>> Setting Sampler to ddim
>> Usage stats:
>>   1 image(s) generated in 57.73s
>>   Max VRAM used for this generation: 5.17G. Current VRAM utilization: 4.36G
>>   Max VRAM used since script start:  5.17G

(ldm) G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion>python scripts/dream.py --precision=float16
* Initializing, be patient...
Using faster float16 precision
>> Model loaded in 20.29s
>> Max VRAM used to load the model: 2.17G
>> Current VRAM usage:2.17G
dream> brown corgi, clean background, flat design -s20 -W512 -H512 -C7.5 -Addim -S1
>> Usage stats:
>>   1 image(s) generated in 52.57s
>>   Max VRAM used for this generation: 3.35G. Current VRAM utilization: 2.21G
>>   Max VRAM used since script start:  3.35G

Result is total black
(ldm) G:\Programs\SDAI\StableDiffusionManual\repo\lstein\stable-diffusion>python scripts/dream.py --precision=autocast
* Initializing, be patient...
Using more accurate float32 precision
>> Model loaded in 16.59s
>> Max VRAM used to load the model: 4.28G
>> Current VRAM usage:4.28G
dream> brown corgi, clean background, flat design -s20 -W512 -H512 -C7.5 -Addim -S1
>> Usage stats:
>>   1 image(s) generated in 60.59s
>>   Max VRAM used for this generation: 5.04G. Current VRAM utilization: 4.36G
>>   Max VRAM used since script start:  5.04G

Same black picture.
Comparing with other repo: it runs faster despite the fact it splits model into 3 parts:
(ldo) G:\Programs\SDAI\StableDiffusionManual\repo\basujindal\stable-diffusion>python optimizedSD/optimized_txt2img.py --
prompt "brown corgi, clean background, flat design" --H 512 --W 512 --seed 1 --n_iter 1 --n_samples 1 --ddim_steps 20 --
precision full --turbo --unet_bs 1
Running PLMS Sampling with 20 timesteps
memory_final =  3.628544
Samples finished in 0.65 minutes and exported to outputs/txt2img-samples\brown_corgi,_clean_background,_flat_design

But results are not identical:

For the last command, results are not identical because it looks like optimized_txt2img.py is using (defaulting to) PLMS Sampling. So it's fine/expected since the dream.py command I showed was with -Addim, i.e. using the DDIM sampler.
In terms of ways forward, here's something to try. Following the link you posted there's this very relevant and fixed issue in pytorch https://github.com/pytorch/pytorch/issues/58123 and the fix landed in cudnn8.2.2. However, if following the standard install you're likely to end up with cudnn8.2.0. You can verify the version you have with:
>>> import torch
>>> torch.backends.cudnn.version()
8200

I would say to try updating pytorch but with a note it's possible to break things. Here's a command to try:
conda install pytorch cudatoolkit=11.3 -c pytorc

Make note of what versions it updates from so you can more easily undo. If updating doesn't help you can also try updating cudatoolkit. Potentially useful commands: conda search pytorch::pytorch and conda search cudatoolkit to show you what versions you can try but also note that the pytorch build (like py3.9_cuda11.3_cudnn8.3.2_0) must match your python and the cudatoolkit version). The default versions are here: https://github.com/invoke-ai/InvokeAI/blob/development/environment.yaml

@mh-dm
Thanks for the info about the ways forward, I have already started following:

Install updated version of pytorch with cuda toolkit

pip install torch==1.12.0+cu116 torchvision==0.13.0+cu116 --extra-index-url https://download.pytorch.org/whl/cu116

Result is - image is not green, it is black, for all prompts and settings
2. Reset all to defaults and manually downloaded cudnn files and replace in ..\conda\envs\ldm\Lib\site-packages\torch\lib, tested following versions
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>

Libraries are from NVIDIA https://developer.nvidia.com/cudnn
With current versions:
# Name                   Version       Build  Channel
pytorch                  1.11.0          py3.8_cuda11.3_cudnn8_0    pytorch
cudatoolkit              11.3.1         h59b6b97_2
>>> import torch
>>> torch.backends.cudnn.version()
8500

The same black picture result
4. Downgraded torch to 1.10.1+cu102 - stable diffusion could not start
But as image is not green anymore, and NaN tests are passing if I replace cudnn to versions > 8.2 I feel that I am on correct way but something is still missing...
Any ideas?

You mentioned before that you're getting black output in the green-images-fix branch. Is the green-to-black change happening on latest development (where you can now use --precision) with just the cudnn update?

Is the green-to-black change happening on latest development (where you can now use --precision) with just the cudnn update?

Yes, I had run following test with cudnn 8200 and 8500:
development branch, float16, 8200, prompt test image -s1 -S1 => green image
development branch, float32, 8200, prompt test image -s1 -S1 => crash on save
development branch, float32, 8200, free gpu mem, prompt test image -s1 -S1 => normal image
development branch, float16, 8500, prompt test image -s1 -S1 => black image
development branch, float32, 8500, prompt test image -s1 -S1 => normal image
development branch, float32, 8500, free gpu mem, prompt test image -s1 -S1 => normal image

Interesting. Couple things to try before I'm out of ideas:

--precision=autocast at development head
optimizedSD/optimized_txt2img.py with the updated cudnn but without --precision=full

--precision=autocast for me is the same as float32. It crashes 2 times out of 3 after sampling, 1 time produces normal image
with optimized repo I have got interesting results:
cudnn 8500, no full precision, 50 steps  => black image in 2.75 minutes
cudnn 8500, full precision, 50 steps       => normal image in 0.95 minutes
float32 is more then twice fast in optimized repo
I checked on development branch
cudnn 8500, float32, 50 steps => crash  - 0 image(s) generated in 189.93s
cudnn 8500, float32, 50 steps => normal 1 image in 192.18s
cudnn 8500, float16, 50 steps => normal image in 127.70s
Soo... looks like 16xx cards are broken?
e.g. see the same perf issue here: https://github.com/tensorflow/tensorflow/issues/44191
https://stackoverflow.com/questions/69188051/pytorchs-nn-conv2d-with-half-precision-fp16-is-slower-than-fp32
I would really like to get optimized repo speed + invoke-ai features )

"cudnn 8500, float16, 50 steps => normal image in 127.70s"
Did you mean black image? (To match the results you reported before)
Do you get a stack trace for the crashes?

Did you mean black image? (To match the results you reported before)
Opps, copy paste error, it was black.

Stack trace
Traceback (most recent call last):
  File "\stable-diffusion\ldm\generate.py", line 411, in prompt2image
    results = generator.generate(
  File "\stable-diffusion\ldm\dream\generator\base.py", line 72, in generate
    image = make_image(x_T)
  File "\conda\envs\ldm\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "\stable-diffusion\ldm\dream\generator\txt2img.py", line 50, in make_image
    return self.sample_to_image(samples)
  File "\stable-diffusion\ldm\dream\generator\base.py", line 84, in sample_to_image
    x_samples = self.model.decode_first_stage(samples)
  File "\conda\envs\ldm\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "\stable-diffusion\ldm\models\diffusion\ddpm.py", line 1117, in decode_first_stage
    return self.first_stage_model.decode(z)
  File "\stable-diffusion\ldm\models\autoencoder.py", line 421, in decode
    dec = self.decoder(z)
  File "\conda\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "\stable-diffusion\ldm\modules\diffusionmodules\model.py", line 599, in forward
    h = self.up[i_level].block[i_block](h, temb)
  File "\conda\envs\ldm\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "\stable-diffusion\ldm\modules\diffusionmodules\model.py", line 121, in forward
    h = silu(h)
  File "\conda\envs\ldm\lib\site-packages\torch\nn\functional.py", line 2032, in silu
    return torch._C._nn.silu(input)
RuntimeError: CUDA out of memory. Tried to allocate 256.00 MiB (GPU 0; 6.00 GiB total capacity; 4.57 GiB already allocated; 133.20 MiB free;
 4.71 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See
 documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Just have got some success in generating image on GTX 1660 Ti + float16 + cudnn 8.5
Standard installation of InvokeAI
Download cudnn libraries from NVIDIA site
Replace existing cudnn files in conda\envs\ldm\Lib\site-packages\torch\lib
Add missing dependency zlibwapi.dll
After that we get black images instead of green
Add somewhere in init scripts (e.g. after model is loaded):
torch.backends.cudnn.benchmark = True
After that we get normal images in float16 mode.
But, performance of sampling in this mode is 2 - 3 times slower for GTX 16xx cards.

Tonight I completely failed to get today's 2.0 release running on a 1650ti (has 4GB).
I tried using the AUTOMATIC1111 fork since it says it has support for 4GB and even 3GB VRAM GPUs. All of the default runs worked fine in terms of not generating errors, but the resulting images were all black. After looking more closely at how they implemented their medvram and lowvram support (https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/122d42687b97ec4df4c2a8c335d2de385cd1f1a1/modules/lowvram.py), I was able to find a set of command line parameters that worked for me:
--lowvram --precision full --no-half --always-batch-cond-uncond --unload-gfpgan --opt-split-attention

I haven't looked to see how much of what they do to manage models between GPU and CPU, but their approach might be useful to find a path forward.

Tonight I completely failed to get today's 2.0 release running on a 1650ti (has 4GB).

Could you be more specific please? Did you get OOM, other crash, or image was just green/black?
Did you try to use method described in previous message and to put updated cudnn libs to your installation?

Yes. I got OOM errors. I don't have the device in front of me at the moment, but it would fail on allocating a 20MB segment, give me a report out of the VRAM used (typically 3.43GB of 4.0GB) and suggested that I set PYTORCH_CUDA_ALLOC_CONF to something like max_split_size_mb:128 and garbage_collection_threshold:0.6. The garbage_collection_threshold option isn't available in pytorch 1.11, so I had to upgrade to 1.12.1, resolve the conflicts. I didn't try the optimized repo from https://github.com/basujindal/stable-diffusion/, but instead switched to the AUTOMATIC1111 fork.
I was contemplating trying to bring the work done in that fork back to the InvokeAI fork when I found this thread, and figured I would lend my experience back. I did (originally) get just black images from the AUTOMATIC1111 fork, until I altered the command line arguments around precision, lowvram, and the other flags. I didn't experiment with precision autocast or whether I could get away w/o unloading gfpgan or split-attention options as I had been working to get something working for a few hours.
If InvokeAI has a path forward, I won't bother to look more deeply or try to port over their work.


chore: merge in template changes

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.48%. Comparing base (40d1081) to head (4168418).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #305      +/-   ##
==========================================
- Coverage   92.15%   90.48%   -1.68%
==========================================
  Files         344      344
  Lines        5165     5128      -37
  Branches      806      745      -61
==========================================
- Hits         4760     4640     -120
- Misses        400      473      +73
- Partials        5       15      +10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


FR: Menu item to invoke OCR on clipboard content (without enabling full Capture access)

For privacy reasons, it seems best if I copied content to the clipboard myself instead of granting access to this (or any other) app to perform screen recording.
Although I could set a global hotkey to quickly have the clipboard content OCRd, there appears to be no menu item to do exactly that.
If there’s a concern about too many menu items, perhaps the existing Capture menu item could first check if there’s an image on the clipboard, and if (and only if) not, proceed to record the screen.
Is this on the roadmap? If not, would you be willing to consider? Thank you for this awesome tool!

I like the idea, should be easy to implement 👍
Another option that may work for you in the meanwhile, TRex has System Services menu integration:

Awesome, thanks for the quick response!
The Services menu would work well for files, but I am still thinking of screenshots where the content likely comes from a portion of a web page or a (web) doc that’s not a local file. Having to save it to a file first would be a pain.

Thanks for the quick response, I appreciate it! Bought via the Mac App Store in appreciation!


[WIP] Move PathSupportsOverlay() to cluster.Start()

Another leftover which should go into the cluster package.
Also, remove an old TODO, fix an error check plus some minor cleanup.

@schu can we remove the branch here and in #257?


Fix createDirectoryMarkerIfNotExists

Closes #7509

I don't understand the purpose of this change. You still rely on ifNoneMatch("*") since 2 workers may list 0 items and then try to create it at the same time. Why are we bothered by the current implementation?

The fix insure we do not create a directory marker if the directory is not empty - there's no attempt to solve an issue of multiple writers


Added a friend page.

Your pull request seems good, but must be checked by a human maintainer for the following reason(s):

This contains potentially hazardous tags that need manual review.

Please be patient until a maintainer has time to look over your contribution.
This is an automated response, beep boop.
if you think there is a mistake, contact a maintainer

02d5503effa6faeee96fd1708be716f595f8121d

Cute dog, looks good. Thanks!


ISPN-13197 developer guide content re-org

https://issues.redhat.com/browse/ISPN-13197

Personally, I find the developer's guide more confusing. It has a copy of everything in random order. The advantage: I have a single page to search for everything...

Yes it's hard to get the right order but I started with the cache configuration. I think where it gets confusing is the separation between remote and embedded. Personally I think we're always going to have this with any "all-in-one" doc but, taking the hybrid approach with the docs, we will have better separation of remote/embedded in the different guides.
My idea with this PR is to make progress on breaking out the rest of the developer's guide as it exists today into more clearly defined assemblies. This is work that I've been trying to get to and is overdue. But, after that, do we kill off the developer's guide completely? There have been numerous calls for an "all-in-one" so my thinking was that this might address that need (also downstream where we don't even have embedded google search).
 I did recently read that repurposing is often better than killing off a doc in terms of SEO. If we totally drop the dev guide then we'll get 404s from the version drop-down.
I also plan to remove the developer guide link from infinispan.org/documentation so there is no direct entry point on the home page.

After further consideration I really think it is the wrong approach to go back to trying to have an "all-in-one" guide. There's just no clean way to do that. I'm going to stick with what @karesti has suggested in the originating Jira. Adding the preview label until I get the embedded vs Java client content clearly mapped out.

Also, deleting the developer guide html from the build will result in 404 so I propose that we keep the title index there and insert a link back to the index.

Thanks @oraNod


allow searching css independent of config path

This PR aims to allow explicitly overriding config in  one xdg path while keeping the css in another. This is useful to e.g. have an explicit style in ~/.config/rmenu/style.css while keeping the default config that might be in /run/current-system/sw/etc/xdg/rmenu/config.yaml (in the case of nixos; other packaging solutions might put the default config into different locations).
To achieve this, find_config has been refactored into find_xdg_file which searches all xdg config dirs based on the prefix constant XDG_PREFIX and returns an optional string of the extended path. This is then used in both get_config and get_theme to find the specific files respectively.

Looks great! Thanks again for your help :)


Consider uploading some quantized checkpoints to hugginface

Correct me if I'm wrong but quantizing would require loading the models in their unquantized form (as per torch.load in https://github.com/saharNooby/rwkv.cpp/blob/master/rwkv/convert_pytorch_to_ggml.py, line 126). Not to mention how much heavier the unquantized models are on bandwidths.

Only PyTorch -> rwkv.cpp conversion would require to load the whole model in the RAM; quantization is done tensor-by-tensor. You are right about the bandwidth tho.
I'll consider it, thanks for the suggestion!

I have uploaded some quantized RWKV-4-Raven models to HuggingFace at LoganDark/rwkv-4-raven-ggml. Conversion took about 2 hours, and upload took about 24 hours and 500GB of disk space.
At the time of writing, the available models are:

Name
f32
f16
Q4_0
Q4_1
Q4_2
Q5_1
Q8_0

RWKV-4-Raven-1B5-v11-Eng99-20230425-ctx4096
Yes
Yes
Yes
No
Yes
Yes
Yes

RWKV-4-Raven-3B-v11-Eng99-20230425-ctx4096
Yes
Yes
Yes
No
Yes
Yes
Yes

RWKV-4-Raven-7B-v11x-Eng99-20230429-ctx8192
Yes
Yes
Yes
No
Yes
Yes
Yes

RWKV-4-Raven-14B-v11x-Eng99-20230501-ctx8192
Split
Yes
Yes
No
Yes
Yes
Yes

Feel free to create a discussion if you have a request.


LESS files are not rebuilt on saved changes

Renderer
v3
Browser
Chrome
Operating System
MacOS
What happened?
When making changes to LESS files in the repo, and saving the file, the server is reloaded but the /build/ files are not updated.  The server has to be stopped and go through the build process again before the changes get pushed through.
I can't find the issue in buildHomebrew.js after looking through Nodemon and Vitreum docs/readme.  I've run npm install and npm update.
At this point I'd like to just confirm that others have this issue?  I got a new mac with silicon chip and while i've assumed that isn't the issue (I have no idea why it would be) I guess it'd just feel good to confirm that, too.
This will basically be enough for me to avoid trying to create themes in the meantime (not that I was really laying them on thick to begin with...)
Code
No response

I'm getting a rebuild (on Windows 10 x64) when making changes to /shared/naturalcrit/nav/nav.less (specifically, commenting out the nav .navDropdownContainer .navDropdown .navItem:vertical-align property, as VSCode flags that property as ignored due to display setting).
Commented:

Uncommented:

The styles.less files in themes

Ok, I haven't touched this for a while (and it's probably due for a re-factor), but looks like we may need to again tweak this script, because we actually have 3 interacting pieces here that I didn't fully understand:

Nodemon - This is the watchFile function at the bottom. Monitors those directories for changes, and if found, it will restart the server.
LiveReload - This is the livereload function at the bottom. Monitors that directory for changes, and if found, it interacts with an old Chrome Extension to refresh the browser page. This extension doesn't work anymore, so this line is useless.
Watchify - This is the dev : isDev && build, line, and is a Browserify plugin (our code bundler). This monitors any source file mentioned in paths: or included by the first argument to pack ('./client/homebrew/homebrew.jsx'). When it detects a change, it re-bundles './client/homebrew/homebrew.jsx', then runs sends those bundles to the function listed in the dev :  property, which is build().

So, build, we rebuild most .less files if they are associated with the larger bundle.css file (nav.less is part of that bundle because it gets include'd by something else in the bundle). But all the separate files (themes CSS, etc.) we are handling outside of the build() function. Sooo.... it's probably just a matter of pulling those steps (everything under the compile themes comment) out of their anonymous async function and inserting them into build() at the right place (probably just at the end).
Also note, as for LiveReload not working anymore, there is a very similar Chrome extension at Live Reload which actually works (I tested it), but instead of connecting to your server to detect changes, you just give it a path to the files you want it to monitor. In this case, just set up the extension to look for all source files:

Okay, that pretty much confirms what I had figured out last night, but I wasn’t sure enough to say “pretty sure this stuff is wack right now”. I sort of assumed I was not really understanding it at all.

OK, I think I understand how it works now. Long story short, if we shuffle lines 48 through 100 of the current script up a few lines so that they're inside the build function, then they will get executed every time a change is detected.
Unrelated to this particular change, I suspect that bringing the building of the /admin pages into the main buildHomebrew.js script would probably need to happen in a similar fashion.


Pass arguments to select_related calls to UserProfile objects.

First three commits are to remove select_related calls for Client and UserProfile objects where no foreign keys are used.
Next eight commit are to pass "realm" as argument in different places where we query UserProfile objects.
Next two commits are to pass "realm" and "bot_owner" arguments to select_related calls in different functions defined in models.py which are used to query user profile by email, id or api-key and while populating cache.
Last commit is to update the code to directly use ID of foreign key fields instead of full objects at some places.

Fixes:  https://chat.zulip.org/#narrow/stream/3-backend/topic/select_related.28.29.20with.20no.20args

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

I also noticed one more bug while working on this here where we do not want to send the PM if bot tries to send to a empty stream for system bots or cross-realm bots as per the comment but we do send them the message.
This is because the cross-realm bots did not have owner previously and this code was added considering that, but then we changed the code to set bot_owners for these bots to the bot themselves.

@sahil839 lgtm aside from this comment, the commit messages would ideally clarify that the concern is more about adding non-nullable foreign key fields to Realm in the future; but I think it's basically fine, going to go ahead and merge to keep momentum going.


Picture of rendering test cases completed vs to do

I tried to but didn't see a way to do it without having a URL to the image... maybe we can check this out Tuesday and see what the best way is to do this

testing to see if I can comment without closing pull request

You have to clone the wiki's git repo and commit/reference the image there
On Sat, Oct 20, 2018, 17:50 aw3227<EMAIL_ADDRESS>wrote:

testing to see if I can comment without closing pull request
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/borisv13/GitHub-Flavored-Markdown-Eclipse-Plugin/pull/84#issuecomment-431620715,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACwIdDGkWw43YnwIY3qDi2zbu_5ndqiOks5um5qkgaJpZM4XyIei
.

@aw3227 I added this to the wiki.


Enable src/node_modules

[HOLD] not particularly useful without changes in Jest
Partially fixes https://github.com/facebookincubator/create-react-app/issues/607
These are the necessary changes in react-scripts to allow code to live in src/node_modules and allow Jest to run tests in that directory.
However, Jest cannot find these tests on its own -- all files in any node_modules are filtered out even before the file matching patterns are applied. See issue https://github.com/facebook/jest/issues/2145.
Without changes in Jest, there are a few workarounds:
Import the package tests into a file that jest can find
// src/node_modules/Foo/Foo.test.js
it("does a thing", ()=> {
  expect(true).toBe(true)
})

// src/index.js
import "Foo/Foo.test.js"

Whitelist the directories you want to test with the haste.providesModuleNodeModules option
This is not the intended use for this option, but its used as a whitelist for the haste module  resolution, and therefore Jest can find tests in these directories.
TODO

[ ] get related issue in facebook/jest resolved
[ ] figure out how to test this
[ ] rebase, clean up etc

I also love putting application-wide packages in a node_modules directories but it does not seem standard and encouraged by the community which generally uses other solutions such as:

non standard bundler configuration
extraneous linking step (like lerna bootstrap)

I find it sad that we have to find other, more complex solution, while the existing resolution algorithm is enough :/

What is the resolution now that https://github.com/facebook/jest/issues/2145 is closed?

This is becoming sort critical because we want to remove NODE_PATH support due to issues like https://github.com/facebookincubator/create-react-app/issues/1023.

As far as I can see, the issue as been closed as wontfix, not resolved :/
I also created an issue for this (facebook/jest#1995) in which @cpojer answered me by saying that he would not oppose a fix but he does not have time to take care of it himself.
Unfortunately I do not have the time to it either.

I’m sure we can all persuade @cpojer. I can literally bug him all day until he agrees to merge a PR as he moved to the London office. But we need somebody willing to figure out the problem and the solution and submit said PR.

I'll do anything for you Dan but I may ask you for a favor in the future.

A day of work on a Jest issue of your choice?

I updated the PR with the haste: { providesModuleNodeModules: [".*"] } hack, which works, but generates a bunch of spurious error noise on the first test run, which seems like a non-starter.
Also, I still have no idea how i'm supposed to write an automated test for this, or if there even are tests in create-react-app (?!)

Also, I still have no idea how i'm supposed to write an automated test for this, or if there even are tests in create-react-app (?!)

No, we only do artisan tests. On a serious note, if Travis passes it means that a basic project runs webpack and jest correctly without crashing, but you can't automatically test a particular feature right now.

Hey @gaearon @cpojer @modernserf,
I've been working on a monorepo build tool that works with yarn and supports app-level modules under node_modules so I would be happy to help out with config settings or problem/solutions I've run in to while trying to integrate outside tools.
I've found most of the friction that comes with app-level node_modules is just the heavy handed approach people take to ignoring */node_modules/* rather than just /node_modules
https://github.com/knitjs/knit/tree/master/modules/node_modules/%40knit/jest-config-socks
https://github.com/knitjs/knit
https://github.com/boennemann/alle

To give a quick overview:

I didn't initially realize that this is a CRA issue. I'm happy to make Jest work well for use-cases that CRA supports (within reason, of course).
This is all done inside of jest-haste-map and it has 100 % test coverage inside of Jest. If you make a change there and add a good test, we'll be all good.
jest-haste-map does static analysis for Jest and supports FB's module system (this second part is irrelevant for this issue). It fetches a list of all files in the system and extracts metadata like dependencies (for reverse lookup during jest --watch so we only run the affected tests) and stores them in a cache.
It parses all JS files and package.json files except those in node_modules which we actually just skip and don't even put into the virtual file system. This means when Jest is trying to match the testRegex against the virtual file system (HasteFS), it can't find the tests. This is the source of our problems. There is an option retainAllFiles in jest-haste-map that keeps those files around, which may or may not work for this use case. I'm not entirely clear what the correct fix should look like, only that we should definitely support this use case. Please feel free to try some things and send a PR.
While you are changing Jest's haste map, always have npm run watch running in the Jest checkout and run Jest with --no-cache to make sure you are hitting the right code paths in the haste map.
source

Let me know if you need anything else.

@cpojer just got some time to work on this again. I have some changes in jest that work for create-react-app but have a failing test and I don't understand Jest well enough to know why. Can you help me understand?

How's the progress on this? I'd love to help because this is kind of a blocking issue for me now.

@rovansteen get the tests passing on this: https://github.com/modernserf/jest/commit/a687e26c05add184d118cbcb7933e592aadaffbd

@modernserf I think it's best to send a PR to start discussion for better visibility.

Can I make a general inquiry about what's going on here? The best I can tell is that @modernserf took this under his wing but due to the way haste-map is set up in jest this can't be done without some rather complex changes (the jest pr from @modernserf looks completely stonewalled).
I guess what I'm asking is, with all the work around this being prefixed with "Don't work on this, it's being taken care of" but looking like we're stuck until someone (??) fixes haste-map with a non-trivial feature for white-listing nothing will come of this.
There's been no activity for 22 days on any of the issues/pr's related to this. I'd love to try and help on the jest/haste-map side but being that it's new functionality required I don't think it's a great candidate for someone unfamiliar with the jest code base or not in-the-loop with their development goals.
How can I help? Or should I transition my projects off the src/node_modules path?

@Reanmachine the haste map thing will probably not be fixed per se -- I think the idea was that someone should add a testPathIgnorePatterns config option that actually works with node_modules as documented. That someone could be you!


UMD build doesn't work in a browser because of using the process variable

There is process.env.NODE_ENV expression in the bundle. It fails because the process variable is available in a Node environment but not in a browser.
Also the bundle requires the PropTypes package which is not a peer dependency and therefore should be included to the UMD bundle during bundling.
Here is a demo: https://codepen.io/TheFinesse/pen/aXxrWb?editors=1011
The latest version with a working UMD bundle is 0.9.0-beta 🙁

Should be fixed in the next release with this commit: https://github.com/clauderic/react-sortable-hoc/commit/16135dfe66604e6dabb570ecc1ef4a8758de7775

Please re-open this issue if you still see issues with this with version 1.8.2 and higher, going to close it for now as it should be fixed in those versions


Update autorest version to prerelease 3.0.0-alpha.20240426.4.x

Triggered from pull request: https://github.com/Azure/autorest.csharp/pull/4663

API change check
API changes are not detected in this pull request.

The changed files have nothing to do with mgmt changes in https://github.com/Azure/autorest.csharp/pull/4663


version: Fail gracefully if git describe cannot get version.

When running the ./tools/cache-zulip-git-version script on Travis, the script
fails because Travis gets a shallow clone of the repository, and not a full
clone. This commit changes the script to fail gracefully, if we are unable to
get the version information using git describe.

Awesome, merged, thanks @punchagan!


UI appearing randomly on landscape mode

The UI appears on my device randomly while playing videos on landscape mode, without touching the screen at all, even when the screen is locked (the lock icon appearing on the center of the screen). This happens only when I hold the device on my hands, not when it's completely still, like e.g. on a table. By the moment I move the device the UI appears. This doesn't happen on portrait mode.
Device:

Model: Nokia 6.2
OS: Android 11
Version: 0.137

The issue is not happening on my wife's Xiaomi Redmi 9C... 🤔

Sound like a faulty touchscreen.

Sound like a faulty touchscreen.

Faulty only with this app? 😁

I guess it's possible. As a video player, it may use the phone resources and draw power in a special way that triggers some (touchscreen) problem. Just Player already ignores taps near the edges of the screen to prevent accidental touches. Try enabling "Pointer location" in phone's "Developer options" to see if it's really something like this.

Unfortunately, "power location" doesn't show any "accidental" taps. So, that's not the case...

My newly purchased Lenovo P11 Pro Gen 2 tablet has the same problem as you. That proves that the player is having problems

By the way, Next Player , which is also Exoplayer based, doesn't have that bug on my device, so it's certainly a bug.

Are you able to replicate it? The UI is only triggered at media end, by touchscreen or any unknown key event. I Would you be able to capture a bug report (or logcat output only) if I provide a debug build?

Are you able to replicate it? The UI is only triggered at media end, by touchscreen or any unknown key event. I Would you be able to capture a bug report (or logcat output only) if I provide a debug build?

It's always replicated. Give a debug built, and I'll probably be able to send a report.

Try this version: Just.Player.v0.146-controller-debug.apk Does anything (what text?) pop up on the screen when the UI appears?

Try this version: Just.Player.v0.146-controller-debug.apk Does anything (what text?) pop up on the screen when the UI appears?

Yes.  On the contrary, when I do tap the screen the message "Tap" appears.

In this case this should fix it: Just.Player.v0.147.apk

In this case this should fix it: Just.Player.v0.147.apk

Yes! Bug killed! So, it actually was a bug.


beforeUnmount is called twice

Version
3.0.0
Reproduction link
Reproduction 1
https://codesandbox.io/s/vue-unmount-bug-hdi3o?file=/src/index.js
Reproduction 2 (using template, has bug)
https://codesandbox.io/s/vue-unmount-bug-2-vr290?file=/src/index.js
Reproduction 2 (using h, behaves correctly, it could be a bug of vue compiler)
https://codesandbox.io/s/vue-unmount-bug-render-1bhk1?file=/src/index.js
Reproduction 2.1 (a more compact reproduction of R2)
https://codesandbox.io/s/vue-unmount-bug-21-8jo3l
R1 and R2 may be caused by different reasons, please have a look.
Steps to reproduce
follow the link.
What is expected?
beforeUnmount is called once.
What is actually happening?
beforeUnmount is called twice.

It seems different child of a parent instance resolve to the same child instance. The reproduction condition is quite wired.
update: maybe it's related to createBlock.

Reproduction 2 is not fixed by https://github.com/vuejs/vue-next/pull/2173.
Reproduction 2.1 (a more compact reproduction of R2)
https://codesandbox.io/s/vue-unmount-bug-21-8jo3l

Yes, they caused by different reasons, I have fixed it in the latest commit.


Teams Live Event Report for Govt/DoD

The option for Teams Live Event usage report isn't available to govt/DoD customers. Is this something that will be available in the future?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d5372edb-6ffe-34a3-0839-587e48f66f6b
Version Independent ID: 43903b13-37de-7dde-5b8c-739586cc2d87
Content: Microsoft Teams live event usage report - Microsoft Teams
Content Source: Teams/teams-analytics-and-reports/teams-live-event-usage-report.md
Service: msteams
GitHub Login: @CarolynRowe
Microsoft Alias: crowe

Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


Permissions explorer

This PR enhances the wagtail admin UI to allow users (right now, only users who already have permission to view users and groups) to get a complete picture of what permissions another  user has, and how those permissions were granted (via group membership, or individually).
Additions

Removals

Changes

Testing

Screenshots
Notes

Todos

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] Any change in functionality is tested
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated
[ ] Reviewers requested with the Reviewer tool :arrow_right:

👍 convenient to see it all in one place. It's pretty easy to figure out permissions that are redundant (including many of mine!)

Do all new apps need to be added to this list? Is there anywhere else we have a list of our apps?

@chosak I don't think there are other such lists-- a grepped for 'transition_utilities' and didn't find any other lists of first-party apps.
I'll make sure permissions_viewer gets added to isort


MongoDB w/ Mongoose

Save note entries and have tag properties saved for reference purposes.
Save user login for authentication purposes.

Completed


discord.js v12

Change discord.js version to v12, because v11 seems outdated.

Updated in 29486c8


Module 1.1 Assignment

Given what you have learned about Git, why would you want to use Git in your classroom?

👋 Hi @jcervos can you please answer the question as well.


what are the steps to fix the issue

pcodeindia@Pcodes-MacBook-Air GoBackend % go run .
github.com/oasisprotocol/nexus/api/v1/types
api/v1/types/util.go:16:9: undefined: ConsensusEventType
api/v1/types/util.go:18:7: undefined: ConsensusEventTypeStakingTransfer
api/v1/types/util.go:19:3: undefined: ConsensusEventTypeStakingBurn
api/v1/types/util.go:20:3: undefined: ConsensusEventTypeStakingEscrowAdd
api/v1/types/util.go:21:3: undefined: ConsensusEventTypeStakingEscrowTake
api/v1/types/util.go:22:3: undefined: ConsensusEventTypeStakingEscrowDebondingStart
api/v1/types/util.go:23:3: undefined: ConsensusEventTypeStakingEscrowReclaim
api/v1/types/util.go:42:9: undefined: Layer
api/v1/types/util.go:51:9: undefined: Runtime
api/v1/types/util.go:61:10: undefined: GetLayerStatsActiveAccountsParams
api/v1/types/util.go:23:3: too many errors
pcodeindia@Pcodes-MacBook-Air GoBackend %
how to fix it

How To Fix the issue

You're missing the Go types that are autogenerated from the OpenAPI spec (v1.yaml).
See the Makefile. The best way to build locally is make nexus, as is also described in the top-level README.

I am a beginner for nexus and go lang, so please elaborate the step by step guide

run make nexus instead of go build .

pcode@Pcodes-MacBook-Air GoBackend % make nexus
fatal: No names found, cannot describe anything.
bash: oapi-codegen: command not found
ERROR: Installed oapi-codegen is not v1.12.x. See Makefile.
go: updates to go.mod needed; to update it:
go mod tidy
cannot locate oapi-codegen templates dir
make: *** [codegen-go] Error 1

oh shoot yeah, we don't have any docs about setting up the openapi tools


Update package

You need to update the package in npmjs.com, because the old version is installed from there.

Done


Send number of participants as FBP message

Should be done by monitoring participants on the msgflo discovery queue. Requires participants to send discovery messages periodically (#30). Coordinator should then send some FBP protocol messages out.
Maybe keep this as graph node metadata?

Has not been needed, closing until someone argues that it is


At first, didn't solve it but now openroad -gui works for me when mounting the image with --privileged switch. Thank you.

          At first, didn't solve it but now `openroad -gui` works for me when mounting the image with --privileged switch. Thank you.

Originally posted by @yunuseryilmaz18 in https://github.com/The-OpenROAD-Project/OpenLane/issues/1525#issuecomment-1367747166
can you expand the full command

Issue template


[Event Request] codeunit 311 "Item-Check Avail." - OnBeforeTransLineCheck

Can you add event OnBeforeTransLineCheck in codeunit 311 "Item-Check Avail."? Just like for SalesLineCheck
    procedure TransferLineCheck(TransLine: Record "Transfer Line") Rollback: Boolean
    // >>>>>>>>>>>>>>>>>
    var
        IsHandled: Boolean;
    // <<<<<<<<<<<<<<<<<
    begin
        // >>>>>>>>>>>>>>>>>
        IsHandled := false;
        OnBeforeTransLineCheck(TransLine, Rollback, IsHandled);
        if IsHandled then
            exit(Rollback);
        // <<<<<<<<<<<<<<<<<
        NotificationLifecycleMgt.RecallNotificationsForRecordWithAdditionalContext(
          TransLine.RecordId, GetItemAvailabilityNotificationId(), true);
        if TransferLineShowWarning(TransLine) then
            Rollback := ShowAndHandleAvailabilityPage(TransLine.RecordId);
    end;

    [IntegrationEvent(false, false)]
    local procedure OnBeforeTransferLineCheck(TransferLine: Record "Transfer Line"; var Rollback: Boolean; var IsHandled: Boolean)
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Analyzer to spot static abstracts in unsupported langversion

Describe the feature
Analyzer to spot static abstracts in unsupported langversion

It's probably clear to just let the compiler report on it


envoy: use errors.Is(..., net.ErrClosed) instead of string matching

Instead of comparing error strings use errors.Is to check whether err or
some error that it wraps is net.ErrClosed (available since Go 1.16, see
https://golang.org/pkg/net/#ErrClosed).
Same for syscall.EINVAL.
 I'll send a PR bumping the VMs.

test-me-please

test-me-please

test-me-please

All required tests passed. The only failing test is the checkpatch test in BPF checks which complains about the commit subject being too long: https://github.com/cilium/cilium/pull/15080/checks?check_run_id=2030318483
Marking as ready to merge.


Multiple Subscriptions are created as separate orders

If a customer has multiple subscriptions which are triggered on the same date, multiple orders are created.
This

invalid issue, #58 is the right one so I'm closing the current.


[DOCFIX] Update metrics system doc

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11494/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/11495/
Test PASSed.

LGTM


release responsibleai-text 0.2.1

Description
release responsibleai-text 0.2.1
Checklist

[ ] I have added screenshots above for all UI changes.
[ ] I have added e2e tests for all UI changes.
[ ] Documentation was updated if it was needed.

Codecov Report

Merging #2337 (eebec52) into main (97416ba) will decrease coverage by 4.50%.
Report is 1 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #2337      +/-   ##
==========================================
- Coverage   88.57%   84.07%   -4.50%
==========================================
  Files         134       12     -122
  Lines        7615      961    -6654
==========================================
- Hits         6745      808    -5937
+ Misses        870      153     -717

Flag
Coverage Δ

unittests
84.07% <100.00%> (-4.50%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed
Coverage Δ

responsibleai_text/responsibleai_text/version.py
100.00% <100.00%> (ø)

... and 122 files with indirect coverage changes


508: Error Suppression Check Box Labels

Labels for the select checkboxes are generic, all have an aria label that says 'select'

[ ] Add Record ID to check box label

Private Zenhub Image

Testing complete, chekboxes now have unique aria labels


Inconsistent custom fields - Rule Builder

As a result of the migration, we have transferred our own rules for shipping cost control from the old SW5 shop. Although we have selected  ‘Available in shopping cart’ in the relevant custom fields, these are not available for use in the Rule Builder.
!Bildschirmfoto 2024-11-20 um 12.01.34.png|width=1244,height=359,alt="Bildschirmfoto 2024-11-20 um 12.01.34.png"!

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-39711
[created from MIG-1089, comment 514616]


PHP7 & session_write_close issue

Version 1.1.9
Problem: Warning: session_write_close(): Failed to write session data (user). Please verify that the current setting of session.save_path is correct (/tmp)
Though snc-redis configured and runs flawlessly on php5.6.
Resolution: http://stackoverflow.com/questions/34117651/php7-symfony-2-8-failed-to-write-session-data
in short: Redis session storage write method should return true, like in master branch.

Fixed in 1.1.10.


Facebook Hashtag

Hi,
Having an issue with Facebook hashtags being pulled into the feed. Seems like @names work, but not Hashtags.  Seems like its the same case on the demo when using #elonmusk.
Some info:

If I use this I get posts from the @name value:

facebook:{      accounts: ['@teslamotors','#teslamotors'],

If I use this, I don't get anything at all.

facebook: {        accounts: ['#finishgreat','#teslamotors'],

If I use this, I don't get anything at all.

facebook: {          accounts: ['#finishgreat'],
Any idea how I can solve this?

Little late, but if I'm not mistaken, the Facebook API does not support posts based on hashtag anymore (source).

I confirm and it's no more available in social-feed doc.


Update default log level as "info" from debug

Issue/Feature Description:
Current default log level for gelato is debug.
Default log level for a service need to be "info".
Why this issue to fixed / feature is needed(give scenarios or use cases):

To avoid redundant info in log
Performance impact

How to reproduce, in case of a bug:
NA
Other Notes /  Environment Information: (Please give the env information, log link or any useful information for this issue)
NA

@kumarashit @rajat-soda @PravinRanjan10


Support building for Android on aarch64 macOS hosts

The NDK puts universal binaries under the darwin-x86_64 directory.
Verified that it works by patching Cargo.toml to use this branch while building on an M1 macbook.

The build error is a new clippy lint, fixed by https://github.com/EmbarkStudios/physx-rs/pull/217

@tgolsson Thanks, added a changelog entry now.


Better warning log on allocation failures.

Allocation failure is based on a threshold of available ram and not the total ram of the system.  Enhanced the log message to make that more obvious.

Perhaps "available"?

Actually "available" again may be misinterpreted, wondering if users will be confused between total available memory vs free memory?
'Free' explicitly states it is the 'free' memory that we are talking about.

OK free seems indeed better, since that's what sysinfo doc is saying (http://man7.org/linux/man-pages/man2/sysinfo.2.html)

Thank you!


Fix 'About' link in 'Chapters' and correct the position of 'Blogs' in it.

Sorry for creating a new issue. But I can't find who fixed the 'About' link in Chapters and also corrected the blogs position.

Yes @ishita80 , it was resolved. No worries, if there is any other enhancements in any section , pls go ahead :)


Problem with log in

Cannot log in into my account.
When I try to log in into my account with the automatic process ( button with Log In (Opens in browser) under Accounts under cog icon) firefox does not open twitch.tv anymore to log me in.
Screenshots
https://imgur.com/LN7jIlr.png
OS and Chatterino Version
GNU/Linux Ubuntu 21.04, Chatterino 2.3.4 nightly 9 October (same with regular version)

firefox does not open twitch.tv anymore to log me in.

If you mean that the button doesn't open https://chatterino.com/client_login/, then that's the link which you can go to manually.
If you actually meant the twitch part, and you aren't getting re-directed to twitch, I'm unsure the reason for that as I don't use Linux.

Chatterino 2.3.4 nightly 9 October

Now that I see this, that's likely to be the same issue as #3207.


[DEV] Upgrade Django to 3.0

Development Sprint for: ...

[ ] Data Model
[ ] UI
[ ] API
[ ] Functionality
[ ] Integration
[x] System
[ ] Other - [specify]

Description of this sprint tasks
Attempt to upgrade Django from 2.2.8 to 3.0.x
It seems like there are vulnerabilities in the current version of Django we are using. The requirements.txt file specifies django>=2.2.8,<3.0.0. The initial reason for not upgrading to 3.0 was due to some issues with either djangorestframework or django-filter not working correctly. These packages were trying to import modules that only existed in Django 2.0.
With recent updates to the above framework it may be worth upgrading Django and testing the current system
Expected Deliverable
Django 2.2.8 -> Django 3.0.x
How will this be tested?
Running the server to see if it starts
How will this be reviewed?
Link(s) to necessary documentation
https://docs.djangoproject.com/en/3.0/howto/upgrade-version/
Above link may not be necessary as we don't have a test suite yet. Could be part of another dev sprint
Other information

Updated master with Django 3.x


Routes work only once

Hi!
I am new to react and trying to figure out with this project, and stucked with a problem when add new route it can be viewed only once.
so added files and new route https://gist.github.com/hunterman/1f46436face225b25425 and then I can transit to any page only once after page loaded, after that page is not rerendering, but only pushState works.

Maybe because you only have name=... but don't have path=... attributes on the <Route> declarations.

@RickWong thank you for your answer, but as you can see here https://github.com/rackt/react-router/blob/master/examples/transitions/app.js there is no path also, and here https://github.com/rackt/react-router/blob/master/docs/api/components/Route.md here is saying that

The path used in the URL. If left undefined, the path will be defined by the name, and if there is no name, will default to /.

also, why does it work only once? after I open page in browser I can navigate to another page, but after I've navigated I can't make it again.
ps to be sure added paths to routes definition, and it works the same, so not solved the issue

Do you get any console errors (in browser or in your terminal)? I too find it strange that it works just once.

@RickWong no errors, made video https://dl.dropboxusercontent.com/u/19321079/routes.mov
About.jsx is simplest component as it can be
import React from 'react';

const About = React.createClass({
    render() {
        return (
            <div className="container">
                Hello World
            </div>
        );
    }
});

export default About;

Can you log the value that is passed to this.transitionTo(...) each time you click?

Closing issue because the problem isn't likely to be caused by the starterkit.


Ensure 'W' and 'A' update for hijacked peer at the same time leads to 'W'

Is your feature request related to a problem? Please describe.
There is the very rare case in some hijacked peers, when the hijack is withdrawn (route-wise), that the peer/monitor sees a final hijack update at exactly the same time as the actual withdrawal (very rare case).
Describe the solution you'd like
Make sure that in that case the withdrawal has precedence, and the update is effectively ignored.
Add a new test for this, and make sure it always succeeds.

Moving to next release, non-replicable (it could also be simply a bug on the monitor's side sending a 'W' and 'A' for the same prefix and peer at the same time, which does not make any sense.

Non replicable, closing for now (will re-open on a need-to basis). Does not affect deployments.


CI images need updated Mono

Currently, CI fails on all Mono legs because it's running on an old Mono. We need to move to machine images that have a newer one installed.
That might mean that we need to ask for such images to be created.
@mmitche This can wait until 2.1 RTM madness is over, but when you get a chance we'll probably need your help with getting updated Mono installed on CI images.

@radical, could mono be used in an app local fashion, without the need for a machine wide installation? Similar to how we get dotnet or repotoolset.


Add the prefix absorb_ to the names of the absorb axioms

Our goal is to add the prefix absorb_ to the names of the absorb axioms to make it clear that these axioms are about absorb and to make their naming consistent with the naming of the lift axioms and the core bisimilarities.

We are retiring the ♮-calculus in favor of the Þ-calculus.


Options for Graphviz-Formatter are being ignored

I want to omit generating a dependency graph image by running deptrac with the following options:
php deptrac.phar --formatter-graphviz=0 --formatter-graphviz-display=0

but it will still open image previewer with the dependency graph.

this is symfony related, you cant pass arguments to the default command. try php deptrac.phar analyze depfile.yml --formatter-graphviz=0 --formatter-graphviz-display=0
would be awesome if you could add a hint to the docs :)


fix: Resets the current verb after the action has finished

Simplemouse only sets the verb if it is changed and thus it is set to nothing when an action finished and resets the verb

Very nice fix!


Using a memmap for the dictionary

Is your feature request related to a problem? Please describe.
It's harder to support lower-end hardware (with limited memory) particularly with bigger dictionaries.
Describe the solution you'd like
I would like the option to be able to use a memory map instead to refer to an uncompressed dictionary since storage is usually cheaper than memory. The application I need does not need extreme performance so I feel like the IO penalty would be acceptable. If the dictionary gets processed by Vibrato into something else, it would also be nice to be able serialize it to a file and memmap it as well. fst offers something like that: https://docs.rs/fst/latest/fst/#example-stream-to-a-file-and-memory-map-it-for-searching
Describe alternatives you've considered
None that I'm aware of.
Additional context
None

Actually, this might not make sense since it's a niche requirement. I'll explore it in my own fork though.

@kirawi I think the Vaporetto tokenizer is a better choice for small devices.
Lattice based tokenizers (including Vibrato) require large dictionaries, while pointwise tokenizers (including Vaporetto) work with smaller models.
There is an example that works on STM32F3DISCOVERY.
https://github.com/daac-tools/vaporetto/tree/main/examples/embedded_device


Foreign Key Filtering/Retrieval

Given the following schema
// todo.go
type Todo struct {
	ent.Schema
}

func (Todo) Fields() []ent.Field {
	return []ent.Field{
		field.Text("text"),
		field.Bool("done").Default(false),
	}
}

func (Todo) Edges() []ent.Edge {
	return []ent.Edge{
		edge.From("user", User.Type).
			Ref("todos").
			Unique(),
	}
}

// user.go
type User struct {
	ent.Schema
}

func (User) Fields() []ent.Field {
	return []ent.Field{
		field.String("name").
			Default("unknown"),
	}
}

func (User) Edges() []ent.Edge {
	return []ent.Edge{
		edge.To("todos", Todo.Type),
	}
}

If I run the following code
todos, err := client.Todo.Query().All(ctx)
There is no easy way to retrieve the user_id field without sending an additional query for each row right now.
Use Case:
Graphql Query that returns users then uses a data loader to fetch users. Ideal SQL:
select name, done, user_id from todo;
select name from users where user_id in (1,2,3,4);

Option 1:
Remove edge relation and manage fk manually.
Easy, works right now but loses a lot of the advantages especially around filtering
Option 2:
Duplicate fk column:
func (Todo) Fields() []ent.Field {
	return []ent.Field{
		field.Text("text"),
		field.Int("user_id_fk").StorageKey("user_id"),
		field.Bool("done").Default(false),
	}

(user_id column name has to be used to prevent function collisions)
        previous declaration at ent/todo_create.go:29:6
ent/todo_update.go:72:23: (*TodoUpdate).SetUserID redeclared in this block
        previous declaration at ent/todo_update.go:41:6
ent/todo_update.go:254:27: (*TodoUpdateOne).SetUserID redeclared in this block
        previous declaration at ent/todo_update.go:223:6

This also breaks because we get conflicting column definitions
Option 3:
Embed a readonly fk column that can be used for queries and retrieval without joining on the users table.
type Todo struct {
	config `json:"-"`
	// ID of the ent.
	ID int `json:"id,omitempty"`
	// Text holds the value of the "text" field.
	Text string `json:"text,omitempty"`
	// UserIDFk holds the value of the "user_id_fk" field.
	UserIDFk int `json:"user_id_fk,omitempty"`
	// Done holds the value of the "done" field.
	Done bool `json:"done,omitempty"`
}

I haven't fully fleshed out this idea but a good precursor might be readonly fields. For example audit fields.

Filtering on edges can be done using the edge predicates.

Embed a readonly fk column that can be used for queries

Some edges, like M2M have a join table. Hence, FK is useless in these cases. For example:
If you have a users<-->groups relation, you want to be able to get the groups of users easily and not the FK in the join table.
About edge retrieving - this is part of the eager-loading feature and I think it's better to move the discussion to one place. Are you fine with closing this issue and continue the discussion in #91?

Hi, thanks for the response.
Edge predicates is an option but isn't the extra join a waste in this case?
todos, err := client.Todo.Query().Where(
		todo.HasUserWith(user.IDIn(1, 2))).All(ctx)

emits
SELECT DISTINCT `todos`.`id`, `todos`.`text`, `todos`.`done`
    FROM `todos` WHERE `todos`.`user_id` IN (
        SELECT `id` FROM `users` WHERE `users`.`id` IN (?, ?)
    )

Eager loading would solve some of these issues but also introduces an extra join.
I am looking for a solution where I don't need that join. Also I am happy if you simplicity sake the library does not support this or chooses not too but it seems like a very common use case to me

Thanks for your comment @cliedeman and sorry for the delay (holiday season until Oct 23rd).

Edge predicates is an option but isn't the extra join a waste in this case?

The FK doesn't always exist on the same table (M2M, O2M and some O2O) and JOIN (or sub-query) is required. It's also required in all cases when querying other fields (besides IDs).
However, I agree that cases like this should be optimized in entc. But in general, FKs are storage internal details and we're not planning to expose them as part of the API.
If a user wants to apply an SQL specific predicate he can do it today. Taking you example:
pets := client.Pet.
	Query().
	Where(predicate.Pet(func(s *sql.Selector) {
		// Assuming you know that "owner_id" exists
		// on the "pets" table.
		s.Where(sql.InInts(pet.OwnerColumn, 1, 2, 3))
	})).
	AllX(ctx)
// Emits:
// SELECT DISTINCT `pets`.`id`, `pets`.`name` FROM `pets` WHERE `owner_id` IN (?, ?, ?)
// args=[1 2 3]

Eager loading would solve some of these issues but also introduces an extra join.

Like above, in some cases JOIN is required. Avoiding JOIN when the edge exists on the same table is an implementation optimization, but I don't think we should change the public API for this.

The FK doesn't always exist on the same table (M2M, O2M and some O2O) and JOIN (or sub-query) is required. It's also required in all cases when querying other fields (besides IDs).

This is interesting to me because I
Yes my use case is verify specific. Normally you would just handle this with eager loading and call it a day. But with graphql resolvers it's a bit awkward knowing you will do that query anyway.
Some example code:
func (r *queryResolver) Todos(ctx context.Context) ([]*Todo, error) {
	client, _ := EntClientFromContext(ctx)

	dbTodos, _ := client.Todo.Query().Where(
		todo.HasUserWith(user.IDIn(1, 2))).All(ctx)

	var todos []*Todo

	for _, t := range dbTodos {

		todos = append(todos, &Todo{
			ID: t.ID,
			Text: t.Text,
			Done: t.Done,
			// Pass user id to child resolver
			User: &User{
				ID: t.UserID,// This field does not exist
			},
		})
	}

	return todos, nil
}

func (r *todoResolver) User(ctx context.Context, todo *Todo) (*User, error) {
	loader := ctx.Value(userLoaderKey)
	return loader.(*UserLoader).Load(todo.User.ID)
}

(Example using gqlgen with ent)
So here we query todos, and then use a data loader to batch fetch the user objects.
So in this case eager loading would fix the issue but would do an extra join. A micro optimisation for sure but it's good to know what the library will and will not support.

However, I agree that cases like this should be optimized in entc. But in general, FKs are storage internal details and we're not planning to expose them as part of the API.

This is a bit of a contradiction :). So I'm not sure what you mean.

Like above, in some cases JOIN is required. Avoiding JOIN when the edge exists on the same table is an implementation optimization, but I don't think we should change the public API for this.

Happy with the filtering example you are presenting here. If the code needs to be optimized so much for this to matter it should probably be hand written sql anyway. To be honest I don't have a good solution for the retrieval side. I think adding a foreign key field is bad idea and will be confusing
E.g.
type Todo struct {
  User User,
  UserId: string
}

So in summary. I would have liked to have foreign keys retrieved when the Todo table is queried but I can't see a clean way to do it that won't mess with the api. Feel free to close

So in this case eager loading would fix the issue but would do an extra join.

What I tried to say is: with minor change in entc (ent codegen) the code can be optimized, and subquery can be avoided for cases like:
dbTodos, _ := client.Todo.Query().Where(todo.HasUserWith(user.IDIn(1, 2))).All(ctx)

(Because user_id exists in the todos table. This is a M2O relation. It will be addressed soon).

This is a bit of a contradiction :). So I'm not sure what you mean.
Happy with the filtering example you are presenting here.

Again, M2O relations (and some O2O cases) will be improved by entc.  Also, the custom filtering won't be needed after this change in order to achieve better performance.

So in summary. I would have liked to have foreign keys retrieved when the Todo table is queried but I can't see a clean way to do it that won't mess with the api.

Give a look on the eager loading proposal. This will make edge loading much simpler and faster.
BTW, it's not part of our roadmap, but we want to auto-bind this into gqlgen as well. I'll try to keep this issue updated with our solution.

Any updates?

@a8m:

However, I agree that cases like this should be optimized in entc. But in general, FKs are storage internal details and we're not planning to expose them as part of the API.

Is it possible to re-open this issue?
I have a use case where I have a tree that's a O2M. Almost exactly like the one in the documentation. https://entgo.io/docs/schema-edges/#o2m-same-type.
With the key difference that I've manually specified the StorageKey.
The only difference is that all nodes belongs to another thing. So I have an additional edge.From entry.
What I essentially want to do is to fetch all items that belongs to that thing, and have the client figure out the tree. For this specific use case it makes more sense to let the client handle it. But to do that I need access to the primary key.
I also have another use case, where I am utilising the relationship between the nodes.
Maybe FKs could be optionally exposed if they have a name specified?

Hey @Zn4rK!

What I essentially want to do is to fetch all items that belongs to that thing

Are you trying to avoid the sub-query? If that's the case, I agree we need to optimize this.

Maybe FKs could be optionally exposed if they have a name specified?

I don't really want to expose the FKs in the default codegen. If the reason is perf-tuning, we need to address these optimizations in the default behavior.
However, I think we can allow anyone who wants it, to get it using an extension for the codegen - let's say using external-templates and edge-annotations.

Are you trying to avoid the sub-query? If that's the case, I agree we need to optimize this.

Not necessarily, but sure, it's an added bonus. The main reason is that GraphQL - which I'm using as the transport layer -  doesn't really support recursive types in queries. So I'm trying to keep it flat and let the client/webapp figure out the relations.
I essentially don't want to "QueryParent()" to figure out the parent id - since I, in theory should already know It, since I've already fetched all Nodes that belongs to specific Thing

A node belongs to a thing
A node can also belong to another node

So in my use case, the relationship here is not an internal implementation detail.

I don't really want to expose the FKs in the default codegen. If the reason is perf-tuning, we need to address these optimizations in the default behavior.

The reason is not perf-tuning in my case, even though I agree with you on this point.
I've managed to hack it for now with an external template, as you suggested. But I'm fairly certain that there are other use cases where it would be befinitial to have access to the FK in the default codegen. But, I understand your point of view.

I'm facing a similar problem having no access to the underlying foreign keys. I'm working with two tables. One products table with 30k records and one product_attributes table with around 1.3 million records.
In order to implement a fast lookup by id I need to prefetch all database records and map them by their ids. I first fetch all products, then all product attributes and then iterate over both collections and map them by their ids.
type cacheEntry struct {
	Product        *ent.Product
	AttributesById map[int]*ent.Attribute
}

cache := make(map[int]cacheEntry)
products, _ := client.Product.Query().All(ctx)
productAttributes, _ := client.Attribute.Query().All(ctx)

for _, product := range products {
	cache[product.ID] = productCacheEntry{
		Product: product,
		AttributesById: make(map[int]*ent.Attribute),
	}
}

for _, attribute := range productAttributes {
	if _, ok := cache[1]; ok { // 1 because attribute.ProductID does not exist on attribute
		cache[1].AttributesById[attribute.ID] = attribute
	}
}

The whole code takes about 150ms to execute.
If I instead use eager loading to fetch the products along with their attributes the code takes 10x longer, about 1.5 seconds.
productsCache := make(map[int]productCacheEntry)
products, _ := client.Product.Query().WithAttributes().All(ctx)

for _, product := range products {
	productsCache[product.ID] = productCacheEntry{
		Product:        product,
		AttributesById: make(map[int]*ent.Attribute),
	}

	for _, attribute := range product.Edges.Attributes {
		if _, ok := productsCache[product.ID]; ok {
			productsCache[product.ID].AttributesById[attribute.ID] = attribute
		}
	}
}

@a8m Do you see using custom extensions being the only way in order to get this running? Or am I missing something?

I'm facing a similar problem having no access to the underlying foreign keys. I'm working with two tables. One products table with 30k records and one product_attributes table with around 1.3 million records.
In order to implement a fast lookup by id I need to prefetch all database records and map them by their ids. I first fetch all products, then all product attributes and then iterate over both collections and map them by their ids.
type cacheEntry struct {
	Product        *ent.Product
	AttributesById map[int]*ent.Attribute
}

cache := make(map[int]cacheEntry)
products, _ := client.Product.Query().All(ctx)
productAttributes, _ := client.Attribute.Query().All(ctx)

for _, product := range products {
	cache[product.ID] = productCacheEntry{
		Product: product,
		AttributesById: make(map[int]*ent.Attribute),
	}
}

for _, attribute := range productAttributes {
	if _, ok := cache[1]; ok { // 1 because attribute.ProductID does not exist on attribute
		cache[1].AttributesById[attribute.ID] = attribute
	}
}

The whole code takes about 150ms to execute.
If I instead use eager loading to fetch the products along with their attributes the code takes 10x longer, about 1.5 seconds.
productsCache := make(map[int]productCacheEntry)
products, _ := client.Product.Query().WithAttributes().All(ctx)

for _, product := range products {
	productsCache[product.ID] = productCacheEntry{
		Product:        product,
		AttributesById: make(map[int]*ent.Attribute),
	}

	for _, attribute := range product.Edges.Attributes {
		if _, ok := productsCache[product.ID]; ok {
			productsCache[product.ID].AttributesById[attribute.ID] = attribute
		}
	}
}

@a8m Do you see using custom extensions being the only way in order to get this running? Or am I missing something?

I'm facing a similar problem having no access to the underlying foreign keys. I'm working with 3 tables. One products table with 30k records, an attributes table with only 3k records, and the pivot table product_attributes table with around 1.3 million records. Products and attributes are M2M.
In order to implement a fast lookup I need to prefetch all product_attributes entries and map them by their foreign keys (product_id and attribute_id). First, I fetch all product attributes and then iterate over all 1.3M records and map them by their ids.
type cacheEntry struct {
	Product        *ent.Product
	AttributesById map[int]*ent.ProductAttribute
}

cache := make(map[int]cacheEntry)
products, _ := client.Product.Query().All(ctx)
productAttributes, _ := client.ProductAttribute.Query().All(ctx)

for _, product := range products {
	cache[product.ID] = productCacheEntry{
		Product: product,
		AttributesById: make(map[int]*ent.Attribute),
	}
}

for _, attribute := range productAttributes {
	if _, ok := cache[1]; ok { // 1 because attribute.ProductID does not exist on attribute
		cache[1].AttributesById[attribute.ID] = attribute
	}
}

The whole code takes about 150ms to execute.
If I instead used eager loading to fetch the products along with their attributes the code takes 10x longer, about 1.5 seconds.
productsCache := make(map[int]productCacheEntry)
products, _ := client.Product.Query().WithAttributes().All(ctx)

for _, product := range products {
	productsCache[product.ID] = productCacheEntry{
		Product:        product,
		AttributesById: make(map[int]*ent.Attribute),
	}

	for _, attribute := range product.Edges.Attributes {
		if _, ok := productsCache[product.ID]; ok {
			productsCache[product.ID].AttributesById[attribute.ID] = attribute
		}
	}
}

@a8m Do you see using custom extensions being the only way in order to get this running? Or am I missing something?

I'm facing a similar problem having no access to the underlying foreign keys. I'm working with 3 tables. One products table with 30k records, an attributes table with only 3k records, and the pivot table product_attributes table with around 1.3 million records. Products and attributes are M2M.
In order to implement a fast lookup I need to prefetch all product_attributes entries and map them by their foreign keys (product_id and attribute_id). First, I fetch all product attributes and then iterate over all 1.3M records and map them by their ids.
type cacheEntry struct {
	Product        *ent.Product
	AttributesById map[int]*ent.ProductAttribute
}

cache := make(map[int]cacheEntry)
products, _ := client.Product.Query().All(ctx)
productAttributes, _ := client.ProductAttribute.Query().All(ctx)

for _, product := range products {
	cache[product.ID] = productCacheEntry{
		Product: product,
		AttributesById: make(map[int]*ent.Attribute),
	}
}

for _, attribute := range productAttributes {
	if _, ok := cache[1]; ok { // 1 because attribute.ProductID does not exist on attribute
		cache[1].AttributesById[attribute.ID] = attribute
	}
}

The whole code takes about 150ms to execute.
If I instead used eager loading to fetch the products along with their attributes the code takes 10x longer, about 1.5 seconds.
productsCache := make(map[int]productCacheEntry)
products, _ := client.Product.Query().WithAttributes().All(ctx)

for _, product := range products {
	productsCache[product.ID] = productCacheEntry{
		Product:        product,
		AttributesById: make(map[int]*ent.Attribute),
	}

	for _, attribute := range product.Edges.Attributes {
		if _, ok := productsCache[product.ID]; ok {
			productsCache[product.ID].AttributesById[attribute.ID] = attribute
		}
	}
}

@a8m Do you see using custom extensions being the only way in order to get this running? Or am I missing something?


two fixes:  #14, and #16 apart from documentation issues

replace strict floating equality by \approx 4 times at end of test_complex.jl unit tests.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (5b5c16c) 93.84% compared to head (9c80042) 93.82%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #15      +/-   ##
==========================================
- Coverage   93.84%   93.82%   -0.02%
==========================================
  Files           7        6       -1
  Lines         341      340       -1
==========================================
- Hits          320      319       -1
  Misses         21       21

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Enhancement: Normalize composer.json

This PR

[x] normalizes composer.json

💁♂ I ran
$ composer global require ergebnis/composer-normalize

followed by
$ composer normalize

For reference, see https://github.com/ergebnis/composer-normalize.

@localheinz Thanks! Used in 633cc6865736d2eb63cb60e1aa4b7b8fc05f59f2.

Thank you, @hason!


[flutter_tools] add flutter build IPA

Description
A WIP/draft example of building an IPA from the command line.

Why has this been closed? Is there an alternative PR to follow?

The appropriate place to follow this is on the issue for flutter build ipa


Whether the files stored in tonutils-storage can be downloaded in the original storage-daemon

May I ask whether the files stored in tonutils-storage can be downloaded in the original storage-daemon?

yes they can, they are compatible


users are not found

Hello
the project is very well developed but the i cant find any users i have enabled the database {
"rules": {
".read": true,
".write": true
}
}
and
userInformationRef.keepSynced(true)
registeredUsernamesRef.keepSynced(true)
reportedUsersRecordRef.keepSynced(true)
are in AuthController.swift but still cant see any users can you please help me
thank you

Hi! Are you running the app on your own Firebase database, and have you downloaded your GoogleService-Info.plist? It sounds like you are, if you are setting the rules from the Firebase side.
Here are the exact rules I have set on the Firebase console. Let me know if these work for you:
{
  "rules": {
    "registeredUsernames": {
      ".read": true,
      ".write": true
    },
    "userInformation": {
      ".read": true,
      ".write": true
    },
    "conversations": {
      ".read": "auth != null",
      ".write": "auth != null"
    },
    "userConversationList": {
      ".read": "auth != null",
      ".write": "auth != null"
    },
    "userInactiveConversations": {
      ".read": "auth != null",
      ".write": "auth != null"
    },
    "reportedUsersRecord": {
      ".read": "auth != null",
      ".write": "auth != null"
    }
  }
}

Hello
Thank you for your reply
well i added the rules in the firebase console but still. i cant see the users?

Hello @joelrorseth where you able to check what I sent before
Thank you


config repo is private

I cannot copy the contents from this config, can you provide an example?
https://github.com/martinabrle/aks-java-demo/blob/65da0fb47066f71693d4fbfd6b02cce84fd9fc33/docs/aks-github-actions.md?plain=1#L8

Set the config repo private, please retest


Use golang 1.17

What this PR does / why we need it:

Some package in https://github.com/gojek/merlin/pull/221 requires newer version of golang (i.e. 1.15) . This PR is to update golang version in CI, docker, and development environment to 1.17
Which issue(s) this PR fixes:

None
Does this PR introduce a user-facing change?:
None
Checklist

[x] Added unit test, integration, and/or e2e tests
[x] Tested locally
[x] Updated documentation

LGTM.
However, you need to update Makefile for remaining test recipes: https://github.com/gojek/merlin/blob/main/Makefile#L71-L81
You need to add -gcflags=all=-d=checkptr=0 flag.


[SPIKE] Using AKS and EKS Terraform configuration directly with Epiphany.

Is your spike related to a problem or idea? Please describe.
Currently we have modules to deploy EKS and AKS alongside an Epiphany cluster. As these Terraform deployments work separate from each other both have separate Terraform states making it hard to maintain the 2 configured together.
Describe the outcome you'd like
With the following issue completed: #2976 it should be possible to deploy custom Terraform templates alongside the ones Epicli generates. With this spike we want to deploy an Epiphany cluster and see if we can deploy AKS and EKS alongside it using custom Terraform which can be partially re-used from the modules:
Modules:
https://github.com/epiphany-platform/m-azure-kubernetes-service
https://github.com/epiphany-platform/m-aws-kubernetes-service
Note that this should also take into account the custom things we did with '''k8s_as_cloud_service''' to facilitate the original Epiphany+ modules deployment.
The outcome should preferable consist of the abstracted Terraform from the AWS and Azure modules, documentation how to deploy them. This can be a separate repo on our GitHub.
What is the reason or source for the spike
The module approach might not be something we want to continue to develop long-term. This approach should make it easier to integrate EKS/AKS with Epiphany and more maintainable from the Terraform perspective.
Additional context
None

DoD checklist

[ ] Reader is able to understand the results of spike
[ ] The results of the spike are presented in a table (to show simply what are compared or researched parameters) / not applicable
[ ] Each value / cell in the results table is described more deeply below
[ ] Demo of the spike (automated as much as possible)
[ ] Design doc updated

As a result of this Spike, there are 3 PRs:

https://github.com/epiphany-platform/k8s-modules/pull/1 -> separate repository with Terraform template files and documentation how to use them
https://github.com/epiphany-platform/epiphany/pull/3146 -> update of modules documentation in main Epi repo
https://github.com/epiphany-platform/epiphany/pull/3141 -> updated of template for filebeat for k8s_as_cloud_service

As a result of this Spike, there are 3 PRs:

AKS and EKS modules k8s-modules#1 -> separate repository with Terraform template files and documentation how to use them
[2.0.1] k8s-modules: update documentation #3146 -> update of modules documentation in main Epi repo
[2.0.1] Filebeat: update template for new version #3141 -> updated of template for filebeat for k8s_as_cloud_service

4 PRs: https://github.com/epiphany-platform/k8s-modules/pull/2


sidekick dice roll delays

Hi,
Could I ask what would be a solution for sidekick having delays in showing results of dice rolls? Currently we are having an ongoing game and I'm using sidekick. I rolled and then got the result after around 20 minutes. Thanks

Pretty much the same as above. We've experienced one 30 minute delay from entering a roll command to getting Sidekick's response, and failures to respond entirely on subsequent attempts. I'm not certain there's anything we as end users can do; simply drawing attention to the issue and confirming it's not just one person.

We've seen temporary delays in #120. Might be a Discord and/or internet connectivity issue.
As usual, providing the exact timestamp when the problem was observed might help.

Closing as answered.

cf #129


Version Issue

Hi @ashaywalke
Which python version is supported for this project i faced a lot of issues while done with python3 and i was unable to install cStringIO, re ,csv ,pdfminer, BeautifulSoup, urllib2, spacy some of these requirements in python2 can you help me to resolve..
Thanks and Regards,
Manikantha Sekhar..

Hi @mady143
Through this reference link, I was able to make progress.
you can simply change the line: from cStringIO import StringIO to from io import StringIO for python3