blacklist pull requests: Improve regex for Metasmoke TP/FP statistics

For example, https://github.com/Charcoal-SE/SmokeDetector/pull/430#issuecomment-270069822 indicates a problem with word boundaries.  The test should probably add something like (^|[^A-Za-z0-9_]) before the phrase and ($|[^A-Za-z0-9_]) after it.

For future reference, the relevant code is https://github.com/Charcoal-SE/metasmoke/blob/master/app/controllers/github_controller.rb#L65.

Other things which work poorly are \W which is used a lot in blacklist requests now, lookaheads / lookbehinds (no support for these in MySQL) and the fact that we should probably search title and body at the same time.  Should there be more than one link? Or should the search be extended to have a "blacklist request support" mode where both title and body are searched, and some regex extensions are hacked in? (We could replace \W with [^A-Za-z0-9_], for example.)

I also note that Smoke Detector (somewhat whimsically IMHO) replaces a space with \s before adding something to the blacklist.  I don't think that's the way to go, really.

Could you expand on that last comment @tripleee? IIRC we had a discussion about that a long time ago. Why isn't it good to generalize whitespace matching?

I think it violates the rule of least surprise. Either we support full regex and accept exactly the original input, or we have our own formalism (escaping all dots by default would seem more useful than the space to \s replacement, for example; but I think no replacing makes more sense).
I'm also thinking that if we remap stuff in the first place, we should remap in the opposite direction when we link to Metasmoke. For example, we should remap \W to something like [^a-z0-9_] in the searches we generate in PRs from users who don't have code admin privileges (... I think we call them).
Bottom line, if we do this at all, we should have a disciplined rationale rather than random organically grown syntactic sugar crust.

(Sorry for repeating myself; the mobile Github app I'm trying out is otherwise nice, but doesn't show me earlier messages in the thread when composing a new comment. Also, no editing of the comment I just submitted.)

Note that the regex substitutions which I introduced in gitmanager a while ago were to maintain compatiability with MS's basic regex capabilities, etc. The way the blacklist command works has changed a lot since then so it could do with updating.

Actually works now


update license-maven-plugin

For deprecated configs.

Codecov Report

Merging #11665 (dda548f) into master (2a3a171) will decrease coverage by 48.62%.
Report is 4 commits behind head on master.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master   #11665       +/-   ##
=============================================
- Coverage     63.04%   14.42%   -48.62%
+ Complexity     1121      201      -920
=============================================
  Files          2343     2343
  Lines        125705   125703        -2
  Branches      19309    19309
=============================================
- Hits          79252    18135    -61117
- Misses        40808   106038    +65230
+ Partials       5645     1530     -4115

Flag
Coverage Δ

integration
?

integration1
?

integration2
?

java-11
?

java-17
14.42% <ø> (-48.48%)
:arrow_down:

java-20
?

temurin
14.42% <ø> (-48.62%)
:arrow_down:

unittests
14.42% <ø> (-48.62%)
:arrow_down:

unittests1
?

unittests2
14.42% <ø> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1515 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Add a Toggle for Low pt Studies + Warnings Printout Toggle

Will post timing/performance plots soon to show that the toggle doesn't change anything.

@slava77 is there anything wrong with this approach? You can specify the pt cut now by saying
sdl_make_tracklooper -mc -P 0.4 for example. It all seems to run fine.

is there anything wrong with this approach?

Have you tried this in the CMSSW setup? When I try to run, I get:
In file included from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Hit.h:4,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Event.h:4,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/LST.h:15,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/RecoTracker/LST/plugins/alpaka/LSTProducer.cc:19:
/home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Constants.h:150:54: error: 'PT_CUT' was not declared in this scope
  150 |     ALPAKA_STATIC_ACC_MEM_GLOBAL const float ptCut = PT_CUT;
      |                                                      ^~~~~~
In file included from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Hit.h:4,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Event.h:4,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/LST.h:15,
                 from /home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/RecoTracker/LST/plugins/alpaka/LSTProducer.cc:19:
/home/users/evourlio/LSTinCMSSW/cgpu-1/CMSSW_13_3_0_pre3/src/../../../TrackLooper/SDL/Constants.h:150:54: error: 'PT_CUT' was not declared in this scope
  150 |     ALPAKA_STATIC_ACC_MEM_GLOBAL const float ptCut = PT_CUT;
      |                                                      ^~~~~~
gmake: *** [config/SCRAM/GMake/Makefile.rules:1793: tmp/el8_amd64_gcc11/src/RecoTracker/LST/plugins/RecoTrackerLSTPluginsPortableCpuSerialSync/alpaka/LSTProducer.cc.o] Error 1

Unless this is not yet ready to be run (I though it was, at least at the level I was testing it).

I think that can be sort of fixed with a simple ifdef in the config file, i.e. if PT_CUT is not defined then it just goes to a default value of 0.8. I'll look into it more though, not sure if there is a better fix that allows you to select any PT_CUT in CMSSW too.

Marked this PR as ready for review. This is just the addition of two toggles, so there are no changes to timing/performance when the two toggles are switched off:
This PR Timing:

This PR: Here
Comparison: Here

Quick question: Is this meant not to change the pT cuts for the CMSSW setup? I don't see any of changes in the LST.cc/h, which are the equivalents of the trkCore.cc.

Yes, this is just for the standalone version for now. I don't see a reason at this stage to add this to the CMSSW setup, and based on what you said at the last meeting there is an issue with lowering the pt cut which would require more work to sort out.

When I run it with the -w option I'm getting the following error. Do you know why this might be happening? It seems to be unrelated to this PR, though. As far as I can tell, this might happen with the DNN, because if you disable it then it works fine.
ERROR
 hittype: 0
 trk.pix_simHitIdx().size(): 162652
 trk.ph2_simHitIdx().size(): 213716
162652 0
206907 0
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 206907) >= this->size() (which is 162652)

@ariostas Can you give your full compile command? I don't see this on my end.

@GNiendorf I'm just using sdl_run -f "-mcw" -s PU200 -n -1 -t test -d

sdl_run -f "-mcw" -s PU200 -n -1 -t test -d

What does the code do when you add the -w flag with no number assigned to it?

What does the code do when you add the -w flag with no number assigned to it?

There's two different -w flags: one for sdl_make_tracklooper and one for sdl. The one I'm using above is for sdl_make_tracklooper and it doesn't take any argument.

@GNiendorf I'm just using sdl_run -f "-mcw" -s PU200 -n -1 -t test -d

It seems like this error only occurs when the write_ntuple variable is set to 1. I'll have to look into it more, thank you for pointing this out. Not sure if it is a problem with this PR yet.


Allow passing dynamic timezone to duckling ISSUE-2801

Proposed changes:
Issue: #2801

Allow to dynamically give timezone to the NLU duckling component using message metadata time_zone property.

Status (please check what you already did):

[ ] added some tests for the functionality
[ ] updated the documentation
[x] updated the changelog (please check changelog for instructions)
[x] reformat files using black (please check Readme for instructions)

Hey @wochinge, would be awesome if you could merge this! Please let us know what is missing so we can work on it. It's a feature that is much needed for duckling and hasn't been implemented (while the issue is quite old...). Thanks


Apple IR Remote support?

Would it be possible to support the Apple IR Remote? That’d be amazing.
There’s some XCode classes here:
http://www.martinkahr.com/2007/07/26/remote-control-wrapper-20/index.html
And a step-by-step for creating and using it as a Framework (albeit using Ruby) here:
http://www.rubycocoa.com/appleremote/1
I’d suggest the following key mappings:
play/pause = enter fullscreen
left = page left
right = page right
menu = exit fullscreen
Thank you!

Hah, Closing this.


Recognition Training Pytorch

Bug description
I want to use docTR for training of Recognition of Turkish language data. I have created a dataset as specified. The images in the dataset are mixed shapes but all images widht and heights are bigger than 12. And I have also added the vocab in vocabs.py.
Can someone explain why I am getting this error and what has to be changed?
Code snippet to reproduce the bug
python .\references\recognition\train_pytorch.py crnn_vgg16_bn --epochs 1 --train_path doctr-train --workers 4 --vocab turkish --push-to-hub --pretrained
Error traceback
Validation set loaded in 0.044s (2160 samples in 34 batches)
Train set loaded in 11.48s (395608 samples in 6181 batches)
Traceback (most recent call last):------------------------------------------------------------------------| 4.00% [247/6181 00:59<24:01 Training loss: 1.238]52]
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\references\recognition\train_pytorch.py", line 467, in
main(args)
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\references\recognition\train_pytorch.py", line 378, in main
fit_one_epoch(model, train_loader, batch_transforms, optimizer, scheduler, mb, amp=args.amp)
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\references\recognition\train_pytorch.py", line 116, in fit_one_epoch
train_loss = model(images, targets)["loss"]
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\train_venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\doctr\models\recognition\crnn\pytorch.py", line 228, in forward
out["loss"] = self.compute_loss(logits, target)
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\doctr\models\recognition\crnn\pytorch.py", line 185, in compute_loss
ctc_loss = F.ctc_loss(
File "C:\Users\Zahide.Tastan\Desktop\doctr-19012023\train_venv\lib\site-packages\torch\nn\functional.py", line 2628, in ctc_loss
return torch.ctc_loss(
RuntimeError: Expected tensor to have size at least 33 at dimension 1, but got size 32 for argument #2 'targets' (while checking arguments for ctc_loss_gpu)
Environment
DocTR version: 0.7.1a0
TensorFlow version: N/A
PyTorch version: 1.13.1+cu117 (torchvision 0.14.1+cu117)
OpenCV version: 4.7.0
OS: Microsoft Windows Server 2022 Standard
Python version: 3.9.13
Is CUDA available (TensorFlow): N/A
Is CUDA available (PyTorch): Yes
CUDA runtime version: 11.7.99
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin\cudnn_ops_train64_8.dll
Deep Learning backend

from doctr.file_utils import is_tf_available, is_torch_available
print(f"is_tf_available: {is_tf_available()}")
is_tf_available: False
print(f"is_torch_available: {is_torch_available()}")
is_torch_available: True

I went through the same thing when training a Korean model. I didn't have this problem when I trained with the vitstr model, using the ce loss rather than the ctc loss.

@Bing-su I am also working on korean model. Lets collaborate, I am working on multiple languages and already achieved korean recognition model

I also have trained Korean models. see:
https://huggingface.co/Bingsu/vitstr_small_230116_002756
https://huggingface.co/Bingsu/doctr-vitstr_base-recognition-korean-230116
The readme file is left as default, but these are korean model.
Character decomposition to resolve class imbalance problem in Hangul OCR
These are models that predict by consonant unit rather than character by referring to this paper.

@Bing-su , I am not able to evaluate it. How is the performance for handwritten korean? Did you train recognition model on multiple GPUs?

use from_hub. doctr doesn't have Korean vocabs yet. or manullay input them.
VOCABS["korean"] = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~°£€¥¢฿ᄀᄁᄂᄃᄄᄅᄆᄇᄈᄉᄊᄋᄌᄍᄎᄏᄐᄑ하ᅢᅣᅤᅥᅦᅧᅨᅩᅪᅫᅬᅭᅮᅯᅰᅱᅲᅳᅴᅵᆨᆩᆪᆫᆬᆭᆮᆯᆰᆱᆲᆳᆴᆵᆶᆷᆸᆹᆺᆻᆼᆽᆾᆿᇀᇁᇂ"

or
from doctr.models import from_hub

rec = from_hub("Bingsu/vitstr_small_230116_002756")

Hi @Bing-su 👋 ,
short question does the model work well for you?
After the implementation i haven't had the time to train it full 😅 So i am curious to get your feedback :)

Thanks for your help @Bing-su 👋,
But what i want to do is starting the training with the --pretrained as True. The pretrained model url information is only available for crnn_vgg16. In others, the pretrained model cannot be loaded because the model url is None. Is there a point I missed here?

For printed documents and handwriting, this works pretty well if the detection model predicts the bbox well.
case 1. well predicted bbox.
In this case, the model showed good accuracy. There is a problem with the image synthesis function, so the result is not displayed properly.

predict
>>> result.pages
[Page(
   dimensions=(3500, 2469)
   (blocks): [
     Block(
       (lines): [Line(
         (words): [
           Word(value='국', confidence=0.12),
           Word(value='세', confidence=0.12),
           Word(value='징', confidence=0.16),
           Word(value='수', confidence=0.12),
           Word(value='법', confidence=0.16),
           Word(value='시행', confidence=0.12),
           Word(value='규', confidence=0.17),
           Word(value='칙', confidence=0.1),
           Word(value='[별지', confidence=0.066),
           Word(value='제94', confidence=0.087),
           Word(value='호', confidence=0.15),
           Word(value='서', confidence=0.14),
           Word(value='식', confidence=0.13),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='No.', confidence=0.11)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='0', confidence=0.18),
           Word(value='4', confidence=0.17),
           Word(value='7', confidence=0.29),
           Word(value='26', confidence=0.16),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='납세증', confidence=0.12),
           Word(value='명', confidence=0.24),
           Word(value='서', confidence=0.17),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='*', confidence=0.082),
           Word(value='색상이', confidence=0.1),
           Word(value='에', confidence=0.11),
           Word(value='두운', confidence=0.11),
           Word(value='난은', confidence=0.055),
           Word(value='신', confidence=0.11),
           Word(value='청', confidence=0.13),
           Word(value='인이', confidence=0.11),
           Word(value='작성', confidence=0.091),
           Word(value='하지', confidence=0.092),
           Word(value='않습', confidence=0.16),
           Word(value='니', confidence=0.12),
           Word(value='다', confidence=0.15),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='발급', confidence=0.14),
           Word(value='번1', confidence=0.085),
           Word(value='호', confidence=0.22),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='처리기간', confidence=0.22)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='즉시', confidence=0.15),
           Word(value='(단,', confidence=0.18),
           Word(value='해', confidence=0.14),
           Word(value='ᄋᄋᄋᄋᄌᄌᄋ용', confidence=0.12),
           Word(value='10일)', confidence=0.2),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='성', confidence=0.15),
           Word(value='명', confidence=0.17),
           Word(value='(상,', confidence=0.17),
           Word(value='호)', confidence=0.16),
           Word(value='노', confidence=0.14),
           Word(value='답', confidence=0.22),
           Word(value='패', confidence=0.2),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='주민', confidence=0.11),
           Word(value='등', confidence=0.22),
           Word(value='록', confidence=0.17),
           Word(value='번', confidence=0.087),
           Word(value='호', confidence=0.1),
           Word(value='(사업,', confidence=0.28),
           Word(value='자등', confidence=0.11),
           Word(value='록', confidence=0.18),
           Word(value='번', confidence=0.11),
           Word(value='호)', confidence=0.17),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='인적사항', confidence=0.095),
           Word(value='주', confidence=0.11),
           Word(value='주소ᅩ사업', confidence=0.098),
           Word(value='장)', confidence=0.15),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='전라북도', confidence=0.23),
           Word(value='남구', confidence=0.15),
           Word(value='원곡', confidence=0.16),
           Word(value='동', confidence=0.13),
           Word(value='덕', confidence=0.22),
           Word(value='포', confidence=0.21),
           Word(value='진로', confidence=0.17),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='관', confidence=0.21),
             Word(value='공서', confidence=0.21),
             Word(value='제', confidence=0.17),
             Word(value='출', confidence=0.13),
           ]
         ),
         Line(
           (words): [
             Word(value='1', confidence=0.12),
             Word(value=']', confidence=0.16),
             Word(value='해', confidence=0.081),
             Word(value='외', confidence=0.099),
             Word(value='이', confidence=0.084),
             Word(value='이ᅮᅮ', confidence=0.11),
           ]
         ),
         Line(
           (words): [
             Word(value='[,', confidence=0.15),
             Word(value=']', confidence=0.16),
             Word(value='기타', confidence=0.1),
           ]
         ),
         Line(
           (words): [
             Word(value='유', confidence=0.1),
             Word(value='효기간', confidence=0.16),
           ]
         ),
         Line(
           (words): [
             Word(value='정', confidence=0.082),
             Word(value='한', confidence=0.084),
             Word(value='사유', confidence=0.072),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='중', confidence=0.1),
             Word(value='명', confidence=0.18),
             Word(value='서의', confidence=0.15),
           ]
         ),
         Line(
           (words): [
             Word(value='사용', confidence=0.096),
             Word(value='목적', confidence=0.15),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='20', confidence=0.13),
           Word(value='87', confidence=0.16),
           Word(value='년', confidence=0.12),
           Word(value='3', confidence=0.24),
           Word(value='3', confidence=0.22),
           Word(value='월', confidence=0.12),
           Word(value='3', confidence=0.24),
           Word(value='9', confidence=0.2),
           Word(value='일', confidence=0.13),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='증명서의', confidence=0.16)]
         ),
         Line(
           (words): [
             Word(value='유', confidence=0.14),
             Word(value='효기간', confidence=0.18),
             Word(value='유회', confidence=0.12),
             Word(value='효기간을', confidence=0.19),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value=']', confidence=0.18),
           Word(value='국', confidence=0.14),
           Word(value='세', confidence=0.09),
           Word(value='징', confidence=0.12),
           Word(value='수', confidence=0.11),
           Word(value='법', confidence=0.14),
           Word(value='시행', confidence=0.12),
           Word(value='령,', confidence=0.17),
           Word(value='1', confidence=0.17),
           Word(value='제', confidence=0.12),
           Word(value='96', confidence=0.13),
           Word(value='조', confidence=0.15),
           Word(value='제', confidence=0.1),
           Word(value='1', confidence=0.2),
           Word(value='항', confidence=0.15),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='기타', confidence=0.098),
             Word(value='(사유:', confidence=0.32),
           ]
         ),
         Line(
           (words): [Word(value='기간', confidence=0.14)]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value=')', confidence=0.19)]
         ),
         Line(
           (words): [Word(value='세액', confidence=0.13)]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='연', confidence=0.14),
             Word(value='장', confidence=0.16),
             Word(value='.', confidence=0.11),
             Word(value='유예', confidence=0.1),
             Word(value='연', confidence=0.11),
             Word(value='장', confidence=0.17),
             Word(value='유예', confidence=0.1),
           ]
         ),
         Line(
           (words): [Word(value='종류', confidence=0.098)]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='과세,', confidence=0.11),
           Word(value='기간', confidence=0.13),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='세목', confidence=0.083)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='납부기한', confidence=0.13)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='가산금', confidence=0.096)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='연', confidence=0.13),
           Word(value='장', confidence=0.19),
           Word(value='유', confidence=0.11),
           Word(value='예', confidence=0.14),
           Word(value='체납', confidence=0.066),
           Word(value='처분유예', confidence=0.073),
           Word(value='20', confidence=0.088),
           Word(value='3', confidence=0.22),
           Word(value='8', confidence=0.26),
           Word(value='6', confidence=0.23),
           Word(value='4', confidence=0.13),
           Word(value='20', confidence=0.087),
           Word(value='81', confidence=0.22),
           Word(value='6', confidence=0.25),
           Word(value='20', confidence=0.12),
           Word(value='지방소득세', confidence=0.33),
           Word(value='20', confidence=0.11),
           Word(value='7418', confidence=0.18),
           Word(value='3', confidence=0.17),
           Word(value='44', confidence=0.15),
           Word(value='2', confidence=0.24),
           Word(value='원', confidence=0.082),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='내역', confidence=0.16)]
         ),
         Line(
           (words): [
             Word(value='(단위:', confidence=0.28),
             Word(value='원)', confidence=0.13),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='체납', confidence=0.07),
           Word(value='처분유', confidence=0.086),
           Word(value='예', confidence=0.2),
           Word(value='20', confidence=0.16),
           Word(value='37', confidence=0.19),
           Word(value='41', confidence=0.16),
           Word(value='20', confidence=0.072),
           Word(value='96', confidence=0.12),
           Word(value='17', confidence=0.31),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='취득세', confidence=0.17),
           Word(value='204', confidence=0.14),
           Word(value='7', confidence=0.27),
           Word(value='7', confidence=0.25),
           Word(value='9', confidence=0.23),
           Word(value='244', confidence=0.15),
           Word(value='6', confidence=0.27),
           Word(value='원', confidence=0.083),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='7', confidence=0.18),
           Word(value='기한', confidence=0.17),
           Word(value='안연', confidence=0.16),
           Word(value='장', confidence=0.27),
           Word(value='21', confidence=0.18),
           Word(value='0', confidence=0.23),
           Word(value='48', confidence=0.1),
           Word(value='49', confidence=0.16),
           Word(value='20', confidence=0.082),
           Word(value='4', confidence=0.16),
           Word(value='738', confidence=0.17),
           Word(value='자동차', confidence=0.14),
           Word(value='쎄', confidence=0.13),
           Word(value='20', confidence=0.14),
           Word(value='4360', confidence=0.12),
           Word(value='49', confidence=0.11),
           Word(value='5', confidence=0.19),
           Word(value='4', confidence=0.16),
           Word(value='원', confidence=0.082),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='위', confidence=0.11),
             Word(value='탁자', confidence=0.098),
           ]
         ),
         Line(
           (words): [
             Word(value='도', confidence=0.14),
             Word(value='신', confidence=0.11),
             Word(value='순', confidence=0.14),
           ]
         ),
         Line(
           (words): [Word(value='함과성', confidence=0.12)]
         ),
         Line(
           (words): [
             Word(value='나', confidence=0.14),
             Word(value='표표', confidence=0.26),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='과세기간', confidence=0.18)]
         ),
         Line(
           (words): [
             Word(value='20', confidence=0.11),
             Word(value='3', confidence=0.23),
             Word(value='7', confidence=0.26),
             Word(value='6', confidence=0.25),
             Word(value='지방소득세', confidence=0.37),
           ]
         ),
         Line(
           (words): [
             Word(value='20', confidence=0.13),
             Word(value='7', confidence=0.24),
             Word(value='5', confidence=0.24),
             Word(value='9', confidence=0.24),
             Word(value='8', confidence=0.21),
             Word(value='지방소득세', confidence=0.42),
             Word(value='세', confidence=0.16),
           ]
         ),
         Line(
           (words): [
             Word(value='20', confidence=0.12),
             Word(value='9', confidence=0.18),
             Word(value='9', confidence=0.24),
             Word(value='983', confidence=0.14),
             Word(value='자동차세', confidence=0.23),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='세목', confidence=0.11)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='납부기한', confidence=0.085)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='세액', confidence=0.078)]
         ),
         Line(
           (words): [
             Word(value='3', confidence=0.24),
             Word(value='7', confidence=0.28),
             Word(value='89', confidence=0.11),
             Word(value='원', confidence=0.11),
           ]
         ),
         Line(
           (words): [
             Word(value='49', confidence=0.18),
             Word(value='6', confidence=0.24),
             Word(value='4', confidence=0.26),
             Word(value='원', confidence=0.1),
           ]
         ),
         Line(
           (words): [
             Word(value='38', confidence=0.13),
             Word(value='5', confidence=0.25),
             Word(value='9원', confidence=0.26),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='가산금', confidence=0.1)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='물적납세의무', confidence=0.68)]
         ),
         Line(
           (words): [
             Word(value='체', confidence=0.087),
             Word(value='납내', confidence=0.1),
             Word(value='역', confidence=0.12),
           ]
         ),
         Line(
           (words): [
             Word(value='(단위:', confidence=0.29),
             Word(value='원)', confidence=0.13),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='ᄋ', confidence=0.084),
             Word(value='국세', confidence=0.14),
             Word(value='징', confidence=0.08),
             Word(value='수', confidence=0.088),
             Word(value='법', confidence=0.11),
             Word(value='제', confidence=0.11),
             Word(value='제08조', confidence=0.24),
             Word(value='및', confidence=0.12),
             Word(value='같은', confidence=0.12),
             Word(value='법', confidence=0.14),
             Word(value='시행"', confidence=0.14),
             Word(value='령', confidence=0.14),
             Word(value='제', confidence=0.1),
             Word(value='195조에', confidence=0.32),
             Word(value='따라', confidence=0.16),
             Word(value='발급', confidence=0.13),
             Word(value='일', confidence=0.11),
             Word(value='현재', confidence=0.11),
             Word(value='위의', confidence=0.094),
             Word(value='연장', confidence=0.15),
             Word(value='5', confidence=0.098),
             Word(value='유예', confidence=0.15),
             Word(value='액', confidence=0.13),
             Word(value='또:', confidence=0.14),
             Word(value='는', confidence=0.13),
             Word(value='부:', confidence=0.11),
             Word(value='가', confidence=0.19),
           ]
         ),
         Line(
           (words): [
             Word(value='가치', confidence=0.07),
             Word(value='세', confidence=0.21),
             Word(value='법', confidence=0.14),
             Word(value='1', confidence=0.16),
             Word(value='제', confidence=0.15),
             Word(value='3조', confidence=0.14),
             Word(value='의ᄀ', confidence=0.11),
             Word(value='및', confidence=0.13),
             Word(value='1종', confidence=0.18),
             Word(value='합부동', confidence=0.15),
             Word(value='산세법', confidence=0.17),
             Word(value='제', confidence=0.13),
             Word(value='7조', confidence=0.12),
             Word(value='의2', confidence=0.12),
             Word(value='및', confidence=0.13),
             Word(value='제', confidence=0.11),
             Word(value='1.', confidence=0.15),
             Word(value='2조', confidence=0.18),
             Word(value='의', confidence=0.16),
             Word(value='12에', confidence=0.17),
             Word(value='따른', confidence=0.12),
             Word(value='수', confidence=0.13),
             Word(value='탁자의', confidence=0.1),
             Word(value='물', confidence=0.15),
             Word(value='적', confidence=0.13),
             Word(value='납세', confidence=0.12),
             Word(value='의', confidence=0.14),
             Word(value='무', confidence=0.13),
             Word(value='와', confidence=0.13),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='관', confidence=0.19),
           Word(value='련', confidence=0.2),
           Word(value='된', confidence=0.14),
           Word(value='체', confidence=0.14),
           Word(value='납액', confidence=0.11),
           Word(value='을', confidence=0.11),
           Word(value='제', confidence=0.083),
           Word(value='외', confidence=0.096),
           Word(value='하고는', confidence=0.15),
           Word(value='다른', confidence=0.092),
           Word(value='체', confidence=0.14),
           Word(value='납', confidence=0.091),
           Word(value='액', confidence=0.13),
           Word(value='이', confidence=0.093),
           Word(value='없', confidence=0.18),
           Word(value='음', confidence=0.15),
           Word(value='을', confidence=0.23),
           Word(value='증명', confidence=0.097),
           Word(value='합ᄂ', confidence=0.094),
           Word(value='니', confidence=0.14),
           Word(value='다', confidence=0.12),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='담', confidence=0.14),
             Word(value='당', confidence=0.21),
             Word(value='부서', confidence=0.091),
           ]
         ),
         Line(
           (words): [
             Word(value='담', confidence=0.2),
             Word(value='당자', confidence=0.13),
           ]
         ),
         Line(
           (words): [
             Word(value='연', confidence=0.12),
             Word(value='락처', confidence=0.13),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='년', confidence=0.098)]
         ),
         Line(
           (words): [Word(value='월', confidence=0.17)]
         ),
         Line(
           (words): [Word(value='일', confidence=0.1)]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='인', confidence=0.094),
           Word(value='그', confidence=0.12),
           Word(value='서', confidence=0.2),
           Word(value='장', confidence=0.12),
           Word(value='직인', confidence=0.1),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='21(', confidence=0.12),
           Word(value='Ommm', confidence=0.19),
           Word(value='X', confidence=0.23),
           Word(value='29', confidence=0.16),
           Word(value='7mm', confidence=0.18),
           Word(value='[백', confidence=0.13),
           Word(value='상지', confidence=0.12),
           Word(value='80g/m2', confidence=0.22),
           Word(value='또는', confidence=0.13),
           Word(value='중질', confidence=0.1),
           Word(value='지', confidence=0.11),
           Word(value='80g/m2', confidence=0.17),
         ]
       )]
       (artefacts): []
     ),
   ]
 )]

case 2. poorly predicted bbox
In this case, the prediction results are also messed up.

predict
>>> result.pages
[Page(
   dimensions=(905, 640)
   (blocks): [
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='창의적', confidence=0.13),
             Word(value='인주시민을', confidence=0.33),
             Word(value='기르는', confidence=0.15),
             Word(value='확신며래ᄀᄀ욕', confidence=0.2),
           ]
         ),
         Line(
           (words): [Word(value='서울특별시교육청', confidence=0.53)]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='처얼서ᄅᄅ', confidence=0.14)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='수신', confidence=0.17),
           Word(value='ᄋ', confidence=0.074),
           Word(value='사단법인', confidence=0.18),
           Word(value='환글문화연대대효', confidence=0.19),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='(경유)', confidence=0.33)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='제목', confidence=0.13),
           Word(value='서울시ᄀᄀ육용', confidence=0.32),
           Word(value='홈페이지', confidence=0.16),
           Word(value='교육좡이', confidence=0.15),
           Word(value='메뉴', confidence=0.14),
           Word(value='소ᅮ긴아ᄋᄇᆼ거ᄀᆼᄇᄋᆼ히ᄅ(대리인ᄉ', confidence=0.15),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='귀험인의부궁한앙전물기원안니ᅵᅡ', confidence=0.21)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='부ᄀᄎᄀᆫᄉᄉᆫ(ᅡᅡᅡᅡᄋ법인', confidence=0.16),
           Word(value='봔급분화연뎌ᄋᄋ히41000503434378', confidence=0.1),
           Word(value='(2020.01', confidence=0.13),
           Word(value='11', confidence=0.18),
           Word(value='16.)', confidence=0.27),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='ᄋᄋᄅ라는', confidence=0.16),
           Word(value='손ᅳᆨᄋᄋᆫᅩᆨ새하ᄋ을', confidence=0.14),
           Word(value='사용해주십시오]', confidence=0.36),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='3.', confidence=0.11),
             Word(value='사울사ᄀᄀᄋᄋᆨᆼᄇᆼᅭᅳᆼ', confidence=0.2),
             Word(value='흠페이지(0ᅢc.ᄋ.snnD0KK-Lr.ᄋᄋᄋ황안ᄂᄂᄀ고육ᅲᄒᄒ이ᅵ나ᅲ으를', confidence=0.11),
             Word(value='사용하기기훈', confidence=0.18),
           ]
         ),
         Line(
           (words): [Word(value='(ᅮ리하ᅡᆯ로ᅩᆫᆫᅥᅧᆼᄒ는ᅳ안에ᅢᅢ예겸보ᅩᅦᅡᅡ걸긍구ᅮᆨᄋᆸᄀᆨᆨᄋᄋᄋᄋ알국이인다듬은루ᄀᆨᆨᄋ으ᄋᄀᄀᆨᆨ이ᅮᆫᄒᄒ자료져ᅧ)에', confidence=0.1)]
         ),
         Line(
           (words): [
             Word(value='2020.', confidence=0.12),
             Word(value='ᄌ..ᅵ.ᄀ(댁내ᄃᄃᄋᄀ굴ᄀ정ᆼᄎᆼᆼᄋ(으ᅲᅲᄅᄅ오', confidence=0.16),
             Word(value='.', confidence=0.14),
             Word(value='교육화ᄌᄋᆼ장ᆼᅩ', confidence=0.14),
             Word(value='으래ᄇᄇᆫᅧᅢᆫ정으져ᅧᆼᄌ이ᅵᅩ이ᅳ', confidence=0.11),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='나와있는', confidence=0.31),
           Word(value='ᄌᄋᆼ장ᄉᆼᄋ이ᅡᆫᄃᄋᄋᄋᆫᅩᅮᆫ오순화화고자아ᅡᅵ다ᄃ다', confidence=0.11),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [
             Word(value='보ᄀ고해주시기ᅵ', confidence=0.22),
             Word(value='바랍니다.', confidence=0.35),
           ]
         ),
         Line(
           (words): [
             Word(value='1이ᄋ', confidence=0.28),
             Word(value='이ᄆ', confidence=0.082),
             Word(value='가동은앙ᆼ고ᄀᄀ자ᄅ로비(,', confidence=0.15),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='성ᄋᆯ', confidence=0.12)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='JH', confidence=0.061)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='-', confidence=0.13),
           Word(value='ᄋ', confidence=0.13),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='홍문타ᄌᄇᆼ', confidence=0.16)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='홈무과장', confidence=0.32)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='2020..1.2..', confidence=0.29)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [
         Line(
           (words): [Word(value='뎜조자', confidence=0.13)]
         ),
         Line(
           (words): [
             Word(value='사항', confidence=0.2),
             Word(value='3ᅩᆼ--8-2626', confidence=0.082),
           ]
         ),
         Line(
           (words): [Word(value='03178', confidence=0.18)]
         ),
         Line(
           (words): [
             Word(value='전화', confidence=0.11),
             Word(value='002-3899-2221', confidence=0.14),
           ]
         ),
       ]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='(2000.01', confidence=0.17),
           Word(value='11.24.)', confidence=0.14),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='서울특별시', confidence=0.25),
           Word(value='종로구', confidence=0.25),
           Word(value='공궐길', confidence=0.18),
           Word(value='애인신ᆫ욘거ᅥᄉ조호독수ᄉᄉᄉ', confidence=0.21),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='목정)', confidence=0.28)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='WNW', confidence=0.14),
           Word(value='sen.', confidence=0.15),
           Word(value='QO.kr', confidence=0.11),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='(전ᄉᆼ', confidence=0.16)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [Word(value='ihelimgsnnno..r', confidence=0.29)]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='ᄋ', confidence=0.19),
           Word(value='개', confidence=0.16),
         ]
       )]
       (artefacts): []
     ),
     Block(
       (lines): [Line(
         (words): [
           Word(value='2020.11.24', confidence=0.24),
           Word(value='11149', confidence=0.15),
           Word(value='총무과', confidence=0.14),
         ]
       )]
       (artefacts): []
     ),
   ]
 )]

So, I think it is necessary to train a detection model, so I plan to train it.
here is the wandb log. https://wandb.ai/bingsu/doctr/overview?workspace=user-bingsu
cer and mer are metrics from torchmetrics.
I modified the code to use the image augmentation of albumentations, but it was cumbersome to modify the metric, so I used it instead.

@zahidetastan Isn't loading via from_hub good enough? This function brings the model and weights together.
Or you can use torch.hub.load_state_dict_from_url, but this requires you to set the vocabs.
from doctr.models import from_hub

rec = from_hub("Bingsu/vitstr_small_230116_002756")

import torch
from doctr.models import vitstr_base, vitstr_small

model = vitstr_small(pretrained=False, vocab=VOCABS["korean"])
url = "https://huggingface.co/Bingsu/vitstr_small-korean/blob/main/pytorch_model.bin"
state_dict = torch.hub.load_state_dict_from_url(url)
model.load_state_dict(state_dict)

@Bing-su  So, do you have any idea about the cause of the error I mentioned in the issue? I did not encounter a problem when I trained with a dataset containing 40k data before.

RuntimeError: Expected tensor to have size at least 33 at dimension 1, but got size 32 for argument #2 'targets' (while checking arguments for ctc_loss_gpu)

Try to modify the  self.max_length to some higher value which contains the max length of your ground truth data. The default value is set to 32, try to set to the max length of the strings in your training data containing ground truth.
This is to be done in doctr/models/recognition/crnn/pytorch.py file.

@Asad-Raza Thanks for your answer. It worked for me 👍

Hi @all :wave:
Looks like the problem is solved :)
Feel free to comment for reopening

@zahidetastan
Hi Zahide would you mind to share trained model to evaluate and use?
Best


Formatting adds an extra new line in some cases

Note this is not a dup of https://github.com/microsoft/vscode-python/issues/1877. Cause of both issues mentioned below is likely the same.
Have a file where the last line needs formatting
1    print   ("yes")

1. BLACK FORMATTER
Formats document with black
Actual behavior:
1    print("yes")
2
3

Expected behavior:
1    print("yes")
2

Formatting again in this case leads to the expected behavior once https://github.com/microsoft/vscode-python/pull/10061/files is merged.
2. YAPF FORMATTER
Formats document with yapf
Actual behavior:
1    print("yes")
2

Expected behavior:
1    print("yes")

I just ran into this issue.
Here's an example repository that reproduces the issue: https://github.com/chdsbd/vscode-black-example/blob/master/main.py
GitHub appears to hide trailing whitespace, but if you download the repository there will be trailing whitespace in main.py. When formatting runs from VSCode on save, the trailing whitespace will remain. But when black is run from the command line the trailing whitespace will be removed.
Looking in the logs it seems like VSCode is running the following command:
~/projects/format-example/.venv/bin/python3 ~/.vscode/extensions/ms-python.python-2020.8.101144/pythonFiles/pyvsc-run-isolated.py black --diff --quiet ~/projects/format-example/main.py

Which provides the following output
--- /Users/chris/projects/format-example/main.py	2020-08-19 21:01:56.296536 +0000
+++ /Users/chris/projects/format-example/main.py	2020-08-19 21:02:24.683804 +0000
@@ -1,4 +1,3 @@
 def main() -> None:
     print("hello")

-

It seems like the diff isn't be applied correctly because the diff removes the trailing whitespace.

Disable formatting from the python extension and try ms-python.black-formatter extension. Available here: https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter
I no longer see the issue with that extension.


Rules are not being applied for a particular situation (See the comment for more details)

Reproduction steps
Type 1

In Atom, open some files that are ruled by editorconfig (e.g. indent_size). In order to make issue appear obvious, rules in .editorconfig should contradict with atom's own editor settings (e.g.  indent_size = 2 vs Atom's Tab size = 4)
Reload Atom (leave editor tabs open). i.e. Window: Reload in Command Palette or Ctrl-Shift-F5
Select tab of file that is ruled by editorconfig

Type 2

In Atom, open some files that are ruled by editorconfig (e.g. indent_size). In order to make issue appear obvious, rules in .editorconfig should contradict with atom's own editor settings (e.g.  indent_size = 2 vs Atom's Tab size = 4)
Restart Atom (leave editor tabs open). i.e. Close Atom window then re-launch it
Select tab of file that is ruled by editorconfig

Expected behaviour
After step 2 of each type, editorconfig still has effect these editor tabs (e.g. tab makes 2 spaces)
Actual behaviour
After step 2, editorconfig has no effects on opened tabs, only Atom's own editor settings applies (e.g. tab makes 4 spaces)
Workaround
Closed opened tabs, re-open their files

Hi @KSXGitHub - thanks for your investigation. Your issue (at least "Type 1") is related to #176.
I would like to change this behavior, but i haven't had the time to dig into it.

Another temp workaround: disable and enable the package

Closing as Atom is abandoned.


ARROW-8158: [Java] Getting length of data buffer and base variable width vector

For string data buffer and base variable width vector can we have a way to get length of the data?
For instance, in ArrowColumnVector in StringAccessor we use stringResult.start and stringResult.end, instead we would like to get length of the data through an exposed function.
Now we have getStartOffset in BaseVariableVector and getElementStartIndex/getElementEndIndex in BaseListVector. To be consistent, we add getEndOffset for BaseVariableVector.

+1 thanks.  LGTM.


DestinationRule affects Deployment/Pod with unmatched workloadSelector

Is this the right place to submit this?

[X] This is not a security vulnerability or a crashing bug
[X] This is not a question about how to use Istio

Bug Description
I have two namespaces:

echo-server with echo server (xDS);
echo-client with echo service client (xDS).

Both client and server use proxyless xDS setup with grpc-agent sidecar.
In echo-server namespace I have DestinationRule and PeerAuthentication with enabling mTLS for echo service.
In this configuration echo-client works fine.
After creating another DestinationRule in echo-client namespace with host * and workloadSelector for unrelated Deployment (with exactly same trafficPolicy) echo-client lost mTLS setting for echo.echo-server.svc.cluster.local service.
Version
$ istioctl version
istioctl version
client version: 1.22.3
control plane version: 1.20.2
data plane version: 1.20.2 (509 proxies)

$ kubectl version
Client Version: v1.30.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.11-eks-db838b0

Additional Information
Resources looks like:
---
apiVersion: v1
kind: Service
metadata:
  name: echo
  namespace: echo-server
spec:
  type: ClusterIP
  clusterIP: None
  ports:
  - appProtocol: grpc
    port: 8888
    protocol: TCP
    targetPort: 8888
  selector:
    service: echo-server
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: echo
  namespace: echo-server
spec:
  host: 'echo'
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    outlierDetection: {}
    tls:
      mode: ISTIO_MUTUAL
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: echo-foo
  namespace: echo-client
spec:
  host: '*'
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    outlierDetection: {}
    tls:
      mode: ISTIO_MUTUAL
  workloadSelector:
    matchLabels:
      app.kubernetes.io/instance: echo-foo
      app.kubernetes.io/name: echo-foo
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/instance: echo-client
    app.kubernetes.io/name: echo-client
  name: echo-client
  namespace: echo-client
spec:
  ...

Thanks for the report. This is definitely not looking correct

I can't reproduce this issue.

The bug is that if there is a DestinationRule that matches the hostname in the current namespace (even if it does not match the workloadSelector), then the DestinationRule will not be searched in the namespace with the service:
https://github.com/istio/istio/blob/a1ff4e10a8d39d51fe54af40cf4157043fce92ef/pilot/pkg/model/push_context.go#L1128-L1149


required=True does not enforce field value existence

Describe the bug
box('input something', required=True) does not seem to actually require a value. It just adds a red asterisk above the text input, but when the "continue" button is clicked, the form submits with no value (or with an empty value?)
Expected behavior
Much like the native text input in modern browsers, provide a tooltip to the user saying "this field is required" or "{NAME} is required".
Actual behavior
The form submits without a value.
To Reproduce
Modify the hello world example, change one of the box line to:
feel = await view(box(f'How do you feel today, {name}?', mode='text', required=True))

Run this example and click all the "continue" buttons without typing anything into the text inputs.
Screenshots
I did not take any sceenshots.
Environment (please complete the following information):

Nitro version: 0.9.0
Python version: 3.10.4
OS and version: macOS 12.4
Browser and version: Vivaldi 5.2

Additional context
None.

@JokerQyou Thanks for trying out Nitro :-)
Currently, the required= field simply marks the field with an asterisk. I agree that most would expect it to also validate and block submission if missing.
It's on the list of things to do - thanks for bringing it up.


misssing Chinese translation docs: content/en/blog/_posts/2022-04-28-Increasing-the-security-bar-in-Ingress-NGINX/

This is a Feature Report
Problem:
misssing Chinese translation docs: content/en/blog/_posts/2022-04-28-Increasing-the-security-bar-in-Ingress-NGINX
Proposed Solution:
add Chinese translation docs in website Kubernetes Blog
Comments
/language zh
/assign

English page: https://kubernetes.io/blog/2022/04/28/ingress-nginx-1-2-0/

/triage accepted


Avoid rebuilds on makefile change

Resolves https://github.com/ascoderu/lokole/issues/419

Codecov Report

Merging #421 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #421   +/-   ##
=======================================
  Coverage   76.52%   76.52%
=======================================
  Files          43       43
  Lines        2718     2718
=======================================
  Hits         2080     2080
  Misses        638      638

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f85a763...c2d0c7d. Read the comment docs.


the plugin doesn't work properly with the very first opened/created file every time st2 restarts

repro:

quit ST2 WITHOUT any ts file opening
open ST2 and create a new file, save it as a.ts, and write some code, you'll find it's laggy, and you can not build it with Cmd + B (nothing with the plugin works, actually)
create another file saved as b.ts, it's working properly.
keep b.ts as the current active file, and quit ST2 with Cmd + Q
reopen ST2 and you'll find the issues come with b.ts, but a.ts works properly.
close b.ts and reopen it, you'll find it working properly.

Item
Version

ST
2.0.2 build 2221

plugin
0.1.14

System
OS X 10.11.5

@zhengbli any thoughts?

I just encountered a very similar issue with Build 3114 on OSX.
Once I added a tsconfig.json file:

The second file I had created was working on responded properly to Build (Cmd-B)
Cmd-B did absolutely nothing for the first file I had created.

After restarting Sublime, both of them work as expected.
(I just started using TypeScript a few hours ago, so I assumed I was doing something wrong. This certainly seems like a natural case a beginnner like me would find themselves in.)


Dev->Main: Foundational Structure

Add linting / README updates before merging
Changes:

Basic skeleton of latis-python-client.
Organized around LatisInstance objects.
Requires base url ('https://lasp.colorado.edu/lisird/latis')  and latis version (2 or 3).
LatisInstance objects on creation make a Catalog object.
Catalog objects can search for datasets using a key word or return a full list (instance.catalog.list or instance.catalog.catalog).
Dataset object can be created using dataset latis name ('https://lasp.colorado.edu/lisird/latis').
Dataset objects on creation make a Metadata object.
Work flow for Dataset objects is as follows:

Get metadata from Metadata object (dataset.metadata.metadata)
Create query with optional selection, projection and operation arrays
Get data in Pandas or Numpy fromat
Download data in csv or other format

Testing:

See/run example.py for basic usage of all features
Test cases are outdated don't run testClient.py/test.py

We can chat about this on Zoom later today, but I'll point out now that it should be possible to do something clever with our catalog/metadata classes such that we avoid the duplicated instance.catalog.catalog and instance.metadata.metadata:
https://realpython.com/inherit-python-dict/

Just about ready to merge. A few small things first:

The extra latis-python-client/ dir isn't needed; let's just move client/ and tests/ to the root dir
You updated example.py but not the README
We'd rather use "get dataset" instead of "create dataset" to clarify that we aren't creating data from scratch. Please change the relevant function names/comments/documentation?

Just about ready to merge. A few small things first:

The extra latis-python-client/ dir isn't needed; let's just move client/ and tests/ to the root dir
You updated example.py but not the README
We'd rather use "get dataset" instead of "create dataset" to clarify that we aren't creating data from scratch. Please change the relevant function names/comments/documentation?

The text above the README's datasets block still says "Create datasets." not "Get".
Lines 35 & 36 of example.py still say "Creating/Create`.
We should go ahead and rename LatisInstance.__createCatalog() to __getCatalog()

Make those changes please then we're good to merge as far as I'm concerned!

Does your IDE do any error checking for Python code? The last commit introduced a bug because line 14 of latis.py is still self.catalog = self.__createCatalog() while that function name was changed.

Does your IDE do any error checking for Python code? The last commit introduced a bug because line 14 of latis.py is still self.catalog = self.__createCatalog() while that function name was changed.

Sometimes? I use an open-source version of vscode on my arch machine. Usually, the best way to verify things is to just run it.

Squashing them is what I'd do.


Generate simple line for OneToOne relations, add title-font-size mgmt command param

SSIA - just needed this

Thank you for your contribution!


SAMZA-1832: Fix race condition between StreamProcessor and SamzaContainerListener

The PR addresses the following issues

We have few scenarios where there is a race condition between SamzaContainerListener and StreamProcessor that results in incorrect application status being propagated to client

Consider the scenario when the container runs into an exception in run loop and triggers shutdown sequence and at the same time the user triggers a stop on the StreamProcessor. The user request comes in and notices that the container is already shutting down and proceeds to shutdown the job coordinator which in turns shuts down successfully and the processorListener.onStop() is invoked. The container eventually invokes the SamzaContainerListener callback and updates the exception state after the StreamProcessor has finished shutting down.

Currently, we only propagate failures to processorListener when containerException is not null. It is possible for the samza container to take longer than task.shutdown.ms to shutdown in which case, we need to propagate a timeout exception to the processorListener as opposed assuming the shutdown was successful.

Make container shutdown idempotent since its only setting a boolean flag to true and there is no need to throw an exception on subsequent invocations.
1. It simplifies the interaction of other components (StreamProcessor) with container and prevents unnecessary check on container state to determine if its safe to call shutdown or not.
2. Enables to reason about the impact of container state change on StreamProcessor easily since we restrict container interaction w/ StreamProcessor only via SamzaContainerListener.

Looks like there're compilation errors in new tests, please fix them.


Image Customisation: Pangeo-style image with VNC/Linux desktop feature

This PR attempts to create the desired image for the Alabama Water Institute CIROH hub by following the instructions here: https://github.com/2i2c-org/infrastructure/issues/1444#issuecomment-1187405324

Copied Dockerfile from COESSING image: https://github.com/2i2c-org/coessing-image/blob/main/Dockerfile
Added lines to Dockerfile to install TurboVNC
Edited environment.yml file to install websockify and jupyter-remote-desktop-proxy

I am struggling to install TurboVNC with the following error:
  E: Invalid archive signature
  E: Internal error, could not locate member control.tar{.zst,.lz4,.gz,.xz,.bz2,.lzma,}
  E: Could not read meta data from /home/jovyan/turbovnc.deb
  E: The package lists or status file could not be parsed or opened.

After 6d4f05c (#1), image now builds successfully


GRL OIT support

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/15966/merge
Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15966/merge/index.html
To test a playground add it to the URL, for example:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15966/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/15966/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/15966/merge
https://gui.babylonjs.com/?snapshot=refs/pull/15966/merge
https://nme.babylonjs.com/?snapshot=refs/pull/15966/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/15966/merge#BCU1XR#0

Visualization tests for WebGPU (Experimental)
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15966/merge/testResults/webgpuplaywright/index.html

WebGL2 visualization test reporter:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/15966/merge/testResults/webgl2playwright/index.html

@Popov72 Hello! Is this as simpke as changing the fragmentOutputs output color name to support OIT or do I need to do more? Thanks!

It's a bit more involved:

you have to add two includes in your shader code (look for "oit" in default.fragment.fx for an example).
you have to add some JS code in your material. Look for "oit" in standardMaterial.ts to know what to do (you don't need the PrepareDefinesForPrePass call).

Also, the PG test link is for WebGL, whereas you only updated the WGSL shaders, so it doesn't show your changes. Another thing is that there's a "Cannot add an uniform after UBO has been created. uniformName=viewProjection" error with this link. It probably doesn't come from your PR, but it should be fixed nonetheless.

@Popov72
Since the GRL material is a material plugin are those two additional steps really needed? Isn't OIT handled by the standard and pbr material base shaders?

No, it's not needed, but I think greasedLine.fragment.fx is used by the standalone material, not the plugin?

Yes, it is a ShaderMaterial. I just added the OIT "support" shader code to a shader to be able to show the changes I'm going to make to all GRL shader involved in OIT and I didn't really choose the best one. Sorry to confuse you.
On the other hand since the GrasedLineSimpleMaterial doesn't support alpha I believe there is no need to implement OIT support in this material. Is this correct?

Yes, in that case you don't have to modify greasedLine.fragment.fx.
Regarding the plugin, you should inject your code in CUSTOM_FRAGMENT_BEFORE_FRAGCOLOR and not CUSTOM_FRAGMENT_MAIN_END. That way, it will automatically work with OIT/the pre-pass renderer.
With this change, instead of gl_FragColor, you should update color for standard materials, and finalColor for PBR materials. The easiest way to do this is to add something like that at the start of the shader code injected at CUSTOM_FRAGMENT_BEFORE_FRAGCOLOR:
#ifdef PBR
    #define grlFinalColor finalColor
#else
    #define grlFinalColor color
#endif

and use grlFinalColor instead of gl_FragColor.


Enable Catch-All Nginx-Proxy-Manager

The problem
hello, sorry to bother you, but I can't figure out if there's some problem or I'm not understanding the "Enable Catch-All Nginx-Proxy-Manager" feature. I'll explain.
I'm trying to pass all the traffic coming from the tunnel through the nginx proxy manager by enabling the voice. but I realize that even if I stop the nginx service it continues to route the traffic through the tunnel directly to HA through port 8123. So I wonder, is it right? Does it have to work like this? or is something wrong?
What version of Cloudflared has the issue?
5.2.2
What was the last working version of Cloudflared?
No response
What type of installation are you running?
Home Assistant OS
Add-on YAML Configuration
external_hostname: xxx.xx.xxx
additional_hosts:
  - hostname: xx.xxx.xx
    service: http://<IP_ADDRESS>
tunnel_name: tunnel-to-ha
nginx_proxy_manager: true
log_level: debug

Anything in the logs that might be useful for us?
DEBUG: Validating created config file: {"tunnel":"4f1f2f95-ada1-4bdc-9c0b-cd7d8a1e20d1","credentials-file":"/data/tunnel.json","ingress":[{"hostname":"xxx.xxx.xx","service":"http://homeassistant:8123","originRequest":{"noTLSVerify":true}},{"hostname":"xxx.xxx.xxx","service":"http://<IP_ADDRESS>","originRequest":{"noTLSVerify":true}},{"service":"http://a0d7b954-nginxproxymanager:80","originRequest":{"noTLSVerify":true}}]}

Steps to reproduce the issue
a
Additional information
a

solved


Conversion of .txt tracks to .csv to work with CellTracksColab

Hi all,
I managed to get Trackastra to work on my 4D dataset (515x512 xy with 81 z-steps and 5 timepoints), however I'd like to convert the tracking file from .txt to .csv format so I can work with it in CellTracksColab. However, my data is in this form:

1 0 1 0
2 0 4 0
3 0 4 0
...
(Full .txt file: res_track.txt)

And I don't think I'm fully understanding the format of this result; the first column should be unique track ID and the second may be timepoints, but the meaning of the third column eludes me as some values jump up to 1339. I can't figure out how to get x/y/z coordinates out of this result, which I need for CellTracksColab. Alternatively, would appreciate pointers to any tracking analysis packages you may know which can work with this .txt file (if I did everything correctly).

Hi,
The output file is in CTC format, i.e.
text file representing an acyclic graph for the whole video. Every line corresponds
to a single track that is encoded by four numbers separated by a space:

L B E P where

L - a unique label of the track (label of markers, 16-bit positive value)
B - a zero-based temporal index of the frame in which the track begins
E - a zero-based temporal index of the frame in which the track ends
P - label of the parent track (0 is used when no parent is defined)

Hope that helps!


Special Variables

Special variables like MAKE should be included for compatibility with make files

Added .RECIPEPREFIX

Added .DEFAULT_GOAL


Download/use the same podman as for the cluster on linux

This code https://github.com/crc-org/snc/blob/2a4aee82cf2eba7d0d1d51c634f15b72c1de8b44/createdisk-library.sh#L391 which was added in https://github.com/crc-org/snc/commit/f4d7855ff6cb452b9d8d4d3ee3bce8d27eaf7a51 is probably no longer needed.

Merged


[vtk[qt]:linux/qt5-x11extras] build failure

Host Environment

OS: [ubuntu 16]
Compiler: gcc/g++ 8.4

To Reproduce
Steps to reproduce the behavior:
./vcpkg install vtk[qt]:x64-linux or ./vcpkg install qt5-x11extras:x64-linux
Failure logs
Computing installation plan...
The following packages will be built and installed:
qt5-x11extras[core]:x64-linux
Starting package 1/1: qt5-x11extras:x64-linux
Building package qt5-x11extras[core]:x64-linux...
-- Using cached /home/super/Downloads/vcpkg-master/downloads/qtx11extras-everywhere-src-5.12.8.tar.xz
-- Using source at /home/super/Downloads/vcpkg-master/buildtrees/qt5-x11extras/src/5.12.8-6f63f291eb
-- Configuring x64-linux-rel
-- Configuring x64-linux-rel done
-- Configuring x64-linux-dbg
-- Configuring x64-linux-dbg done
-- Package build-x64-linux-dbg
-- Package build-x64-linux-rel
-- Package install-x64-linux-dbg
-- Package install-x64-linux-rel
-- Installing: /home/super/Downloads/vcpkg-master/packages/qt5-x11extras_x64-linux/share/qt5-x11extras/copyright
-- Performing post-build validation
The folder /include is empty or not present. This indicates the library was not correctly installed.
Found 1 error(s). Please correct the portfile:
/home/super/Downloads/vcpkg-master/ports/qt5-x11extras/portfile.cmake
-- Performing post-build validation done
Error: Building package qt5-x11extras:x64-linux failed with: POST_BUILD_CHECKS_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: qt5-x11extras:x64-linux
Vcpkg version: 2020.02.04-unknownhash
Additionally, attach any relevant sections from the log files above.
Additional context
I have update libx11 by sudo apt-get install libx11-xcb-dev libxt-dev .
I have remove all qt5-* packages and reinstall qt5-*. The errors still ocurred.

Missing system libraries. Try installing:
https://github.com/microsoft/vcpkg/blob/c21893b4dc68430ebcf028ffdb190e586fc3d639/scripts/azure-pipelines/linux/provision-image.sh#L8-L24
remove qt5-base und reinstall qt5-base. qt5-base must be installed with the correct system libraries installed so that qt5-x11extras build successfully.

@Neumann-A  Thank you. Now the vcpkg can install qt5 and vtk[qt] successfully on Ubuntu 16.  : )
However there is a small question. After installing provision-image.sh, Ubuntu cannot mount ntfs disk in nautilus, showing "no object for d-bus interface". The ubuntu forum suggests using gparted but warn "may lose data". Finally I mount ntfs disk by running sudo mount in bash.


update to coverage_attribute feature

Change Summary
Fixes CI issues after upstream nightly change https://github.com/rust-lang/rust/pull/114656
Related issue number
N/A
Checklist

[ ] Unit tests for the changes exist
[ ] Documentation reflects the changes where applicable
[ ] Pydantic tests pass with this pydantic-core (except for expected changes)
[ ] My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

please review


message_row: Reduce space above collapsed Show More button.

This only impacts collapsed Show More buttons in a message with a sender.
Fixes #31221
Screenshots and screen captures:

Before
After

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[x] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[x] Responsiveness and internationalization.
[x] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

Looks great, marked to merge once CI passes. Thanks @karlstolley!


update required focus-core version in package.json

I don't agree with this one, focus-core should be a dev dependency, and it's up to date there.


Support barge moves via the up command

As a user, I'd like to be able to change my barge value in harbor-compose.yml and run up and have it do to the right thing.
Under the covers, harbor-compose should detect the barge change, bring the current stack down with the down command, update the barge setting on the ec2 provider, and then bring up the new stack on the new barge.

Moving to terraform.


AUTHORING DOCS: Indicate that NO PTs in a nested RT should be required

If a nested RT includes required PTs, this will cause validation errors if no value given for the nested-template PT!!

@briesenberg07 wiki has been updated


Return Commentary Generation working

It is now possible to generate @return Parameters!
And the type is only shown if there is one entered - no empty {} anymore!

awesome thanks!
Going to add some more features soon!


Show Facebook icon in navigation bar

This PR adds the Facebook Icon into the navigation bar, provided a Facebook URL is set up...
Fixes #776

This does need some additional design work, though... 😊
Probably at least the spacing between the facebook and twitter icons needs to increase

Hey @goranche, thanks for your contribution. But we just fixed the issue in 3fa7f25147c92da992fa589bed53f36d79990f7e

Hey @goranche, thanks for your contribution. But we just fixed the issue in 3fa7f25

How could you? 😁
great, thanks 😉  👍


Bug in the demo

On the Demo website
If you check "Custom emoji" then click on the flag tab then uncheck "Custom emoji"
The emoji picker freeze with an error in the console :
picker.js:843 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'id')

Thank you, confirmed. Looks like a bug when dynamically adding or remove custom emoji.


tests for analysis and util

Add in unittests for:

[x] spaghetti.analysis.py

[x] spaghetti.util.py

all functions covered (100%) in #95

resolve in #98


Use Cart rather than the model

The call to Cart::conditions() fails in admin.order.paymentProcessed without this.
Unless I'm misunderstanding - this shouldn't be referencing the model as it doesnt provide a conditions() method

Good spot


Getting this error (The provided information does not map to a role assignment.) when trying to run the start-ambacleanup.ps1 script

Check for previous/existing GitHub issues

[X] I have checked for previous/existing GitHub issues

Description
We deployed to the Connectivity and Management Management Groups and I remediated the policies in the Connectivity Management Group. I have run this cleanup script before multiple times and have never gotten this issue.

Hi @lmjaanes-mck, thank you for your feedback. Since it usually works, is there anything different in your current deployment? Could you provide the command you used to deploy AMBA, so I can check the version? I'll review the script and try to replicate the issue.

Hi @lmjaanes-mck, I've reviewed the script and I havent been able to replicate the issue.
However, due to a separate issue, the script has been updated. Could you try running it again with the newest version?

Thank you for your feedback. I'll close the issue since we couldn't replicate it. Please inform us if it happens again.


Added isInPast and isInFuture methods

Hi! It would be awesome if we can add these 2 methods to make the code more beautiful :P Sometimes we need to check if the date isInPast to reflect changes in the UI.
Very useful library, thanks! Good job :clap:

Thanks!


cannot compile macros that contain vim namespace

Some of my macros contain various vim namespace functions used to simplify the macro signature:
; this is *not* a macro, it used in macros
(fn scope [opt]
  "Get the scope of an option"
  (let [opt# (tostring opt)]
      (. (vim.api.nvim_get_option_info2 opt# {}) :scope)))

these do not compile with nfnl, saying that vim is an unknown identifier.
.nfnl.fnl file
{:source-file-patterns ["test/fnl/**/*.fnl"
                        "fnl/**/*.fnl"]}

the macros are imported with import-macros for the file i am attempting to compile

So this is actually a vanilla Fennel compiler thing that Aniseed insulated people from by making some large Assumptions for everyone. Those assumptions could cause issues for others, so now I leave everything in the Fennel compiler as default and let you choose if you want to make these changes.
You're running into the compiler sandbox I think, something that prevents you from referring to things that aren't good known global values at compiler / macro time. You can either but _G. in front of each of them or disable these checks with one of these options in your .nfnl.fnl. I can't remember which one it is in this situation so you might have to try one and then the other (from the Fennel API docs):
{:compiler-options {:allowedGlobals ["vim"] ;; or just false to disable checks
                    :compilerEnv _G}}

There may be an argument to be made here to change the default :allowedGlobals to ["vim"] if that works for you. And maybe add some more things we know exist in the Neovim ecosystem. I suspect however it's because you're reaching outside of the compiler environment / sandbox without a _G. prefix or :compilerEnv not being disabled.
One of these should fix the issue I hope!

setting :compilerEnv to _G seems to work just fine, the function gets fired as it should during compilation. might be good to mention this in the docs

Yay! I hope the readme updates are helpful too 😄

looks good, thanks!


🎨(api) convert camelCase variables to snake_case

Purpose
Ensures PEP8 compliance without disabling PyLint.
Proposal
Implemented camelCase aliases on attributes or routes' parameters.
Let me know if I am being too pedantic, and this breaking change is not worth it.

@lebaudantoine some camelCase remain in Pydantic model in models. To align with this PR, we could standardise all our Pydantic models, but that could be done in another PR

Issue opened @quitterie-lcs https://github.com/openfun/ralph/issues/439


how to get final precision when test model?

Hello, thank you for your contributions. I am using your project and encountered a problem. I have checked all the issues and did not solve my problem. My problem is: When I tested the model, I did not get the precision data. Only gts, dets, recall, ap, my data is in voc format, I have used --eval mAP, it still has no precision, in addition, using --eval mAP recall, the following problems occur
   metric = dataset.evaluate(outputs, **eval_kwargs)   File "/home/ailab/anaconda3/envs/mm/lib/python3.7/site-packages/mmdet-2.18.0-py3.7.egg/mmdet/datasets/voc.py", line 60, in evaluate     assert len(metric) == 1 AssertionError
1.How can we get precision through the model inference in voc data.
2.If not, how can I get it through training and verifying the model.
3.dets is all the boxes predicted by the model, how to get the final number of predicted boxes

What command did you run?

What command did you run?

os.system(f"python {test} {config} {checkpoint} --eval mAP") os.system(f"python {test} {config} {checkpoint} --eval mAP recall")

1.How can we get precision through the model inference in voc data.

python {test} {config} {checkpoint} --eval mAP command is correct. And the mAP is the precision. Do not add recall after that. The evaluation currently only support one eval type.

3.dets is all the boxes predicted by the model, how to get the final number of predicted boxes

What do you mean final number of predicted boxes? If you want to get the number of boxes with a higher score threshold, just modify the score_thr in the config file.

1.How can we get precision through the model inference in voc data.

python {test} {config} {checkpoint} --eval mAP command is correct. And the mAP is the precision. Do not add recall after that. The evaluation currently only support one eval type.

3.dets is all the boxes predicted by the model, how to get the final number of predicted boxes

What do you mean final number of predicted boxes? If you want to get the number of boxes with a higher score threshold, just modify the score_thr in the config file.

Thanks for the answer, in fact, what I mean is that the number of dets is significantly higher than the number of gts. The detection frame needs to be filtered, nms, so that many invalid frames can be eliminated, and then we can calculate the recall and precision based on the remaining detection frames. I want to know The recall value obtained here, whether the prediction box used has been processed。

Because of the large number of dets, the recall calculated based on it should be very high, and the corresponding precision should be very low.

These boxes are already filtered by the test_cfg set in your config. If you want to filter more boxes, just modify your config such as changing score threshold or nms iou threshold.

1.How can we get precision through the model inference in voc data.

python {test} {config} {checkpoint} --eval mAP command is correct. And the mAP is the precision. Do not add recall after that. The evaluation currently only support one eval type.

3.dets is all the boxes predicted by the model, how to get the final number of predicted boxes

What do you mean final number of predicted boxes? If you want to get the number of boxes with a higher score threshold, just modify the score_thr in the config file.

I modified mean_ ap.py  ,  i can  get the precision data. The result is the same as expected, that is,

you use all the detection boxes to calculate the recall and precision, so the recall is very high and the precision is very low.
According to the intuitive feeling, the AP should also be very low, but the result is just the opposite.
In addition, when evaluating the model, i also saved the pictures and found that there are not many prediction boxes, It means that you eliminate the invalid prediction bounding box after calculating the evaluation index(recall , recision), which is also my question. Why not calculate the evaluation after that.
How can I get the coordinate information and label information of the last prediction box？

#1231
#506
+-------+-------+-------+--------+-----------+-------+
| class | gts   | dets  | recall | precision | ap    |
+-------+-------+-------+--------+-----------+-------+
| class1  | 1108 | 3585 | 0.998  | 0.308     | 0.951 |
| class2  | 95   | 935  | 0.981  | 0.100     | 0.807 |

These boxes are already filtered by the test_cfg set in your config. If you want to filter more boxes, just modify your config such as changing score threshold or nms iou threshold.

ok， Modified score_thr, changed 0.05 to 0.25, the number of prediction boxes obtained was reduced a lot, and the result looked normal. It is recommended to increase the default value.


Final map for Presentation Night

Hi, Matt and Hector.
This is my map for tomorrow.  Since you last saw the map, I added:

popups
records for >90 orgs.  this is nearly 40% of the orgs.  data collection is killing me.
a whole bunch of design stuff, including: hero image, about section (in a modal), and footer

I also made some adjustments in the JS file.
Please let me know if/what needs to change before tomorrow.
Thanks,
Christina
Addresses #19

Looks great, @cahdeemer!
I'm going to merge this and deploy it so that you can poke around to make sure everything works as expected on the live site.


Subscriber groups/shared subscriptions

@dergraf Ready for review

policies implemented: random and prefer_local. We could consider adding only_local as well.
enqueue_many with options only implemented in 'online' state. Should probably be implemented in others as well?
needs to be verified against the MQTTv5 spec.
must be squashed before merging.

LGTM! :)

Squashed and rebased the subscriber group onto master and changed the message compatibility layer so the cluster com is compatible with pre-subscriber group nodes (need to test this)

Link this PR to the related issues: #230 and #221

I think as the official spec isn't final yet, this is good to merge, we can change it later, and mark this feature as beta.
comments?

I think that's fair. I'll do some mixed-cluster testing and merge if that looks OK. Then I'll start documenting it (and include a note about the beta status etc).

Hi @dergraf it looks good to me - my mixed-cluster tests didnt't reveal any issues with the cluster comm. I added an intermediate entry to the changelog so we can document features as we add them and then we'd only need to change the heading when we do the actual release.
Could you take a quick look at the changelog? If the content there is OK with you I'll merge it (or feel free to do it)

guys,
Is it included in the release?
Trying it in 0.15.3 and don't see it's working

Oh, I see. Thanks for the heads-up!


Post logout redirect only works when browser is set to block third party cookies.

Core Library
MSAL.js v2 (@azure/msal-browser)
Core Library Version
2.14.1
Wrapper Library
MSAL Angular (@azure/msal-angular)
Wrapper Library Version
2.5.4
Public or Confidential Client?
Public
Description
I have ran into an issue when redirecting on post logout when in non incognito mode in chrome and Firefox.
It works perfectly fine when I set my browser to block third part cookies but when I allow all cookies I do not get redirected.
This also works fine in Safari.
Error Message
No error message, the browser never get's redirected to login page after logout.
Msal Logs
No response
MSAL Configuration
{
    auth: {
      clientId: "b5c2e510-4a17-4feb-b219-e55aa5b74144",
      authority: "https://login.microsoftonline.com/common",
      redirectUri: '/',
      postLogoutRedirectUri: '/login',
    },
    cache: {
      cacheLocation: BrowserCacheLocation.LocalStorage,
      storeAuthStateInCookie: false, // set to true for IE 11. Remove this line to use Angular Universal
      secureCookies: false,
    },
    system: {
      loggerOptions: {
        logLevel: LogLevel.Verbose,
        loggerCallback: (level, message, containsPii) => {
          console.log(message);
          },
        piiLoggingEnabled: false,

      }
    }
  }

Relevant Code Snippets
logout(popup?: boolean) {
    if (popup) {
      this.authService.logoutPopup({
        mainWindowRedirectUri: "/"
      });
    } else {
      this.authService.logoutRedirect();
    }
  }

Reproduction Steps

Have chrome browser cookies set to block third party cookies in incognito mode only.
Login to app in NON incognito mode.
Logout of app and the window never gets redirected to the /login endpoint.

Expected Behavior
For the logout to redirect the user to the post logout redirect URL when in non incognito mode in Chrome and Firefox.
Identity Provider
Azure AD / MSA
Browsers Affected (Select all that apply)
Chrome, Firefox
Regression
No response
Source
Internal (Microsoft)

Hey @benaguitar ! Are you seeing this issue in both logout with redirect as well as popup?

Hi @lalimasharda! Thank you for looking at this.
I am seeing this with both logout with redirect as well as popup.
What is weird is I can get logout with redirect to work when I set my third party cookies to be blocked in chrome and firefox.
Have anything I can try that would make a difference when have the cookies set to allow all?

I am a little confused about the problem. In your description you said that when you set "Allow all cookies" to true, your application does not redirect you back to login page after logging out. But in your repro steps you are asking to set "Block third-party cookies in Incognito" to true instead and the application doesn't redirect back to login page.
In any case, I tried out our angular 6 sample app with both the settings and I am not able to reproduce it. Can you please share logs?

Hello,
The first step can be confusing, I was saying that to make sure your chrome browser has third party cookies only blocked in incognito mode or allow all cookies, sorry about that.
In the angular 6 sample app it works as expected but I am testing this with the Angular14-rxjs7-sample-app and see the issue.
I have attached the console logs, let me know if you want the network .har file and I can email you the file.
login.microsoftonline.com-console-logs.log

@lalimasharda I did get this to work by creating a user within Active Directory verses using the admin user of the Active Directory or using an external user.
I have read that this was an issue in the past but seems to still be an issue that's not on the MSAL Angular side.
Any thoughts?

@benaguitar Is this same as #5783?

Hi @sameerag Yes this is the same issue we determined.

I will be closing this and link it to #5783 for traction in that case. Thanks @benaguitar.


os.path.isfile(unix socket) always returns False

https://github.com/ateska/ramona/blob/master/ramona/socketuri.py#L128
assert not os.path.isfile(fname)

os.path.isfile always returns False on a unix socket file (at least in my system):
>>> sys.version
'2.7.3 (default, Mar 13 2014, 11:03:55) \n[GCC 4.7.2]'
>>> os.path.exists("unix.sock")
True
>>> os.path.isfile("unix.sock")
False
>>>

I think, you should consider changing this assertion, as it always succeed.

Hi,
Very good observation.
I'll change that assertion.
Thanks,
Ales


in navigation bar the remaining distance always show in metric.

even instantiating the units Navigation Options  as metric, the navigation panel always show the distance in imperial units.

The navigation are getting the android systems locale.
So if you set the locale to US, the navigation will be in imperial (no care if you select metric).
Do you know if the sdk has a way to override android system units and use the user configured ones?

It uses the device settings in some scenarios. Adjust your device to see if it picks it up. I have a PR coming that fixes this issue

Yes, if i set my device to english UK  (instead of US) it works in metric systems.
do you have some idea when you will push the PR?

@fabioqueiros  i set   language: "en-uk",  do the job for me.

Stale issue message


HTTP 202 - Accepted for GET and POST

I have a case where a POST can start creating a resource, and give a URL for where it's going to exist, but the client shouldn't expect it to be complete yet. I believe the right response code is 202.
GETs to the resource can provide an indication of the progress of making the resource but shouldn't give response code 200 until it's actually complete, at which point the body will represent the final resource. I believe 202 is also the right response code here.
From http://restpatterns.org/HTTP_Status_Codes/202_-_Accepted it sounds like it's not necessarily the actual resource location that gives the progress info, but if GET can respond with 202, I don't think there's anything wrong with that (the progress info fits into my data model anyway).
Is this something cowboy's REST handler could be made to support? Why is the delete_complete callback so specific?
For GET and POST (and HEAD, I guess) I'd propose an is_complete callback.
JD

For "POST 202 Accepted" possible solution mentioned in #757
While the RFC does not appear to specify which methods can result in 202, since it means "the request has been accepted for processing", it does not seem appropriate for GET.
The REST cookbook recommends if you POST and receive 202, a subsequent GET can return:

Still processing: 200 with status
On successful completion:  303 to redirect
On task failure: 200 with error message

There's only wording about 202 for DELETE, which we handle. There's no wording for other methods so it's up to you. I would recommend continuing to use cowboy_req:reply/2 along with a stop tuple when you need a 202 for non-DELETE methods. Let's keep cowboy_rest close to the standards.
Closing, thanks!


Groups should not exist before managing groups that don't exist in tests

Attempting to solve an intermittent issue, or at least help identify it.
http://manhattan.ci.chef.co/view/Chef/job/chef-test/architecture=x86_64,platform=windows-2008r2,project=chef,role=tester/153/console
1) Chef::Resource::Group group manage action when there is no group does not raise an error on manage
     Failure/Error: expect { group_resource.run_action(:manage) }.not_to raise_error
       expected no Exception, got #<ArgumentError: group[t-2013] (dynamically defined) had an error: ArgumentError: A member could not be added to or removed from the local group because the member does not exist.> with backtrace:
         # ./lib/chef/util/windows/net_group.rb:38:in `modify_members'
         # ./lib/chef/util/windows/net_group.rb:89:in `local_set_members'
         # ./lib/chef/provider/group/windows.rb:78:in `manage_group'
         # ./lib/chef/provider/group.rb:154:in `block in action_manage'
         # ./lib/chef/mixin/why_run.rb:52:in `call'
         # ./lib/chef/mixin/why_run.rb:52:in `add_action'
         # ./lib/chef/provider.rb:170:in `converge_by'
         # ./lib/chef/provider/group.rb:153:in `action_manage'
         # ./lib/chef/provider.rb:135:in `run_action'
         # ./lib/chef/resource.rb:578:in `run_action'
         # ./spec/functional/resource/group_spec.rb:380:in `block (5 levels) in <top (required)>'
         # ./spec/functional/resource/group_spec.rb:380:in `block (4 levels) in <top (required)>'
     # ./spec/functional/resource/group_spec.rb:380:in `block (4 levels) in <top (required)>'

Meh. Unrelated intermittent failure on Travis. Ugh.
First chef-test green in a while though. I'm for merging this even though I'm not confident on the benefit yet @chef/client-core.

:+1: go for it

@jdmundrawala We saw this failure again on last nights build on one system. Still not sure this is the right fixed, but was green for me and seems like an improvement.

go for it :+1:
not sure why travis is sad. It seems to be trying to get the wrong commit

Merged as  c58162b9ce165bde444ff396a95668666884c052


SFI-Swin: Symmetric Face Inpainting

Dear reaserchers, please also consider checking our newly introduced face inpainting method to address the symmetry problems of general inpainting methods by using swin transformer and semantic aware discriminators.
Our proposed method showed better results in terms of fid score and newly proposed metric which focus on the face symmetry compared to some of the state-of-the-art methods including lama.
Our paper is availabe at:
https://www.researchgate.net/publication/366984165_SFI-Swin_Symmetric_Face_Inpainting_with_Swin_Transformer_by_Distinctly_Learning_Face_Components_Distributions
The code also will be published in:
https://github.com/mohammadrezanaderi4/SFI-Swin

Thanks!


Getting error when opening Houdini

Hey there, currently I am getting the following error each time I open Houdini. I tried in v18.348, 353 and 355 and it did it in all of them.
Error running event handler:
Traceback (most recent call last):
  File "opdef:/hammer::Sop/group_expand::1.0?ViewerStateModule", line 1, in <module>
ImportError: No module named universal_state

Error running event handler:
Traceback (most recent call last):
  File "opdef:/hammer::Sop/group_expand::1.0?ViewerStateInstall", line 1, in <module>
  File "C:/PROGRA~1/SideFX/HOUDIN~1.355/houdini/python2.7libs\viewerstate\utils.py", line 1099, in register_pystate_embedded
    template = node_type.hdaViewerStateModule().createViewerStateTemplate()
  File "C:/PROGRA~1/SideFX/HOUDIN~1.355/houdini/python2.7libs\hou.py", line 39278, in __getattr__
    def __getattr__(*args): return _hou.HDAViewerStateModule___getattr__(*args)
AttributeError: 'module' object has no attribute 'createViewerStateTemplate'

Thanks,
-MH

Oh, I'm sorry that I noticed your problem just now. Fixed in the dev branch.


Create Superadmin mode.

Show all the private datasets/files/folders if only the user switches to Super admin mode. Else, the user should be able to see only it's own and shared datasets/files/folders.

Fixed in #819


Update links in red tile blog (EN)

This PR resolves issue #2229 "Links in "What should you do if you see a red tile?" (EN)". It aligns https://www.coronawarn.app/en/blog/2021-12-15-cwa-red-tile-guidance/ with the changes made in the equivalent German language blog https://www.coronawarn.app/de/blog/2021-12-15-cwa-red-tile-guidance/ (see PR #2202).

@MikeMcC399 Thanks for providing PR.


Readme file variable name mistake in code

In your readme file, the sample text creates array name bandArray but adds object into fruitsArray. Both of them should be same

Done.


Custom language for "Current Usage"

When displaying the current usage column in a list of assets, it just takes up a lot of room horizontally. While the usage count just takes up 1 or 2 digits, "Current Usage" is 13. I'd like a setting to be able to change the column header. In my case I just want it to be "U" to take up as little horizontal space as possible.

@chadcrowell the labels are a bit large. Do you think naming the columns "Used" and "In use" will be better and still clear enough? And naming the values "{count} times"?

Ideally, I think a plugin setting for the column label would be the best, maybe with a default of "Usage" (for American english, anyway, that would be the shortest term that makes sense with just the count in the table). If you don't want to provide a setting than "Usage" would be my preference, and just the count in the table as it currently is.

@chadcrowell, there are 2 types of columns with different numbers, which is why i'm trying to label only one as "Usage".
Currently the numbers are rendered as "Used {count} times", so i don't know why you are seeing just the numbers. You can change all texts from your own translations folder, which is why i'm reluctant with adding a setting for labels.
I was thinking about changing to the following labels, can you give your opinion on this?
'Used' // Old usage table
'{count} time' and '{count} times' // are the time/times additions nedded?
'Unused' // should this be just 0?
'In use' // Could be Usage, anyway this is the current column

I never noticed there were 2 columns available via your plugin. Showing them both here: https://d.pr/i/0DaIli - I guess the one already says "Usage". I had only previously enabled the "Current Usage" column and never noticed the other one. What is the difference between them?

OK re-read the docs... got it. "The "Usage" field shows all relations and the "Current Usage" field shows relations excluding revisions and deleted elements." Honestly I just need the Current Usage field and would like that label to be shorter... but I think you got that from my initial request.
Maybe Total Usage & Live Usage would be appropriate?