Training with very small objects

Originally posted by @VincentDuf in https://github.com/NVIDIA/retinanet-examples/issues/8#issuecomment-602027330
Hi,
I have the same issue using the fine-tune method. I changed the learning rate twice (0.0005 and even 0.0001) but still after few ten's of thousand of iterations it prints "no detections" and I confirmed it after shutting the training down and try using "retinanet infer"...
I'm wondering if my objects aren't too small, I'm trying to detect aphids on plants (see image linked, https://fromsmash.com/5_svMDKrmh-ct) and I have a small dataset of 2840 images for training and 720 for validation.
(2 precisions:

I only labelled the aphids on the leaf not on the ground, there were too much of them
I have 2 classes of aphids, green and pink, but on an image there are really similar so maybe I should merge them as one).
So do you think I should do more augmentations because Retinanet can handle that size of object of should move to segmentation algorithm because this is too small ?

Thank you

Those objects are very small. Can you try training again, but this edit model.py here:
https://github.com/NVIDIA/retinanet-examples/blob/master/retinanet/model.py#L29
Can you set
self.scales = [1, 2, 4]

Please let me know if that works.
If not, try changing these lines:
https://github.com/NVIDIA/retinanet-examples/blob/master/retinanet/box.py#L290
https://github.com/NVIDIA/retinanet-examples/blob/master/retinanet/box.py#L291
https://github.com/NVIDIA/retinanet-examples/blob/master/retinanet/box.py#L301
So that all of the 0.5 or 0.4 values become 0.2.

One more idea -- try this before changing the hardcoded values in box.py: Please also try padding your annotations by a few pixels (eg four)
So if your existing annotation is
x, y, w, h

Try training with new annotations as
x - 2, y - 2, w + 2, h + 2

Ok thanks, so if I sum up, I should try:

training with modified model.py file
training with padding annotations (with original or modified model.py ?)
training with modified box.py (again with original model.py and annotations or with the changes ?)

Thanks

Thinking about it, I'd try padding the annotations first, then try reducing self.scales in model.py (with the padded annotations.
Let me know how that goes before you try changing the IoU threshold in box.py.

Hi,
I just finished with the self.scales modified training but it didn't help much (I only trained on 30.000 iterations as I'm fine-tuning and not full-training). Should I keep the learning rate equal to 0.0005 or should I try 0.0001 ?
I'll add the padded annotations to see how it goes, and increase to 45k iterations.

What focal and box loss are you seeing? While we're experimenting, I'd ramp your learning rate up as high as you can (eg 0.001) so that you can see if your network is learning.
The problem is that your targets aren't registering as foreground objects because the IoU between your targets and the anchors is below the threshold. We are trying three things:

Reducing the size of the anchors, which will increase the IoU with your targets.
Increasing the size of your targets (the padding), to also increase the IoU.
Decreasing the IoU threshold at which the network regards an object as foreground, so starts to predict it.

We will find a combination that works. Perhaps try with more padding (eg 10 pixels each side), show that it works, and then reduce the padding and see what effect that has.
What size is your original image? Are you using the default --resize 800? You could increase this (and also increase --jitter during training) to make your targets bigger with respect to the anchors.

So far I see a low value for the first few iterations and then it falls to 0 (both bbox and focal loss).
Okay so I just tried the 2 first option (padded + anchors scale), I'll now add the IoU threshold and increase the learning rate.
My images are 1080p (1920x1080px), below that it was too hard to label the object manually. I don't use the resize option so it takes 1333p as a max from what I understand.
I didn't get what the jitter option would be useful for..
I keep on trying and I'll let you know how it goes. I'll increase the padded value to.

Those losses are too low -- your targets are disappearing into the background.
Try adding --jitter 1000 1200 --max-size 2000 --resize 1080 to your train command. This will resize your training images so that the short length is between 1000 and 1200 pixels and the long length is a maximum of 2000 pixels.
We want to see your losses greater than 0.1 for the first 30k iterations, otherwise it's not learning.
Have you labelled all of the aphids in the scene?
What padding did you use? Maybe add a little more until we see it learning properly.

@VincentDuf As a reminder, make sure that you run pip install -e . from within the container, or rebuild the container, after you change the code.

Ah yeah I forgot to rebuild the repo after the changes...I don't use the docker but still I didn't do it.
Ok I'll add the options.
I labelled the aphids that are on the plants only to create the association plant/aphid otherwise it would have been a titan work to label all of them especially the one on the ground. More over my interest is to detect aphids on the plant (the part on the ground isn't really representative of a real density of aphids around plants).
I applied the padding twice so far so it's already [x–4,y-4,w+8,h+8].
I rebuild the whole thing and launch another try.

"Detect object A only when it is on object B" is a hard training signal -- the network is set up for "detect objects A and B" and then post processing to detect an overlap.
Another thing to consider is to crop your training images to the area that you've fully labelled (for now), then the task becomes just object detection. Remember to update the values in --jitter 1000 1200 --max-size 2000 --resize 1080 to make sure they make sense with your smaller dataset.
I'd try all of these things (smaller anchors, padding, crop your image to the area labelled) before you adjust the IoU threshold.

Oh okay I supposed falsely that when training a network the "context" mattered, like in my case where an aphid on a plant is considered so because it's essentially a grey dot in a green area (gross idea) and a grey dot in a brow area (the ground) wouldn't be consider as an aphid..
Ok yeah cropping the images is a good idea, I'm working in parallel on a simple plant detector with classic image processing techniques and that could be a part of 2 stages detector. I'll dig into that.
It will probably be hard to delete all the non labelled object from the image but I can definitely reduce the number.
Okay thank you very much for your extensive help. I learned a lot 👍

One last thought: One of the 80 COCO classes is "potted plant". You can look at some examples here: http://cocodataset.org/#explore
We provide trained models (trained on COCO) for this network, under the 'release' tab. You might be able to use that network to detect your plants, then crop your images in an automated fashion.
I'll close this now, but please open a new issue if you have problems.

Oh so you would crop the images using the network then augment the dataset and then train a new one using retinanet to detect aphids on that plant ?
I'll have a look to how I can do that.
Thank you again I'll let you know how things go with the training.

Hi,
I'm facing an error I don't really get what's the problem:
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/numpy/core/function_base.py", line 117, in linspace
    num = operator.index(num)
TypeError: 'numpy.float64' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/greenshield/.virtualenvs/cv/bin/retinanet", line 11, in <module>
    load_entry_point('retinanet==0.1', 'console_scripts', 'retinanet')()
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/main.py", line 181, in main
    worker(0, args, 1, model, state)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/main.py", line 127, in worker
    metrics_url=args.post_metrics, logdir=args.logdir, verbose=(rank == 0))
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/train.py", line 154, in train
    mixed_precision=mixed_precision, is_master=is_master, world=world, use_dali=use_dali, is_validation=True, verbose=False)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/infer.py", line 141, in infer
    coco_eval = COCOeval(data_iterator.coco, coco_pred, 'bbox')
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/pycocotools/cocoeval.py", line 76, in __init__
    self.params = Params(iouType=iouType) # parameters
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/pycocotools/cocoeval.py", line 527, in __init__
    self.setDetParams()
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/pycocotools/cocoeval.py", line 507, in setDetParams
    self.iouThrs = np.linspace(.5, 0.95, np.round((0.95 - .5) / .05) + 1, endpoint=True)
  File "<__array_function__ internals>", line 6, in linspace
File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/numpy/core/function_base.py", line 121, in linspace
    .format(type(num)))
TypeError: object of type <class 'numpy.float64'> cannot be safely interpreted as an integer.

I didn't touch anything from the database and it's the second time it occurs at the exact same number of iteration.
If you want the full log file I have it.

That error is coming from pycocotools -- my guess is it doesn't like something in your dataset.
Can you see if you can find an error in your data format?
If not, please download the latest version of this repo (you are on version 0.1, latest is 0.2.1) and then open a new issue if you're sure it's not a dataset problem?

Hey,
I found the issue, it was caused by numpy, the version 1.18 isn't compatible with pycocotools, here is the thread that helped me:
https://stackoverflow.com/questions/59493606/pytorch-and-tensorflow-object-detection-evaluate-object-of-type-class-nump
It is finally learning and I even have those printed information after a first epoch:
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.094
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.238
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.053
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.094
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.020
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.098
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.126
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.126
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

Can't tell what it really means but it looks like the process is finally running properly.

Hi,
I'm facing a space issue, what's the procedure to correct this ?
(To be noted I have more images than previously..)
Traceback (most recent call last):
  File "/home/greenshield/.virtualenvs/cv/bin/retinanet", line 11, in <module>
    load_entry_point('retinanet==0.1', 'console_scripts', 'retinanet')()
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/main.py", line 181, in main
    worker(0, args, 1, model, state)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/main.py", line 127, in worker
    metrics_url=args.post_metrics, logdir=args.logdir, verbose=(rank == 0))
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/train.py", line 80, in train
    cls_loss, box_loss = model([data, target])
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/apex/amp/_initialize.py", line 197, in new_fwd
    **applier(kwargs, input_caster))
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/model.py", line 111, in forward
    return self._compute_loss(x, cls_heads, box_heads, targets.float())
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/model.py", line 158, in _compute_loss
    cls_loss = self.cls_criterion(cls_head, cls_target)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/retinanet-0.1-py3.5-linux-x86_64.egg/retinanet/loss.py", line 15, in forward
    ce = F.binary_cross_entropy_with_logits(pred_logits, target, reduction='none')
  File "/home/greenshield/.virtualenvs/cv/lib/python3.5/site-packages/torch/nn/functional.py", line 2126, in binary_cross_entropy_with_logits
    return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
RuntimeError: CUDA out of memory. Tried to allocate 52.00 MiB (GPU 0; 7.94 GiB total capacity; 7.27 GiB already allocated; 32.56 MiB free; 7.44 GiB reserved in total by PyTorch)

Thanks

What batch size are you using? Try reducing it to the number of GPUs you are using, so if one GPU, try --batch 1

Thank you it worked so far.
Another question: I have now 4200 images for the training of one classe only do you think I can try a full training instead of fine-tuning the pre-trained model ?
Thanks

Both would work, but I expect fine-tuning to work better. Try both if you have the time / compute resources.

Hey,
I just ran a complete fine-tune 90K iteration training. The losses look right (they are all > 0.1 for the whole process).
At the end I have these annotations:
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.665
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.138
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.264
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.029
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.217
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.372
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.372
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

How should I interprete those number ?
Thanks

This explains the pycocotools output:
https://medium.com/@timothycarlen/understanding-the-map-evaluation-metric-for-object-detection-a07fe6962cf3
What do you mean about 3/4 times more annotations? In your training set?

This explains the pycocotools output:
https://medium.com/@timothycarlen/understanding-the-map-evaluation-metric-for-object-detection-a07fe6962cf3
What do you mean about 3/4 times more annotations? In your training set?

No, I meant the network annotates 3 to 4 times more objects than the validation dataset. It finds way more objects than I want him to.
Should I reduce first the number of training iteration or is it more linked to the NonMaxSuppression parameter (how to change it in retinanet?) or the IoU levels you mentionned ?
Thanks

How are you plotting your results images? Only plot those with score about a threshold. Increase that threshold to get fewer detections.
It looks like you threshold is very low. What is it at the moment?

Oh okay I thought the output already took the threshold in account. I'll have a look, I'll let you know.
I'm using this script btw to plot the results:
import json
import cv2
import matplotlib.pyplot as plt
import numpy as np
import os

def getImageId(name,data):
    images = data['images']
    for image_data in images:
        if image_data['file_name']==name:
            return image_data['id']

def showBoxes(image_path,annotation_file):
    image = cv2.imread(image_path)
    (h1, w1) = image.shape[:2]
    center = (w1 / 2, h1 / 2)
    with open(annotation_file) as f:
        data = json.load(f)
    image_id = getImageId(image_path[image_path.rfind('/')+1:],data)
    boxes = []
    for annot in data['annotations']:
        if annot['image_id']==image_id:
            boxes.append((annot['category_id'],annot['bbox']))
    cloneImg = image.copy()
    for box in boxes:
        (_,(x,y,w,h)) = box
        cv2.rectangle(cloneImg,(int(x),int(y)),(int(x+w),int(y+h)),(255,0,0),1)
    cloneImg = cloneImg[:-50,int(center[0]-300):int(center[0]+550)]
    return cloneImg[:,:,::-1]

listfile = os.listdir('validation/')
del listfile[listfile.index('annotations.json')]

for name in listfile:
	img = showBoxes('validation/'+name,'validation/annotations.json')
	img_trained = showBoxes('validation/'+name,'valid_detect.json')
	buff = np.concatenate((img,img_trained),axis=1)
	#cv2.imshow('Comparaison',buff)
	#cv2.waitKey(0)
	cv2.imwrite('compare/'+name,buff)
	print(name)

print('done')

It's coming from this repo: https://github.com/joheras/CLoDSA

How are you plotting your results images? Only plot those with score about a threshold. Increase that threshold to get fewer detections.
It looks like you threshold is very low. What is it at the moment?

Okay I modified my script to take into account the score and it looks better. To find the best threshold I assume I need to use the COCO mAP metrics right ? I'll implement these using the article on Medium that you provided.

Hi,
I have a question I can't find the answer to: when I use the mAP metrics I have some numbers out, but it looks like it only takes in account IoU and mAP.
But with what threshold of confidence (the 'score') are they calculated on?
I mean the mAP of the network with a threshold of 5% won't be the same with a threshold of 80% right ? I can't get what is the relation between mAP/IoU and the "score" of the boxes (and so the threshold I can choose to filter).
My idea would be to evaluate the mAP of the network for different threshold on the score to find the best threshold. Does it make sense ?
Thanks

There's a whole bunch of discussion about the best way to report metrics for object detection. I personally favour a precision/recall curve. In this repo we use pycocotools mAP because it's the standard for COCO object detection. You might want to use your own metric (F1, F2, etc) if you have a specific use case.
Alternatively there's a lot more you can do using pycocotools. You could edit infer.py and change the call there.

Okay I rephrase my question, it wasn't clear.
When the mAP metrics are calculated, they are using the model trained but there is no restriction on the "score" right ? So the mAP are "valid" for the boxes with the confidence score > 0 only.
My question was then: is it possible to re-compute the mAP metrics but incorporating a threshold on the confidence score ? For example I want to test the mAP for boxes with a score > 0.15, is it possible (by modifying the infer.py for example) ?
Thanks

Hi,
I just noticed a weird behaviour using the "infer.py" script to evaluate the model precision.
When I use the --with-dali option, the time/img decreased significantely (by 2 at least) and the precision metrics increase a little.
Here are the two results metrics, using the exact same model and images/annotations but one with pytorch loader and the other with dali:
retinanet infer rn50fpn/rn50fpn_crop.pth --images validation_crop/ --annotations validation_crop/annotations.json
Loading model from rn50fpn_crop.pth...
     model: RetinaNet
  backbone: ResNet50FPN
   classes: 1, anchors: 9
Preparing dataset...
    loader: pytorch
    resize: 800, max: 1333
   backend: pytorch
    device: 1 gpu
     batch: 2, precision: mixed
Running inference...
[90/90] 0.347s/2-batch (fw: 0.344s), 5.8 im/s
Gathering results...
Writing detections.json...
Evaluating model...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.371
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.829
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.251
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.371
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.034
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.280
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.472
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.472
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

$ retinanet infer rn50fpn/rn50fpn_crop.pth --images validation_crop/ --annotations validation_crop/annotations.json --with-dali

Loading model from rn50fpn_crop.pth...
     model: RetinaNet
  backbone: ResNet50FPN
   classes: 1, anchors: 9
Preparing dataset...
    loader: dali
    resize: 800, max: 1333
   backend: pytorch
    device: 1 gpu
     batch: 2, precision: mixed
Running inference...
[90/90] 0.714s/2-batch (fw: 0.712s), 2.8 im/s
Gathering results...
Writing detections.json...
Evaluating model...
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.398
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.863
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.278
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.398
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.032
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.293
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.512
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.512
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

Any ideas why it does this and what are the "real" metrics ?
Thanks

The difference in mAP is explained by the two dataloaders behaving slightly differently. Both scores are valid.
DALI would normally increase your inference speed, but it may be that your dataset is so small that the overhead of setting up DALI outweighs the increase in data loading speed.
What GPU are you using?
Have you considered using the new MobileNetV2FPN backbone? It should run more quickly.

Ok it's good to know thanks for the explanation, it just feels weird that the metrics can be impacted by the dataloader..
Here is my config:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro M4000        Off  | 00000000:01:00.0 Off |                  N/A |
| 46%   31C    P8    10W / 120W |     24MiB /  8125MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1221      G   /usr/lib/xorg/Xorg                            21MiB |
+-----------------------------------------------------------------------------+

I didn't see you added MobileNet, I'll try it right away !
Thanks

Hey,
I'd like to know wether the focal loss is calculated on the test dataset or the validation one ?
I also would like to modify this focal loss to tune it for my case. I want to add a term taking into account the precision of the center of my predicted bounding boxes. In what file should I add this term ?
Thank you


Conversion Issue!!

Hi I'm new to KCC and I tried to use it to convert a volume of One Piece that I wanted to upload to the kindle, however, despite the fact that I created a special folder for the volume and put inside the chapter folders, when I go to convert it I get a mobi file with all the messy pages.
(I tried to convert the folder with both a directory and the .zip format of it but it always gives me the same problem)
I hope you can help me!  (I use KCC 5.6.3)

What do you mean by messy?

What do you mean by messy?
I mean that if I start to read the first chapter the first page it may be okay but the others have a different order and I can't read them
Upload an image of your kcc settings

I've the same error, when I try to convert a pdf to mobi, the mobi file have a wrong index of the pages. example normal reader---> pag1->pag2->pag3-pag4->pag5 actual issue-----> pag1->pag30->pag31->pag2->pag3 I try to convert the same pdf with the version 5.6.2 and I don't have this issue

I don't have a pdf file I have those folders that I mentioned in the first post but I'll try to download the 5.6.2 version of kcc

@n3nuzzzz let me know if you solve

@vincenzo1993 ok I tried downloading the 5.6.2 but I can't even open it.
It says that the file is damaged and it can't be opened (I have a mac, of course I downloaded the macos file)

@n3nuzzzz I use windows so I don't know why the program 5.6.2 don't start.

@vincenzo1993 so I MAYBE found the issue with the folders... I need to number all the pages with different numbers and removing the gap between the letters and the numbers.
It's not the best solution but it works in this way... idk how I'll manage to number all the pages in One Piece but that's it I think

Please try this build that uses natural sort:
Windows: https://github.com/axu2/kcc/actions/runs/6123144358
Mac: https://github.com/axu2/kcc/actions/runs/6123143150

How should I build it?

YOU SAVED ME @axu2
It works!!

@n3nuzzzz Consider using manga mode and mozJpeg mode. Also considering installing from source since Apple Silicon native python is twice as fast for the conversion step.

@n3nuzzzz Also please change the title to something like "Pages out of order" or similar to make it easier for others to find.

@n3nuzzzz don't close issues until fixes are officially merged in a PR


Assertion failed: Exit events not matched by an enter

This issues was originally reported as https://github.com/phetsims/hookes-law/issues/18, but it's a general issue, so I'm moving it to scenery.

@samreid reported:
Most likely a scenery issue but I saw this in Hooke's Law so I'll report here first.  With the dev tools open, if I open the Systems screen and move the slider, then mouse out and back in, I see this problem.  It happened 2-3 times in a row so I thought I should report it:

Uncaught Error: Assertion failed: Exit events not matched by an enter

I'll reassign to @pixelzoom to see if he can reproduce the problem.
I'm on OSX 10.10.4 running Chrome Version 44.0.2403.130 (64-bit).  Testing a requirejs version of hookes law out of master.  Tagging @jonathanolson in case he knows anything helpful about this kind of error.

@samreid reported:
This error happened again on the next run (4th run) but not the following 2 runs.

I commented:
Thanks for reproducing this. I've seen it a few times (over the past few months?). I couldn't reproduce it, so assumed (perhaps incorrectly) that it was transient Chrome problem.  No idea what the cause is, so assigning to @jonathanolson.  If this is indeed a scenery issue, then it's probably high priority.
@samreid There's no URL shown in your screenshot, and you didn't mention how you were running the sim. But I'm assuming you were running with assertions enabled (?ea), correct?

@samreid replied:
Yes, ?ea
You can see that it says enabling assert in the console

@jonathanolson commented:
I've spent about 15-20 minutes trying to reproduce across platforms (I have OS X 10.9.5, maybe that is a difference), for OS X, Windows and all browsers, and I haven't been able to cause it.
Any more detail on:

open the Systems screen and move the slider, then mouse out and back in

Mouse out (of window?)

@samreid commented:
I tested with the built and non-built versions of hookes law and could not reproduce the problem again.
Not sure what else I can do here, I'll send it back to @pixelzoom to decide what's next, if anything.

I commented:
Hmmm... Now I can't reproduce reliably either.  But I have seen this repeatedly with hookes-law and other sims over the past few months.  So I'm hesitant to just close this as "cannot reproduce".
Searching for the error message, this assertion comes from scenery.ButtonListener, line 104:
assert && assert( this._overCount > 0, 'Exit events not matched by an enter' );
@jonathanolson Does that provide any additional clues about what might be going on here, or how to reproduce it reliably?

@jonathanolson commented:
Yes, it would generally indicate an inconsistent state for input handling. It could be either a browser issue or Scenery issue.
It's the type of bug where it would be very helpful to record Scenery input events so that we could play back the failing sequence once recorded.

@samreid commented:
In my case, I was mousing out to the dev tools, not out of the entire browser. Not sure if that was a critical ingredient or not.

@jonathanolson commented:

In my case, I was mousing out to the dev tools, not out of the entire browser. Not sure if that was a critical ingredient or not.

That's what I was testing. Just tried it out a few more times on OS X / Win, and still wasn't able to reproduce. If we can reproduce even somewhat regularly, I think it would be useful to log all input events.
I'll leave assigned to me, and look more into bringing Scenery event recording back up-to-date.

@jonathanolson commented:
Pulling latest. If you're able to reproduce, please do the following:

In 'chronicle', npm install
In 'chronicle', node js/server.js
Run sim with ?recordInputEventLog&ea
If assertion is hit, resume, interact just a bit more, then close the sim browser tab (it uses a websocket, but the message won't be hit right when the debugger pauses it).
Check chronicle/data for a .txt file, and get it to me.
Record exact SHAs, to help reproduce?

I'll hopefully be able to reproduce from a log.

I commented:
I've been playing around with the sim, but can't reproduce the problem. Mac OS X 10.10.5, Chrome 4.0.2403.155 (64 bit).
@samreid you said:

With the dev tools open, if I open the Systems screen and move the slider, then mouse out and back in, I see this problem.

Do you recall if you moved into the dev tools window with the mouse still down on the slider? Or did you release the mouse in the browser window, then move the mouse (still up) into the dev tools window?

@samreid replied:
I believe the mouse was up, but, like I said in #18 (comment) I can no longer reproduce the problem.

I commented:
As I mentioned in https://github.com/phetsims/scenery/issues/456#issuecomment-134246310, this assertion is occurring in scenery.ButtonListener. So it seems like we should be able to isolate it, or at least hypothesize about what's causing it. But I've been unable to reproduce it, and I haven't heard any thoughts about how this might be related to scenery.ButtonListener. So I'm going to unassign this issue. If someone experiences the problem, follow @jonathanolson's instructions in https://github.com/phetsims/scenery/issues/456#issuecomment-134246604.

@phet-steele recently noticed this issue with HSlider in Isotopes and Atomic Mass and Bending Light.  Unable to reproduce issue locally, but it is very easy to reproduce on phettest.  I will try to go through https://github.com/phetsims/scenery/issues/456#issuecomment-134246604 since I can reproduce with some consistency.

@jonathanolson I am not able to get chronicle server working to capture the input event log.  We can reproduce this issue easily on phettest, if that helps track this down.  Please let me know if I can provide other information that might help with this.  Assigning to @jonathanolson to review comments.

The event server is located at scenery/tests/event-logs/server/server.js.  https://github.com/phetsims/chronicle is a deprecated PhET-iO thing.

Thank you for clarifying @samreid!  Assigning back to me to see if I can capture with the event server.

This issue is beyond my knowledge of scenery, removing assignment, but let me know if  I  can help.

This issue has surfaced again in https://github.com/phetsims/balloons-and-static-electricity/issues/406.
In Firefox only, the "Report a Problem" and "PhET Website" PhET menu buttons are throwing this assertion when clicked. @jonathanolson I am not sure if the problem is in scenery or joist this time, but I thought I would post here.

On 6/18/18 and 6/18/18, this assertion failure is being reported by several sims in CT:
area-model-decimals
area-model-decimals
area-model-introduction
build-a-fraction
fractions-intro
fractions-mixed-numbers
I think these are all sims that @jonathanolson is responsible for, so perhaps there's a common use case that's failing.

Labeling for developer meeting, since this issue is going on 3 years old and has been re-reported several times in the past few months.

I think these are all sims that @jonathanolson is responsible for, so perhaps there's a common use case that's failing.

Yes, there's an interaction with PressListener that is sometimes erroring out that I'll need to solve.

Labeling for developer meeting, since this issue is going on 3 years old, has been re-reported several times in the past few months, and is appearing in CT.

The error message basically just indicates that the state of the listener is incorrect. This can happen due to a wide variety of causes, of which I've fixed at least 8-10 since this was originally reported (iframe handling, pointer capture, logic errors, etc.), and a few have been maintenance released. https://github.com/phetsims/scenery/issues/803 is an example of an unfixed issue that can cause this (which is separate from the cause that is happening for area-model/fractions).

Untagged from developer meeting. @pixelzoom can you respond to the above comment? The recommendation during dev meeting was that we create a new issue for this whenever it occurs, and that this "general" issue could be closed.

+1 for dev meeting recommendation.

Closing, will handle future issues on a case-by-case basis.


Fixed issue jBuilder not loaded in test_app environment

Fixed issue : jBuilder not loaded in test_app environment
Fixed post controller spec

@mwlang
In gem jbuilder, Last line of jbuilder.rb is require 'jbuilder/railtie' if defined?(Rails).
Which register :jbuilder template in ActionView handler on initialise.
Its required rails loaded before requiring 'jbuilder'.


Description field on Child Email from Case changed

Expected behavior and actual behavior.
The Description field should not be touched, but we have seen that the Description field form a Child Email record is unwillingly changed to the exact text:"< body dir="LTR" contenteditable="true" onfocusout="parent.setEmailRange();" lang="en-US" style="overflow-wrap: break-word;">"
Steps to reproduce the problem (screenshots are really usefull)
When a parent Case is changed we first fill a custom field on the Case
Then we always update two text fields on the same child Email twice in a row

,

CRM Version - online/onpremise - WorkflowTools version
CRM Online version: <IP_ADDRESS>33 Tool version: <IP_ADDRESS>

This is what is written in the Description field on the child Email:
"< body dir="LTR" contenteditable="true" onfocusout="parent.setEmailRange();" lang="en-US" style="overflow-wrap: break-word;">"

Hi pokkemonner,
The code doesn't update the Email description field. maybe there is any issue in other part, or you should open a case support with Microsoft.
thanks!


[desktop_drop] Fix broken gestures when used with modified MainFlutterWindow

When used in conjunction with plugins that replace the FlutterViewController in macOS Flutter apps, such as flutter_acrylic and macos_ui, desktop_drop causes some trackpad gestures to break. I found that pinch-to-zoom no longer worked, but there could be other weird behavior I missed.
This PR fixes the issue by recursively searching the view controller hierarchy starting at MainFlutterWindow.contentViewController for a FlutterViewController to attach its child view to.

LGTM, Thanks for the fix.


Blueprint members in parameters

I need to know what is the right syntax to enter a members for a paramater members
I tried
@Parameter("role_name", type="string", required=true, description="The name of role to list users by.", members="foo,bar")
but it gives me an error that it must be an array. How I should fix it ?

@mohammedibrahim check this conversation https://github.com/dingo/blueprint/pull/5

@mohammedibrahim The linked PR should answer your question for you. Closing this out


[feat] Add configuredir variable to the Autotools buildsystem

I am writing a test suite using Reframe for a project with the following structure (simplified):
src/
data/
configure
reframe/checks.py

The project is built using Autotools.
At first I simply set sourcesdir = '..', but this did not work well in practice since copying the whole source tree for each build can be quite expensive. Especially on a file system with limited quota.
With the change in this PR, I am able to do an out of source build without copying the entire source tree:
self.build_system.configuredir = os.path.join(self.prefix, '..')

Using this method, the build output is generated in the stage directory while the source files are read from the original source directory.
Please let me know what you think!

Can I test this patch?

Codecov Report

Merging #2460 (63b8e2c) into master (ca9ebfd) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #2460   +/-   ##
=======================================
  Coverage   85.71%   85.71%
=======================================
  Files          56       56
  Lines       10536    10537    +1
=======================================
+ Hits         9031     9032    +1
  Misses       1505     1505

Impacted Files
Coverage Δ

reframe/core/buildsystems.py
96.21% <100.00%> (+0.01%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca9ebfd...63b8e2c. Read the comment docs.


WDC working on Simulator but hangs on Tableau desktop 2018.1 (does not load data on Desktop)

I developed a WDC which has basic auth, it works well on the simulator and also connects to tableau desktop 2018.1 but when I hit Update Now/ Automatically Update it does not load data (it keeps on loading but never shows up data) I'm loading only two rows of data for testing purpose.
I tried using the debugger but even debugger hangs leaving me with no option to close the application.
This is something related to CORS, I don't any control on the server side. I used the proxy server but did not work! How do I deal with CORS on Tableau browser?
Please help!

Most likely a duplicate of https://github.com/tableau/webdataconnector/issues/252

Please check this as it fixed my similar Issue
https://community.tableau.com/thread/222842


[stable/eventrouter] allow sink configuration

What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:

/ok-to-test

/assign @gianrubio

/lgtm


Update strategy link

Following the discussion about improving the comparison table, the strategy pdf has been updated and this pull request points to the new pdf.

Sorry, I missed this one.


Slow dev server when using vite-plugin-importer

Hi 👋 ,
I'm looking into vite (coming from webpack) and quickly find your plugin to be very helpful for builds. When running the dev server I do notice the following pattern: The server is started, page loads and it refreshes again because of a new dependency being found.
12:37:44 PM [vite] new dependencies found: @material-ui/core/esm/CircularProgress, @material-ui/core/esm/Box, updating...
12:37:45 PM [vite] ✨ dependencies updated, reloading page...
12:37:47 PM [vite] new dependencies found: @material-ui/core/esm/Grid, updating...
12:37:48 PM [vite] ✨ dependencies updated, reloading page...
12:37:53 PM [vite] new dependencies found: @material-ui/core/esm/Fade, @material-ui/core/esm/Slide, updating...
12:37:55 PM [vite] ✨ dependencies updated, reloading page...
12:37:58 PM [vite] new dependencies found: @material-ui/core/esm/useMediaQuery, updating...
12:37:59 PM [vite] ✨ dependencies updated, reloading page...
12:38:04 PM [vite] new dependencies found: @material-ui/core/esm/IconButton, updating...
12:38:05 PM [vite] ✨ dependencies updated, reloading page...
12:38:08 PM [vite] new dependencies found: @material-ui/core/esm/Button, updating...
12:38:09 PM [vite] ✨ dependencies updated, reloading page...

I have in my config the following
import tsconfigPaths from 'vite-tsconfig-paths';
...
    pluginImport({
      libraryName: '@material-ui/core',
      libraryDirectory: 'esm',
      camel2DashComponentName: false,
    }),
    pluginImport({
      libraryName: 'lodash-es',
      libraryDirectory: '',
      camel2DashComponentName: false,
    }),

Sorry, I can't solve it.Because I use the babel-travel to process

Hi 👋 ,
I'm looking into vite (coming from webpack) and quickly find your plugin to be very helpful for builds. When running the dev server I do notice the following pattern: The server is started, page loads and it refreshes again because of a new dependency being found.
12:37:44 PM [vite] new dependencies found: @material-ui/core/esm/CircularProgress, @material-ui/core/esm/Box, updating...
12:37:45 PM [vite] ✨ dependencies updated, reloading page...
12:37:47 PM [vite] new dependencies found: @material-ui/core/esm/Grid, updating...
12:37:48 PM [vite] ✨ dependencies updated, reloading page...
12:37:53 PM [vite] new dependencies found: @material-ui/core/esm/Fade, @material-ui/core/esm/Slide, updating...
12:37:55 PM [vite] ✨ dependencies updated, reloading page...
12:37:58 PM [vite] new dependencies found: @material-ui/core/esm/useMediaQuery, updating...
12:37:59 PM [vite] ✨ dependencies updated, reloading page...
12:38:04 PM [vite] new dependencies found: @material-ui/core/esm/IconButton, updating...
12:38:05 PM [vite] ✨ dependencies updated, reloading page...
12:38:08 PM [vite] new dependencies found: @material-ui/core/esm/Button, updating...
12:38:09 PM [vite] ✨ dependencies updated, reloading page...

I have in my config the following
import tsconfigPaths from 'vite-tsconfig-paths';
...
    pluginImport({
      libraryName: '@material-ui/core',
      libraryDirectory: 'esm',
      camel2DashComponentName: false,
    }),
    pluginImport({
      libraryName: 'lodash-es',
      libraryDirectory: '',
      camel2DashComponentName: false,
    }),

You can try vite-plugin-importus, vite-plugin-importus use es-module-lexer to transform codes internally so it has higher performance.


Configure Azure Pipelines

This pipeline configuration builds the application and produces a build for the Release configuration

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: sahayagodson sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


DNS for cosmos on privatelink not resolving via twingate resolver - linux

We employ a zero trust network provider called twingate. Our cosmosdb instances employ azure privatelink/private endpoint.  I can't access instance privately while using this extension. It forces me to the public interface where I am blocked by firewall. I know that the twingate setup is correct because I can nslookup and get the twingate address for it.  We have defined the public name, the privatelink name, and the private ip address inside of twingate resources. Is there something in this extension that is preventing OS level dns resolution of the cosmosdb name? I am running linux - pop_os 22.04.

I have been informed this is a problem potentially on the twingate side.. so awaiting more info / closing for now. https://forum.twingate.com/t/failing-to-connect-to-resources-outside-of-the-browser-ie-cli-and-dev-tools/1015/2

Managed to get a support call in with Azure networking and twingate. Leaving this note in case someone else runs into this -- And maybe someone from the vscode team can explain / shed some light on requirements for request payload -- but connection to cosmosdb via databases extension didn't work for privatelink while on active twingate connection until we provided all four domains as resources in twingate -- from the associated zone file:
xxx.documents.azure.com
xxx.privatelink.azure.com
AND ALSO THE REGION ENDPOINT:
xxx-southcentralus.documents.azure.com
xxx-southcentralus.privatelink.documents.azure.com
and then finally the private IPs..
We were missing the southcentralus /region based endpoints defined as twingate resources as well. Why does vscode database extention require these endpoints? Thx!

Unfortunately, the database extension doesn't handle DNS resolution so for such kind of issues, there isn't much we can help. You will need to fix the network configuration by yourself.
I did read your links and I have a hypothesis. The documentation of Azure Cosmos DB says there are two kinds of regions for a Cosmos DB instance, a read region and a write region. It's said that write operations will be redirected to clusters in the write regions if the request goes to a read region. You might want to check if South Central US is indeed one of the write regions of your instance. If so, the client might be redirected to it. Since twingate doesn't recognize it, it delegates the connection to the normal network interface and exposes the public IP of the client.


multiselect  with async data sources and models

Are there any working multiselect examples that use async data sources and models? Thanks for any input!

+1


update urls to use https; add sha256 checksum

I have updated the package on chocolatey with these changes.  I think if you want to update it further, just bump the last number.  I could not seem to add another number - perhaps a length constraint.

Hi @msarahan didn't get a notification about the PR, sorry about that! Saw you already updated the package but is failing the automatic validation. I'll create a new branch with what I was working on, but I still can't make the uninstaller work!

Ok, made it work! Pushing new version.


feat(cli): customize the idKey

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/datorama/akita/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Can customize the idKey name in the package.json.
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

Thanks, could you please change the commit type to chore?


added 'symfony-assets-dir' option

Added the 'symfony-assets-dir' option which enables installing assets to
a different directory from the 'symfony-web-dir' directory.
Would it be possible to merge this please? I have a use case in which this option is quite useful.

@fabpot Could you consider merging this PR again ?
Currently https://github.com/sensiolabs/SensioDistributionBundle/issue/#117 and https://github.com/sensiolabs/SensioDistributionBundle/pull/118 haven't solve this issue...
@gentisaliu Have you found a workaround meanwhile ?


TASK: AggregateRoot: remove trait in favour of base class

Resolves #17

@albe yes: https://github.com/neos/Neos.EventStore/pull/12 :-)


west.yml: MCUboot synchronization from upstream

Update Zephyr fork of MCUboot to revision:
02267cfdb5b368c700e3ca6bf47ecc97924c9a56
Brings following Zephyr relevant fixes:

14961292 boot: zephyr: Add optional MCUboot boot banner
7174dd2b boot: zephyr: boards: actinius: enable multithreading
in config
556b32a6 boot: Removed unnecessary if-statement
da2e2ab4 boot: Enforce TLV entries to be protected
ea1cdfde boot: Add tlv query for protected region
8c0e36c8 boot: zephyr: esp32: rename boards to meet hwmv2
f06bc711 bootutil/crypto: Builtin ECDSA key support for PSA
Crypto backend
e369784b bootutil: Allow the usage of builtin keys in
verification
c5a528ba New OVERWRITE_ONLY_KEEP_BACKUP option

Notes on process:

The MCUboot update from mcu-tools/mcuboot/main with the SHA used in west.yaml is already synchronized to zephyrproject-rtos/mcuboot/upstream-sync branch, and is available in the Zephyr fork of MCUboot.
The [DNM] on this PR should be kept until the PR passes all tests and is accepted.
Once the PR passes all tests and gets accepted, the upstream-sync branch should be fast-forward merged to zephyrproject-rtos/mcuboot/main branch and [DNM] should be removed.
After the main branch gets updated, this PR does not require further changes and should be merged as is.

CC @PetrBuchtaNXP


add attributions

adding references to the linked list library I got code from

Codecov Report

Merging #64 into master will increase coverage by 1.47%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           master    #64      +/-   ##
========================================
+ Coverage   98.52%   100%   +1.47%
========================================
  Files          25      2      -23
  Lines        2230     13    -2217
  Branches      307      0     -307
========================================
- Hits         2197     13    -2184
+ Partials       33      0      -33

Impacted Files
Coverage Δ

ArrayUtilsLib/truffle/contracts/Array8Lib.sol

...lsLib/truffle/contracts/ArrayUtilsTestContract.sol

...ib/truffle/contracts/ArrayUtilsTestContractTwo.sol

...Crowdsale/truffle/contracts/DirectCrowdsaleLib.sol

ArrayUtilsLib/truffle/contracts/Array256Lib.sol

WalletLib/truffle/contracts/WalletGetterLib.sol

ArrayUtilsLib/truffle/contracts/Array64Lib.sol

TokenLib/truffle/contracts/ThrowProxy.sol

...DirectCrowdsale/truffle/contracts/CrowdsaleLib.sol

...dsale/truffle/contracts/EvenDistroCrowdsaleLib.sol

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af25255...c30a3f5. Read the comment docs.


Issue with compiling tools/analysis-graph-generator

When I try to run ./build.sh in tools/analysis-graph-generator, the gradle task to build WALA 1.5.7-SNAPSHOT fails.
My Setup

Docker Ubuntu 20.04 LTS
Java 11 (openjdk)

Steps to reproduce
$ chmod +x build.sh
$ ./build.sh
Cloning into 'WALA'...

...

> Task :com.ibm.wala.cast.java.test.data:compileTestJava
/root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/javaonepointfive/Varargs.java:63: warning: non-varargs call of varargs method with inexact argument type for last parameter;
                varargs(2, 3, new String[] { "hello", "world" });
                              ^
  cast to Object for a varargs call
  cast to Object[] for a non-varargs call and to suppress this warning
Note: /root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/JLex/Main.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/JLex/Main.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning

> Task :com.ibm.wala.core:downloadKawa
Download https://ftp.gnu.org/pub/gnu/kawa/kawa-3.0.zip

> Task :com.ibm.wala.util:compileJavaUsingEcj
----------
1. ERROR in /root/analysis-graphs/WALA/com.ibm.wala.util/src/main/java/com/ibm/wala/util/intset/IntSetUtil.java (at line 33)
        MutableIntSetFactory<?> intSetFactory = intSetFactoryClass.newInstance();
                                                                   ^^^^^^^^^^^^^
The method newInstance() from the type Class<capture#3-of ? extends MutableIntSetFactory<?>> is deprecated
----------
1 problem (1 error)

> Task :com.ibm.wala.util:compileJavaUsingEcj FAILED

> Task :com.ibm.wala.core:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':com.ibm.wala.util:compileJavaUsingEcj'.
> Process 'command '/opt/java/openjdk/bin/java'' finished with non-zero exit value 255

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1m 38s
91 actionable tasks: 68 executed, 23 up-to-date

I believe this has to do with WALA. Changing the build.sh script to the following works well:
# get WALA
git clone https://github.com/wala/WALA
cd WALA
git checkout java11
./gradlew clean assemble publishToMavenLocal
cd ..

mvn clean install

What Java version are you using?
On Fri, Jun 25, 2021, 19:05 Rahul Krishna @.***> wrote:

When I try to run ./build.sh in tools/analysis-graph-generator, the
gradle task to build WALA 1.5.7-SNAPSHOT fails.
My Setup

Docker Ubuntu 20.04 LTS
Java 11 (openjdk)

Steps to reproduce
$ chmod +x build.sh
$ ./build.sh
Cloning into 'WALA'...
...

Task :com.ibm.wala.cast.java.test.data:compileTestJava
/root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/javaonepointfive/Varargs.java:63: warning: non-varargs call of varargs method with inexact argument type for last parameter;
varargs(2, 3, new String[] { "hello", "world" });
^
cast to Object for a varargs call
cast to Object[] for a non-varargs call and to suppress this warning
Note: /root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/JLex/Main.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: /root/analysis-graphs/WALA/com.ibm.wala.cast.java.test.data/src/test/java/JLex/Main.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 warning
Task :com.ibm.wala.core:downloadKawa
Download https://ftp.gnu.org/pub/gnu/kawa/kawa-3.0.zip
Task :com.ibm.wala.util:compileJavaUsingEcj

ERROR in /root/analysis-graphs/WALA/com.ibm.wala.util/src/main/java/com/ibm/wala/util/intset/IntSetUtil.java (at line 33)
MutableIntSetFactory intSetFactory = intSetFactoryClass.newInstance();
                                                                ^^^^^^^^^^^^^
The method newInstance() from the type Class> is deprecated

1 problem (1 error)

Task :com.ibm.wala.util:compileJavaUsingEcj FAILED
Task :com.ibm.wala.core:compileJava
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':com.ibm.wala.util:compileJavaUsingEcj'.> Process 'command '/opt/java/openjdk/bin/java'' finished with non-zero exit value 255
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 1m 38s
91 actionable tasks: 68 executed, 23 up-to-date
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/IBM/Project_CodeNet/issues/26, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMTO7Q2L75Q2JHB63HRHJLTUUDT5ANCNFSM47KWG52A
.

I was able to atleast put off that error until later in the execution (although related ones appear first) by adding in the call to the default constructor:
        MutableIntSetFactory<?> intSetFactory = intSetFactoryClass.getConstructor().newInstance();

This shifts the first error to a related issue:
1. ERROR in /home/ijcon40/RCOS/wala/WALA/com.ibm.wala.dalvik/src/main/java/com/ibm/wala/dalvik/util/AndroidManifestXMLReader.java (at line 261)
        this.item = item.newInstance();
                         ^^^^^^^^^^^^^
The method newInstance() from the type Class<capture#2-of ? extends AndroidManifestXMLReader.ParserItem> is deprecated

This one is a little more complicated as item.getConstructor() and item.getDeclaredConstructor() do not compile as intended for ParserItem. I'm running this with openjdk11. I'm running ./gradlew clean build publishToMavenLocal on the java11 branch to simulate the compilation from CodeNet without recloning the repo.


Citation numbers should be after the full stop instead of before

See e.g. Wikipedia.

requires prompt engineering, as this is returned by the LLM

@melissasamworth who's in charge of prompt engineering?

@bryceerobertson afaik there is not a single person, but Matthew  (@matthew1970 on Discord) has been poking around with prompt engineering. AFAIK he is not on Github

I'm not sure it's possible to cross-link between projects, but here's a try
https://github.com/StampyAI/stampy-ui/issues/645

Update: @LeMurphant will give this a go using the following recommendation from Matthew:

Most frustratingly, I can't run the code on my local machine (outdated MacOS won't support all the necessary packages), so I can't test it.

As I wrote in the github pull request notes:
"This was simply a matter of changing one line in stampy-chat/api/src/stampy_chat/settings.py
QUESTION_PROMPT = (
"In your answer, please cite any claims you make back to each source "
"using the format: [1], [2], etc. If you use multiple sources to make a claim "
"cite all of them. For example: "AGI is concerning [1, 3, 8]."\n\n"
)
to
QUESTION_PROMPT = (
"In your answer, please cite any claims you make back to each source "
"using the format: [1], [2], etc. If you use multiple sources to make a claim "
"cite all of them. For example: "AGI is concerning. [1, 3, 8]" (note the period placement)\n\n"
)
I am unable to run this codebase on my local machine (due to dependencies issues caused by an outdated MacOS, I think), but I'm 98% confident this will work, since the reason for the unwanted behaviour was so explicit."
So perhaps rather than me iterating over variations of QUESTION_PROMPT via github, someone could with a better setup could just try a few variations. I would suggest changing


Chapter 4 SQL Alchemy NoSuchModuleError

I know that this error has been addressed before, but when I run the :
flask db migrate -m "users table"
command. It returns this error:
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:sqlit
If you notice with this error that the module name isn't "sqlite", it's "sqlit" with no 'E'. After going through the packages, I found that I do indeed have the package for sqlite to work, but not sqlit. Does anyone know why SQL Alchemy is looking for this specific package?

(venv) C:\Users\T41765\microblog>flask db migrate -m "users table"
Traceback (most recent call last):
File "C:\Python\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Python\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\T41765\microblog\venv\Scripts\flask.exe_main.py", line 9, in
File "c:\users\t41765\microblog\venv\lib\site-packages\flask\cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "c:\users\t41765\microblog\venv\lib\site-packages\flask\cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 1137, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\click\decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\flask\cli.py", line 412, in decorator
return ctx.invoke(f, *args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\flask_migrate\cli.py", line 90, in migrate
rev_id, x_arg)
File "c:\users\t41765\microblog\venv\lib\site-packages\flask_migrate_init.py", line 95, in wrapped
f(*args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\flask_migrate_init.py", line 215, in migrate
version_path=version_path, rev_id=rev_id)
File "c:\users\t41765\microblog\venv\lib\site-packages\alembic\command.py", line 176, in revision
script_directory.run_env()
File "c:\users\t41765\microblog\venv\lib\site-packages\alembic\script\base.py", line 427, in run_env
util.load_python_file(self.dir, 'env.py')
File "c:\users\t41765\microblog\venv\lib\site-packages\alembic\util\pyfiles.py", line 81, in load_python_file
module = load_module_py(module_id, path)
File "c:\users\t41765\microblog\venv\lib\site-packages\alembic\util\compat.py", line 82, in load_module_py
spec.loader.exec_module(module)
File "", line 678, in exec_module
File "", line 219, in call_with_frames_removed
File "migrations\env.py", line 87, in
run_migrations_online()
File "migrations\env.py", line 70, in run_migrations_online
poolclass=pool.NullPool)
File "c:\users\t41765\microblog\venv\lib\site-packages\sqlalchemy\engine_init.py", line 466, in engine_from_config
return create_engine(url, **options)
File "c:\users\t41765\microblog\venv\lib\site-packages\sqlalchemy\engine_init.py", line 425, in create_engine
return strategy.create(*args, **kwargs)
File "c:\users\t41765\microblog\venv\lib\site-packages\sqlalchemy\engine\strategies.py", line 57, in create
entrypoint = u._get_entrypoint()
File "c:\users\t41765\microblog\venv\lib\site-packages\sqlalchemy\engine\url.py", line 156, in _get_entrypoint
cls = registry.load(name)
File "c:\users\t41765\microblog\venv\lib\site-packages\sqlalchemy\util\langhelpers.py", line 221, in load
(self.group, name))
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:sqlit

NVM solved it!

Just curious how did you solve it? I have been having issues of init importing the Config file for som reason.

It had nothing really to do with my problem. Just a typo :)


Need CI in repo

This repository needs CI implemented to run tests.
This issue can be closed when:

[ ] CI is set up
[ ] CI is called when a new PR is created.

I recommend using GitHub actions to keep things contained in this repository instead of calling a third-party.

This Issue can now be closed because PR #6 was merged.


[Doc] EI6 does not allow to add policies to a proxy service

Doc link: https://docs.wso2.com/display/IS540/Configuring+WSO2+EI+Entitlement+Mediator+with+Identity+Server
Description:
Configuring the EI entitlement mediator step 5-  Could not find "Policies"  to remove the applied policies from the Binding Hierarchy to continue the scenario in EI 6.1.1.
The content needs to be updated accordingly or older and new EI versions as this is done through dev studio in EI6.1.1.
Doc link to for applying security to a proxy service for EI6 : https://docs.wso2.com/display/EI611/Applying+Security+to+a+Proxy+Service#ApplyingSecuritytoaProxyService-Importingaproxyservice
Suggested Labels
Bug

Thank you for your contribution!
We are closing this issue since it has not been prioritized for a long time. Chances are that it has already been solved in more recent versions. If not, we will be re-evaluating this when it becomes a priority.


Remove unused RFC gem

the RFC gem doesn't support ruby 3.3, but we don't seem to use it anyway

Thanks for contributing to cloud_controller_ng. To speed up the process of reviewing your pull request please provide us with:

A short explanation of the proposed change:

An explanation of the use cases your change solves

Links to any other associated PRs

[ ] I have reviewed the contributing guide

[ ] I have viewed, signed, and submitted the Contributor License Agreement

[ ] I have made this pull request to the main branch

[ ] I have run all the unit tests using bundle exec rake

[ ] I have run CF Acceptance Tests

glad removing this rather than the terrible things I did in https://github.com/cloudfoundry/cloud_controller_ng/compare/main...terrible-things-to-get-to-ruby-3.3


safeExtGet looks for parameters in the wrong location

It seems to me that safeExtGet in android/build.gradle looks for parameters in the wrong place.  It looks for example for rootProject.ext.minSdkVersion.  When I print out variables of a library during an Android app build, rootProject.ext.minSdkVersion is undefined but rootProject.minSdkVersion contains the expected value.
I ran into this when debugging this build issue.  The first library I fixed had hard-coded the minSdkVersion and I changed it to read rootProject.minSdkVersion and it works fine. The next library read it from rootProject.ext.minSdkVersion which caused it to use the default value. When I change it to use rootProject.minSdkVersion it works as fine.
To reproduce, add following lines to node_modules/<react-native-some-library>/android/build.gradle and run an app build:
    println "minSdkVersion"
    println rootProject.ext.hasProperty('minSdkVersion') ? rootProject.ext.minSdkVersion : 0
    println rootProject.hasProperty('minSdkVersion') ? rootProject.minSdkVersion : 0

Sorry, I misread the generated code.  The generated code uses has instead of hasProperty, which works as expected.


python310Packages.meilisearch: 0.26.0 -> 0.27.0

Diff: https://github.com/meilisearch/meilisearch-python/compare/refs/tags/v0.26.0...v0.27.0
Changelog: https://github.com/meilisearch/meilisearch-python/releases/tag/v0.27.0
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 235496 run on x86_64-linux 1

4 packages built:

python310Packages.meilisearch
python310Packages.meilisearch.dist
python311Packages.meilisearch
python311Packages.meilisearch.dist


Fixed imports for inference

What does this PR do?
In short, provide a summary of what this PR does and why. Usually, the relevant context should be present in a linked issue.

[x] Addresses issue (#issue)

    from .nvidia import NVIDIAInferenceAdapter
  File "/localhome/local-cdgamarose/llama-stack/llama_stack/providers/remote/inference/nvidia/nvidia.py", line 37, in <module>
    from .openai_utils import (
  File "/localhome/local-cdgamarose/llama-stack/llama_stack/providers/remote/inference/nvidia/openai_utils.py", line 11, in <module>
    from llama_models.llama3.api.datatypes import (
ImportError: cannot import name 'CompletionMessage' from 'llama_models.llama3.api.datatypes' (/localhome/local-cdgamarose/.local/lib/python3.10/site-packages/llama_models/llama3/api/datatypes.py)
++ error_handler 62

Test Plan
llama stack build

Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Ran pre-commit to handle lint / formatting issues.
[ ] Read the contributor guideline,
Pull Request section?
[ ] Updated relevant documentation.
[ ] Wrote necessary unit or integration tests.

The change looks good, but the test plan does not have enough details to know for sure if things will work or not. @mattf had shared a test plan in his earlier PRs but I wonder if those are executable (since we may need access to nvidia hosted servers.)

Got it, I've run the tests and updated the description with the test plan and output.


Run func tests with multiple platforms

В tests/functional/tests/config.py в platforms указываем типа ["ubuntu-14.04-x64", "chrome:v0.0.2"] и получаем прогон каждого теста на каждой платформе.

Codecov Report

Merging #287 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #287   +/-   ##
=======================================
  Coverage   55.86%   55.86%
=======================================
  Files          43       43
  Lines        3498     3498
  Branches      403      403
=======================================
  Hits         1954     1954
  Misses       1457     1457
  Partials       87       87

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4840dfb...80f1691. Read the comment docs.

!test

!test

!test

!test


import pandas fails on trunk

Just upgraded to trunk, import pandas fails now.
In [1]: import pandas
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<...>/<ipython-input-1-d6ac987968b6> in <module>()
----> 1 import pandas

<...>/lib/python2.7/site-packages/pandas-0.7.0.dev_063ff4b-py2.7-linux-i686.egg/pandas/__init__.py in <module>()
      8
      9 try:
---> 10     import pandas._tseries as lib
     11 except Exception, e:  # pragma: no cover
     12     if 'No module named' in str(e):

<...>/inference.pyx in init pandas._tseries (pandas/src/tseries.c:87762)()

AttributeError: 'module' object has no attribute 'float16'

In [2]:

You must be on an old version of NumPy, no? I'll push a quick fix for this.

Should be OK now. Do confirm though. FYI I am going to cease supporting NumPy < 1.6.1 within a month or two

Numpy 1.5.1 (upgraded to 1.6 already except on one machine)
Now i get AttributeError: 'module' object has no attribute 'float128', instead of 'float16'.

Ah, apparently float128 and complex256 are NumPy >= 1.6 only. Should work now (the Cython file causing problems is pandas/src/inference.pyx)

I pushed the fix, BTW

It's ok now.


feat(mlx_lm): support batch input in generate()

The prompt argument can now be either a str or list[str].
This is based on @willccbb's implementation at https://github.com/willccbb/mlx_parallm; I noticed that it aligned with the KVCache upgrades in #911.
The change to generate() is backwards-compatible.
The changes to generate_step(), top_p_sampling(), and min_p_sampling() are backwards-incompatible in order to unify shapes; this could be changed by adding a few if-statements, if preferred.

Kind of interesting: for quantized models, the throughput is doesn't go up a lot between bs=1,2,3,4, but then it starts to go up a lot, which is the opposite of what I expected intuitively. I observe the same on @willccbb's original repo.

I think it makes sense to minimize the complexity to the generate function (which is becoming a bit spaghetti) to split out the batched generation into a separate function called batch_generate. I would simplify that function to have fewer arguments (like no formatter, no printing during generation, verbose only prints the timings (e.g. as you have it now).
Also maybe more tricky is the fact that I think for this to be correct, the causal masks need to consider the left padding in the input (please correct me if I'm wrong about that). This has two implications:

Probably we'd need to add a mask parameter to the model __call__ functions and provide an appropriately constructed mask for the batch case.
The Rotating KV cache will be broken in this case (it keeps the initial tokens which would be the padded tokens) and when rotates the mask would need to be updated to consider the padding (which is a bit complicated/tedious). In this case I may suggest disabling this option entirely..

Let me know what you think about the above.

I think it makes sense to minimize the complexity to the generate function (which is becoming a bit spaghetti) to split out the batched generation into a separate function called batch_generate. I would simplify that function to have fewer arguments (like no formatter, no printing during generation, verbose only prints the timings (e.g. as you have it now).

Makes sense to me, will implement.

Also maybe more tricky is the fact that I think for this to be correct, the causal masks need to consider the left padding in the input (please correct me if I'm wrong about that). This has two implications:

Probably we'd need to add a mask parameter to the model __call__ functions and provide an appropriately constructed mask for the batch case.

Yes, this sounds straightforward enough.

The Rotating KV cache will be broken in this case (it keeps the initial tokens which would be the padded tokens) and when rotates the mask would need to be updated to consider the padding (which is a bit complicated/tedious). In this case I may suggest disabling this option entirely..

I'll do a bit of thinking if there's an easy way to handle this, otherwise I'll remove that parameter in batch_generate.
Will update when these changes are ready!

@llllvvuu are you coming back to this?

hey @awni , sorry for the delay, I'd been job hunting this month. I should be able to get back to this in ~a week

No worries, just checking. I'll follow up in a week or so.

Just realised the attention mask has been mentioned in this PR, which is the reason I raised this issue #1044

Sorry for the delay @awni . I took advantage of https://github.com/ml-explore/mlx-examples/pull/1173 to update this PR. It is pending https://github.com/ml-explore/mlx/pull/1726 for the mask dtype.
I noticed one other potential issue: For absolute/rotary positional encodings, the position IDs of padded prompts won't start from 0. I'm not sure what the priority of this is or if it requires any change to mx.fast.rope


Support fully qualified module names

Given these two lines of code:
Hello::World
::Hello::World

Bust-a-gem currently only finds a symbol in the first line. This PR updates the regexes to support the latter as well.
No worries if you want to change the wordPattern to something else, in my tests the changes I made work fine, but there may be some edge cases I missed.

Hey, thanks for the PR! I'll take a peek this weekend. Looks simple enough. :)

OK, I think I understand what you'd like to fix and the problem with the PR.
First of all, the intent behind wordPattern is to snatch "words" from the text document and try to find matches. The current word pattern allows for symbols (which start with :). That will never provide a definition, but that's the whole point. You wouldn't want to click on :puts and have it take you to def puts by mistake. You can change the regex to allow for :: at the front and strip it off later, but we still need the ability to query (and fail) on symbols too. The regex probably needs to be changed to something like /:*[A-Za-z]... to grab all the colons at the front. Or maybe /:{0,2}[A-Za-z]....
Second, we need to fix the test. checkDefinition looks for a call site in something.rb, runs provideDefinition, and asserts that it matches the definition site. You can add another test, but you'll have to edit something.rb as well to add the call site (and maybe the definition site, but not in this case). This little test set is not documented, sorry. Add an explaining comment as well if you feel up for it. :)
Third, you should run the tests in VS code before submitting the PR. That way you'll know that things are working! Use Start Debugging, but make sure to set the current launch config to Extension Tests, not Extension which is unfortunately the default. I typically do this using the dropdown that appears in the bottom blue/orange bar when you start debugging. Then you have to stop/start debugging to get it to run Extension Tests. Maybe there's an easier way. Once you've picked the right config you can easily run the tests over and over again. This is a good skill to master because it'll set you up to do more extension PRs. You can find the VS code docs here - https://code.visualstudio.com/docs/extensions/developing-extensions, though personally I don't find them very useful on this particular topic.
Thanks @dustym - let me know if you need any more help.


Game is freezing after first round scores, displays "Next Round In" but no timer

Happens regardless of player actions on multiple attempts.

It's impossible to proceed in the game past this point.


Error

Committing binding changes to IIS
System.ComponentModel.Win32Exception (0x80004005): A specified logon session doe
s not exist. It may already have been terminated
at Microsoft.Web.Management.Utility.HttpApiWrapper.CreateSSLBinding(IPEndPoin
t endPoint, String hostName, HTTP_SERVICE_CONFIG_SSL_PARAM_MANAGED allSSLData, S
slFlags sslFlags)
at Microsoft.Web.Management.Utility.HttpApiWrapper.CreateSSLBinding(IPEndPoin
t endPoint, String hostName, Byte[] certificateHash, String certificateStoreName
, SslFlags sslFlags)
at Microsoft.Web.Administration.BindingManager.BindingTransaction.Commit()
at Microsoft.Web.Administration.BindingManager.Save()
at Microsoft.Web.Administration.ServerManager.CommitChanges()
at LetsEncrypt.ACME.Simple.IISPlugin.Install(Target target, String pfxFilenam
e, X509Store store, X509Certificate2 certificate) in C:\Users\Bryan\Documents\le
tsencrypt-win-simple\letsencrypt-win-simple\Plugin\IISPlugin.cs:line 116
at LetsEncrypt.ACME.Simple.Program.Auto(Target binding) in C:\Users\Bryan\Doc
uments\letsencrypt-win-simple\letsencrypt-win-simple\Program.cs:line 259
at LetsEncrypt.ACME.Simple.Program.Main(String[] args) in C:\Users\Bryan\Docu
ments\letsencrypt-win-simple\letsencrypt-win-simple\Program.cs:line 180
It looks like the cert was created and added to IIS, the site is encrypted working fine after recreating the binding.  The task scheduler routine did not run though.  Not sure what is up with the c:\users\bryan, doesn't exist.

Looks like this is a duplicate of issue #31

Thank you!


new synonym request: post-synaptic; hind limb; fore limb; ...

Synonyms

Have the term with post-synaptic in stead of postsynaptic as a synonym for any term that contains that word. There're undoubtedly other terms that contain unhyphenated pre or post that can benefit from same. Some people write pre- and post- with hyphens.
Similar-wise, any term that has unhyphenated and unseparated hind or fore prefix could do with a synonym where hind or fore is separated. Currently only MP:0001513 limb grasping has a synonym that separates hind (hind limb extension reflex | hindlimb extension reflex ). Some people write hindlimb as one word, some as two (and some probably hyphenated). Microsoft Word actually marks hindlimb as a spelling error! And the problem is that if you search the database for hind limb you get much fewer results than you get with searching for hindlimb. Searching for hind-limb yields the same as hind limb.
There are probably other prefixes that are not spelled the same by everyone.

Existing MP terms/synonyms where pre-synaptic was added as a synonym:
failure of neuromuscular synapse presynaptic differentiation
abnormal synaptic bouton morphology (abnormal presynaptic terminal morphology)

Existing "hindlimb" MP terms where hind needs to be separated and/or hyphenated.
hindlimb paralysis
abnormal hindlimb morphology
limb grasping (hindlimb extension reflex)
paraparesis (hindlimb weakness)
absent hindlimb
small hindlimb buds
maximal tonic hindlimb extension seizures
abnormal hindlimb stylopod morphology
abnormal hindlimb zeugopod morphology
abnormal hindlimb bud morphology
absent hindlimb buds

Words containing the fore prefix are not as commonly separated or hyphenated e.g., forebrain, forehead, forelimb.


RFC: Snapshot API Proposal

Motivation
"Snapshotting" is a method commonly used in native app development to provide an efficient way to visually "clone" view hierarchies in a side-effect free way. Having these capabilities allows for more nontrivial animations, especially those that cross boundaries of react component hierarchies, and perhaps need multiple visual copies of the element in order to produce the desired effect.
Potential Use Cases

Reflection View
Shared Element Transitions
"Explosion" Effect
Screen Chooser
...
Profit?

Interface
SnapshotRef
A SnapshotRef is an opaque data structure that acts as a light-weight and serializable identifier for a snapshot stored in the native context. Because
there is no way to practically inspect this data structure for an actual image
in JavaScript, we are allowing ourselves to create and use these references synchronously from the JavaScript thread.  The only thing you can do with a SnapshotRef that is of use, is pass it to an <Image /> component as a source prop.
Additionally, the developer is expected to remember to dispose of a SnapshotRef, or else the actual snapshot will be held in memory outside of the JS context.
Snapshot: React.Component
Static Methods
Snapshot.create(node: NodeHandle) => SnapshotRef
Snapshot.dispose(ref: SnapshotRef) => void
Snapshot.update(ref: SnapshotRef) => SnapshotRef returns a new SnapshotRef representing an freshly-taken snapshot of the same view the passed in SnapshotRef was a snapshot of.
Notes:

If Snapshot.update gets passed a snapshot of a view that no longer exists, the snapshot won't be able to be taken. We could warn the user, but we can't throw because we wouldn't know synchronously (unless we can ask whether a node exists in JS? Hmmm.... Maybe we can?).
It's unclear if Snapshot.update should dispose of the passed in ref or not. There are definitely cases where we wouldn't want to, but I wonder if those cases should be opt out?

Component Props
...View.propTypes
snapshot: SnapshotRef
Notes:

There are a lot of cases where we probably want to render a snapshot alongside a currently-being-rendered react element, and we don't have the nodeHandle yet as a result. We may want to build Snapshot in such a way that it is resilient to having null or something passed in as the node handle?
We will also want to export a Snapshot.Animated component or something.

Usage
Mirror / Clone Effect:
import { Snapshot } from 'react-native';

class SomeComponentWithMirror extends React.Component {
  render() {
    const mirrorStyle = {
      transform: [ ... ],
    };
    return (
      <View>
       <SomeComponent ref="foo" ... />
       <Snapshot
         snapshot={Snapshot.create(this.refs.foo)}
         style={mirrorStyle}
       />
      </View>
    );
  }
}

Note: what's nice about this is you could actually make a generic component that did this and didn't know anything about the component it was rendering.
Shared Element Transitions:
The code here is a bit more complicated for a simple example, but the basic idea would be that you would annotate views in one screen and another screen and pair them together. You would take snapshots of them all, and then create a new "animation container" view that you'd put all of the snapshot views in, positioned absolutely... and then animate them manually.
Implementation
This API is taking advantage of highly efficient snapshotting APIs that exist on both the iOS and Android platforms. On iOS, we will be using the snapshotViewAfterScreenUpdates method, and on Android we will be using the View::getDrawingCache method.
JavaScript:
// Snapshot.js
const { SnapshotModule } = require('NativeModules');

const createRef: nodeHandle => ({
  __isSnapshot: true,
  __nodeHandle: nodeHandle,
  __snapshotId: uuid(),
});

class Snapshot extends React.Component {
  static create: (nodeHandle) => {
    const ref = createRef(nodeHandle);
    SnapshotModule.create(ref.__nodeHandle, ref.__snapshotId);
    return ref;
	};
  static dispose: (ref) => {
    SnapshotModule.dispose(ref.__snapshotId);
  };
  static propTypes: {
    ...View.propTypes,
    snapshot: PropTypes.shape({
      __isSnapshot: PropTypes.bool.isRequired,
    }).isRequired,
  };
  render() {
    return <NativeSnapshotView {...this.props } />;
  }
}

module.exports = Snapshot;

Potential additional component that takes a node, and handles lifecycle for you:
const Snapshot = require('Snapshot');

class NodeSnapshot extends React.Component {
  static propTypes: {
    ...View.propTypes,
    node: NodePropType.isRequired,
  };
  constructor(props) {
    super(props);
    this.state = { snapshot: Snapshot.create(props.node) };
  }
  componentWillReceiveProps(props) {
    if (props.node !== this.props.node) {
      Snapshot.dispose(this.state.snapshot);
      this.setState({
        snapshot: Snapshot.create(props.node),
      });
    }
  }
  componentWillUnmount() {
    Snapshot.dispose(this.state.snapshot);
  }
  render() {
    return <NativeSnapshotView {...this.props } snapshot={this.state.snapshot} />;
  }
}
module.exports = NodeSnapshot;

iOS:
struct Snapshot {
  let id: String
  let snapshot: UIView
}

class SnapshotModule: RCTBridgeModule {
  private let snapshotMap: [String: Snapshot]
  func create(nodeHandle: Int, snapshotId: String) {
    if let view = UIManager.getViewByNodeHandle(nodeHandle) {
      let snapshot = view.snapshot(afterUpdates: true)
      snapshotMap[snapshotId] = Snapshot(
        id: snapshotId,
        snapshot: snapshot
      )
	  }
  }
  func dispose(snapshotId: String) {
    snapshotMap[snapshotId] = nil
  }
}

class SnapshotViewManager: RCTViewManager {
	 // details still a little fuzzy here. But essentially,
  // we will be rendering a view that will be the
  // size of the snapshot, with the snapshot inside of it
}

cc @janicduplessis

:+1: I was thinking if a similar API too. Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead? I guess it would make is possible to display the same snapshot in multiple views but I dont know if its a use case worth adding this complexity.

very interesting 👍
the reason I created react-native-view-shot was that it was not going to make it to Android (when I PR the equivalent API of the current iOS takeSnapshot ). My current use-case being just limited to just "I want to export a rendering to share an image result to social networks".
I really hope we can push this idea to be built in RN, I especially like your dispose & update API proposal. if it can't be, I still imagine it could be possible to implement as a third party?
If you need another use-case: I could definitely imagine it could be a solid API to implement caching of consuming views like OpenGL views. For instance to implement something like https://github.com/gre/gl-react-dom-static-container (in WebGL, it was mandatory to have this, as you are limited in maximum concurrent WebGL contexts)
One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?
cheers

@janicduplessis

Is there any advantage to expose the lower level api of create/update/dispose snapshot instead of having it managed by the component using lifecycle methods and just passing the ref directly as a prop instead?

I thought about this a bit. I think it's a good idea to have a snapshot component that just accepts a node and handles all of the lifecycle for you... but I don't think that it is enough to do just that. For the most part, when you are using snapshots you might want to reuse and unmount/mount etc. and the second we call "dispose", we will never be able to get it back again. It'd be nice if we could come up with a component that handles the most common use cases without people needing to deal with disposal, but i don't think that we can only provide that component.
@gre

if it can't be, I still imagine it could be possible to implement as a third party?

Yes. Nothing here requires us to use react internals... as a result, we can make a 3rd party solution. This RFC is mainly to identify whether or not we want to put it in core, and to discuss refinements on the proposed API.

One question: do you think the update can actually be implemented faster than dispose(old) + create(new) ? Is it just so you don't "leak" images that you have designed the update or do you think it can be implemented in a way that's it's actually faster to keep snapshotting the same view & reusing same image?

The main purpose for the "update" API is to allow a consumer of a SnapshotRef to get an "update" of the original image/view, without needing a reference to the node directly. There is no performance benefit, though.

@lelandrichardson Sounds good, I guess we could make it so that snapshot prop can accept either a node or a snapshot object.

@janicduplessis that's how I first wrote up my example... but i thought it made things a bit confusing so i split it out.

@facebook-github-bot label Icebox

Hi there! This issue is being closed because it has been inactive for a while.
But don't worry, it will live on with ProductPains! Check out its new home: https://productpains.com/post/react-native/rfc-snapshot-api-proposal
ProductPains helps the community prioritize the most important issues thanks to its voting feature.
It is easy to use - just login with GitHub.
Also, if this issue is a bug, please consider sending a PR with a fix.
We're a small team and rely on the community for bug fixes of issues that don't affect fb apps.

@facebook-github-bot close


Fix note animations not accounting for ModTimeRamp speed adjustments

Fixes #90
This dumps using virtual track in favor of DI-ing the WorkingBeatmap, and getting the track rate from there.
Actual laned notes will now be consistent throughout the whole track.
Unfortunately due to the touch-based notes using a fade/scale in instead of moving, invalidating them causes jarring differences. I have disabled them for now as they don't really benefit much from the adjustments (TouchHold is duration based, and Touch is based on HitWindows, both depend on real time)

In my eagerness to fix this problem, I reintroduced another problem that was resolved.
Which is that hitobject animations now also account for the playback speed in Autoplay and Replays.

Reverting this PR since the problems this thing causes is way more obvious.


avoid lossy conversion to/from cents

Reticketing from https://github.com/gratipay/gratipay.com/pull/3470#discussion_r31279938.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/18571971-avoid-lossy-conversion-to-from-cents?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github).

During payday, this is.


Set minimal workflow permissions

Fixes #53296
As per the linked issue, this PR sets top-level read-only permissions on all workflows. This reduces risks in case any workflow dependencies are compromised in a supply-chain attack.
Some of the workflows seem to require contents: write permissions to push directly to the repository. These have been given read-only top-level permissions and then write permissions at the job level. This is for future proofing, in case additional jobs that do not require write access are added to the workflow.

Is there a reason to do it this way as opposed to changing the global settings (on our end) to start as read-only and then explicitly grant permissions?

So, seems like well take this, flip the repo root setting, then remove all of the top-level ones as they will be redundant.

Sent #53298 now that the repo has changed (which reverts some of this PR); let me know if it looks good.


Camera-focus crash on React Native 0.50.1

Hey there react-native-camera team. Thanks for such an amazing package, it's done wonders for us!
We've recently upgraded to React Native 0.50.1 and have started getting an app crash when attempting to focus using react-native-camera: Unsupported top level event type "topFocusChanged" dispatched.
Exact repro steps:

Open view with camera
Tap on camera to focus
Crash

Full stack trace:
Unhandled JS Exception: Unsupported top level event type "topFocusChanged" dispatched
reactConsoleErrorHandler @ ExceptionsManager.js:73
console.error @ YellowBox.js:71
logToConsole @ RCTLog.js:48
logIfNoNativeHook @ RCTLog.js:31
__callFunction @ MessageQueue.js:302
(anonymous) @ MessageQueue.js:116
__guard @ MessageQueue.js:265
callFunctionReturnFlushedQueue @ MessageQueue.js:115
(anonymous) @ debuggerWorker.js:72

Here is our react-native-camera component code:
      <RNCamera
        ref={c => this.camera = c}
        style={{ flex: 1 }}
        deafultOnFocusComponent={true}
        onFocusChanged={e => console.log('FOCUS')}
        keepAwake />

package.json:
"react-native": "0.50.1",
"react-native-camera": "^0.10.0"

Unclear if this is an issue on the react-native-camera side of things or on the RN side of things. Any help would be greatly appreciated!

We’re experiencing this issue as well on the exact same versions of react-native and react-native-camera

Me too
On 6 Nov 2017 21:52, "Cesar Figueroa"<EMAIL_ADDRESS>wrote:
We’re experiencing this issue as well on the exact same versions of
react-native and react-native-camera
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lwansbrough/react-native-camera/issues/969#issuecomment-342299633,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AV6rxIVaPb27_L3Uta0d1xzdYqs-Bfm9ks5sz3-IgaJpZM4QSNM0
.

This is because a breaking change in react-native

Appears to be fixed on react-native 0.50.3 and react-native-camera 0.12.0.


Add a release workflow

Describe the solution you'd like
Add a GitHub workflow action that automates the release process.
When a tag is created, the docker images for the Dockerfiles in the package-tooling-image and root directories should be created and pushed to ghcr.io/vmware-tanzu.

This can be closed as the release workflow was added with this pr https://github.com/vmware-tanzu/build-tooling-for-integrations/pull/34


logstash couldn't show the content of the rabbit message

I create config for logstash rabbitmq plugin
rabbitmq {
host => "xx.xx.xx.xxx"
user => "rabbit"
password => "xxxxxxxxxxxxxx"
exchange => "nova"
key => "#"
queue => "logstash-queue"
durable => true
}
then start logstash, it does work and can collect rabbitmq message.  but can't show the content, just show like this (I use stdout output):
2015-07-17T15:51:13.767+0000 %{host} %{message}
2015-07-17T15:51:13.836+0000 %{host} %{message}
2015-07-17T15:51:13.872+0000 %{host} %{message}"
Why so?

@baoyonglei you want to try and inspect  the message with the 'rubydebug' codec on stdout. Try:
output {
  stdout { codec => rubydebug }
}

and let us know if that works or not.

No response, closing.


Function 2 documentation (update_grades.py)

Write the documentation for function 2: update_grades.py.

Assigned to Shirley Zhang
Due Friday, January 13, 2023 @ 9 PM PST

** Remember to create a branch and add useful commit messages!

[x] Done
[x] Have pull request approved

Hi @shlrley I reviewed the PR and approved to merge to main branch