EfficientDet pretrained arch transfer learning

📓 New <Tutorial/Example>
Request for an example
What is the task?
Object detection using transfer learning for the whole architecture. Are there some defined methods to load fastai model and change its head to a different number of classes, similar to this?
I was able to run the Faster-RCNN example using this example trained on COCO dataset and evaluate its mAP.
The EfficientDet workflow seems not to be yet ready. Has there been some update on that?
I was able to create EfficientDet with pretrained encoder and train it myself on COCO. I'm now trying to do transfer learning for a different number of classes. Loading model through fastai, expectedly, throws an error:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-22-cf184f01fec3> in <module>
----> 1 learn.load('coco_local')

~/anaconda3/envs/icevision/lib/python3.8/site-packages/fastai/learner.py in load(self, file, with_opt, device, **kwargs)
    293         if self.opt is None: self.create_opt()
    294         file = join_path_file(file, self.path/self.model_dir, ext='.pth')
--> 295         load_model(file, self.model, self.opt, device=device, **kwargs)
    296         return self
    297

~/anaconda3/envs/icevision/lib/python3.8/site-packages/fastai/learner.py in load_model(file, model, opt, with_opt, device, strict)
     47     hasopt = set(state)=={'model', 'opt'}
     48     model_state = state['model'] if hasopt else state
---> 49     get_model(model).load_state_dict(model_state, strict=strict)
     50     if hasopt and with_opt:
     51         try: opt.load_state_dict(state['opt'])

~/anaconda3/envs/icevision/lib/python3.8/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
   1049
   1050         if len(error_msgs) > 0:
-> 1051             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
   1052                                self.__class__.__name__, "\n\t".join(error_msgs)))
   1053         return _IncompatibleKeys(missing_keys, unexpected_keys)

RuntimeError: Error(s) in loading state_dict for DetBenchTrain:
	size mismatch for model.class_net.predict.conv_pw.weight: copying a param with shape torch.Size([819, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([99, 64, 1, 1]).
	size mismatch for model.class_net.predict.conv_pw.bias: copying a param with shape torch.Size([819]) from checkpoint, the shape in current model is torch.Size([99]).

Is this example for a specific model?
EfficientDet
Is this example for a specific dataset?
COCO transfer learning

Don't remove
Main issue for examples: #39

The EfficientDet workflow seems not to be yet ready.

Outdated comment :sweat_smile: It's going to be removed in upcoming commits
Actually, when you call efficientdet.model it already loads pretrained weights from COCO. This is what we grab by default, you can take a look there for all the sizes with pretrained weights.

The EfficientDet workflow seems not to be yet ready.

Outdated comment :sweat_smile: It's going to be removed in upcoming commits
Actually, when you call efficientdet.model it already loads pretrained weights from COCO. This is what we grab by default, you can take a look there for all the sizes with pretrained weights.

If you want to load your own weights and change the head for fine-tuning, take a look here for insights on how to do it.
The key concept there is model.reset_head

If you want to load your own weights and change the head for fine-tuning, take a look here for insights on how to do it.
The key concept there is model.reset_head

Thanks for the update!
For now I wrote this workaround function to load pretrained weights with matching parameter names and shapes:
@patch
def load_matching(self:Learner, model_name:str):
    this_model = self.model.state_dict()
    trained_model = torch.load(f'models/{model_name}.pth')['model']

    for (this_module, this_param), (loaded_module, loaded_param) in zip(this_model.items(), trained_model.items()):
        assert(this_module==loaded_module), f'Models differ: {this_module}, {loaded_module}'
        if this_param.shape==loaded_param.shape:
            this_model[this_module]=loaded_param
        else:
            print(f'Weights not loaded: {this_module}: {this_param.shape=}, {loaded_param.shape=}')

    return self.model.load_state_dict(this_model)

Thanks for the update!
For now I wrote this workaround function to load pretrained weights with matching parameter names and shapes:
@patch
def load_matching(self:Learner, model_name:str):
    this_model = self.model.state_dict()
    trained_model = torch.load(f'models/{model_name}.pth')['model']

    for (this_module, this_param), (loaded_module, loaded_param) in zip(this_model.items(), trained_model.items()):
        assert(this_module==loaded_module), f'Models differ: {this_module}, {loaded_module}'
        if this_param.shape==loaded_param.shape:
            this_model[this_module]=loaded_param
        else:
            print(f'Weights not loaded: {this_module}: {this_param.shape=}, {loaded_param.shape=}')

    return self.model.load_state_dict(this_model)

The EfficientDet workflow seems not to be yet ready.

Outdated comment sweat_smile It's going to be removed in upcoming commits
Actually, when you call efficientdet.model it already loads pretrained weights from COCO. This is what we grab by default, you can take a look there for all the sizes with pretrained weights.

any ETA on those commits? I was trying to reproduce the mAP results from the source you provided, but the problem is that apart from loading weights, the head gets modified and parameter values are not verbatim
class_net.predict.conv_pw.weight model_loaded.size=torch.Size([810, 64, 1, 1]) model_current.size=torch.Size([819, 64, 1, 1])
class_net.predict.conv_pw.bias model_loaded.size=torch.Size([810]) model_current.size=torch.Size([819])

I guess this is because we add background to the ClassMap.
Or perhaps you have some other suggestion on how to obtain the same mAP in validation?

The EfficientDet workflow seems not to be yet ready.

Outdated comment sweat_smile It's going to be removed in upcoming commits
Actually, when you call efficientdet.model it already loads pretrained weights from COCO. This is what we grab by default, you can take a look there for all the sizes with pretrained weights.

any ETA on those commits? I was trying to reproduce the mAP results from the source you provided, but the problem is that apart from loading weights, the head gets modified and parameter values are not verbatim
class_net.predict.conv_pw.weight model_loaded.size=torch.Size([810, 64, 1, 1]) model_current.size=torch.Size([819, 64, 1, 1])
class_net.predict.conv_pw.bias model_loaded.size=torch.Size([810]) model_current.size=torch.Size([819])

I guess this is because we add background to the ClassMap.
Or perhaps you have some other suggestion on how to obtain the same mAP in validation?

this thread suggest the models were pretrained using yxyx.

Model specific formatting happens inside the dataloader here (this is how we can be agnostic to any model implementation)

I've made sure that I'm using the same size and config parameters as in the original (padding color, normalization).

This is all being done on COCO right? can you share the code? I can take a look and we can figure out what is lacking to achieve the reported results
Actually, let us make this clear: Are you training from scratch on coco or loading the model with the pre-trained weights and checking the mAP?

this thread suggest the models were pretrained using yxyx.

Model specific formatting happens inside the dataloader here (this is how we can be agnostic to any model implementation)

I've made sure that I'm using the same size and config parameters as in the original (padding color, normalization).

This is all being done on COCO right? can you share the code? I can take a look and we can figure out what is lacking to achieve the reported results
Actually, let us make this clear: Are you training from scratch on coco or loading the model with the pre-trained weights and checking the mAP?

any ETA on those commits?

I meant a commit only to remove that comment :sweat_smile: , did you have something else in mind?

any ETA on those commits?

I meant a commit only to remove that comment :sweat_smile: , did you have something else in mind?

Thanks for pointing to the place in dataloader.
I'm trying to reproduce the results of a trained model to make sure that I'm using the transfer learning correctly.
All done on COCO Dataset downloaded from the official link.
One thing that surprised me: I changed the order in the build_train_batch function (also used by build_valid_batch) from yxyx to xyxy and re-run the evaluation metrics and results are the same! The only difference is the loss function increase in the changed example.
#default - yxyx
learn.validate()

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.177
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.258
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.275
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.114
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.409

(#2) [0.6418642997741699,0.12209737428903714] # ValLoss, COCOMetric

# modified - xyxy
learn.validate()

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.177
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.258
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.275
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.114
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.409

(#2) [2.570807933807373,0.12209737428903714] # ValLoss, COCOMetric

Maybe its a problem with the COCOMetric again? My search continues..

Thanks for pointing to the place in dataloader.
I'm trying to reproduce the results of a trained model to make sure that I'm using the transfer learning correctly.
All done on COCO Dataset downloaded from the official link.
One thing that surprised me: I changed the order in the build_train_batch function (also used by build_valid_batch) from yxyx to xyxy and re-run the evaluation metrics and results are the same! The only difference is the loss function increase in the changed example.
#default - yxyx
learn.validate()

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.177
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.258
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.275
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.114
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.409

(#2) [0.6418642997741699,0.12209737428903714] # ValLoss, COCOMetric

# modified - xyxy
learn.validate()

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.122
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.227
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.113
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.203
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.279
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.177
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.258
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.275
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.114
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.338
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.409

(#2) [2.570807933807373,0.12209737428903714] # ValLoss, COCOMetric

Maybe its a problem with the COCOMetric again? My search continues..

I'm thinking this is something specific to the Effdet + COCO dataset, cause I was able to train the Effdet model using icevision workflow and obtain good mAP results.
I calculated mAP from preds and records accumulated in the metric (before conversion to COCO API) using another library and got nearly the same results, here is how I did it:
def raw_to_odm():
    groundtruth_bbs = []
    detected_bbs = []
    for pred, record in zip(preds, records):
        image_name = record['filepath'].name
        for bbox, label in zip(record['bboxes'], record['labels']):
            class_id=label
            coordinates = bbox.xywh
            bb = BoundingBox(image_name=image_name, class_id=class_id, coordinates=coordinates)
            groundtruth_bbs.append(bb)

        for score, bbox, label in zip(pred['scores'], pred['bboxes'], pred['labels']):
            class_id=label
            coordinates = bbox.xywh
            bb = BoundingBox(image_name=image_name, class_id=class_id, coordinates=coordinates,
                             bb_type=BBType.DETECTED, confidence=score)
            detected_bbs.append(bb)
    return groundtruth_bbs, detected_bbs

get_coco_summary(groundtruth_bbs, detected_bbs)

{'AP': 0.11017058248779392,
 'AP50': 0.2117803965667843,
 'AP75': 0.09722278750266831,
 'APsmall': 0.005122517036826111,
 'APmedium': 0.1955153436184948,
 'APlarge': 0.26638203179285247,
 'AR1': 0.15844023950095418,
 'AR10': 0.22851812631946697,
 'AR100': 0.2428790814116496,
 'ARsmall': 0.07777300144455561,
 'ARmedium': 0.33974585757257,
 'ARlarge': 0.37168434508668985}

I'm still investigating the Effdet repo to see what kind of tweaking they do to achieve these mAP results. Any help is welcome!

I'm thinking this is something specific to the Effdet + COCO dataset, cause I was able to train the Effdet model using icevision workflow and obtain good mAP results.
I calculated mAP from preds and records accumulated in the metric (before conversion to COCO API) using another library and got nearly the same results, here is how I did it:
def raw_to_odm():
    groundtruth_bbs = []
    detected_bbs = []
    for pred, record in zip(preds, records):
        image_name = record['filepath'].name
        for bbox, label in zip(record['bboxes'], record['labels']):
            class_id=label
            coordinates = bbox.xywh
            bb = BoundingBox(image_name=image_name, class_id=class_id, coordinates=coordinates)
            groundtruth_bbs.append(bb)

        for score, bbox, label in zip(pred['scores'], pred['bboxes'], pred['labels']):
            class_id=label
            coordinates = bbox.xywh
            bb = BoundingBox(image_name=image_name, class_id=class_id, coordinates=coordinates,
                             bb_type=BBType.DETECTED, confidence=score)
            detected_bbs.append(bb)
    return groundtruth_bbs, detected_bbs

get_coco_summary(groundtruth_bbs, detected_bbs)

{'AP': 0.11017058248779392,
 'AP50': 0.2117803965667843,
 'AP75': 0.09722278750266831,
 'APsmall': 0.005122517036826111,
 'APmedium': 0.1955153436184948,
 'APlarge': 0.26638203179285247,
 'AR1': 0.15844023950095418,
 'AR10': 0.22851812631946697,
 'AR100': 0.2428790814116496,
 'ARsmall': 0.07777300144455561,
 'ARmedium': 0.33974585757257,
 'ARlarge': 0.37168434508668985}

I'm still investigating the Effdet repo to see what kind of tweaking they do to achieve these mAP results. Any help is welcome!

Bingo! I figured it out.
When passing target images to the EfficientDet model, icevision preserves the original image sizes of targets and those values are processed by the effdet itself (no padding remembered). I forced those values to be 512 (as in the padding I use) and the metric show correct results now:
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.489
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.328
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.095
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.356
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.503
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.270
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.418
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.177
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.523
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.659

This also explains why I was able to train on my own Dataset with aspect_ratio 1 and no resize or padding used in the tfms.
    def forward(self, *args, **kwargs):
        args[1]['img_size'] = torch.full_like(args[1]['img_size'], 512.0)
        return self.model(*args, **kwargs)

I will try to figure out a PR on how to fix this the right way.

Bingo! I figured it out.
When passing target images to the EfficientDet model, icevision preserves the original image sizes of targets and those values are processed by the effdet itself (no padding remembered). I forced those values to be 512 (as in the padding I use) and the metric show correct results now:
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.311
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.489
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.328
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.095
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.356
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.503
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.270
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.418
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.177
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.523
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.659

This also explains why I was able to train on my own Dataset with aspect_ratio 1 and no resize or padding used in the tfms.
    def forward(self, *args, **kwargs):
        args[1]['img_size'] = torch.full_like(args[1]['img_size'], 512.0)
        return self.model(*args, **kwargs)

I will try to figure out a PR on how to fix this the right way.

Amazing work @potipot, I was going to investigate this right now but you already solved it! hahahah
So, if I understood correctly, the error is on this line? Here we are setting the image size without padding, but instead we need to set the image size with padding? If so we just need to take the image size from the images tensor images.shape[2:]

Amazing work @potipot, I was going to investigate this right now but you already solved it! hahahah
So, if I understood correctly, the error is on this line? Here we are setting the image size without padding, but instead we need to set the image size with padding? If so we just need to take the image size from the images tensor images.shape[2:]

Another question, how was your experience using this other library for metrics? Is it a good replacement for pycocotools?

Another question, how was your experience using this other library for metrics? Is it a good replacement for pycocotools?

So, if I understood correctly, the error is on this line? Here we are setting the image size without padding, but instead we need to set the image size with padding? If so we just need to take the image size from the images tensor images.shape[2:]

I think this would be the place to insert it and rely directly on the input tensor; however, what they do at EffDet is they use some box scaling and resizing, to scale it up to the match with the original target image shape and only then, calculate the metric.
Note the difference in parameters passed to _batch_detection in train and validation
I'm not sure how this improves the results they get on COCO. I guess it can have impact on the box size and whether it is classified as small or large but otherwise?

So, if I understood correctly, the error is on this line? Here we are setting the image size without padding, but instead we need to set the image size with padding? If so we just need to take the image size from the images tensor images.shape[2:]

I think this would be the place to insert it and rely directly on the input tensor; however, what they do at EffDet is they use some box scaling and resizing, to scale it up to the match with the original target image shape and only then, calculate the metric.
Note the difference in parameters passed to _batch_detection in train and validation
I'm not sure how this improves the results they get on COCO. I guess it can have impact on the box size and whether it is classified as small or large but otherwise?

Another question, how was your experience using this other library for metrics? Is it a good replacement for pycocotools?

It was way easier to implement but I think slightly slower. The API is self explanatory with single call to
get_coco_summary(groundtruth_bbs: list[BoundingBox], detected_bbs: list[BoundingBox])

I could take a look later on how to speed it up. I expect there is room for improvement.

Another question, how was your experience using this other library for metrics? Is it a good replacement for pycocotools?

It was way easier to implement but I think slightly slower. The API is self explanatory with single call to
get_coco_summary(groundtruth_bbs: list[BoundingBox], detected_bbs: list[BoundingBox])

I could take a look later on how to speed it up. I expect there is room for improvement.

I could take a look later on how to speed it up. I expect there is room for improvement.

Do you think there is value in us trying to implement these metrics ourselves?

I could take a look later on how to speed it up. I expect there is room for improvement.

Do you think there is value in us trying to implement these metrics ourselves?

Note the difference in parameters passed to _batch_detection in train and validation

I still have to look deeper into this, but here is a conversation I had with ross that can be helpful.

img_scale is used to move coorodinates between what I think of as the 'model canvas', the img_size * img_size input image size of the model. Umages are scaled down maintaining aspect to fit in that square, located at the origin, upper left corner, the rest is padded if the original image aspect is not a square. The img_scale stores ratio needed to move the output coordinates of the model back to the original image coordinate space for coco evaluator. you can just set img_scale to 1 to not use it. and the image size values used to crop bbox to (img_size, img_size) if you want to handle of the image sizing, scaling, evaluation yourself.

Note the difference in parameters passed to _batch_detection in train and validation

I still have to look deeper into this, but here is a conversation I had with ross that can be helpful.

img_scale is used to move coorodinates between what I think of as the 'model canvas', the img_size * img_size input image size of the model. Umages are scaled down maintaining aspect to fit in that square, located at the origin, upper left corner, the rest is padded if the original image aspect is not a square. The img_scale stores ratio needed to move the output coordinates of the model back to the original image coordinate space for coco evaluator. you can just set img_scale to 1 to not use it. and the image size values used to crop bbox to (img_size, img_size) if you want to handle of the image sizing, scaling, evaluation yourself.

Btw @potipot, have you joined our forum? If not, consider joining, we have lots of interesting discussions happening there =)
https://discord.gg/JDBeZYK

Btw @potipot, have you joined our forum? If not, consider joining, we have lots of interesting discussions happening there =)
https://discord.gg/JDBeZYK

Note the difference in parameters passed to _batch_detection in train and validation

So, we want to handle re-scaling ourselves, we have the option to set img_size and img_scale as None, from what I'm seeing this will then not rescale the bboxes [1] [2]
Before we continue, can you try that and see if you still get the correct results?

I'm not sure how this improves the results they get on COCO. I guess it can have impact on the box size and whether it is classified as small or large but otherwise?

Yeaaah, I'm still not quite sure as well. What we have to be careful about is that when we call CocoMetric.accumulate records and preds should have the same img_sizes and bboxes scaled accordingly. When effdet is scaling the predictions internally it might be messing this up

Note the difference in parameters passed to _batch_detection in train and validation

So, we want to handle re-scaling ourselves, we have the option to set img_size and img_scale as None, from what I'm seeing this will then not rescale the bboxes [1] [2]
Before we continue, can you try that and see if you still get the correct results?

I'm not sure how this improves the results they get on COCO. I guess it can have impact on the box size and whether it is classified as small or large but otherwise?

Yeaaah, I'm still not quite sure as well. What we have to be careful about is that when we call CocoMetric.accumulate records and preds should have the same img_sizes and bboxes scaled accordingly. When effdet is scaling the predictions internally it might be messing this up

Maybe we could try more OO approach?

For sure! The code for the conversion was implemented in a hush, a better way to implement this functionality is to use the [visitor[(https://refactoring.guru/design-patterns/visitor) pattern in the RecordMixins.
I opened this issue to keep track of that

Maybe we could try more OO approach?

For sure! The code for the conversion was implemented in a hush, a better way to implement this functionality is to use the [visitor[(https://refactoring.guru/design-patterns/visitor) pattern in the RecordMixins.
I opened this issue to keep track of that

The code would be much cleaner if we used the library I linked before.

The only disadvantage is that it's a bit slower right? Probably because it's fully implement in python while pycocotools uses C.

The code would be much cleaner if we used the library I linked before.

The only disadvantage is that it's a bit slower right? Probably because it's fully implement in python while pycocotools uses C.

COCOMetric gives correct results for validation with this change:
def build_valid_batch(records, batch_tfms=None):
    (images, targets), records = build_train_batch(
        records=records, batch_tfms=batch_tfms
    )

-   img_sizes = [(r["height"], r["width"]) for r in records]
-   targets["img_size"] = tensor(img_sizes, dtype=torch.float)
+   # passing the size of transformed image to efficientdet, necessary for its own scaling and resizing, see
+   # https://github.com/rwightman/efficientdet-pytorch/blob/645d84a6f0cd837703f98f48179a06c354902515/effdet/bench.py#L100
+   targets["img_size"] = tensor([image.shape[-2:] for image in images], dtype=torch.float)
    targets["img_scale"] = tensor([1] * len(records), dtype=torch.float)

    return (images, targets), records

COCOMetric gives correct results for validation with this change:
def build_valid_batch(records, batch_tfms=None):
    (images, targets), records = build_train_batch(
        records=records, batch_tfms=batch_tfms
    )

-   img_sizes = [(r["height"], r["width"]) for r in records]
-   targets["img_size"] = tensor(img_sizes, dtype=torch.float)
+   # passing the size of transformed image to efficientdet, necessary for its own scaling and resizing, see
+   # https://github.com/rwightman/efficientdet-pytorch/blob/645d84a6f0cd837703f98f48179a06c354902515/effdet/bench.py#L100
+   targets["img_size"] = tensor([image.shape[-2:] for image in images], dtype=torch.float)
    targets["img_scale"] = tensor([1] * len(records), dtype=torch.float)

    return (images, targets), records

Effdet inference will be resolved by https://github.com/airctic/icevision/pull/630


ARROW-3445: [GLib] Fix libarrow-glib link for libparquet-glib

It went further, but it misses orc now https://gist.github.com/kszucs/a272080f512bd52a31fb13e6cd623ee1

Are the binary packages affected by this?

AFAIK We don't have c_glib binaries for OSX

@kszucs Thanks for confirming it but the failure is unrelated. Because the original CXXLD libparquet-glib.la failure is built after GISCAN Arrow-1.0.gir and the GISCAN Arrow-1.0.gir worked well. So I think that the build just missedd something. (environment variable?)

I've confirmed this change fix this case on macOS.

Thanks @kou !


lib not accepted as an argument of adatper

Hey, fellow
I'm trying to set TerraDraw on top of a react-map-gl/map-libre example on a Typescript-NextJs project like this:
terra version : 0.0.1-alpha55
import { MapLibreGL as lib } from "maplibre-gl";

import {
  TerraDraw,
  TerraDrawMapLibreGLAdapter,
  TerraDrawRectangleMode,
} from "terra-draw";

useEffect(() => {

    if (onMapLoad && mapRef?.current) {
      onMapLoad(mapRef.current);
      if (enableDraw) {

        const draw = new TerraDraw({
          // Using the MapLibre Adapter
          adapter: new TerraDrawMapLibreGLAdapter({
            map: mapRef.current.getMap(),
            lib
          }),

          // Add the Rectangle Mode
          modes: [new TerraDrawRectangleMode()],
        });

        console.log('draw',draw);

        draw.start()
      }
    }
}, [onMapLoad, mapRef, enableDraw]);

I get a type error on lib saying this:
Object literal may only specify known properties, and 'lib' does not exist in type '{ map: Map; } & BaseAdapterConfig'.
looking at TerraDrawMapLibreGLAdapter types definition, I don't see lib, and without it nothing happens
export declare class TerraDrawMapLibreGLAdapter extends TerraDrawBaseAdapter {
    private mapboxglAdapter;
    constructor(config: {
        map: Map;
        /// NO lib?
    } & BaseAdapterConfig);

Hi @ralexrdz, thanks for raising this! You are right, there is no need to pass this in. I will update the guides, I think this was an oversight.

@ralexrdz just checking you didn't get this code from somewhere else other than the guides?

It was from the guides, but I see it fixed, now

Yes I've updated it now. Thanks for your help!


Config flow could not be loaded: 500 Internal Server Error Server got itself in trouble

Config flow could not be loaded: 500 Internal Server Error Server got itself in trouble
It gives this error if I try to connect it

Can you look in the home-assistant.log and see if there's something that refers to the okokscale integration?

2024-01-26 14:33:21.631 WARNING (SyncWorker_2) [homeassistant.loader] We found a custom integration okokscale which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
File "/config/custom_components/okokscale/config_flow.py", line 90, in async_step_user
if device.supported(discovery_info):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/bluetooth_sensor_state_data/init.py", line 66, in supported
self._start_update(data)
File "/config/custom_components/okokscale/okokscale.py", line 84, in _start_update
name = human_readable_name("OKOK Scale", None, address)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
maybe to add I have the TRiSTAR WG-2440, but you mentioned it to work with Tristar

I also use the TRiSTAR WG-2440, but just found out mine is also no linger working. I don't use it very often...
Anyways, I'll look into it

I think it should work now, please give the latest version a try

Cheers, it worked!
Can you provide an other service to tip you? Don't have a creditcard. You saved me a few bucks, I might as well share it.

If you like you can send me something on PayPal :-) paypal.me/seekingtheedge


Broken dependencies - blobs/__init__.py

In the master branch, blobs/__init__.py tries to import blobs.maxout and blobs.rnn modules which do not currently exist (they did in previous commits). This causes import errors.
I'm not sure if the blobs.maxout and blobs.rnn modules have become redundant, but deleting the import statements in blobs/__init__.py allows the MNIST example to run without import errors.
See https://github.com/kundajelab/deeplift/blob/master/deeplift/blobs/init.py

Thanks for catching this; those modules exist on a private dev branch and the import statements + cached modules lingered when I merged into the private branch. I will set up continuous integration eventually to avoid introducing this in the future.


[scintilla] Initial port.

Scintilla port
Scintilla is a free source code editing component for Win32, GTK+, and OS X.
Notes

Scintilla v3.7.6 (v.4 branch is unstable yet)
Uses Scintilla Visual Studio project to build (win32/SciLexer.vcxproj)
No patches applied

Thanks!


[Feature] Add watcher for running plot processes

In the config.yaml it should be possible to define a certain number of chia plotting processes (e.g. chia_plot_processes_warning_threshold) and if the number of active processes falls below the threshold a warning notification should be sent out.
Similarly it should be possible to add a plot_process_warning_time_m and if a plot process ever goes over that time a warning notification should be sent out.

Thank you for the suggestion. There are many other tools specialising in plotting and monitoring these processes. Chiadog specializes in monitoring the harvesters (for now).


Update FileSelector parameters when setting path

Adds an _on_set method to all Parameters, this allows Parameters to trigger actions when one of its attributes changes. This is used to update the list of valid paths when (Multi)FileSelector.path is set.
Fixes https://github.com/holoviz/param/issues/412

My main question is whether this mechanism might be better used to generate warnings instead of errors?

Reverted any changes to validation on setting but still factored out separate validation methods for different attributes which should make it straightforward to readd the validation later.

Makes sense. Ready for review?

Yes, note this PR also continues some style and exception cleanup. Should have kept that separate sorry.

Reviewing with PR was a little tricky with all the formatting changes but every code change looks like an improvement to me. Happy to see this merged!


Fix #29 – Update rollup-stream and adjust bundler

Intentionally update rollup-stream to 1.24.1+ and adjust bundler to resolve the issues related to resolve issues related to a breaking change in the rollup-stream package.
Starting with release 1.19.1 rollup-stream dropped support for moduleName,  which resulted in unexpected key 'moduleName'  errors as npm fetched the latest (incompatible) patch version due to the use of caret ranges.
Fixes ngryman/speedracer#29

Good catch! Thanks 👏


Add auto-correct for Style/RaiseArgs cop

This change adds auto-correct for Style/RaiseArgs. I also took the opportunity to add some more documentation to the cop.
Examples:
raise StandardError.new("message")

will be corrected to:
raise StandardError, "message"

... and ...
raise StandardError, "message"
raise StanrardError, arg1, arg2, arg3

will be corrected to:
raise StandardError.new("message")
raise StandardError.new(arg1, arg2, arg3)

The changes look OK overall. You'll have to rebase on top of the current master.

@bbatsov: Made corrections according to your comment, rebased, and extracted the message strings into frozen constants for consistency with the rest of the code base.

Looks good. Just squash the two commits together and we're good to go.

@bbatsov: squashed


Local Capture and Review -> Optional Remote Upload

I love this tool, and am planning to use it for all kinds of demos, training, and process documentation.  But, I need to be able to run it on local machines.
Ideally, I could have it installed on the notebooks of many field staff (mostly linux), where the sessions could be captured and reviewed locally.  After review, sessions could be uploaded to either a company server or the showterm.io site.
Architecturally I see this as pretty simple:

The local machine runs it's own private server, that let's the user locally review the sessions (http://localhost:3000/).
For any given session, the user can choose to upload it to an alternate network connected server.

Command wise, it could look like this:
showterm --list
# shows a listing of sessions on $SHOWTERM_SERVER
showterm --name  "New Name" http://localhost:3000/9d4f95fade1c52ad86837
# adds "New Name" as an attribute of 9d4f95fade1c52ad86837 that appears in the --list
showterm --copy http://localhost:3000/9d4f95fade1c52ad86837  --server http://my.corp:3000/
# copies the session from  up to --server
Lastly, having a simple script to setup the local instance.  Ideally using something like sqlite with full setup by the script, and no expectation for conflict with other apps on the system.
Thanks for creating this great tool!  Keep it up. :-)

I've been using showterm a lot this pandemic year for exactly @alingse's use case. I feel a little bad leaning my paid work so heavily on it, too, but it's just so much more useful and tightly done than Zoom or https://asciinema.org/ or using the mac or gnome screenrecorders.
I also want more control over the files though. Personally, I don't need showterm --list, but I would like to be able to store session as regular files, say *.scriptreplay? *.showterm (themselves just zip files)?,

saving a .zip of the script and timing files before or instead of uploading (so that they can be archived and reviewed by scriptreplay)
prompting about if we want to upload the file or not
uploading after-the-fact (showterm upload ...)
downloading that .zip from the website

I can extract it from the javascript but my teammates can't; it'd be nicer if this was just built-in.

@kousu I'd absolutely accept changes so that downloading a zip works (by just adding .zip to the end of the URL?)
I'd prefer to not prompt by default (but happy to have local saving as an option so you could do showterm -o tutorial-3.zip or similar, then showterm --upload tutorial-3.zip).
(P.S. One problem I ran into with your previous commit is that showterm.io is on an unsupported heroku stack, and upgrading that requires upgrading (at least) the version of ruby (but probably also more things..). So while I'd love to deploy a new version, we need to do some work and testing to make that happen. (Maybe time to rewrite in go? :D))

You're right, -o is better than prompting. And .zip on the end of the url is super, that's genius.
(It's only been, what, a decade? Enough time for a language to fade away I guess :P)


Prevent collision of imported type with exported declarations in current module

Fixes #31031

@andrewbranch rebased. Thank you.

@typescript-bot user test this

User tests only turn up failures in VS Code: https://github.com/collin5/TypeScript/pull/1/files#diff-0e62d568d28b82cee9e66b2b8fafd290R7

I hate to be the https://xkcd.com/1172/ person but this is actually something I liberally use to work around --isolatedModules-related limitations when re-exporting types under the same name 🤣
// a.ts
export interface A {}

// index.ts
import { A } from './a'
export type A = A

No big deal, it'll just take some, uh, intensive codemodding.

The best way of dealing with this and correctly be able to elide type-only reexports with --isolatedModules / @babel/preset-typescript would probably be to have an export { type A } from './a' syntax, à la Flow.


Closing the Signals help menus insert settings into my settings file

Testing #205384

Run "List Signal Announcements"
Close the picker
A bunch of settings are inserted

And accessibility.signals.chatResponseReceived has a warning on "announcement" that says Property announcement is not allowed.
This also happens with "List Signal Sounds" but only with accessibility.signals.format

It's expected that they're inserted as I'm updating the configuration.
Tracking the issue you mentioned with chatResponseReceived here #205736


DOCS Add Recipes section, add module version breakdown to 4.5.2

This PR:

Introduces a section to the documentation covering what recipes are, how they differ from modules, and what our core recipes entail.
Adds a message to the changelog index page explaining that changelogs relate to recipe versions as of Silverstripe CMS 4.
Adds a breakdown of the module versions included in 4.5.2.

These changes are intended to alleviate some confusion we've been seeing in the community around version information recently, as the 4.5.2 hotfix release diverges module versions significantly (some modules stayed at 4.5.1, whilst framework was already ahead a patch and shifted to 4.5.3). Other work is also underway to put more emphasis on prioritising display of recipe versions within the CMS and related tooling, in order to make it easier to determine whether projects are safely running the latest release.
Feedback on the structure and wording of these changes is most welcome 🙂

Merged ASAP in the interest of getting this information out to the community. Iterative feedback is of course still welcome!


Material Theme White font Issue

Hi I installed this extension and after applying all of the 4 material themes all of my fonts are white.  I wonder is there anything I could possibly be missing or over looking.  Please help if you have any suggestions or fixes, I have VSC 1.3.1

vsc 1.3.1?

I have 1.3.1 is it beta?

It's a older release. Now The public release is 1.13 and material theme support vsc 1.12+. Please update ant let me know if you still have the issue.

Omg I have no idea its that old and the funny thing is when I check for updates it says its the most current one -_- I will install the new one and keep you updated.  Thanks!

@NoahHuynh Do you fixed this?


[BUG]:card is not visible in price section

Fixes:  #2275
Description
card is not visible properly in price section
Type of PR

[x] Bug fix

Screenshots / videos (if applicable)

Hi @huamanraj,
Please check my PR. If you need any changes, please let me know.
Thank you!

@nikhilnishadoo7 , the issue has not been assigned to you yet. Please do not start working on it until it is officially assigned to you.
This is the last time i am considering it

Dear @RitiChandak ,
Thank you for bringing this to my attention. I apologize for any confusion caused. I will ensure to wait for the official assignment before beginning work on the issue.
Thank you for your understanding.
Best regards,
@nikhilnishadoo7

@nikhilnishadoo7 the content is overlapping with buy now correct it and open a new PR or reopen this one


Add packaging and deployment script

The script needs to do the following:

tarball the code
move the code to the rpmbuild directory
run rpmbuild
upload the built rpm to the production repo

First version of this is added in https://github.com/sirgraystar/mandyville-data/commit/314868a134bbb0b2bf41d0f0163981b9c99abd64


-r flag not working in Travis CI

https://travis-ci.org/oreqizer/creepx
Somehow tape thinks that -r is meant to be a file. When I tried -r=babel-register, it also complained that it's not a module. It works fine locally, only broken in Travis CI.
Here's the error report:
$ npm run clean && npm test && npm run test:flow && npm run lint && npm run build
><EMAIL_ADDRESS>clean /home/travis/build/oreqizer/creepx
> rimraf es lib
><EMAIL_ADDRESS>test /home/travis/build/oreqizer/creepx
> npm run test:base | faucet
module.js:544
    throw err;
    ^
Error: Cannot find module '/home/travis/build/oreqizer/creepx/-r'
    at Function.Module._resolveFilename (module.js:542:15)
    at Function.Module._load (module.js:472:25)
    at Module.require (module.js:585:17)
    at require (internal/module.js:11:18)
    at /home/travis/build/oreqizer/creepx/node_modules/faucet/node_modules/tape/bin/tape:5:5
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/home/travis/build/oreqizer/creepx/node_modules/faucet/node_modules/tape/bin/tape:4:23)
    at Module._compile (module.js:641:30)
    at Object.Module._extensions..js (module.js:652:10)
    at Module.load (module.js:560:32)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>test:base: `tape -r babel-register 'src/**/__tests__/*.spec.js'`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>test:base script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/travis/.npm/_logs/2017-11-18T06_53_29_241Z-debug.log

(-r arg is the proper form, fwiw)
Can you share your travis.yml? It's pretty strange that it wouldn't interpret that properly.

language: node_js

node_js:
  - "stable"

script:
  - npm run test:coverage
  - npm run test:flow
  - npm run lint

after_success:
  - bash <(curl -s https://codecov.io/bash)

and presumably you have babel-register in your package.json? If it's installed locally but not actually listed in your dependencies, that would cause it to not be installed in travis, which might mean it's treated as a file path instead.

Yes, here's my whole package.json:
https://github.com/oreqizer/creepx/blob/master/package.json
Didn't want to paste the whole thing so it's not bloated here

HMM @oreqizer @ljharb will reach out to Travis and see what's good.

@snuggs any news?

Not yet will ding again. Otherwise may have to take a visit. :-) @jameswomack

@jameswomack will also reach out internally next week if we don't hear back by then.
https://twitter.com/snuggsi/status/934735828571578368

Thank you Snuggsi. I dig your Twitter profile BTW... ✈ 💰

@jameswomack well since thanksgiving i'll have to change that to 🚀  💰  #bitcoin lol followback

@jameswomack Increasing pressure by 20% ;-)

@oreqizer @jameswomack pressure application successful. What next? Who's got the keys 🔑 🔑 . Should we call DJ Khalid? ;-)

ok this is getting a bit chatty; let's please keep the noise to a minimum.
If there's an update about this issue, please post it - if not, please quietly continue pursuing leads in separate channels :-)

@ljharb please take this over yourself. I've done all I can do. Do you have access to SSH? If so the ball is in your court. Please and thank you! Good luck with Travis sir.

@snuggs i cant ssh into the build vm of “not my repo”, no. I appreciate your help here, but Github notifications are very noisy already, and light chatter makes it harder for actual work to get done.
If someone could provide a failing test case - that fails in travis-ci consistently - I’m happy to fix it, otherwise I’ll have to close this since it can’t be reproduced.

Try to run tape -r babel-register 'src/**/__tests__/*.spec.js' in Travis. This fails in all of my repos that have that script.

https://travis-ci.org/ljharb/tape/builds/310646335 passes (https://github.com/ljharb/tape/commit/efbf84c6ac5dca2f0b39f667d1a183c847c425df). That's not invoking tape directly, it's invoking node path/to/tape/bin, but I'm not sure how that would be different. I also ran https://travis-ci.org/ljharb/tape/builds/310653043 with babel-register specifically, and it still passes.
I then added a pre-script run of ./bin/tape -r babel-register test/require/a.js and that also passed: https://travis-ci.org/ljharb/tape/builds/310654127


Drop unnecessary swizzling

Hi, I think the swizzling is unnecessary. The result is able to be generated on NSTextCheckResult lazily.
Since no swizzling, there is no need for objc. And it's better store such info on NSRegularExpression, since it's all about pattern.
I tried refactoring with less locs and all tests passed.

Fixed


hi,how to get my all roles or permissions with jstree?

hi,how to get my all roles or permissions with jstree.
https://github.com/vakata/jstree
jstree needs a json data like :
[{
"id":1,"text":"Root node","children":[
{"id":2,"text":"Child node 1","children":true},
{"id":3,"text":"Child node 2"}
]
}]
thanks.

Hello,
The new version will include a tree-structured array. At the moment, you can use descendants and pass it through a filter to make it recursive.

On Jul 21, 2015, at 3:59 AM, hepanming007<EMAIL_ADDRESS>wrote:
hi,how to get my all roles or permissions with jstree.
https://github.com/vakata/jstree https://github.com/vakata/jstree
jstree needs a json data like :
[{
"id":1,"text":"Root node","children":[
{"id":2,"text":"Child node 1","children":true},
{"id":3,"text":"Child node 2"}
]
}]
thanks.
—
Reply to this email directly or view it on GitHub https://github.com/OWASP/rbac/issues/71.

thanks

+1 @hepanming007 for pointing me to JsTree
Code Example
Using JsTree version 3.2.1
I found it more easy to do it with the Alternative JSON format. It uses the parent uniq key to generate the tree.
Javascript Node (Docs)
// Alternative format of the node (id & parent are required)
{
  id          : "string" // required
  parent      : "string" // required
  text        : "string" // node text
  icon        : "string" // string for custom
  state       : {
    opened    : boolean  // is the node open
    disabled  : boolean  // is the node disabled
    selected  : boolean  // is the node selected
  },
  li_attr     : {}  // attributes for the generated LI node
  a_attr      : {}  // attributes for the generated A node
}

PHP
header('Content-Type: application/json; charset=utf-8');

$roles = \phpSec\SQL("SELECT ID AS id , Title AS text, Description AS description  FROM `phprbac_roles`",array());

                        foreach($roles as &$role){
                            $parent =  $rbac->Roles->parentNode($role['id']);
                            $role['parent'] = $parent['ID'];
                            $role['depth'] = $rbac->Roles->depth($role['id']);
                            if($role['id'] == 1){
                                $role['text'] = 'System';
                                $role['description'] = 'Account with limited capabilities managing the system';
                                $role['parent'] = '#';
                            }
                        }

 print(json_encode($roles));

Pushing this further:
Trying to find out the best way to modify the context menu  items plugin so it can handle :

description (add, edit)
remove and recursive  remove

Adding description to tree nodes
To display the description i used Twitter Bootstrap Tooltip plugin. If you use similar solution you have to know that you can add attributes on the fly to the <a/> link using the node a_att attribute, example using an angularJs resource  (with angular-ui bootstrap )
 var roles = Roles.query(function () {
                        for (var i = 0; i < roles.length; i++) {
                            roles [i].state = {};
                            roles[i].a_attr = {
                                // Angular-ui bootstrap
                                tooltip: roles[i].description,
                                // Regular Twitter Bootstrap
                                 title : roles[i].description
                            };
                           ....
                           roles [i].state.opened = true;
                      }
                      $scope.rolesList = roles;
});

Note that if parent is closed child nodes are not created so you will need to initialise the plugin when parent is opened :
// in angularJS `angular.element` warps jquery's `$`
angular.element('#myTree').jstree({
                            ...
                        }).bind("changed.jstree", function (e, data1) {
                            console.log(data1.changed.selected); // newly selected
                            console.log(data1.changed.deselected); // newly deselected
                        }).bind('ready.jstree', function () {
                                //update when tree is ready
                                // [AngularJS]
                                $compile(angular.element('#myTree').find('a'))($scope);
                               // [no-AngularJS]
                                 $(function () {
                                        $('#myTree')..find('a').tooltip()
                                      })
                        }).bind('after_open.jstree', function () {
                            // Same as above
                        });
                        ...
});


fn map8(fuzz_a: Fuzzer<a>, .., fuzz_h: Fuzzer<f>, f: fn(a, ..., h) -> result) -> Fuzzer<result>

https://github.com/aiken-lang/fuzz/pull/7


vagrant: 2.2.14 -> 2.2.15

Motivation for this change
ChangeLog: https://github.com/hashicorp/vagrant/blob/v2.2.15/CHANGELOG.md#2215-march-30-2021
Also had to revert a change which improved plugin-errors a bit since it
changed the dependency resolution in a way that broke vagrant-libvirt
from $out/vagrant-plugins/plugins.d.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 118138 at 13bb7556 run on aarch64-linux 1

1 package built successfully:

vagrant

4 suggestions:

warning: missing-patch-comment
Please add a comment on the line above, explaining the purpose of this patch.
Near pkgs/development/tools/vagrant/default.nix:54:5:
   |
54 |     ./unofficial-installation-nowarn.patch
   |     ^

warning: missing-patch-comment
Please add a comment on the line above, explaining the purpose of this patch.
Near pkgs/development/tools/vagrant/default.nix:56:5:
   |
56 |     ./0004-Support-system-installed-plugins.patch
   |     ^

warning: missing-patch-comment
Please add a comment on the line above, explaining the purpose of this patch.
Near pkgs/development/tools/vagrant/default.nix:55:5:
   |
55 |     ./use-system-bundler-version.patch
   |     ^

warning: missing-patch-comment
Please add a comment on the line above, explaining the purpose of this patch.
Near pkgs/development/tools/vagrant/default.nix:57:5:
   |
57 |     ./0001-Revert-Merge-pull-request-12225-from-chrisroberts-re.patch
   |     ^

@SuperSandro2000 fixed :)


[AUTO-CHERRYPICK] nodejs and python-tensorboard: patched vendor tarball to fix CVE-2024-21538 - branch 3.0-dev

This is an auto-generated pull request to cherry-pick commit 6fa4340db2366ab7724ab95059935c68111a60c2 to 3.0-dev. Original PR: #11140

Hello,
We are currently utilizing the image mcr.microsoft.com/cbl-mariner/base/nodejs:18 and have encountered the vulnerability CVE-2024-21538. Could you please provide an estimated timeline for the availability of a fix?


Add Spring Runner plugin component ID

[x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
[x] Update the CHANGES log.

https://github.com/apache/skywalking-java/pull/594

ID 150 was just registered, please use 151 instead, thanks.


[backport] ChainerMN: add an error message when mpi4py is missing

Backport of #5559

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 3015f433428699b777649a1130cdc13f8a8a57dc, target branch v5) failed with status FAILURE.
(For contributors, please wait until the reviewer confirms the details of the error.)

LGTM!


Read stdin only one time (fix for #7002).

Read stdin only if not already been readed.
Avoid a bug where we read multiple times stdin when using upgrade with --install option. Indeed, the second time, the standard entry is empty.
Fix: https://github.com/helm/helm/issues/7002

@Yannig Is there a reason why you pushed this PR when PR #7640 is in progress to fix #7002 ?

@hickeyma no sorry. I didn't realised that there was already a fix. You can close it as you wish.

@Yannig I'll close it then. You can add your comments to #7640


Setting gcloud auth to use Python2.7 rather than the global use of 3.x

Works as expected - including in #222


Changing theme colors

Sometimes the syntax colors change when working. For example, the color of the name of getters and setters in JS changes to the color of normal variables and the text becomes poorly distinguishable.

/duplicate - see https://github.com/microsoft/vscode/issues/92308#issuecomment-596885893 for a workaround.


Can we use OAuth with webview

Hi i seek solution about using OAuth with Webview in flutter but not found
Can this package using with OAuth

Yes, why not? Just implement your logic to capture the access token from the URL itself or cookies or localStorage or wherever it is.

@pichillilorenzo  any example here ? can your give some example

If the access token is set in the redirect URL after the user login, just take the access token from it in the onLoadStart or onLoadStop event.
If the server sets the access token as a cookie, just use CookieManager.
It depends how it is implemented by the server.

@pichillilorenzo thank I will try

I have gone through this problem but unable to solve by logic, then found that google disabled OAuth in webView. And able to make it working by removing wv from userAgent in crossPlatform options. So if you are yet to solve can try by replacing '; wv' with ''. And if you already solved by logic please share your logic, I need this.

I'm facing the security error doing the same "Refused to display 'https://accounts.google.com/' in a frame because it set 'X-Frame-Options' to 'deny'."


SPProductUpdate and SPLanguagePack conflict

Details of the scenario you try and problem that is occurring:
This was originally reported on the "SharePointDSC Best Practices and Gotchas Post" http://nikcharlebois.com/sharepoint-dsc-gotchas-and-best-practices/
In theory, it should be possible for someone to install, in order, a PU, a Language specific patch, and then run the config wizard. That scenario works just fine if done manually. However, with the SPDSC module, if you specify two SPProductUpdate blocks in a row, and then call the SPConfigWizard, it will skip over the second one because of the way we use B2B and will throw the following message:
Write-Verbose -Message ("An upgrade is pending. " + `
"To prevent a possible loop, the install will be skipped")
If this can be done manually, then we should look at having the DSC module allow it as well, given that Language specific patches are now separate from the main CU/PU.
The DSC configuration that is using the resource:
N/A
Version of the Operating System and PowerShell the DSC Target Node is running:
Win 2016/WMF 5.1
Version of the DSC module you're using:
<IP_ADDRESS>

The ProductUpdate resource is using the server version information in the Config database. This information is updated when the Config Wizard is executed. This means that after installing an update, at a next run the resource is unable to check if it already is installed but it is able to detect that the Config Wizard is required.
To prevent a loop (or error messages), the resource does not try to install the update when the Config Wizard is required.
This can be resolved if we find a way to detect the version information locally. So far I haven't got a solution for this problem.

Unless I am mistaken, the [Hive]/ISAPI/Microsoft.SharePoint.dll file will always get updated on a server the moment a CU/PU is applied. We could then check the minor version of that DLL in order to detect if the patch has been applied locally can't we?

When we try to check the version number of a file, we have to be certain, that this file was part of the patch. If I understand this post by Stefan Gossner correctly, the file versions do not reflect the installed patches.
In cases we install patches manually, there is no check whether an update was installed prior. The updates its self checks, if the update is already in place. See #579. The Updater will exit with an exit code 17025.
Maybe we can work with this result in the test method or try to get information about the method that is used in the installer.

I was wondering whats the difference between a manual install of an update and the way we do it with DSC. With SharePointDSC there are assumptions made (Please correct me if I'm wrong):

Each ProductUpdate has to be followed by Config Wizard
Each Set-TargetResource function of the ProductUpdate Resource checks if there is a pending upgrade. Usually, a user would not do so.
Config Wizard can only be used once (properly) in a configuration

Just some ideas on how to resolve this issue:

Implement something similar to the Package Resource, that checks via the Package GUID, if the update is installed or not
Do not check if there is a pending upgrade; Install the update anyway
Try to reinstall the update and use the exit code 17025 to return exit successfully
Load all information from the update package and compare it with all dlls in place

@andikrueger, Yes it should be as straight forward as this in theory. The challenge is that simply by looking at the PU/CU package we need to obtain information about what version it is trying to install. Even if we add logic to look inside the package, the .xml file included normally provides this information in the  node, but I am not sure if this truly is our best option.
Also, the bigger challenge here is that unless the file continues to exist, the Test-TargetResource will always return false (also true for language packs). If the user deletes the PU/CU package after installation, then the method can't figure out what version we're trying to install and will throw an error. There are no ways around it.
Unless we add a parameter to the resource to specify what version the SPProductUpdate is for, there's no real way for us to obtain this information from the installer file and then to check to see if it is locally installed.

Our only real option at this point would be to do what you suggested, which is to attempt to install the patch, not matter what, then catch exit code 17025 and threat it as a "Package already installed on server" message. I am just afraid of what that will cause to the performance of the resource. Normally a DSC config should not have more than one SPProductUpdate resource block in it, but some users will not follow best practices and attempt this.

One of the issues we have here is:
I install SP and the most recent CU. If I then install a Language Pack, I have to reapply the CU in order to have that LP updates to the latest patch level. The current resources are able to do that.
@andikrueger To answer your questions:

That is true, but you only need one SPConfigWizard resource in a configuration. That will pick up both Language Pack and CU installs. It does have to be specified early in the configuration, so you are sure it will be executed.
Correct, that is because of the way we check the installed updates: Using the Config DB. When an update is installed, but the Config Wizard has not been executed yet, the build numbers in the Config DB are not updated yet. So if SPProductUpdate detects that the Config Wizard is needed, it will refuse to execute to prevent installing the update over and over again.
The Config Wizard is only needed in a configuration just once, there is no need to specify it multiple times.

If we can find a way to detect if the update is needed on the environment by not using the Config DB, I am all for it. Unfortunately so far, I haven't found a way yet.

@NikCharlebois I agree with you. Trying to install the update and wait for the error code is not the best option we have. This will even break the ability to run Test-DscConfiguration as the Test-TargetResource of product update will return false. Also the user has to have the update in place.
@ykuijs Thank you very much for the answers.
What would happen, if we change the ProductUpdate Resource to something like this:

Parameter
Attribute
DataType
Description
Allowed Values

Patch
Key
String
The Name of the patch like: "KB4011244"
All KB patches alailable for SharePoint

SetupFile
Write
String
The name of the update setup file

ShutdownServices
Write
Boolean
Shutdown SharePoint services to speed up installation

BinaryInstallDays
Write
String[]
Specify on which dates the installation is allowed
mon, tue, wed, thu, fri, sat, sun

BinaryInstallTime
Write
String
Specify in which time frame the installation is allowed

Ensure
Write
string
Present to install SharePoint. Absent is currently not supported
Present, Absent

InstallAccount
Write
String
POWERSHELL 4 ONLY: The account to run this resource as, use PsDscRunAsCredential if using PowerShell 5

In case we use a script similar to this one, we could  optain all installed patches:

Using this method has one major issue: When you deployed a CU and then decide to add a Language Pack, the CU (which is multilingual) will never be applied to that language pack.
I have no clue how the CU itself is doing its checks to see if it needs to be installed or not. It might be an idea to reflector the setup file to "borrow" that logic ;-)

That's something I was already up to. But I was not able to get into the inside of the msiexec file (hope this is the right one). It's not .Net...
The part about the language pack is interesting though. I think I ran into an issue,the moment I tried to reapply a CU after a Language pack. I documented the outcome in #579. Maybe there is more magic in the CU installer or even less magic than we currently can think of. :-)
BTW. What about the SP2016 updates that got split up to start and was files?

Now it has been a while since we revisited this issue, but here are my thoughts. For the Test-TargetResource to work properly, we need to pieces of information:
1 - What is the current patch level on the server (even if not applied to the Config Database with PSConfig yet),
2 - What is the version that the current patch is trying to apply to our server:
The first piece of information can easily be obtained by calling the following:
(Get-SPDscFarmVersionInfo).Highest
This will return the highest versioned patch installed (even if not committed by psconfig).
The second piece of information can be obtained by extracting the .exe installer, and by looking at the //MSIPatch/SequenceData/Sequence Node value from the sts-x-none.xml file contained within it.
If the second piece of information is greater than the first one, we install the patch, otherwise we assume it is already installed "cumulatively".
Any reasons issues you folks can think of with this approach?

Getting the patch level from SP is the way the SPProductUpdate resource currently is doing it, with the exception that I used the Lowest instead of the Highest (Lowest specifies the CU patch level). This is however depending on having the Config Wizard executed.
Getting the patch level from the file is done a little easier in the current resource, by retrieving the product version file property of the setup file.
The challenge is how to retrieve the installed CU on a server, without using the ConfigDB. If we can find a way to get that info from the server instead from the ConfigDB, we are there.
When installing a CU on a system which already has that CU installed, you are prompted that installation isn't necessary. So the patch itself contains somekind of checking mechanism. @andikrueger already tried to reverse engineer the patch, but was unsuccessful so far. I will give it a try as well.

Thank you for the new thoughts. Unfortunately I haven't come up with any other idea yet. I still hope, we will find a way to mimic the checks made by the installer...
The second part of your solution is promising. We could use that information to come against the current installation.
I have some thoughts about the first part and hope I'm mistaken:
Isn't Get-SPDscFarmVersionInfo dependent on a Farm in place? If there is this dependence: e.g. we won't be able to install language packs and reapply a CU prior to the farm creation.
In cases a Farm is in place: SPProductVersions is obtained from config DB. I haven't tried the command on a SharePoint Server yet. I hope these information get updated by a timer job and not PSConfig (Let's hope for the best!). In case it changes: Your solution looks great.

@ykuijs Ireally hope, you are lucky with the reverse engineering part of the installer.

Ok, a quick update: I used ProcessMonitor to check what the install is doing. As it turns out, the installer contains a msp and xml file for each patch. During installation, the package extracts the XML file, reads a GUID from that XML file and check if that GUID is installed.
Unfortunately the GUID is not used exactly as it is stored in the XML file, so have dive into this a little deeper to figure out that one.
Also: I don't think we can reuse this method 100%, since this requires extracting the package each time we validate the resource. But maybe it is an idea to create a script that generates an versioning file based on the package (only once and placed in the same dir as the package) and have the resource use that file. Does that sound as an option?

Awesome idea with ProcessMonitor!
I really like the idea of the patch version file. This could be a lot of work in the first place to collect all version information fomr SP 2013 and SP 2016 patches, but is well worth it.
With that file in place we are able to use one configuration continually and it would even work in cases, the user deletes the patch file.
What kind of information would you use as key property: The filename of the patch?
Could you give us some more information about the part: "the Guid is not used exactly as stored in the xml file".

My idea was not to create one patch file which includes the information for all available patches. That is costing too much maintenance on our end. My idea was to create a script (or a method in our module) to have the user create a patch version file as needed. That script extracts the patch, reads all information from the XML files and writes that to the patch file.
During installation, the resource then reads the patch file and checks if each patch is installed on the server.
Looking up the GUID info now......my VM in Azure auto shutdown last night, so lost the information.....have to reproduce the issue ;-)

@ykuijs That's a good point with the maintenance cost. Why don't we create that file the first time SPProductUpdate or SPLanguagePack are used? This would lower the requirements on the users side and we know for sure the patch-info-file is in place.
Auto shutdown is sometimes a real pain :)

Extracting the package can take a very long time (30 minutes or more). Implementing this as part of the resource will introduce a long delay. But of course we can do both: Offer the option to generate the file manually beforehand or generate it when it does not exist.
The GUID in the XML file is: 90150000-115D-042C-1000-0000000FF1CE
The GUID the installer is looking for in the registry is: 00005109D511C2400100000000F01FEC
If you compare these GUIDs, you can see some patterns:
90150000-115D-042C-1000-0000000FF1CE
00005109-D511-C240-0100-000000F01FEC
The first three parts are reversed, in the fourth the first two are reversed (maybe more, but 00 reversed is still 00 ;-)) and in the last part they are also reversed in pairs of two characters.
Right now we have to figure out how to convert the GUID in the file to the GUID used in the registry

We can use this function: https://gist.github.com/MyITGuy/d3e039c5ec7865edefc157fcd625a20a

...and here is the description of what is done: https://blogs.technet.microsoft.com/odsupport/2009/12/17/how-to-convert-an-office-guid-or-office-patch-guid-to-a-compressed-guid/

Ok, I've got the code written to obtain the GUID and will commit it into the Util module. Great catch Colombo @ykuijs

We may still have some issues trying to determine if a patch is already installed due to their cumulative nature. For example, I spin off a plain vanilla SP2016 environment, and directly installed the November 2017 CU on it. Based on the XML file, the GUID for that patch is:
**4F2C8C6B-8A0D-46D9-8D34-C9EEE665B03C**

which will get formatted down to:
**B6C8C2F4D0A89D64D8439CEE6E560BC3**

Now, I can find a registry key matching that Formatted GUID under
**HKLM\SOFTWARE\Classes\Installer\Patches**

which tells me it has been installed. However, if after installing the November 2017 CU, I download the April 2017, I need to have a way to determine that it doesn't need to be installed because it has been cumulatively applied by a more recent patch.
The April 2017 formatted patch GUID is:
**E9AC635FD91CBDA4F9A6E02D1B2E1BBA**

But that is nowhere to be found in the Registry. This means that there are no easy way to figure out if a patch needs to be applied or not given that something more recent may be on the server.

In that case we would have to stick with a database (patch info file), that contains all information about when a cu was released and the guid (or several guid for each component contained in that cu).

What happens if you try to install the older cu? Will the installer succeed?

Interesting. With an environment that has the November 2017 CU already installed on it, if I extract the April 2017 CU, and run the extracted .msp, then it installs anyway, and creates a registry entry with the Formatted GUID, even if there is a more recent patch applied.
However, if I run the "unextracted" .exe path, then I get a prompt saying that the patch is already installed. Therefore the .exe is what has the logic to check to see if it is already installed. Maybe an option here is to force the users to extract the CUs (just like we are forcing them to do it for the LP), and have the .msp run no matter what if the associated registry key is not found.

Are there any files changed when the msp gets installed? If that's the case, we would roll back any newer version of the patched file and have to reapply the newer patch. I think this is the reason why we have to reinstall the current CU when we install any language pack and had a CU already installed. Is this right?

A CU can contain hunderds of MSP's (in SP2013 anyways). So I don't like forcing the user to extract the CU package.
I have analyzed the ProcMon trace a little more and with a patch that is not required, the logic checks several reg keys which all do not exist.
When a patch is required, is checking more regkeys, including a DisplayVersion key. The XML file that is supplied with the MSP also contains a version number and I believe the logic is comparing these versions.
The logic also does some other checks, but I am not sure what theses are for yet.

Today I spent some time analyzing two SharePoint 2016 patches, that are not needed on my system. I did the following:

Compared procom traces of both setups.
Checked the opatchinstall.log files
the XML file corresponding to a patch
tried to understand the logic that is used.

What are my findings with the following file: sts2016-kb4011217-fullfile-x64-glb.exe

Double Click the .exe
Accept the Eula
Next
will lead to this screen:

After reviewing the opatchinstall.log I could trace back, what the installer does:
After my step 1 (minimized log output)
OPatchInstall: Will extract the file with ID '$(PATCH.FILEID.EULA)' to the path '$(PACKAGE.PATH.TEMP)\eula.txt'
...
OPatchInstall: Will extract the file with ID 'supplementaleula_sharepoint_en-us.txt' to the path 'C:\Users\admak\AppData\Local\Temp\OWPA29D.tmp\eula.txt'
...
OPatchInstall: Reading EULA from file 'C:\Users\admak\AppData\Local\Temp\OWPA29D.tmp\eula.txt'

After my step 2:
OPatchInstall: Will extract the file with ID 'sts-x-none.xml' to the path '$(PACKAGE.EXTRACT.FOLDER)\sts-x-none.xml'
...
OPatchInstall: Adding the patch '$(PACKAGE.PATH.TEMP)\sts-x-none.msp' to the queue of patches to install
OPatchInstall: Will log the installation to '$(parentPath:LOGPATH)\sts-x-none.LOG'
OPatchInstall: Will use the command line '$(PACKAGE.PATCHES.DEFAULT.ARGUMENTS) $(PATCH_ARGS)'
...

When the .xml file is present the registry will be checked for the following registry keys, which are partially obtained from the xml file (contained in the patch):
<MsiPatch xmlns="http://www.microsoft.com/msi/patch_applicability.xsd" SchemaVersion="<IP_ADDRESS>" PatchGUID="{358FACB8-17F1-4C89-A9BC-5CD2C479B53C}" MinMsiVersion="5">
	<TargetProduct MinMsiVersion="405">
		<TargetProductCode Validate="true">{90160000-1014-0000-1000-0000000FF1CE}</TargetProductCode>
		<TargetVersion Validate="true" ComparisonType="Equal" ComparisonFilter="MajorMinorUpdate">16.0.4351.1000</TargetVersion>
		<TargetLanguage Validate="false">0</TargetLanguage>
		<UpdatedLanguages>0</UpdatedLanguages>
		<UpgradeCode Validate="true">{00160000-1014-0000-1000-0000000FF1CE}</UpgradeCode>
	</TargetProduct>
	<TargetProductCode>{90160000-1014-0000-1000-0000000FF1CE}</TargetProductCode>
	<SequenceData>
		<PatchFamily>sts</PatchFamily>
		<Sequence>16.0.4600.1002</Sequence>
		<Attributes>1</Attributes>
	</SequenceData>
</MsiPatch>

The TargetProductCode 90160000-1014-0000-1000-0000000FF1CE will lead to this reg key:

There is a folder Patches:

With the Name of that folder we can navigate to the patches folder with the same name:

and get the path to the local .msp file of the current patch installed: C:\Windows\Installer\ea43.msp
This msp file contains the needed information to compare the current PatchSequence to the patch installed:
Patch we try to install:
	<SequenceData>
		<PatchFamily>sts</PatchFamily>
		<Sequence>16.0.4600.1002</Sequence>
		<Attributes>1</Attributes>
	</SequenceData>

Path that is installed:

With that information the installer knows whether to install or not to install.
This means we have to look into the patch, open the registry, get the installed patch GUID, open the patches folder and compare the PathSequences.

Following my last post, I will start to investigate if the patch version wich is going to be applied can be obtained from the .exe file information. If so, and if this version number matches each of the contained patches, we would be able to determine if an update is necessary or not, by just comparing the version against the version in the registry.
The thing I did not verify yet: what about language packs that need to be patched. Is it the same procedure or not.
@NikCharlebois @ykuijs any further ideas what to look for?

Linking  #708 to this issue.

I did some more testing to verify my previous post.
The Farms latest patch is the Januar CU 2018 (16.0.4639.1002). Today I installed the Feb 2018 CU. In the following screenshot are some more details on how the patch installer verifies if the patch is installed:

The product code
The installer of the previous patch (see above
The current patch and its GUID + Patch Order
The previous patch and the well formatted GUID (As Compressed GUID: 22C0E2B49ABF8A24A82FD7B8059314FD)

This is the patch data of the Feb 2018 CU:
<SequenceData>
		<PatchFamily>sts</PatchFamily>
		<Sequence>16.0.4654.1000</Sequence>
		<Attributes>1</Attributes>
	</SequenceData>

The Sequence or version number above equals the one in the file information dialog:

I see the following ways to determine if we have to apply a patch or not:

The filenames for SP 2016 Patches do start with sts2016 (language independent) or wssloc2016 (language dependant).
a. STS patches are targeting the following product GUID
<TargetProductCode Validate="true">{90160000-1014-0000-1000-0000000FF1CE}</TargetProductCode>
b. WSSLOC: The GUID for the Target Product varies with every language. The installed languages can be obtained from:[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\16.0\InstalledLanguages] "1033"="16.0.4351.1000" "1031"="16.0.4351.1000"
I did not find a way to get either the GUID or the compressed GUID for the Language Packs. Even with an English installation there is an Englisch Language Pack in place. Currently I only see the option to save all compressed or well formatted GUIDs for the language packs.
With the information about the registry keys we can now look for the latest patch and its Sequence.
Compare that sequence to the one present in the file information.
a. If the current is lower: Install the patch
b. If they match or the one in the patch  is lower: Do nothing.

Currently there is still one big challenge left. For SharePoint 2016 there are 49 Language Packs and therefor we would have to know every product guid to find the right registry keys.
By now I was not able to find a way on how we could determine the right registry keys. I see the following ways to solve this shortcoming:

Something in brute force style: @NikCharlebois has something in the SharePointDSC.Reverse Script that could be very useful to finde every installed language pack. the SharePointDSC.Reverse Script uses the uninstall Information to gather the installed components.
We could store every Language Pack Guid in our resources. This means we would have to maintain this information too.

I tend to [1] and would start with a draft implementation based on the findings above. Please let me know, if you see any other way.

@andikrueger Great job on this by the way buddy! I tend to prefer option 1 as well, however we need to build an approach that will also support whatever comes out for SP2019 at this stage of the game. We do have a commitment to support SP2019 from Day 1. I do believe the GUID format will remain the same for the 2019 wave, but it's something we will have to verify internally first. So in the meantime, I say it's safe to use the Registry RegEx approach.

There is some news:
I did more reverse engineering on the update process and am very confident to have a solution for this ready soon.
Currently there is one big constraint, I did not see coming... For SharePoint 2013 there are security updates - how did I forget about these???
For SharePoint 2016 there are not.
The SharePoint 2013 Security Updates are named like "wdsrvloc2013", "coreserverloc2013" and so on. They patch a single SharePoint part and do have an own product code.
Currently I'm stuck at this point. I do not want to store every possible product code, but I can't think of a way, that does not need, to extract the patch file.

@ykuijs could you assign this topic to me and add an enhancement label? I'm still working on it...

@andikrueger Since you do not have the correct permissions level in this repo, I cannot assign the issue to you. But it is all yours! Please also checkout #839, since this might have a relation with this issue.

@ykuijs I updated my branch for this issue with the latest v3.0 changes and will add unit testing fpr the current changes.
Parts of issue #839 should be fixed by the new  Get-SPDscLocalVersionInfo function as we are evaluating the state of a single server and not the farm.

I'm done with integration testing on SharePoint Server 2016 - Why are SharePoint update THAT time consuming?
The updated mechanism does a great job. It's possible to leave all prior SPProductUpdates and SPInstallLanguagePack Resources in a configuration :-)
Please find attached my test configuration and the log of the DSC configuration.
SPProductUpdateTestConfiguration.txt
log.txt

Today Pester got updated with the ability to add a TestRegistry: https://github.com/pester/Pester/pull/1112 This should make unit testing of this resource easier.
I'm looking forward to update my recent PR.

Will be fixed by PR #1046

Closed via PR #1046
A huge thanks to @andikrueger for his work on investigating and implementing this issue!!!


[markdown mode] codeBlocks & setext improvements

more CommonMark compliance improvements, and some setext regression fixes
changed state.prevLine/thisLine value a bit
also closes https://github.com/codemirror/CodeMirror/issues/4137

Thanks! Merged as c06c273afc78...55aa50667c2


Feature Request: select column as list or set

use case is:

you have a list of objects by category
you want to display a category list that can be used to filter your objects

to do this in a normal DB, I'd do a select distinct of a column and order it alphabetically and include a where clause if it's a subcategory.
There's no way to do this w/o pulling all the objects into memory with the current API I think?

This should already work, Did you try something like:
data.select(<fields>).distinct().where(<conditions>).get().toList()
?

Went through the wiki and didn't see an example of this in the query examples.  How do you specify fields in the select?

It's the same as any select statement multiple or single attributes can be specified. https://github.com/requery/requery/blob/master/requery-test/src/main/java/io/requery/test/FunctionalTest.java#L786

Thanks...now I feel stupid :-)
The query examples in the wiki should probably include one of those samples...

Sure, the examples will be updated with more content soon.


Expose delimiter to commander?

Allows for later change delimiter in the process of a command. Similarly, it'd good to expose all other run-time related system variables for more featured plugins.
Say, it would be awesome to replace the delimiter after a login?

Good idea. I'm currently working on a refactor (authentication fork), and I'll throw the delimiter feature into that. Out of town for about a week, but should be able to jump on this after that.
Could you please amplify on what you mean by exposing related system variables? One of the refactors of this fork is that the this in a Command action exposes a Session object which custom functions, so I should be able to tack on additional data to it pretty cleanly. Do you mean environmental variables? Please expand on that.

@dthree I am currently working on a user/group(linux like) module for vantage(including login stuff). In the process, I found it would be useful to get hands on delimiter. For the rest of it, I haven't come up with other stuff yet.
I know I can get a lot of information through this. But there are some variables (and some I don't know about it yet) are pre-configured (like delimiter), that are not mutable during runtime.

@dthree it would great to make vantage command and use command as plugable middlewares. So developers can use it if they need it.

Gotcha. Should be back in a few days. If you are still working on auth middleware before then, try to use what I have so far for basic auth as a model for this:
https://github.com/dthree/vantage/blob/authentication/lib/vantage-auth-basic.js
The auth fork is broken right now, but the general form for an auth middleware function is pretty solid as given in the above.

@joesonw this is now good to go - do you want to try this out?


LOG4J2-1901: Fix travis-ci build

remove apt-get, use trusty dist
not sure this will be enough

Now it fails with the socket option test that has been fixed. Have you synced with upstream?

Yes. The problem is that these tests also fail on java 7 when run on travis.

Added assumes to skip these tests on travis-ci.
Now a test of og4j-api-scala_2.11 fails:
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.228 s <<< FAILURE! - in org.apache.logging.log4j.scala.LoggerTest
[ERROR] org.apache.logging.log4j.scala.LoggerTest  Time elapsed: 0.55 s  <<< ERROR!
java.lang.NoClassDefFoundError: org/apache/logging/log4j/scala/Logger$
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.scala.Logger$
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)

That's strange. However, the Scala modules will soon be remove from this repository (as they live in the logging-log4j-scala repo). So maybe we should not spend time on fixing that problem in this repo.

Not finding the Logger$ class sounds like the Scala code isn't being compiled properly. That's the companion object class name.

It may be due to macros. When using Scala macros, you need to compile the macro definitions separately from whatever using the macros. If src/main and src/test are compiled together, it will not work.
But why is it differently from Jenkins? Isn't both Jenkins and Travis CI just running the Maven build? Or does Travis CI tries to be "smart" and actually do something stupid?


No errors shown when having undefined components

An error should be shown that Text is undefined.
To reproduce:
import React from 'react'

import { View } from 'react-native'

export default class Root extends React.Component {
  render() {
    return (
      <View>
        <Text>Hello!</Text>
      </View>
    )
  }
}

Rules:
{
  "extends": "plugin:react/recommended",
  "env": {
    "es6": true,
    "browser": true,
    "node": true
  },
  "plugins": [
    "react",
    "babel"
  ],
  "parserOptions": {
    "ecmaFeatures": {
      "jsx": true
    },
    "sourceType": "module"
  }
}

Actually I've encountered more strange behaviour, unless I'm missing something:

I believe lowercased components are just treated as HTML element tag strings - ie, no error is expected there.

This is happening because Text is a global in the browser environment.
Here's the data that ESLint uses to set up the globals for the environments: https://developer.mozilla.org/en-US/docs/Web/API/Text
And here's more information on the Text API: https://developer.mozilla.org/en-US/docs/Web/API/Text

Yep, It's an edge case an there is nothing we can do to fix this.


--check mode does not write cache

I noticed that the cache is not written when --check mode is on. This makes sense if the check fails, but if the file being checked is well-formatted, it would seem to be safe to write the cache. I propose that in --check mode we write to the cache only if the checked file is well-formatted.
My proposed behavior is useful when running Black in CI with --check, where we will frequently check the same file, and almost all will usually be well-formatted.

Agreed :) happy to merge a PR about this

Good catch, this is going to make things run faster.


Allow undefined stereochemistry on molecule export

Description
Fixes #58 by allowing undefined stereochemistry during molecule export.
Questions

[ ] Is this safe to do or should we skip these records?

Status

[ ] Ready to go

I think we should actually skip these exports (mark them as failed jobs). The user that reported the issue can't share the molecule that it happened on, and this doesn't appear to be a stereochemistry case that I've ever seen before. So even if we get past this step, this case might just cause an error upon trying to load this molecule in the next step.

Closing as we decided to go with #65. Thank you for preparing this and launching the solution, @jthorton!


Footer links not arranged properly

Link: https://nextjs-reusables.vercel.app/forms

Thanks. I'll make a push for that. It's not an error on the form library but on the examples website itself. Layout Footer here


docs: update cloudfunction guide && images preview

Refine the cloudfunction guide according to the version 1.0 of laf,
Add picture preview function.

There are conflicts to be dealt with.  @nightwhite


Fix broken integration tests

12 integration tests consistently fail.  We should fix them or delete them.
Example of a failing build: https://travis-ci.org/search/grails-spring-security-core
Failure details:
- Running tests for configuration static with GEB: chromeHeadless and Grails 3.3.0
 -- Test there are no roles initially(specs.RoleSpec)
 -- Test add a role(specs.RoleSpec)
specs.RoleSpec > add a role FAILED
    org.openqa.selenium.WebDriverException at RoleSpec.groovy:34
 -- Test edit the details(specs.RoleSpec)
specs.RoleSpec > edit the details FAILED
    geb.error.RequiredPageContentNotPresent at RoleSpec.groovy:45
 -- Test show role(specs.RoleSpec)
specs.RoleSpec > show role FAILED
    geb.error.RequiredPageContentNotPresent at RoleSpec.groovy:76
 -- Test delete role(specs.RoleSpec)
specs.RoleSpec > delete role FAILED
    geb.error.RequiredPageContentNotPresent at RoleSpec.groovy:86
 -- Test there are no users initially(specs.UserSpec)
 -- Test add a user(specs.UserSpec)
specs.UserSpec > add a user FAILED
    org.openqa.selenium.WebDriverException at UserSpec.groovy:34
 -- Test edit the details(specs.UserSpec)
specs.UserSpec > edit the details FAILED
    geb.error.RequiredPageContentNotPresent at UserSpec.groovy:48
 -- Test show user(specs.UserSpec)
specs.UserSpec > show user FAILED
    geb.error.RequiredPageContentNotPresent at UserSpec.groovy:83
 -- Test delete user(specs.UserSpec)
specs.UserSpec > delete user FAILED
    geb.error.RequiredPageContentNotPresent at UserSpec.groovy:93
 -- Test create roles(specs.StaticSecuritySpec)
specs.StaticSecuritySpec > create roles FAILED
    org.openqa.selenium.WebDriverException at StaticSecuritySpec.groovy:29
 -- Test create users(specs.StaticSecuritySpec)
specs.StaticSecuritySpec > create users FAILED
    org.openqa.selenium.WebDriverException at StaticSecuritySpec.groovy:75
 -- Test secured urls not visible without login(specs.StaticSecuritySpec)
 -- Test check allowed for admin1(specs.StaticSecuritySpec)
specs.StaticSecuritySpec > check allowed for admin1 FAILED
    org.openqa.selenium.WebDriverException at StaticSecuritySpec.groovy:172
 -- Test check allowed for admin2(specs.StaticSecuritySpec)
specs.StaticSecuritySpec > check allowed for admin2 FAILED
    org.openqa.selenium.WebDriverException at StaticSecuritySpec.groovy:229
15 tests completed, 12 failed
:functional-test-app:integrationTest FAILED
:functional-test-app:mergeTestReports

Steps to reproduce
Run the following command in the root of the project:
./gradlew -DTESTCONFIG=static -Dgeb.env=chromeHeadless functional-test-app:check

After some investigation, here are my findings I uncovered while testing locally:
Executing the following command in the root of the project (grails-spring-security-core) duplicates the errors seen in Travis:
./gradlew -DTESTCONFIG=static -Dgeb.env=chromeHeadless functional-test-app:check

Executing the following command in the root of the project (grails-spring-security-core) will result in all tests passing (assuming you have firefox installed):
./gradlew -DTESTCONFIG=static functional-test-app:check

The same tests will run and pass if you execute the following command (assuming you have firefox installed) in directory grails-spring-security-core/functional-test-app
grails -DTESTCONFIG=static test-app -integration

And this will fail the same as Travis if run in directory grails-spring-security-core/functional-test-app:
grails -DTESTCONFIG=static -Dgeb.env=chromeHeadless test-app -integration

So.... the problem is with -Dgeb.env=chromeHeadless.  These tests are failing when run in a headless environment.
I will continue to investigate.

Problem solved :)
Integration tests failed if run in a chromeHeadless environment.
Upgrading the chromedriver and geckodriver to more recent versions resolved the issue.
I also added some additional code to build.gradle that users may comment out if they wish to see more test failure details in Travis.  It helps with troubleshooting.

Closed by: https://github.com/grails-plugins/grails-spring-security-core/commit/0a5cc69f27ac0cd3810c49826603336b7fa8e9d2
and:
https://github.com/grails-plugins/grails-spring-security-core/commit/4b859ddf59660b1646990d82d5b93c12d84d8d4d


[translation] Migrate @azure-rest/ai-translation-text to ESM/vitest

Packages impacted by this PR

@azure-rest/ai-translation-text

Issues associated with this PR

https://github.com/Azure/azure-sdk-for-js/issues/31338

Describe the problem that is addressed by this PR
Migrates the @azure-rest/ai-translation-text package to ESM/vitest via automation.
What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?
Are there test cases added in this PR? (If not, why?)
Provide a list of related PRs (if any)
Command used to generate this PR:**(Applicable only to SDK release request PRs)
Checklists

[ ] Added impacted package name to the issue description
[ ] Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
[ ] Added a changelog (if necessary)

API change check
API changes are not detected in this pull request.

@microsoft-github-policy-service rerun

/check-enforcer evaluate