Evaluation on Map Prediction

Hi,
Thanks for this great work!
I see in the provided, saved semantic map there are train and val folders.
Have you tried to use the val data to validate and test the map predictor? I tried to integrate an Evaluation Hook during training, but the results are mostly all zero IoU. I'm not sure if I am wrong.
Also, I tried to run all the 2000 episodes in HM3D val set, but only 30 of them got non-zero map prediction. I am using your official model, is this expected?

That is not expected, the map prediction should never be all zero. Which parts of the code have you modified?

Hi, thanks for the feedback!
I added these two lines in collect.py and didn't modify
while not hab_env.episode_over:
    action = nav_agent.act(observations)
    if nav_agent.agent_states.target_pred.max()>0:
        flag = True
    observations = hab_env.step(action)

I did the evaluation on the 2000 episodes in hm3d-val set. There seem to be only 30 episodes that flag=True.
I also used the visualizer to show the map prediction, most of which looks like this:

I think the value map is the original prediction (Z_t in original paper Eqn (3)) and the Dist Weight is used in Eqn (4)?

I also updated prediction/train_prediction_model.py as follows:
import torch, torchvision
import torch.nn as nn
import torch.nn.functional as F
import mmseg

import mmcv
import os.path as osp
import numpy as np
from scipy.special import expit
from PIL import Image
import matplotlib.pyplot as plt

from mmseg.datasets.builder import PIPELINES
from mmseg.datasets.builder import DATASETS
from mmseg.datasets.custom import CustomDataset
from mmcv.utils import print_log
from mmseg.utils import get_root_logger
from mmseg.models.builder import LOSSES
from mmseg.models.losses.utils import weighted_loss
from mmcv import Config
from mmseg.apis import set_random_seed
from mmseg.utils import get_device
from mmseg.datasets import build_dataset
from mmseg.models import build_segmentor
from mmseg.apis import train_segmentor
from mmseg.core import intersect_and_union

NUM_TARGET_CATEGORIES = 6
NUM_EXTRA_CATEGORIES = 3

def sigmoid(x):
    return expit(x)

@PIPELINES.register_module()
class LoadMapFromFile(object):
    """
    Load semantic maps from file.
    Requires key "img_info" (a dict that must contain the key "filename").
    """

    def __init__(self,
                 to_float32=False,
                 file_client_args=dict(backend='disk'),
                 imdecode_backend='np'):
        self.to_float32 = to_float32
        self.file_client_args = file_client_args.copy()
        self.file_client = None
        self.imdecode_backend = imdecode_backend

    def __call__(self, results):
        """
        Call functions to load image and get image meta information.
        Args:
            results (dict): Result dict from :obj:`mmseg.CustomDataset`.
        Returns:
            dict: The dict contains loaded image and meta information.
        """

        if self.file_client is None:
            self.file_client = mmcv.FileClient(**self.file_client_args)

        if results.get('img_prefix') is not None:
            filename = osp.join(results['img_prefix'],
                                results['img_info']['filename'])
        else:
            filename = results['img_info']['filename']
        maps = np.load(filename)
        if filename[-1] == 'z':
            maps = maps['maps']
        img = maps[results['img_info']['t_idx']].transpose(1, 2, 0)
        img = img.astype(np.float32) / 255.

        results['filename'] = filename
        results['ori_filename'] = results['img_info']['filename']
        results['img'] = img
        results['img_shape'] = img.shape
        results['ori_shape'] = img.shape

        # Set initial values for default meta_keys
        results['pad_shape'] = img.shape
        results['scale_factor'] = 1.0
        num_channels = img.shape[0]
        results['img_norm_cfg'] = dict(
            mean=np.zeros(num_channels, dtype=np.float32),
            std=np.ones(num_channels, dtype=np.float32),
            to_rgb=False)

        mask = (img[:, :, 1] > 0)
        goals = range(4, 4 + NUM_TARGET_CATEGORIES)  # channels of semantic map

        # Setting the "ground-truth" for prediction here
        results['gt_semantic_seg'] = (maps[-1, goals] * (1 - mask)).transpose(1, 2, 0)
        results['seg_fields'].append('gt_semantic_seg')
        return results

    def __repr__(self):
        repr_str = self.__class__.__name__
        repr_str += f'(to_float32={self.to_float32},'
        repr_str += f"imdecode_backend='{self.imdecode_backend}')"
        return repr_str

@DATASETS.register_module()
class SemMapDataset(CustomDataset):

    CLASSES = ['chair', 'couch', 'potted plant', 'bed', 'toilet', 'tv', 'dining-table', 'oven',
              'sink', 'refrigerator', 'book', 'clock', 'vase', 'cup', 'bottle']
    PALETTE = np.array([
        1.0, 1.0, 1.0,
        0.6, 0.6, 0.6,
        0.95, 0.95, 0.95,
        0.96, 0.36, 0.26,
        0.12156862745098039, 0.47058823529411764, 0.7058823529411765,
        0.9400000000000001, 0.7818, 0.66,
        0.9400000000000001, 0.8868, 0.66,
        0.8882000000000001, 0.9400000000000001, 0.66,
        0.7832000000000001, 0.9400000000000001, 0.66,
        0.6782000000000001, 0.9400000000000001, 0.66,
        0.66, 0.9400000000000001, 0.7468000000000001,
        0.66, 0.9400000000000001, 0.8518000000000001,
        0.66, 0.9232, 0.9400000000000001,
        0.66, 0.8182, 0.9400000000000001,
        0.66, 0.7132, 0.9400000000000001,
        0.7117999999999999, 0.66, 0.9400000000000001,
        0.8168, 0.66, 0.9400000000000001,
        0.9218, 0.66, 0.9400000000000001,
        0.9400000000000001, 0.66, 0.8531999999999998,
        0.9400000000000001, 0.66, 0.748199999999999]).reshape((20, 3)) * 255

    def __init__(self, **kwargs):
        super().__init__(img_suffix='.npz', seg_map_suffix='.npz',
                         split=None, **kwargs)
        assert osp.exists(self.img_dir)

    def load_annotations(self, img_dir, img_suffix, ann_dir, seg_map_suffix,
                         split):
        """
        Load annotation from directory.
        Args:
            img_dir (str): Path to image directory
            img_suffix (str): Suffix of images.
            ann_dir (str|None): Path to annotation directory.
            seg_map_suffix (str|None): Suffix of segmentation maps.
            split (str|None): Split txt file. If split is specified, only file
                with suffix in the splits will be loaded. Otherwise, all images
                in img_dir/ann_dir will be loaded. Default: None
        Returns:
            list[dict]: All image info of dataset.
        """

        img_infos = []
        k=0

        for img in self.file_client.list_dir_or_file(
                dir_path=img_dir,
                list_dir=False,
                suffix=img_suffix,
                recursive=True):
            k+=1
            for t_idx in range(10):  # use first 10 timesteps as partial map inputs
                img_info = dict(filename=img)
                img_info['t_idx'] = t_idx
                img_infos.append(img_info)

        img_infos = sorted(img_infos, key=lambda x: x['filename'])

        print_log(f'Loaded {len(img_infos)} images from {k} .npz files', logger=get_root_logger())
        return img_infos

    def get_ann_info(self, idx):
        """
        Get annotation by index.
        """
        # We don't have separate annotation files, everything is in the map sequence
        return None

    def get_gt_seg_map_by_idx(self, index):
        """Get one ground truth segmentation map for evaluation."""
        img_info = self.img_infos[index]
        results = dict(img_info=img_info)
        self.pre_pipeline(results)
        results = self.pipeline(results)
        return results['gt_semantic_seg']

    def pre_eval(self, preds, indices):
        """Collect eval result from each iteration.

        Args:
            preds (list[torch.Tensor] | torch.Tensor): the segmentation logit
                after argmax, shape (N, H, W).
            indices (list[int] | int): the prediction related ground truth
                indices.

        Returns:
            list[torch.Tensor]: (area_intersect, area_union, area_prediction,
                area_ground_truth).
        """
        # In order to compat with batch inference
        if not isinstance(indices, list):
            indices = [indices]
        if not isinstance(preds, list):
            preds = [preds]

        pre_eval_results = []

        for pred, index in zip(preds, indices):
            # Original pred is logit, we need to convert it to probability
            pred = sigmoid(pred)
            seg_map = self.get_gt_seg_map_by_idx(index)
            seg_map = seg_map[0].transpose(2, 0, 1) / 255.
            pre_eval_results.append(
                intersect_and_union(
                    pred,
                    seg_map,
                    NUM_TARGET_CATEGORIES,
                    self.ignore_index,
                    # as the labels has been converted when dataset initialized
                    # in `get_palette_for_custom_classes ` this `label_map`
                    # should be `dict()`, see
                    # https://github.com/open-mmlab/mmsegmentation/issues/1415
                    # for more ditails
                    label_map=dict(),
                    reduce_zero_label=self.reduce_zero_label))

        return pre_eval_results

@weighted_loss
def my_loss(pred, target):
    target = torch.permute(target, (0, 3, 1, 2))
    assert pred.size() == target.size() and target.numel() > 0
    wts = [36.64341412, 30.19407855,<PHONE_NUMBER>6, 25.58503269,<PHONE_NUMBER>,<PHONE_NUMBER>6]  # inverse frequency
    pos_weight = torch.ones(pred[0].shape).to(pred.device)
    for i, wt in enumerate(wts):
        pos_weight[i] = wts[i]

    loss = F.binary_cross_entropy_with_logits(pred, target / 255., reduction='none')  # no weighting
    # loss = F.binary_cross_entropy_with_logits(pred, target / 255., reduction='none', pos_weight=pos_weight)
    return loss

@LOSSES.register_module
class MyLoss(nn.Module):

    def __init__(self, reduction='mean', loss_weight=1.0):
        super(MyLoss, self).__init__()
        self.reduction = reduction
        self.loss_weight = loss_weight

    def forward(self,
                pred,
                target,
                weight=None,
                avg_factor=None,
                reduction_override=None,
                ignore_index=None):
        assert reduction_override in (None, 'none', 'mean', 'sum')
        reduction = (
            reduction_override if reduction_override else self.reduction)
        loss = self.loss_weight * my_loss(
            pred, target, weight, reduction=reduction, avg_factor=avg_factor)
        return loss

    @property
    def loss_name(self):
        return 'loss_bce'

if __name__ == '__main__':

    cfg = Config.fromfile('prediction/configs/pspnet/pspnet_r50-d8_512x1024_80k_cityscapes.py')

    # Since we use only one GPU, BN is used instead of SyncBN
    cfg.norm_cfg = dict(type='BN', requires_grad=True)
    cfg.model.backbone.norm_cfg = cfg.norm_cfg
    cfg.model.decode_head.norm_cfg = cfg.norm_cfg
    cfg.model.backbone.in_channels = 4 + NUM_TARGET_CATEGORIES + NUM_EXTRA_CATEGORIES + 1
    cfg.model.decode_head.num_classes = NUM_TARGET_CATEGORIES
    cfg.model.decode_head.loss_decode = dict(type='MyLoss', loss_weight=1.0)

    cfg.model.auxiliary_head.num_classes = NUM_TARGET_CATEGORIES
    cfg.model.auxiliary_head.loss_decode = dict(type='MyLoss', loss_weight=0.4)
    cfg.model.auxiliary_head.norm_cfg = cfg.norm_cfg

    # Modify dataset type and path
    cfg.dataset_type = 'SemMapDataset'
    cfg.data_root = 'data/saved_maps'

    cfg.data.samples_per_gpu = 8
    cfg.data.workers_per_gpu = 2

    cfg.img_norm_cfg = dict(
        mean=[0, 0, 0], std=[1 ,1, 1], to_rgb=False)

    orig_in_size = 960   # the map size
    in_size = orig_in_size
    cfg.crop_size = (in_size, in_size)
    cfg.train_pipeline = [
        dict(type='LoadMapFromFile'),
        dict(type='Resize', img_scale=None, ratio_range=(in_size / orig_in_size, in_size / orig_in_size)),
        dict(type='Pad', size=(int(in_size * 1.25), int(in_size * 1.25)), pad_val=0, seg_pad_val=0),
        dict(type='RandomCrop', crop_size=cfg.crop_size, cat_max_ratio=1.),
        dict(type='RandomFlip', flip_ratio=0.5),
        dict(type='RandomRotate', prob=1., degree=180, pad_val=0, seg_pad_val=0),
        dict(type='DefaultFormatBundle'),
        dict(type='Collect', keys=['img', 'gt_semantic_seg']),
    ]

    cfg.test_pipeline = [
        dict(type='LoadMapFromFile'),
        dict(
            type='MultiScaleFlipAug',
            img_scale=None,
            img_ratios=[in_size / orig_in_size],
            flip=False,
            transforms=[
                dict(type='Resize', keep_ratio=True),
                dict(type='ImageToTensor', keys=['img']),
                dict(type='Collect', keys=['img', 'gt_semantic_seg']),
            ])
    ]

    cfg.data.train.type = cfg.dataset_type
    cfg.data.train.data_root = cfg.data_root
    cfg.data.train.img_dir = 'train'
    cfg.data.train.ann_dir = None
    cfg.data.train.pipeline = cfg.train_pipeline

    cfg.data.val.type = cfg.dataset_type
    cfg.data.val.data_root = cfg.data_root
    cfg.data.val.img_dir = 'val'
    cfg.data.train.ann_dir = None
    cfg.data.val.pipeline = cfg.test_pipeline

    cfg.data.test.type = cfg.dataset_type
    cfg.data.test.data_root = cfg.data_root
    cfg.data.test.img_dir = 'val'
    cfg.data.train.ann_dir = None
    cfg.data.test.pipeline = cfg.test_pipeline

    # Set up working dir to save files and logs.
    cfg.work_dir =  'data/work_dirs/final_model'

    cfg.runner.max_iters = 60000
    cfg.log_config.interval = 100
    cfg.evaluation.interval = 200
    cfg.checkpoint_config.interval = 2000
    cfg.optimizer = optimizer = dict(type='Adam', lr=0.0005)
    cfg.lr_config.min_lr = 1e-5

    # Set seed to facilitate reproducing the result
    cfg.seed = 0
    set_random_seed(0, deterministic=False)
    cfg.gpu_ids = range(1)
    cfg.device = get_device()

    # Let's have a look at the final config used for training
    print(f'Config:\n{cfg}')

    # Build the dataset
    datasets = [build_dataset(cfg.data.train)]

    # Build the detector
    model = build_segmentor(cfg.model)

    # Add an attribute for visualization convenience
    model.CLASSES = datasets[0].CLASSES

    # Create work_dir
    mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))
    cfg.dump(osp.join(cfg.work_dir, 'cfg.py'))
    model.train()
    train_segmentor(model, datasets, cfg, distributed=False, validate=False,
                    meta=dict())

Basically, I added get_gt_seg_map_by_idx and pre_eval functions in the SemMapDataset class, they will work with the single_gpu_test function in prediction/mmseg/apis/test.py here to enable evaluation:
def single_gpu_test(model,
                    data_loader,
                    show=False,
                    out_dir=None,
                    efficient_test=False,
                    opacity=0.5,
                    pre_eval=False,
                    format_only=False,
                    format_args={}):
    """Test with single GPU by progressive mode.

    Args:
        model (nn.Module): Model to be tested.
        data_loader (utils.data.Dataloader): Pytorch data loader.
        show (bool): Whether show results during inference. Default: False.
        out_dir (str, optional): If specified, the results will be dumped into
            the directory to save output results.
        efficient_test (bool): Whether save the results as local numpy files to
            save CPU memory during evaluation. Mutually exclusive with
            pre_eval and format_results. Default: False.
        opacity(float): Opacity of painted segmentation map.
            Default 0.5.
            Must be in (0, 1] range.
        pre_eval (bool): Use dataset.pre_eval() function to generate
            pre_results for metric evaluation. Mutually exclusive with
            efficient_test and format_results. Default: False.
        format_only (bool): Only format result for results commit.
            Mutually exclusive with pre_eval and efficient_test.
            Default: False.
        format_args (dict): The args for format_results. Default: {}.
    Returns:
        list: list of evaluation pre-results or list of save file names.
    """
    if efficient_test:
        warnings.warn(
            'DeprecationWarning: ``efficient_test`` will be deprecated, the '
            'evaluation is CPU memory friendly with pre_eval=True')
        mmcv.mkdir_or_exist('.efficient_test')
    # when none of them is set true, return segmentation results as
    # a list of np.array.
    assert [efficient_test, pre_eval, format_only].count(True) <= 1, \
        '``efficient_test``, ``pre_eval`` and ``format_only`` are mutually ' \
        'exclusive, only one of them could be true .'

    model.eval()
    results = []
    dataset = data_loader.dataset
    prog_bar = mmcv.ProgressBar(len(dataset))
    # The pipeline about how the data_loader retrieval samples from dataset:
    # sampler -> batch_sampler -> indices
    # The indices are passed to dataset_fetcher to get data from dataset.
    # data_fetcher -> collate_fn(dataset[index]) -> data_sample
    # we use batch_sampler to get correct data idx
    loader_indices = data_loader.batch_sampler

    for batch_indices, data in zip(loader_indices, data_loader):
        with torch.no_grad():
            result = model(return_loss=False, **data)

        if show or out_dir:
            img_tensor = data['img'][0]
            img_metas = data['img_metas'][0].data[0]
            imgs = tensor2imgs(img_tensor, **img_metas[0]['img_norm_cfg'])
            assert len(imgs) == len(img_metas)

            for img, img_meta in zip(imgs, img_metas):
                h, w, _ = img_meta['img_shape']
                img_show = img[:h, :w, :]

                ori_h, ori_w = img_meta['ori_shape'][:-1]
                img_show = mmcv.imresize(img_show, (ori_w, ori_h))

                if out_dir:
                    out_file = osp.join(out_dir, img_meta['ori_filename'])
                else:
                    out_file = None

                model.module.show_result(
                    img_show,
                    result,
                    palette=dataset.PALETTE,
                    show=show,
                    out_file=out_file,
                    opacity=opacity)

        if efficient_test:
            result = [np2tmp(_, tmpdir='.efficient_test') for _ in result]

        if format_only:
            result = dataset.format_results(
                result, indices=batch_indices, **format_args)
        if pre_eval:
            # TODO: adapt samples_per_gpu > 1.
            # only samples_per_gpu=1 valid now
            result = dataset.pre_eval(result, indices=batch_indices)
            results.extend(result)
        else:
            results.extend(result)

        batch_size = len(result)
        for _ in range(batch_size):
            prog_bar.update()

    return results

I got zero IoU during evaluation on the val set, not sure where I was wrong.

If the IoU is taking the argmax over the classes, it may indeed be zero, because the model is trained with binary cross entropy and there is no "empty" class. However, something is definitely wrong with your prediction outputs, because the visualizer should produce stuff like the videos on our project page. Are you sure that the weights are being loaded successfully? And is the navigation success rate OK?

The weights should be loaded successfully, I am a bit confused on how to reproduce your success rate. Seems that I can only do this for HM3d val set (Table 4), but I'm not sure which 500 episodes did you use, there are 2000 episodes in total.
The figure I attached here is from the first episode in HM3d val set. Could you tell me which episode are you using for the figure you attached? I can try that.

It was an old image but I believe it was also the first episode. Table 4 does use the first 500 episodes, but I am not asking for an exact number; I am just wondering if you see that the agent's behavior is normal. For example, you can just check the first 10 episodes, I think there should be 6 or 7 successes

Yes I'm able to get non-zero output now and I'm testing the first 500 episodes.
It turns out that I was using my locally re-trained model (iter_60000.pth), maybe it is the re-training failure.
I'm also testing the prediction accuracy.
Thanks for the help, I'll reach out if I have further questions.

Hi,
I've finished the evaluation on the first 500 episodes, but I only get 0.311 SPL and 0.608 Success. I wonder which version of hm3d scene datasets did you use in Table 4? Is it v0.1? As I'm using v0.2, so I assume the score difference comes from that

Yes, it is v0.1 (see README)

Thanks, v0.1 got close results.


Changes unp_spec crawl test to use a unique temporary directory to av…

…oid conflicts with other temp files

The current directory /var/tmp can cause conflicts because systemd can add root owned files to this which will lead to permission denied errors.

Updated to use Tempfile.  However  Tempfile will by default use the OS's default tmp directory which in this case is '/tmp'.  This would cause the same conflict problems since unp.crawl() will try to crawl the entire directory yielding other temp files.  Therefore we still need to specify a specific directory that is unique to this test for it to consistently work.


Getting a deprecated warning/error with clang 14 and CXXOPTS_USE_UNICODE_HELP ON

Just wanna inform that I'm receiving a deprecated about when ever I build with CXXOPTS_USE_UNICODE_HELP ON on clang 14
It gives no warnings as soon as I don't build with the CXXOPTS_USE_UNICODE_HELP
The version I'm using is the 3.0.0 from releases
I'm compiling with -Werror -Wall -Wextra -O3 set in cmake and compiling with C++17
Btw the reason for the weird path that is because CPM to pull in dependencies.
[build] /home/buildclient/.cache/CPM/cxxopts/7604315a3c6ca9634c8e8a1107796dc9f310074e/cxxopts/include/cxxopts.hpp:122:10: error: 'iterator<std::forward_iterator_tag, int>' is deprecated [-Werror,-Wdeprecated-declarations]
[build]     std::iterator<std::forward_iterator_tag, int32_t>
[build]          ^
[build] /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/stl_iterator_base_types.h:127:12: note: 'iterator<std::forward_iterator_tag, int>' has been explicitly marked deprecated here
[build]     struct _GLIBCXX17_DEPRECATED iterator
[build]            ^
[build] /usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/x86_64-linux-gnu/c++/12/bits/c++config.h:119:34: note: expanded from macro '_GLIBCXX17_DEPRECATED'
[build] # define _GLIBCXX17_DEPRECATED [[__deprecated__]]
[build]                                  ^
[build] 1 error generated.

Thanks, I'll check it out.


Update fantomas in src as well... yep, it is weird

Fantomas needs to be present in two directories. Uhum. Yep, that's how it is. Otherwise, in Visual Studio, it won't work properly with the Format F# plugin.
They also should both have the same version, which is fixed here.

Bet it'll work if you move the sln to top level (which is also more typical, no?)

Yeah, it is more typical, but also makes it harder to add projects using the IDE (but then again, adding projects is not something we do every day for the repo anyway). I might give it a try.


PCC release of mAPS

close #424
close #425

@MaryLJ here is a DRAFT pull-request. You will see that I drafted up the changes for the index.html. Note the content is not there yet.
I am concerned that the previous public comments are still there, but didn't want to remove them.

mAPS international is now here

@MaryLJ here is a DRAFT pull-request. You will see that I drafted up the changes for the index.html. Note the content is not there yet.
I am concerned that the previous public comments are still there, but didn't want to remove them.

I think it is OK to remove them as the comment period is over.

Changes “this” to “these” and “document” to “documents”
Comments on this document should be submitted by July 4, 2024.
Mary Jungers | Documentation Specialist
IHE International | IHE.net
From: John Moehrke @.>
Sent: Tuesday, June 4, 2024 4:04 PM
To: IHE/publications @.>
Cc: Mary Jungers @.>; Mention @.>
Subject: Re: [IHE/publications] PCC release of mAPS (PR #434)
mAPS international is now here
—
Reply to this email directly, view it on GitHub https://github.com/IHE/publications/pull/434#issuecomment-2148412565 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AGDULOJEMQXIX2MOK4WAQ2DZFYTSJAVCNFSM6AAAAABIZJFCLOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBYGQYTENJWGU .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/AGDULOMCAVI4SYZVNJHCDZTZFYTSJA5CNFSM6AAAAABIZJFCLOWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUABYWJK.gif Message ID: @.*** @.***> >

ready for approval and deployment


Strange memory leak detector problem.

I already posted this in the google group but got no answer. The memory leak detector seems to get stuck in an infinite loop for me.
I have to use this hack to make CppUtest work for me:
https://github.com/kaimast/cpputest/commit/834e0bdb95d67135218455995706aa8222edf379

Hi @kaimast,
I believe me and several others did reply via the googlegroup. Didn't you receive these replies?
I looked at your hacks and the only behavior changing code that you made is a check on whether a node is recursive, as in it has itself as the next. If this were true, then obviously it would go in a recursive look, and you did resolve that. However, that case should be impossible.
Now, it is either a bug where this is somehow possible, or it is a memory corruption and you resolved the forever loop by the case that will eventually be true.
In order to figure out which one, could you put a breakpoint or so on the break that you inserted and see where it allocates memory and explore the node to see where it was allocated. The node ought to have file/line information in itself, so you ought to print it in a debugger or just printf the content of the node.
Alternatively, could you write a test that exposes this bug, so we can help with debugging this problem? In order words, why to remove production code out of your test and just send the test that causes this problem.

Hi @kaimast - From the information you provide (also on Google Groups), it is impossible to tell what might be going on, as the code of the test and the code being tested are missing.
As @basvodde suggested, perhaps you could write a brief test (all code in the test; no production code) to expose this behavior?

The memory leak tracking could get corrupted like this if the selected platform doesn't have mutex support but the test tries to allocate memory on multiple threads anyway.

@kaimast - are you still there? What is the status of this problem?

Sorry for the delay. I'm still trying to figure out if the project I am using is corrupting memory.

I assume this is resolved ?


Document use of clippy

Add documentation for how to use clippy with our project.  Ideally:

How to run clippy on the main crate
How to run clippy on our other crates (rusoto_codegen for example)

It'd be nice to document how to run clippy against our generated code to clean that up, too.

The way the features are configured, you can either:
Run clippy on the main crate (or any of the individual crates) by going into the relevant directory and running (on nightly):
cargo build --no-default-features --features nightly-testing

Note that running this command for the rusoto crate will also result in Clippy being called when building the dependency crates (e.g. rusoto_codegen).
Running Clippy will automatically have it check the generated code as well (because Clippy runs after serde).
This should probably be added to the GitBook.


Create attributes using attributes.xml

Description
Please describe your pull request:

Why is it necessary?
Added a way to create plugin attributes with xml
What does it improve?
plugin install/update simplification
Does it have side effects?
Nope

Questions
Answers

BC breaks?
no

Tests pass?
yes

Related tickets?

How to test?

Example File:
<?xml version="1.0" encoding="utf-8"?>
<tables xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:noNamespaceSchemaLocation="../../../../engine/Shopware/Components/Plugin/schema/attributes.xsd">

        <attribute name="test" type="combobox">
            I am a checkbox
            <supportText>supportText</supportText>
            <helpText>helpText</helpText>
            <displayInBackend>true</displayInBackend>
            <translatable>true</translatable>
            <custom>true</custom>
            <updateDependingTables>true</updateDependingTables>
            <arrayStore>
                <option key="1">Yes</option>
                <option key="0">No</option>
            </arrayStore>
        </attribute>

</tables>

Would prefer the keyword "arrayStore" to be "store" to unify it with config.xml.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-17638.
Please use this issue to track the state of your pull request.

Hey @shyim
thanks for your contribution :+1:
as we have to test this properly, we will schedule the ticket for one of the next sprints

@nlubisch Its unifyed

Hi @shyim
we tested your implementation and found a problem which is not easy to solve. If you change the structure of your attributes.xml and then perform an update the old attributes and their data remains in the database. There is also no possibility for renaming your attribute tables and migrate the data.
In case of the other xml files, all the data is actually in the xml and the database rows are referenced to the plugin ids. So the cleanup there is simple.
According to this we have to close this pull request because we do not find a suitable solution for the problem.

@taltholtmann Whats about a table which holds added attributes and from which plugin?

Hi @shyim,
we discussed the feature and found different problems. The most import problem is that some attributes should not be removed even if the plugin is uninstalled, like customer or order data. For that reason we want to keep the responsibility to the plugin developers.

This could be solved by a attribute in config.xml (false or with the secureUninstall question in backend

Would be possible, but we would need to many configuration parameters to solve all possible scenarios ;)
Other scenario would be a shopware version validation ( 5.2.16 => attribute table only exists in this version)

Other scenarios could be solved by using crud service in install method. Attribute.xml should not solve all possible scenarios :P

I'm not a fan of how attributes are added in the Plugin file right now, too. This is too much overhead and code duplication. In my opinion the best way is to add an remove attributes with the migrations that have been announced quite a long time ago - ping @bcremer @dnoegel The migration should offer handy shortcut methods to work with attributes.

@FiveDigital that's true, a migration system for plugins would solve many problems, but that is out of scope for this PR.
I've published a ticket for the plugin migration system where you can up-vote this feature.
https://issues.shopware.com/issues/SW-16207


Exclude/better identify transactions from events and projects by default

For the second time in as many weeks I have had to ensure the mental dissonance caused by this:

I find it confusing and have been confused by how this is presented currently.
I would like it to be clearer that the credit transactions are to projects within the collective. Or to remove them by default from the transaction history, so I actively engage with the concept of projects and events before seeing the artefacts in the transaction list.
I feel like we should also tackle https://github.com/opencollective/opencollective/issues/4806 when tackling this issue.

Related #4360
I think there's a whole deeper round of design thinking that needs to be done about how we handle project and event transactions. There are multiple related issues.


Updated the deps and geth updated to latest master

Signed-off-by: Krasi Georgiev<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build 692979335

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 52.086%

Totals

Change from base Build 673972777:
0.07%

Covered Lines:
1448

Relevant Lines:
2780

💛  - Coveralls


[master/1.14][DCOS-53077] Create TMPDIRs in known locations for various services to avoid probl…

…ect to be marked exec
High-level description
Along with DC/OS Enterprise changes, this allows DC/OS Enteprise to start when temporary directories are marked noexec on master nodes.
This was done with @jongiddy and @timaa2k and with input from @mhrabovcin .
See the Enterprise PR for the manual testing done there.
Once this is approved, I will backport this and make the following follow up issues:

Create a preflight check that /var/lib/dcos/exec is marked exec.
Document that /var/lib/dcos/exec must be marked exec.

Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS-53077 DC/OS cannot be installed if temporary directories are marked noexec.

Checklist for all PRs

[X] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change: As per https://github.com/dcos/dcos/wiki/CHANGES.md-guidelines#major-releases, and our intention to port this to 1.13 (and below) my understanding is that there is no CHANGES entry required here.
[X] Included a test which will fail if code is reverted but test is not. If there is no test please explain here: DCOS-18799 will count as a test for this.
[X] Read the DC/OS contributing guidelines
[X] Followed relevant code rules Rules for Packages and Systemd

In theory this solves the immediate customer issue.
However, we cannot know for sure without CI coverage which is coming later.
That will also cover agents and public agents.

Are we going to add some documentation about the fact that this new directory can potentially grow without being automatically cleaned up by OS?

Thank you for the work @adamtheturtle. I did not look at the details, but would appreciate if @gpaul had a look in addition to Jon's review.

Are we going to add some documentation about the fact that this new directory can potentially grow without being automatically cleaned up by OS?

Documentation issue is at DCOS_OSS-5127 - "Document that /var/lib/dcos/exec is a custom temporary directory and what to do about that".

As per comments on the EE counterpart of this issue I have put the creation of a private directory into a helper function.


changed message template to Hi instead Hello

:star: PR built and available in a preview environment sashkica-azure-ms-pr-1 here


[ACR] swagger updates for private links

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

/azp run automation - sdk

// cc @djyou @yugangw-msft

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

The modelling for privateEndpointconnections and privatelinkresources is in line with the recommended pattern. I think multiple SQL RPs have implemented it like this. Signing off from ARM side.


Disable git fsck for mirrored repos by default

Codecov Report

Merging #6018 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #6018      +/-   ##
==========================================
+ Coverage   38.72%   38.73%   +<.01%
==========================================
  Files         332      332
  Lines       48992    48992
==========================================
+ Hits        18973    18975       +2
+ Misses      27271    27269       -2
  Partials     2748     2748

Impacted Files
Coverage Δ

models/repo.go
47.24% <100%> (ø)
:arrow_up:

models/repo_list.go
64.55% <0%> (+1.26%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c865f3...fb747f6. Read the comment docs.


Added German translation

Please note that the German polite form (Sie) was used.

I don't get why this isn't merged

We have 2 german translations and I'm not sure which one to use. I guess I'll use this one!

Lol TDE5 is great :P


(FEAT) Add ability to define local template dir in build.sh script

When the ENV VAR LOCAL_TEMPLATE_PATH is defined, the build script
will copy the contents of this dir to either templates or
dist/$(GOHOSTARCH)_$(GOHOSTOS)/templates (dependent on args passed).

This should be separated out in to a dedicated script and kept separate from the build process.


The replication queue shows HTTP connection issues

Requests seem to result in "request-sent" message from python.

Fixed in 178b4e1d2be7ed901247ba2f58b0f06d968464b0


CI: Skip tests if there are no changes in the generated project

IMO https://github.com/jhipster/jhipster-lite/pull/8684 is more robust, since we don't rely on cache that could introduce side effects.
But @pascalgrimaud know better than me on the subject :-)

Thanks for trying to fix this complex ticket, but as @murdos said, https://github.com/jhipster/jhipster-lite/pull/8684 is better, so I'm closing this one


Online demo created

Not sure if this is of any use, but I created a small online demo using this tool when I was experimenting with it. You can view it online at
https://captioner.richardson.co.nz/
And the source code for it is at: https://github.com/Rodeoclash/captioner
It might be possible to adapt this for an official demo if you're interested (although it is lacking a few things at the moment, i.e. it only works on video and currently the videos have no audio when playing).

Very nice! I am thinking of adding a section to the Readme with some examples/demos/products using the library, what do you think?

I think being able to try it out easily is very helpful. I'm happy if you want to point to my demo, but you might be better off forking it (or I'm happy to transfer ownership of the repo) if you want a starting point for the demo. I really hacked it together and I think it could do with a bit of polish (which I'm happy to help out with).

Hey guys !
I discover your work yesterday. It's great.
I try to run it but it's not possible in a simple way. I try several things.
First the cdn, the first thing on your repo:
I try to run it with a single page using nodejs and that doesn't run if you put all in an index.html and open it with the computer. After spend the night on it, i think the generation of the model with a ramdom path is an big issue. On an express nodejs server the dynamic generation seems to be an issue. With the index file open in the desktop that push a url error. In each case the worker code is generate.
The two errors are  Error: HTTP error! status: 404 from the worker 0378db2c-af89-499f-877f-7f8d9a12851a:127
and TypeError: Failed to construct 'URL': Invalid base URL
Second the example in moder-vanilla, same test, same result
I test the repo of rodeoclash and it's good too
To Make the things works. Take the word example and put the modification you have doing compare to the moder-vanilla one.
To get it work i have to replace
`const sampleRate = 16000;
const mediaStream = await navigator.mediaDevices.getUserMedia({
    video: false,
    audio: {
        echoCancellation: true,
        noiseSuppression: true,
        channelCount: 1,
        sampleRate
    },
});
const audioContext = new AudioContext();
const source = audioContext.createMediaStreamSource(mediaStream);

const recognizer = new model.KaldiRecognizer(sampleRate, JSON.stringify(['[unk]', 'encen el llum', 'apaga el llum']));

recognizer.on("result", (message) => {
    const result = message.result;
    console.log(JSON.stringify(result, null, 2));

    const newSpan = document.createElement('span');
    newSpan.textContent = `${result.text} `;
    resultsContainer.insertBefore(newSpan, partialContainer);
});
recognizer.on("partialresult", (message) => {
    const partial = message.result.partial;
    console.log(JSON.stringify(message.result, null, 2));

    partialContainer.textContent = partial;
});

partialContainer.textContent = "Ready";

const recognizerNode = audioContext.createScriptProcessor(4096, 1, 1)
recognizerNode.onaudioprocess = (event) => {
    try {
        recognizer.acceptWaveform(event.inputBuffer)
    } catch (error) {
        console.error('acceptWaveform failed', error)
    }
}
source.connect(recognizerNode);`

by
`    const sampleRate = 48000;
const recognizer = new model.KaldiRecognizer(sampleRate);
recognizer.setWords(true);

recognizer.on("result", (message) => {
    const result = message.result;
    // console.log(JSON.stringify(result, null, 2));

    const newSpan = document.createElement('span');
    newSpan.textContent = `${result.text} `;
    resultsContainer.insertBefore(newSpan, partialContainer);
});
recognizer.on("partialresult", (message) => {
    const partial = message.result.partial;

    partialContainer.textContent = partial;
});

partialContainer.textContent = "Ready";

const mediaStream = await navigator.mediaDevices.getUserMedia({
    video: false,
    audio: {
        echoCancellation: true,
        noiseSuppression: true,
        channelCount: 1,
        sampleRate
    },
});

const audioContext = new AudioContext();
await audioContext.audioWorklet.addModule('recognizer-processor.js')
const recognizerProcessor = new AudioWorkletNode(audioContext, 'recognizer-processor', { channelCount: 1, numberOfInputs: 1, numberOfOutputs: 1 });
recognizerProcessor.port.postMessage({action: 'init', recognizerId: recognizer.id}, [ channel.port2 ])
recognizerProcessor.connect(audioContext.destination);

const source = audioContext.createMediaStreamSource(mediaStream);
source.connect(recognizerProcessor);`

With that you can use express with an express server with just that in the index.js and a public folder where you put your project word-vanilla
'const express = require('express');
const path = require('path')
const app = express();
const PORT = 3300;
app.use(express.static('public'));
app.get('/public/test.html', (req, res) => {
res.send("hello")
});
app.listen(PORT, () => console.log(Server listening on port: ${PORT}));'
and Voila just run node index.js of course you have to put the model you have download and set the right name in Vosk.createModel('model.tar.gz');  and add a / behing the name of the file.
You have do realy great job both of us.
I think the best is to make a folder with the js server i provide and remove your vanilla folder and the react too because you have lot of critical warning. Yoou have complicate the think with your react guys lol. Just simple express make the work far more accessible than now.
Again thank you for the job i have now a simple server wich provide the page without other language or framework so simple.
I gonna push the job a little more and gonna modify the code to connect the a websocket and so on. I used rhasspy before for my voice assistant. Now i can use to power a page where the performance of the detection is link to the devices.
Nice job and again thank you

The examples are meant to run in the browser, not nodejs. You need however to serve the files with a server (which you did in the end with express), you can't just open the HTML file with your browser, that is why you got the error.

open with my computer was the hopeless try for me but after all why not.
What i say however still true. You can try your repo, no one work out of the box. I works on an implementation of a websocket listener  of a node-red app i run this morning. If you want i can send you the file. I just finish to handle message in and out. I plan to devellop skill with this tools.
You are for me a prefect replacement for a part of rhasspy. I gonna respect the standard of rhasspy intent because he use a kaldi model too. So i am note sure a this point but i think i could apply them when vosk create the model. I you have more info on that part. i take it.
Since i gonna use tfjs for other stuff than the audio i think i gonna add it to handle wakeword because i am not aware of a tool that run in browser and dont need apikey.
Again thank a lot. for the stuff and sorry for my poor english.
I think i gonna have time to put that stuff in my repo tomorrow or the next day

@TechServiceConseil sounds like you worked it out.
If you are running this on the server then you could also take a look at https://www.npmjs.com/package/vosk which is the node bindings to use it. FYI - I've never used them, I just saw a link to it on the homepage of this repo.

Why did the sample rate need to be updated from 16000 to 48000? Isn't VOSK compatible with 16kHz audio data?


Feature: Open all servers in submenu in new window with split panes in iTerm

I would very much like to open all servers in a submenu with one click (ideal for opening sessions on every node in a cluster)
Ideally this would be in a new (iTerm) window in split-panes if possible (maybe configurable horizontal/vertical panes or tabs)

Thank you for the suggestion. For now, I'd recommend looking into using a combination of tmux and mosh or autossh to manage a situation like that. You could then create a Shuttle menu item that automatically starts that tmux session in a new terminal window.


Enable token streaming for text-generation

We have a few models that already support token streaming.
We could add an extra options
export interface Options {
	streaming?: boolean;
....
}

Here's an example by @julien-c @olivierdehaene using an improved API for EventSource with fetch-event-source.
https://gist.github.com/julien-c/9565fbc85d0baecee9f910017584d157
Available models
[
    "Rallio67/joi2_20B_instruct_alpha",
    "google/flan-t5-xxl",
    "google/flan-ul2",
    "bigscience/bloom",
    "bigscience/bloomz",
    "EleutherAI/gpt-neox-20b"
]

[could be a great first task for Adrien D. who's joining HF on April 10]


Maintaining a plain-text variant of the lists

Hi, I run a Discord bot that I have routed through a Pihole installation. That is, the bot runs a DNS query whenever a link shows up in a Discord message, and if it fails to resolve (aka Pihole blocks it), then the bot deletes the message.
Problem is, Pihole and perhaps a few other platforms use plain text lists. Just one domain name per line, no formatting, no markup.
This list would be super useful to me and surely a lot of other Pihole users, but in its current state we cannot use it without cloning the list and making our own local scripts to edit it into plain text, and then re-host it somewhere where Pihole can find it. Has any consideration gone to maintaining both the original JSON and also a plaintext variant within the repo, presumably using some kind of script to keep both lists in sync at the same time?
Edit: Should clarify, I am aware plain text sources are available in the readme but running on the assumption that they may change over time, I think having your list as a collection of those sources, still available in plain text, would be a nice touch. That said if your objective is purely to JSON-ify it, that's understandable and I'll stick to sources only.

Hey @RedPanda4552 thankyou so much for your Idea.
Generally spoken, I support the Idea of a synced Plain Text List, but I don't have the time to keep it updated and sync with the List. If you or someone else has a Idea on how we could implement a automation to do this for us, you're very welcome to share it, atm i cannot think of a efficient and working solution!

No worries then, I can keep using the sources that you feed off of for my own use case. I don't really have any brilliant ideas on how to maintain it myself, so I'll let this one go for now. Take care!

I reopen the idea, its a good one and perhaps somebody that knows how to do that will see that issue ;)

Keeping two files constantly updated seems hard to me, but you could implement this on your end: get the array of URLs from the JSON, stringify it, remove all the quotes, replace all the commas with line breakes (\n), and you technically have the right string that you can write to a .txt file.
It's just an idea but it might work!

We can use GitHub actions to achieve this. I'll push a PR later.

@RedPanda4552 https://github.com/nikolaischunk/discord-phishing-links/blob/main/txt/domain-list.txt there you go!
1000 thanks to @lucapolesel 😊


Add function to prevent scroll

Hi there 👋
data-lenis-prevent is very useful to allow nested scroll but in some peculiar cases, we don't have the control over an element that is expected to be scrollable so no way to add this attribute.
Here is an example: a third party cookie modal for fine grained controls.

In this situation this element is generated by their script and they do not provide events on modal triggering so I don't really see how I can add the attribute. Here is a solution I could think of:
new Lenis({
  // Prevent scroll on any `.foo` element
  preventScroll: (node) => {
    return node.classList.contains('foo')
  }
})

I can propose a PR for that :)

Hi Clement,
I guess yes, in 99% of cases.
If we want to cover every potential situation, we could define preventScroll this way:
type LenisOptions = {
  // ...
  preventScroll?: string[] | (node: Element) => boolean
}

hey @mickaelchanrion what service provides this cookie modal ?

@mickaelchanrion should be available in v1.1.0

exact @mickaelchanrion, does it suits your needs ? If so i'm closing this issue.


Raise AuthenticationCredentialsNotFoundException if Principal is not present and declared as not optional

Currently when defining a query or mutation method that has a non-nullable Principal argument, it fails with "Unresolved NullPointerException for executionId ..." when no principal is present in the security context.
The same scenario in a @RestController would result in 401 Unauthorized response.
It would be convenient if spring-graphql automatically validated the presence of a principal, when the argument is defined as non-null, and throws an AuthenticationException when not set, which then could be handled in an exception resolver.

Seems like a reasonable enhancement. /cc @rwinch for comment if any.

Superseded by #790.


feat: remove onTimingsCallback for Flutter 3.0

Description
This technically speaking is a breaking change, but since we are already planning on deprecating the FPSCounter the team has internally decided to turn the fps() method into a copy of the functionality from #1595 so that we can remove the method onTimingsCallback already so Flutter 3.0 users wont have any warnings. This method is either not used or a really small portion of people are using it, compared to the FPSCounter.
Checklist

[x] The title of my PR starts with a Conventional Commit prefix (fix:, feat:, docs: etc).
[x] I have read the Contributor Guide and followed the process outlined for submitting PRs.
[x] I have updated/added tests for ALL new/updated/fixed functionality.
[x] I have updated/added relevant documentation in docs and added dartdoc comments with ///.
[x] I have updated/added relevant examples in examples.

Breaking Change

[x] Yes, this is a breaking change, but as we are doing this on purpose I have not added the suffix !.
[ ] No, this is not a breaking change.

Related Issues

:+1: to this change - even with the old FpsCounter it was unclear why the method was exposed in the Game's API, since it was possible to subscribe to the timing events from the mixing itself.

this is a breaking change, but as we are doing this on purpose I have not added the suffix !.

That doesn't sound right though. All breaking changes that we do are deliberate, and are marked with !. Of course, there could be some breaking changes that we introduced without noticing, by accident - those will have no !.


Being able to display a simple geometry on top of buffer manager

Summary
Ability to display box geometry and manipulate its position.
Value
Basic geometry allows us to implement canvas for all other components.
Specification

Geometry should be defined using our GPU buffer manager attributes.
All its properties, like position, should be lazily updated on th GPU side.

Acceptance Criteria & Test Cases

working example scene
tests

Issue moved to luna/basegl #2 via ZenHub


Add analyze output to bug-report

@ostromart: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-notes_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

integ-pilot-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

integ-pilot-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-pilot-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-ipv6-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

integ-pilot-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-pilot-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-ipv6-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

integ-pilot-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-pilot-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-ipv6-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-telemetry-k8s-tests_istio

integ-multicluster-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test analyze-tests_istio

unit-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test unit-tests_istio

integ-security-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-distroless-k8s-tests_istio

release-test_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test release-test_istio

lint_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test lint_istio

integ-pilot-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-pilot-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-ipv6-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-telemetry-k8s-tests_istio

integ-multicluster-k8s-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-multicluster-k8s-tests_istio

integ-operator-controller-tests_istio
8c601051437c41f52984c3a217fad7577b4aab26
link
/test integ-operator-controller-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
b0327d8ccb36a9b6119940157c2178549b8edf26
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@ostromart: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
b0327d8ccb36a9b6119940157c2178549b8edf26
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest


Module scoped, autouse clean_marathon_state

High-level description
This changes clean_marathon_state to module scoped - so that it can exist in EE and OSS, and coexist with module scoped fixtures which create Marathon resources. See the JIRA issue for alternatives discussed.
Corresponding DC/OS tickets (required)

DCOS-45746 tests: autouse=True Marathon fixture breaks module-scoped Marathon app fixtures.

@mesosphere-mergebot changelog-not-required Test only change

Please can an owner run:
@mesosphere-mergebot override-status
teamcity/dcos/test/terraform/aws/onprem/static/group1 https://jira.mesosphere.com/browse/DCOS-51211

@mesosphere-mergebot override-status
teamcity/dcos/test/terraform/aws/onprem/static/group1 https://jira.mesosphere.com/browse/DCOS-51211


feat: Clean up all related messages upon project finalisation

This makes the bot clean up all of its sent messages in relation to projects when a project is accepted, declined, or the original message is deleted.
This will need to wait for #285 to merge as it will involve this logic being shifted over and moved after that PR merges. Until then, it will be in the freezer.
Closes #283.

This functionality was implemented as part of the refactor (#293), so this can be closed now.


Include schemes ws:// and wss:// for web sockets in hostname counts

Do we use any other schemes? E.g. ws:// and wss:// for web sockets?
Originally posted by @tunetheweb in https://github.com/HTTPArchive/data-pipeline/pull/34#discussion_r851631118

These are logged as errors. Could that possibly be contributing to #72, as if it was handled as a fatal error?


How to apply class to table rows

Vue.js version: 2.0
consumed using: webpack

Is there a way to apply bootstrap table class to table row regarding a condition.
For example, apply this class :

...

if one of my data in this row meets a condition.

Not at the moment.
You can fork the repo and change the template to your liking.

Just in case anyone need it, I've faced the same problem and solved it using jquery.

Where #shipments is my vue-tables container div and inside contains() you say what's displayed inside a td.


Autonomous tweaks

Added the flipping for alliance colour. Also changed starting position to be in front of speaker.
There is still a big bug in the generation of trajectories. If you start the sim, then select 2 note auto and run straight away, it will throw an error in the trajectory generation. But if you drive around a bit first in teleop it sometimes works, sometimes fails.
Even then, the trajectories have weird circles at the end of the path.
Help requested!!

I figured it out - bad use of headings when making trajectories for holonomic drivebase.

I think we should also consider storing the note positions and our preferred firing positions (if relevant) in a way that lets us label them (perhaps a NotePositions dataclass?). It would help to reduce the number of magic coordinates and allow us to change firing positions globally.

I think we should also consider storing the note positions and our preferred firing positions (if relevant) in a way that lets us label them (perhaps a NotePositions dataclass?). It would help to reduce the number of magic coordinates and allow us to change firing positions globally.

That's a good idea, but beyond the scope of this PR. Perhaps do this next?


[CSYS-453] Update Button component

Link to task: Card #453 from Jira
Staging app: https://deploy-preview-69--confetti-storybook.netlify.app/
How to test: At the Storybook page, click at the Button section and test its properties and check if they behave as the Figma prototype
Description of your solution: Updating the Button Base Props and its related interfaces as also updating the Button imports and tests.
Screenshots (when applicable):

there are a LOT of missing snapshots.
did you remove them intentionally? if so, why?


mIOU is low

Thanks for sharing the code.
When I used this project to train Pascal VOC for nearly 200 epochs, I got only 0.32 mIOU.
The result on tensorboardX is:

My configuration is:
learning rate=0.0001
backbone=xception
use_sbd=False
And I have set size_average=True in nn.CrossEntropyLoss function.

I noticed that you set use_sbd=False, that means you only use VOC 2012 dataset to train the model. Therefore, maybe training images are insufficient. Pls look into #23 .
You can download SBD dataset and set use_sbd=True to train the model.

Thanks for your reply.
I will use SBD dataset and train again.

You're welcome.


Remove Chef Server from Omnitruck

This just causes confusion as people just pull down an really old out of date CS11. For example:
➜  curl "http://www.getchef.com/chef/metadata-server?prerelease=false&nightlies=false&p=ubuntu&pv=12.04&m=x86_64"
url	http://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.1.6-1_amd64.deb
md5	d07cf230410b55bd8939ab29d65a3cc5
sha256	b0a27a0b04c30c102d65fa7e2f2f40f64e9cdde7ff07a453a91e1113b52929d8

/cc @chef/ociv @chef/client-engineers
This is for @randomcamel :heart:.

I haven't checked the code but sounds like the current behavior is really broken, we should ship this.
Cc: @chef/lob

On Apr 17, 2015, at 8:47 AM, Seth Chisamore<EMAIL_ADDRESS>wrote:
This just causes confusion as people just pull down an really old out of date CS11. For example:
➜  curl "http://www.getchef.com/chef/metadata-server?prerelease=false&nightlies=false&p=ubuntu&pv=12.04&m=x86_64"
url http://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.1.6-1_amd64.deb
md5 d07cf230410b55bd8939ab29d65a3cc5
sha256  b0a27a0b04c30c102d65fa7e2f2f40f64e9cdde7ff07a453a91e1113b52929d8
/cc @chef/ociv @chef/client-engineers
This is for @randomcamel .
You can view, comment on, or merge this pull request online at:
https://github.com/chef/opscode-omnitruck/pull/90
Commit Summary
Remove Chef Server from Omnitruck
File Changes
M .gitignore (2)
M README.md (47)
M app.rb (41)
M backup/README.md (26)
M backup/backup-s3-bucket.sh (10)
D backup/rerelease-server.rb (70)
M backup/reverse-release-json.rb (4)
M config/config.yml.example (3)
M lib/opscode/version_tester.rb (9)
M s3_poller (21)
M spec/app_spec.rb (232)
D spec/data/build_server_list_v1.json (159)
D spec/data/build_server_list_v2.json (611)
M spec/spec_helper.rb (2)
M views/install.sh.erb (3)
Patch Links:
https://github.com/chef/opscode-omnitruck/pull/90.patch
https://github.com/chef/opscode-omnitruck/pull/90.diff
—
Reply to this email directly or view it on GitHub.

Could we go the other direction and upload the current Chef12 artifacts to omnitruck buckets?

@lamont-granquist Not sure it's needed as all platforms we support CS12 on support APT/YUM. IMO we should encourage people to use the platform native repositories offered up at:
https://packagecloud.io/chef/current
https://packagecloud.io/chef/stable
Thoughts @chef/lob?

I can go either way.  In either case (removing the endpoint of having it server chef 12) we need to announce this in advance.
Unlike chef-client, the yum and apt repos cover all of our supported platforms for chef-server, making omnitruck less necessary.

@stevendanna It has never served Chef Server 12 (or EC).

If it simplifies things and we foresee it staying redundant with distro package managers, then go for it, but do we expect to ever be using a platform where using the native package manager is hard? If not, then go for it imo.

We're already working on an AIX chef server, and there's no apt or yum or anything similar for that platform -- in the useful sense of being able to install over the network and follow a channel and depsolve and everything AIX has no package manager.

@lamont-granquist We are?

@lamont-granquist perhaps you're confused about the Linux on POWER port being done? We are porting the server to RHEL 7.1 and Ubuntu 14.04 to Linux on POWER, much simpler than AIX.

Omnitruck's going to be around for a while. The client's going to end up on more platforms, not less, and packagecloud seems unlikely to be fast on supporting all-the-things.
I'd check the splunk logs to see if anyone is hitting this endpoint. The big question is, how do you get older versions of Chef Server if you need them if we turn this off? It looks like package cloud only has 11.1.5 and 11.1.6 from 11.x. Are we effectively taking the old versions off the internet if we remove this endpoint?

@mattray ah you're right, i got those confused.

@btm It would probably be best to publish the old versions to PackageCloud (they exist in Artifactory) and update the download page to serve up the PackageCloud URLs. This will make old OSS Chef Server releases consistent with the other server products.

A simple splunk search:
index=opscode-platform metadata-server OR full_server_list OR chef_server_platform_names OR download-server

shows that the endpoints removed in this PR have been accessed 4000+ times in the last 24 hours.  So, we should definitely make an announcement regardless of the direction we plan to go.  Many people are making requests with version=latest, so updating omnitruck to serve the 12.0 packages would still likely be an impactful event for some as would removing these endpoints altogether.
I'm +1 on making it easier to get Chef 12, just putting this here as a reminder for us to announce this change.

Thx @stevendanna. It should also be noted full_server_list is fetched every time someone the Open Source Chef Server 11 download page (it's used to populate the various drop downs):
https://www.chef.io/download-open-source-chef-server-11/
If we update this download page to work like the other server products (full static generation with PackageCloud URLs) the Splunk logs might tell a different story.

Has there been any movement on this in the last week?

@sdelano No movement. I am planning on cutting the Open Source Chef Server 11 download page over to using packagecloud links sometime this week. That change/decision should be done regardless of whether we completely remove OLD chef-server from Omnitruck...which I still stand by as the right decision.

We definitely either need to remove the old chef-server from omnitruck, or upload chef-server 12 to omnitruck.  The current state is bad.

Kinda feel someone should just make an executive decision here...

@stevendanna I got looking at the chef-web-downloads project yesterday. We're moving the client over to use it because after switching to manhattan for builds we're not getting some of the omnitruck related bits that old-ci used to push that would trigger the API to provide yolo for untested versions. It's not clear how we plan to have all of this work together, so I had 1:1 meetings with a handful of people yesterday to try to determine the state and history of how we got here. There are a bunch of people whose mile-high view of how the release infrastructure looks or is going to look is different, and we might need to pounce on that quickly, because we're looking at problems with parts of that infrastructure differently.

@btm @stevendanna Well the Chef Server download page now contains ALL Chef Server releases including older OSC11:
http://downloads.chef.io/chef-server/
This was done almost a month ago in:
https://github.com/chef/chef-web-downloads/pull/185
https://github.com/chef/chef-web-www/commit/6f91cb89de900a392880e19df855aff399205780
I honestly think we should just rebase this onto master and get it deployed.

It should also be noted full_server_list is fetched every time someone access the Open Source Chef Server 11 download page (it's used to populate the various drop downs):
https://www.chef.io/download-open-source-chef-server-11/
If we update this download page to work like the other server products (full static generation with PackageCloud URLs) the Splunk logs might tell a different story.

Okay, we did that, so did we get the results we expected?
Also, is it possible to return a 410 Gone or better yet 301 Moved Permanently here and have the messaging reach the end user?
I'm in favor of the general idea, we just need to be sure that we make the end-user experience as nice as possible.

Okay, we did that, so did we get the results we expected?

Over 5/26/15 8:00:00.000 AM to 5/27/15 8:22:00.000 AM we see 4533 hits on the following query:
index=opscode-platform sourcetype="nginx-access" metadata-server OR full_server_list OR chef_server_platform_names OR download-server

Some facts about those requests.
Request Breakdown
User-agents

Agent
Count
Percentage

Ruby
4,217
93.029%

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.81 Safari/537.36
29
0.64%

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.65 Safari/537.36
20
0.441%

Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
17
0.375%

There is a long tail of brower-ish user-agents that all clock in at under 1%.
By Endpoint

Endpoint
Count
Percentage

download-server
4,122
90.93%

full_server_list
307
6.7%

metadata-server
104
2.29%

chef_server_platform_names
0
0.0%

Looking at just the download-server requests

http_path
count
percentage

/chef/download-server?p=el&pv=6&m=x86_64&v=latest&prerelease=false&nightlies=false
1,625
39.423%

/chef/download-server?p=ubuntu&pv=12.04&m=x86_64&v=latest&prerelease=false&nightlies=false
1,410
34.207%

/chef/download-server?p=el&pv=6&m=x86_64&v=11.1.6&prerelease=false&nightlies=false
496
12.033%

/chef/download-server?p=ubuntu&pv=12.04&m=x86_64&v=11.1.6&prerelease=false&nightlies=false
190
4.609%

/chef/download-server?p=ubuntu&pv=12.04&m=x86_64&v=11.1.6-1&prerelease=false&nightlies=false
147
3.566%

/chef/download-server?p=ubuntu&pv=12.04&m=x86_64&v=11.1.1&prerelease=false&nightlies=false
90
2.183%

/chef/download-server?p=el&pv=6&m=x86_64&v=11.0.11&prerelease=false&nightlies=false
67
1.625%

/chef/download-server?p=ubuntu&pv=12.04&m=x86_64&v=11.1.3&prerelease=false&nightlies=false
47
1.14%

/chef/download-server?p=ubuntu&pv=12.10&m=x86_64&v=latest&prerelease=false&nightlies=false
45
1.092%

Over 70% of the requests are pulling latests, so changing what that returned would likely impact most of these users.
From the user-agent, my expectation is that these requests are mostly from chef-client runs of people using some version of the chef-server cookbook.

User agent "ruby" is weird because Chef sets the user agent to "Chef Client $version blah blah". Does the chef-server cookbook just use net/http or rest-client directly?

Over 70% of the requests are pulling latests, so changing what that returned would likely impact most of these users.

Contrarily, the documented behavior for "latest" is to get the latest. If these folks actually don't want CS 12, they've been lucking out. Though that doe make it seem like a hard break by returning 410 might be better than making their cookbook possibly do the wrong thing. :frowning:

Does the chef-server cookbook just use net/http or rest-client directly?

Yes: https://github.com/chef-cookbooks/chef-server/blob/chef11/libraries/omnitruck_client.rb#L56
https://github.com/chef-cookbooks/chef-server/blob/chef11/attributes/default.rb#L19
That leads me to believe that almost all uses of that endpoint come from the chef11 version of the chef-server cookbook.

The following is a plan proposed during a HipChat conversation.  The goal is to reduce the number of people we break while also reducing the confusion around omnitruck and chef server.

Make changes to the relevant version line of the chef-server cookbook to convert it to package cloud.
Release the new chef-server cookbook and encourage people to upgrade to it.
Announce a deprecation date for the omnitruck chef-server endpoints
Remove chef-server endpoints from omnitruck on those dates

This plan assumes the following:

Chef Server packages canonical release location is package cloud.
Having the omnitruck endpoints serve Chef 12 would be dangerous as the Chef 11 -> 12 upgrade is non-trivial.
Our end goal is for all chef-server users to install the chef-server packages from our packagecloud repositories or manually by downloading the package.

The default recipe of the chef-server cookbook allows setting the version as an attribute.
Does this work? If so, users can pin their version with the attribute and I don't think we need to make any changes to the cookbook. Unless it is desired to set a version 11 and switch to chef_server_ingredient in the 11-stable branch of that cookbook? I don't think we should make a new release to supermarket - the default (latest) version on the site is 12+.

@jtimberman The context is that we believe the majority of the users of the chef-server omnibus endpoint are users of the 2.x version of this cookbook which uses omnitruck.  If those users can all safely move to the newest version simply by setting the version string to an 11.x version, then that will work; however, it requires that we actually get users on that version.

@adamedx @jdmundrawala Have we gotten this work scheduled?

@charlesjohnson , @mmzyk, looks like people still hit this problem. @echohack has some examples, and given ongoing omnitruck work, now may be a good time to take this.

You can now install any of Chef Software Inc's products using Omnitruck's install script. Full details can be found at https://docs.chef.io/packages.html#omnitruck-install-scripts.


Add Aave v3 Optimism sUSD + Avalanche BTC.b

Aave sUSD has a cap supply cap iirc

Hey @MirthFutures! Yes sUSD has a supply cap which is not met (like DAI, USDC, USDT on Optimism).
If that's not okay for you, would be glad if you could merge this PR and I'll close https://github.com/beefyfinance/beefy-v2/pull/1092.


Add poinnet model

Dear @daavoo, you raised this issue adding PointNet, but included VoxNet so far. What direction is this going?

I'm having trouble to find time to work on pyntcloud. Implementing pointnet is probably one of the first things that I'll do as soon as I find some time.

Hi, @daavoo , I am ready to merge my pointnet repo to the module as well

Hola @hansen7 , glad to hear that! Feel free to open a P.R. whenever you want

Hi @hansen7 @daavoo,
Did this PR pass? I really would like to see this.


Link fix (Layman's Guide)

Domain ftp.rsasecurity.com no longer resolves (NXDOMAIN).
ftp protocol no longer supported natively in browsers.

The link provided links to a formatted PDF version of the paper, at a stable domain (researchgate), using https.

A fix for #51.


Might be helpful to require rich version a bit more loosely

In no way a real blocker - especially as objexplore used primarily/solely in a local dev environment.
Rich has become really popular, and I use it in a lot of my own code, especially in django management commands. I then choose to use objexplore sometimes in debugging.
It would be helpful for me if objexplore defined a range of (prior) rich releases that were compatible (if that were possible), so that the (pinned) rich requirement in my app was not so likely to conflict.
Not a big deal, just means I may sometimes have to reinstall objexplore to use it, as subsequent installs of my apps requirements.txt will have defined a different version of rich, and required me to uninstall objexplore if it conflicts.
If this makes testing/maintenance/development difficult please ignore/close, happy to install when I need/want it.

Yeah that's not a bad idea, I'll set the version >= 10.9.0 because that is where it has been since I started this project


Mobile. "Type your message and select optional the ve..." popup

On iPhone from time to time window with "Type your message and select optional the ve..." popups on the screen.
I believe it's connected to autobeckup somehow. What is that and how to get rid of this popup message?

 So the setting Specify custom commit message on auto backup: on is the reason. If you don't want that, turn the setting off. Now the commit message for auto backup is used.

Thank you!


Question: python newbie upload document example

Hi,
thank you for your work here. I'm new to python and want to implement a migration tool from wiki media files into paperless.
I have a little test python snippet that gives me an error I can't solve myself and the documentation does not show a full example.
Using pypaperless-3.1.15
The script looks like this
import asyncio
from pypaperless import Paperless

file_name = '/tmp/dokument.pdf'

pp = Paperless('https://tpp.example.com','***********************************************')

draft = pp.documents.draft(
       title="New document",
       filename=file_name,
    )
new_pk = asyncio.run(draft.save())
asyncio.run(pp.close())

and produces the following output:
Traceback (most recent call last):
  File "/root/pp2nc/pp_test.py", line 9, in <module>
    draft = pp.documents.draft(
            ^^^^^^^^^^^^
AttributeError: 'Paperless' object has no attribute 'documents'

What am I missing ?
Thanks for pointing me into the rigth direction...
Regards
Joachim

ok, after a lot of trying I found that I was missing
asyncio.run(pp.initialize())

see above


Friendly error when creating a space without identity

[x] when attempting to createSpace before obtaining a HALO identity, client times out

Should be:
Failed to create space.  This device has no HALO identity available. See https://docs.dxos.org/guide/halo

https://github.com/dxos/dxos/pull/2423


Short solution needed: "Example of Redis usage" (redis)

Please help us write most modern and shortest code solution for this issue:
Example of Redis usage (technology: redis)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Example of Redis usage


Add enumStyle option into the openApiToTypeScriptClient

Update NJsonSchema to 10.1.7
Add EnumStyle into the OpenApiToTypeScriptClientCommand.cs

Sorry for the mess in the *.csproj files. Some of them were with BOM and some without it. Now changed are all without BOM.

Ah shit, i just did that
https://github.com/RicoSuter/NSwag/commit/34d2924b06610f2d362db66513fe96c5e85420ec

I will move it on top of develop

Update command and UI: https://github.com/RicoSuter/NSwag/commit/110b107a3aa29bbab48957350ea2033bc5a9c099

I think its fine now.

Thanks! I like your work

I think, this is now resolved https://github.com/RicoSuter/NSwag/issues/1188

Yes, closed. So I will now release a new version of NSwag... ok?

It will be lovely. I tested code generation, and it works like a charm.

Ok, release is done. NuGet pkgs should be available now/soon.


Cannot add feed that includes ccTLD (Country Based URL)

Cannot add a URL that includes a Country Coded TLD.
Example:
ucunion.com works fine, but ucunion.com.au doesn't.
Looks like this is due to inbuilt pattern in the regex. See Screenshot Below

Page:
https://winds.getstream.io/app/personalization-feed
Browser:
Chrome/macOS 10.12
Example URL:
http://ucunion.com.au/feed/
Edit:
This seems to be a DOM only issue, as when I remove it using dev tools it works fine.

Hey @sifex, good find. Do you mind making a PR with a quick fix to the regex pattern? I'll happily accept. Thanks!

Good PR! Thank you! 👍


SCN with sparse matrices

Is there an adaptation to SCN to be made in order to have as an input a sparse matrix as opposed to a dense matrix?

Hi,
PySingleCellNet has supports use of h5ad:
https://github.com/pcahan1/PySingleCellNet


Split kube-vip template to its own template.

Split kube-vip template to its own template.

Do you mind if we just merge this in to your branch for now and continue to follow up in your PR about the readme and the EIP handling? I'll likely end up submitting separate PRs to your branch to address each of those.

@cprivite sure that's fine by me

Is this ready for merging?

Yes, please.


Make stacktrace query result to be serialized using standard data converter only

What was changed
Currently, before the change, stacktrace query uses user-defined data converter to serialize its result.
After the change a standard default converter will always be used for serialization of stacktrace query result.
Why?
Stacktrace query is a built-in query for UI and tctl usage, hence a default converter should be used.

I think we need to revert this. UI has remote decoder to decrypt payloads

@bergundy remote decoder is not our standard setup. Client encoding is. See discussion in the company’s slack. This code predates any remote encoders and is obviously wrong.


Refactor layouts (add )

This PR adds a new <LayoutProvider> component that allows a page to switch from the default layout used on the majority of pages, to other layouts though the use of a useLayout() hook.
The existing layouts are all intact: <Layout> has been renamed to <DefaultLayout> and there is also an option to use an error layout and no layout at all.

Abandoning my PR because of some quirks around setState() and re-rendering. :cry:


Support Utf8View for string function

Is your feature request related to a problem or challenge?
Bit length for example doesn't support utf8view yet.
query error
select bit_length(arrow_cast('a', 'Utf8View'));
----
DataFusion error: Optimizer rule 'optimize_projections' failed
caused by
optimize_projections
caused by
Internal error: Failed due to a difference in schemas, original schema: DFSchema { inner: Schema { fields: [Field { name: "bit_length(arrow_cast(Utf8(\"a\"),Utf8(\"Utf8View\")))", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, field_qualifiers: [None], functional_dependencies: FunctionalDependencies { deps: [] } }, new schema: DFSchema { inner: Schema { fields: [Field { name: "bit_length(arrow_cast(Utf8(\"a\"),Utf8(\"Utf8View\")))", data_type: Int64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }, field_qualifiers: [None], functional_dependencies: FunctionalDependencies { deps: [] } }.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker

statement ok
create table test_source as values
  ('Andrew', 'X'),
  ('Xiangpeng', 'Xiangpeng'),
  ('Raphael', 'R'),
  (NULL, 'R');

# Table with the different combination of column types
statement ok
create table test as
SELECT
  arrow_cast(column1, 'Utf8View') as column1_utf8view
FROM test_source;

query error DataFusion error: Arrow error: Compute error: bit_length not supported for Utf8View
select bit_length(column1_utf8view) from test;

Describe the solution you'd like
I hope we can support utf8view for all the string functions
Describe alternatives you've considered
No response
Additional context
No response

take

Related epic is here:  https://github.com/apache/datafusion/issues/11790
Do you mind if I make this ticket about supporitng bit_length rather than all string functions?

The fix upstream in arrow didn't actually fix this bug (yet) so reopening


Website has no HTTPS (redirect)

What's wrong
http://www.discocrypto.com/
How it should be
https://www.discocrypto.com/by default
Why?
Obviously, it is not really good for a crypto lib not to force the use of HTTPS on their website. See also: HSTS. :wink:

Hey @rugk
There is HTTPS but indeed I forgot to enable HSTS. This is fixed now. Thanks for reporting this.


Update path notation documentation

Prerequisites

[x] I have read the Contributing guidelines
[x] I have read the Code of conduct and I agree with it

Description
Update  most of the empty sections in path notation documentation

Codecov Report

Merging #280 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #280   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          95     95
  Lines         276    276
=====================================
  Hits          276    276

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad436c5...3931539. Read the comment docs.


IMPLEMENTATION_STATUS should/could be generated automatically out of Sharness Tests

https://github.com/ipfs/ipfs/blob/master/IMPLEMENTATION_STATUS.md

Yeah there is already a kind of coverage script in the go-ipfs repo:
https://github.com/ipfs/go-ipfs/blob/master/test/sharness_test_coverage_helper.sh
It shows the existing tests for each ipfs command and each option of each IPFS command. For example, here is some output:
ipfs pin verify
      2 t0085

ipfs pin verify --quiet
ipfs pin verify -q

ipfs pin verify --verbose
      1 t0085

ipfs ping

ipfs ping --count
ipfs ping -n

It shows that ipfs pin verify is used twice in test t0085 while there are no tests for ipfs ping with or without the --count (-n is a shortcut for --count) option.
We could reuse that script to associate IPFS commands (and probably each option of each comamnd) with the tests they appear in and we could tell if they are working on not depending on whether the associated tests pass or not.


Need JDK

I need a JDK tag to use gradle in my dockerfile
But there's just JRE branches

for now i'm installing jdk manually using apk
But it would be better to have a branch for jdk

I used to use JDK in the base image, but I must have changed my mind at some point. Is there one specific Java version that would help you out? I'll investigate what the image size difference would be.

JDK 8 (because my plugins use minecraft 1.8 which runs on java 8)

It turns that as part of #1054 I'll be switching to a JDK base image. I'll get a test image out shortly.

@tiagodinis33 here is a JDK 8 image to test out: itzg/minecraft-server:test-temurin-8

thx, i'll test it

Its working

Keep it as a separate branch

The itzg/minecraft-server:java8-multiarch now includes the JDK (and is based on Eclipse Temurin).


GLTFFileLoader.loadFile does not call onError callback for .GLB files with less than 20 characters

Summary
For corrupt GLB files with less than 20 bytes, GLTFFileLoader.loadFile throws an unhandled exception instead of calling the onError callback (see screenshots below).
Validated bug: https://forum.babylonjs.com/t/gltffileloader-loadfile-does-not-call-onerror-callback-for-files-with-less-than-20-characters/22669
Repro

Bug repro on sandbox.  Please note that this same issue exists in other, customer implementations with @babylonjs/loaders but the generic babylon sandbox is an easy repro.
Expected result:
Upload a .GLB file with >= 20 characters.  Example contents:

{
	"hello": "123abc"
}

In this scenario, the generic error message is displayed and the sandbox state is reset.

Current result:
Upload a .GLB file with < 20 characters.  Example contents:

{
	"hello": "123"
}

In this scenario, no error message is displayed and the sandbox stalls in the loading state.
Screenshots
When uploading a .GLB file with >= 20 characters.

When uploading a .GLB with < 20 characters and waiting a sufficient amount of time.

Browser console

Desktop:

OS: Windows, Mac
Browser: Chrome, Safari, Firefox
Version: Latest

Additional context
I was able to track the issue down to the construction of the Unt8Array in the following codeblock (starting on line 538). Fundamentally the issue is an out-of-bounds error resulting from the combination of byteOffset and byteLength being greater than the file length (in this case, 19).
loaders/src/glTF/glTFFileLoader.ts
  return this._loadFile(scene, fileOrUrl, (data) => {
      const arrayBuffer = data as ArrayBuffer;
      this._unpackBinaryAsync(new DataReader({
          readAsync: (byteOffset, byteLength) => Promise.resolve(new Uint8Array(arrayBuffer, byteOffset, byteLength)),
          byteLength: arrayBuffer.byteLength
      })).then((loaderData) => {
          onSuccess(loaderData);
      }, onError);
  }, true, onError);

While most of the time this won’t be an issue, given the architecture of the loaders, it is nearly impossible to catch the error from the surface of the library so when a <= 19 character file is submitted, a crash is currently unpreventable.

We want that people discuss issues and bugs first on the forum (as stated in the template)
That being said you did a pretty thorough investigations so I'll keep the issue open and validated :)

Thanks @deltakosh. Please see the link at the top of my post. That'll take you to the forum post where @bghgary validated the bug.

I'm dumb :D

@baswelsh Do you have a repro file (i.e. glb file that is less than 20 bytes) that you can attach here?