Bug: Pricepoint.getDemandAt() stuck in while loop

Hi,
With bjorn ive discovered another bug in Pricepoint.getDemandAt()...we are porting REX to PM2.0 whic has a fake agent that is sending out random bids.These are pricepoint bids which are never passed on by the concentrator to the auctioneer.We have started digging and found out that it hangs when converting the pointbid to an arraybid. Constructing an ArrayBid(Pointbid) calls Pricepoint.getDemandAt()
We noticed that cmp variable becomes 0 which causes it to get stuck in the while loop:
`
@Override
public double getDemandAt(Price price) {

    if (pricePoints.length == 1) {
        // Flat bid, send any demand (they are all the same)
        return getMaximumDemand();

    } else if (price.compareTo(getFirst().getPrice()) < 0) {
        // If the price is lower than the lowest price, return the maximum
        // demand
        return getMaximumDemand();
    } else if (price.equals(getFirst().getPrice())) {

        // If the first matcher, it could be that the second is at the same price. If that is the case, use the // second, otherwise the first.

        PricePoint secondPricePoint = pricePoints[1];

        if (price.equals(secondPricePoint.getPrice())) {
            return secondPricePoint.getDemand();

        } else {
            return getMaximumDemand();
        }

    } else if (price.compareTo(getLast().getPrice()) >= 0) {
        // If the price is higher than the highest price, return the minimum
        // demand

        return getMinimumDemand();
    } else {

        // We have a normal case that is somewhere in between the lower and higher demands
       // First determine which 2 pricepoints it is in between
        int lowIx = 0, highIx = pricePoints.length;
        while (highIx - lowIx > 1) {
            int result = highIx - lowIx;

            LOGGER.info("Alex: " + Integer.toString(result));
            LOGGER.info("Alex price: " + price);
            LOGGER.info("Alex pricepoints: " + pricePoints);
            LOGGER.info("Alex high: " + highIx);
            LOGGER.info("Alex low: " + lowIx);

            int middleIx = (lowIx + highIx) / 2;
            PricePoint middle = pricePoints[middleIx];

            LOGGER.info("Alex pricepointMidden: " + middle);

            int cmp = middle.getPrice().compareTo(price);

            LOGGER.info("Alex cmp: " + cmp);

            if (cmp < 0) {
                lowIx = middleIx;
            } else if (cmp > 0) {
                highIx = middleIx;
            } else {
                // Found at least 1 point that is equal in price.
                // This is the special case with an open and closed node. Always the lower demand should be chosen.
                PricePoint nextPoint = pricePoints[middleIx + 1];
                if (price.equals(nextPoint.getPrice())) {
                    return nextPoint.getDemand();
                } else {
                    middle.getDemand();
                }
            }
        }
        PricePoint lower = pricePoints[lowIx];
        PricePoint higher = pricePoints[highIx];

        // Now calculate the demand between the 2 points
        // First the factor (between 0 and 1) of where the price is on the line
        double factor = (price.getPriceValue() - lower.getPrice().getPriceValue())
                        / (higher.getPrice().getPriceValue() - lower.getPrice().getPriceValue());
        // Now calculate the demand
        return (1 - factor) * lower.getDemand() + factor * higher.getDemand();
    }
}

`
Here you can see the output of the logging in the console:
[Lnet.powermatcher.api.data.PricePoint;@7665b1
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex high: 26
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex low: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepointMidden: {15 -> 3.15E5}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex cmp: 0
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex price: Price{priceValue=15}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepoints: [Lnet.powermatcher.api.data.PricePoint;@7665b1
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex high: 26
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex low: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepointMidden: {15 -> 3.15E5}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex cmp: 0
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex price: Price{priceValue=15}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepoints: [Lnet.powermatcher.api.data.PricePoint;@7665b1
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex high: 26
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex low: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepointMidden: {15 -> 3.15E5}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex cmp: 0
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex price: Price{priceValue=15}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepoints: [Lnet.powermatcher.api.data.PricePoint;@7665b1
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex high: 26
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex low: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepointMidden: {15 -> 3.15E5}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex cmp: 0
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex price: Price{priceValue=15}
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepoints: [Lnet.powermatcher.api.data.PricePoint;@7665b1
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex high: 26
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex low: 13
12:10:12.206 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex pricepointMidden: {15 -> 3.15E5}
12:10:12.207 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex cmp: 0
12:10:12.207 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex: 13
12:10:12.207 [pool-2-thread-2] INFO  net.powermatcher.api.data.ArrayBid - Alex price: Price{priceValue=15}
12:10:12.207 [pool-2-thr
I believe it gets stuck here:
                    } else {                         middle.getDemand();                     }
But adding a return before caused another problem; the bidcurve suddenly became asccending and was rejected...

Ok, that is definitely a bug then. Do you by any change have the code that generates the bid and price that causes the problem? Would make debugging a lot easier. :)

You are right Alex, looking at this code in that else statement it should return that middle.getDemand(). The question then is why the resulting bidcurve is ascending. If you can share the input PointBid, we can check why that happens.

Wilco I've sent you the bidGenerator..maybe the problem lies there
i added a return statement before middle.getDemand()
in arrayBid a bid looks as follows:
[1578000.0, 1551000.0, 1551000.0, 1531000.0, 1531000.0, 1530999.9999999998, 1531000.0, 1531000.0,....
you can see there is probably a rounding error somewhere; have to check whether that's in the orignal bidgenerator or later on in the code...

ehm leave it to me for now; I think the problem is in the bidgenerator...it produces really strange pointbids.
however it might accidentily have uncovered a problem in the concentator where a single bid is not directly one on one the same as the aggregatedBid, even though it is a weird Bid.... if there is only one bid in the bidcache the aggregatedBid should turn out the same as the original bid.

it seems the bidgenerator is flawed because it generates multiple pricepoint on the same x coordinate:
[0 -> 10, 0 -> 8, 0-> 6…..
we will fix this...two pricepoints should be okay but more than that is not needed.
But apparantly the powermatcher starts doing weird things when there is multiple points on the same x coordinate. For robustness sake we should perform a check when constructing a bidladder that there is no more than 2 coordinates on the same x coordinate. For instance all points that are in between should be ignored before sending the pointbid onwards...

arghh i'm lost a bit..there does seem to be a bug in the powermatcher.
if I follow the pointBid generated by the bidgenerator (even though there are multiple coordinates on the x axes which is fine for now)
it arrives at the concentrator
at some point the concentrator basematcher is scheduled to aggregate the bid from the bidcache
it retrieves the pointbid correctly
then goes into builder.addAgentbid using the correct pointbid
it goes into addBid with the correct pointbid
and then it goes wrong here:
    public Builder addBid(Bid bid) {
        if (bid.getMarketBasis().equals(marketBasis)) {
            double[] demand = bid.toArrayBid().getDemand();
            for (int ix = 0; ix < marketBasis.getPriceSteps(); ix++) {
                aggregatedBid[ix] += demand[ix];
            }
        }
        return this;
    }

at double[] demand = bid.toArrayBid().getDemand();
getDemand goes into the arrayBid and returns a demandArray which is NOT EQUAL to pointBid in that arrayBid. The PointBid is however the correct pointBid that I was following all the time...
I don't understand where this demandArray variable is created and why it is not equal pointBid variable...does this happen when the concentrator receives at the BaseAgentEndpoint?

okay I think I figured out where it goes wrong!
it is again here in getDemandAt():
`
ArrayBid(PointBid base) {
super(base.getMarketBasis());
    int priceSteps = marketBasis.getPriceSteps();
    demandArray = new double[priceSteps];

    for (int ix = 0; ix < priceSteps; ix++) {
        demandArray[ix] = base.getDemandAt(new PriceStep(marketBasis, ix));
    }

    pointBid = base;
}

`
base.getDemandAt is creating the wrong demandArray from the pointBid. Probably for two reasons:

the pointBid has multiple coordinates on one x coordinate
the pointBid has 189 values >> 100 pricesteps

So either we need to check on these aspects and not allow them or finetune getDemandAt() so that it can handle these cases.

Thanks for looking into this problem @alex254. We've run into more troubles with the two different ways to represent a bid (ArrayBid and PointBId). Although both representations seem equivalent, they differ quite a lot in the details. Issue #199 is also causing problems at our side. My plan was to take some time next week to look into the details of these bugs.
We could try to fix all these individual bugs, but there maybe is an alternative. The PointBid representation is mainly used because it is easier to construct, which is nice if you have to develop a device agent. However, when bids are aggregated we always use the ArrayBid representation (because aggregating ArrayBids is more efficient). I think it would simplify things if we would switch to only using the ArrayBid representation, and offer the PointBid only as a convenience method to easily construct an ArrayBid. In other words: PointBids always get instantly converted into ArrayBids.
The downside of this change is that it would mean a significant API change.
Any thoughts?


Issue with using the extension

For some reason I get this weird message when I try to activate the extension:   NET::ERR_CERT_DATE_INVALID
Is the extension dead?

Looks like the security certificate expired at the end of May, after only being active 3 months. Can that be remedied?

Thanks for letting me know. I updated the certificate!


[Azure] Error when using Standard_NV12 Compute

System Information (please complete the following information):

Model Builder Version: 16.0.2002.1901
Visual Studio Version: VS2017

Describe the bug
When training an image classification model in Azure using Standard_NV12 compute, an error is thrown once training is complete.
Expected behavior
Model trains successfully
Additional Context
Training data consists of weather dataset with 1125 images.
Compute Properties

Dialog Box + Stack Trace

Object reference not set to an instance of an object.
at AzureML.AutoMLRunnerImages.<RunAutoMLAsync>d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.ML.ModelBuilder.AutoMLService.Experiments.AzureImageClassificationExperiment.<ExecuteAsync>d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.ML.ModelBuilder.AutoMLEngine.<StartTrainingAsync>d__28.MoveNext()

Azure driver_log.txt
Starting the daemon thread to refresh tokens in background for process with pid = 125
Entering Run History Context Manager.
Preparing to call script [ hd_image_classification_dnn_driver.py ] with arguments: ['--data-folder', '/mnt/batch/tasks/shared/LS_root/jobs/mbwkspc/azureml/automl_7fdc920f-d244-403b-93f1-d720129104e0_hd_0/mounts/workspaceblobstore', '--labels-file-root', '/mnt/batch/tasks/shared/LS_root/jobs/mbwkspc/azureml/automl_7fdc920f-d244-403b-93f1-d720129104e0_hd_0/mounts/workspaceblobstore', '--model_name', 'seresnext', '--strategy', 'diff_lr']
After variable expansion, calling script [ hd_image_classification_dnn_driver.py ] with arguments: ['--data-folder', '/mnt/batch/tasks/shared/LS_root/jobs/mbwkspc/azureml/automl_7fdc920f-d244-403b-93f1-d720129104e0_hd_0/mounts/workspaceblobstore', '--labels-file-root', '/mnt/batch/tasks/shared/LS_root/jobs/mbwkspc/azureml/automl_7fdc920f-d244-403b-93f1-d720129104e0_hd_0/mounts/workspaceblobstore', '--model_name', 'seresnext', '--strategy', 'diff_lr']

Downloading: "https://automlcesdkdataresources.blob.core.windows.net/models-vision-pretrained/se_resnext50_32x4d-a260b3a4.pth" to /root/.cache/torch/checkpoints/se_resnext50_32x4d-a260b3a4.pth

  0%|          | 0.00/105M [00:00<?, ?B/s]
  0%|          | 144k/105M [00:00<01:18, 1.41MB/s]
  1%|          | 944k/105M [00:00<00:58, 1.87MB/s]
  4%|▍         | 4.01M/105M [00:00<00:41, 2.58MB/s]
  8%|▊         | 8.01M/105M [00:00<00:28, 3.58MB/s]
 12%|█▏        | 12.8M/105M [00:00<00:19, 4.88MB/s]
 14%|█▍        | 14.9M/105M [00:01<00:24, 3.95MB/s]
 16%|█▌        | 16.4M/105M [00:01<00:22, 4.22MB/s]
 17%|█▋        | 17.6M/105M [00:02<00:23, 3.89MB/s]
 18%|█▊        | 18.6M/105M [00:02<00:22, 4.03MB/s]
 18%|█▊        | 19.4M/105M [00:02<00:22, 4.08MB/s]
 19%|█▉        | 20.1M/105M [00:02<00:22, 4.01MB/s]
 20%|█▉        | 20.6M/105M [00:02<00:20, 4.36MB/s]
 20%|██        | 21.2M/105M [00:02<00:18, 4.68MB/s]
 21%|██        | 21.9M/105M [00:03<00:17, 4.89MB/s]
 21%|██▏       | 22.5M/105M [00:03<00:17, 5.07MB/s]
 22%|██▏       | 23.1M/105M [00:03<00:16, 5.37MB/s]
 23%|██▎       | 23.9M/105M [00:03<00:14, 5.74MB/s]
 23%|██▎       | 24.5M/105M [00:03<00:14, 5.73MB/s]
 24%|██▍       | 25.1M/105M [00:03<00:14, 5.76MB/s]
 24%|██▍       | 25.8M/105M [00:03<00:13, 6.00MB/s]
 25%|██▌       | 26.6M/105M [00:03<00:13, 6.34MB/s]
 26%|██▌       | 27.4M/105M [00:03<00:11, 6.82MB/s]
 27%|██▋       | 28.1M/105M [00:04<00:11, 6.76MB/s]
 28%|██▊       | 29.0M/105M [00:04<00:11, 7.25MB/s]
 29%|██▊       | 30.1M/105M [00:04<00:09, 8.00MB/s]
 30%|██▉       | 31.4M/105M [00:04<00:08, 8.81MB/s]
 31%|███       | 32.2M/105M [00:04<00:08, 8.83MB/s]
 32%|███▏      | 33.4M/105M [00:04<00:08, 9.36MB/s]
 33%|███▎      | 34.9M/105M [00:04<00:07, 10.5MB/s]
 34%|███▍      | 36.0M/105M [00:04<00:06, 10.7MB/s]
 36%|███▌      | 37.7M/105M [00:04<00:05, 11.9MB/s]
 37%|███▋      | 39.4M/105M [00:05<00:05, 12.7MB/s]
 39%|███▊      | 40.7M/105M [00:05<00:05, 12.7MB/s]
 40%|███▉      | 42.0M/105M [00:05<00:05, 12.8MB/s]
 42%|████▏     | 43.8M/105M [00:05<00:04, 13.9MB/s]
 43%|████▎     | 45.2M/105M [00:05<00:04, 13.9MB/s]
 45%|████▍     | 47.0M/105M [00:05<00:04, 14.7MB/s]
 46%|████▌     | 48.5M/105M [00:05<00:04, 14.8MB/s]
 48%|████▊     | 50.4M/105M [00:05<00:03, 15.5MB/s]
 49%|████▉     | 52.0M/105M [00:05<00:03, 14.3MB/s]
 51%|█████     | 53.9M/105M [00:06<00:03, 15.2MB/s]
 53%|█████▎    | 56.0M/105M [00:06<00:03, 16.3MB/s]
 55%|█████▍    | 57.6M/105M [00:06<00:03, 16.2MB/s]
 56%|█████▌    | 59.3M/105M [00:06<00:02, 16.3MB/s]
 58%|█████▊    | 60.9M/105M [00:06<00:02, 15.9MB/s]
 60%|█████▉    | 62.8M/105M [00:06<00:02, 16.5MB/s]
 61%|██████    | 64.4M/105M [00:06<00:02, 16.1MB/s]
 63%|██████▎   | 66.4M/105M [00:06<00:02, 16.9MB/s]
 65%|██████▍   | 68.0M/105M [00:06<00:02, 16.8MB/s]
 67%|██████▋   | 70.6M/105M [00:07<00:01, 18.4MB/s]
 69%|██████▊   | 72.4M/105M [00:07<00:01, 18.4MB/s]
 71%|███████   | 74.4M/105M [00:07<00:01, 18.8MB/s]
 72%|███████▏  | 76.3M/105M [00:07<00:01, 18.6MB/s]
 75%|███████▍  | 78.7M/105M [00:07<00:01, 19.7MB/s]
 77%|███████▋  | 80.7M/105M [00:07<00:01, 20.0MB/s]
 79%|███████▉  | 83.4M/105M [00:07<00:01, 21.3MB/s]
 81%|████████  | 85.5M/105M [00:07<00:00, 21.3MB/s]
 83%|████████▎ | 88.0M/105M [00:07<00:00, 20.7MB/s]
 86%|████████▌ | 90.6M/105M [00:08<00:00, 21.6MB/s]
 88%|████████▊ | 92.7M/105M [00:08<00:00, 19.9MB/s]
 90%|█████████ | 95.3M/105M [00:08<00:00, 21.0MB/s]
 92%|█████████▏| 97.3M/105M [00:08<00:00, 19.2MB/s]
 95%|█████████▍| 100M/105M [00:08<00:00, 19.8MB/s]
 97%|█████████▋| 103M/105M [00:08<00:00, 21.2MB/s]
 99%|█████████▉| 105M/105M [00:08<00:00, 20.3MB/s]
100%|██████████| 105M/105M [00:08<00:00, 12.5MB/s]
[model: seresnext, #param: 25519092]
[start training: train batch_size: 80, val batch_size: 80]
Type: Unclassified
Class: RuntimeError
Message: [Hidden as it may contain PII]
Traceback:
[Hidden as it may contain PII]

The experiment failed. Finalizing run...
Cleaning up all outstanding Run operations, waiting 300.0 seconds
2 items cleaning up...
Cleanup took 0.0015499591827392578 seconds
Starting the daemon thread to refresh tokens in background for process with pid = 125
Traceback (most recent call last):
  File "hd_image_classification_dnn_driver.py", line 6, in <module>
    runner.run(automl_settings, multilabel=False)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/azureml/contrib/automl/dnn/vision/classification/runner.py", line 142, in run
    settings=automl_settings)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/azureml/contrib/automl/dnn/vision/classification/trainer/trainer.py", line 138, in train_model
    device=device, multilabel=multilabel)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/azureml/contrib/automl/dnn/vision/classification/trainer/trainer.py", line 240, in _train
    outputs = model_wrapper.model(inputs)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/pretrainedmodels/models/senet.py", line 364, in forward
    x = self.features(x)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/pretrainedmodels/models/senet.py", line 350, in features
    x = self.layer2(x)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward
    input = module(input)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/pretrainedmodels/models/senet.py", line 123, in forward
    out = self.bn3(out)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in __call__
    result = self.forward(*input, **kwargs)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 81, in forward
    exponential_average_factor, self.eps)
  File "/azureml-envs/azureml_a978bb0284b9ce9dc72c2755c7eeaa98/lib/python3.7/site-packages/torch/nn/functional.py", line 1670, in batch_norm
    training, momentum, eps, torch.backends.cudnn.enabled
RuntimeError: CUDA out of memory. Tried to allocate 124.00 MiB (GPU 0; 7.94 GiB total capacity; 7.38 GiB already allocated; 61.00 MiB free; 54.97 MiB cached)

2020/02/19 19:55:52 mpirun version string: {
mpirun (Open MPI) 3.1.2

Report bugs to http://www.open-mpi.org/community/help/
}

@beccamc,
RuntimeError: CUDA out of memory. Tried to allocate 124.00 MiB (GPU 0; 7.94 GiB total capacity; 7.38 GiB already allocated; 61.00 MiB free; 54.97 MiB cached)
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/sizes-gpu#deployment-considerations

@justinormont Can you help with this? The machine has 24 GPU GB based on the specs, but it seems to be running out of memory and failing.

Lets filter out all NV.

Optimizing for multi-GPUs is work in progress.
Two options:

Disallow VMs with <= 14GB of RAM per GPU
Reduce the batch size

I agree w/ @JakeRadMSFT's suggestion. I'd go with the option to filter these VM SKUs for the moment. Generally I'd rather avoid having Model Builder consider hyperparameters like batch size and the closely tied-together learning rate; hyperparameters should be the domain of the AutoML code.


Fix validation message in basket

Fixes stockrecord availability validation message in the case when stock is lower than allocation. Previously the message that appeared contained negative numbers. For example: "a maximum of -1 can be bought". Now it will display: "no stock available"
Fixes #1765

Lovely, ta.


python: required AND nullable

I'm modeling an existing api in an OpenAPI 3.0 spec.
A key thing across most of this particular API (like it or not) is that all fields must be present (they are all required), but most of the time the fields are nullable. Some APIs include the fields and never use them (ie they are always set to null), but try to send a request without the field and it fails. (A replacement API is in the works, but until then, legacy prevails...)
So, I tried to model this by listing the requestBody parameters, and adding all of them to required. Then, I marked the fields that can be null as nullable: true.
However, in the python model template, required results in:
https://github.com/Mermade/openapi-codegen/blob/393d932619fc64776d82e0dd398ae3b9add26040/templates/python/model.mustache#L100-L103
Could this be updated to support nullable? So that if required and not nullable it raises the ValueError?

BTW this issue must be the answer to life the universe and everything.

The question is why did I submit this here instead of in the openapitools repo? (I must have had 42 tabs open or something).
Refiled: https://github.com/OpenAPITools/openapi-generator/issues/579


Add easyset dataloaders

Initially I was going to make easyset a separate package but since the datasets are all pickled up, it doesn't make much sense to provide a whole new dependency. So you might as well just add a module here and have the following function:

def load_set(path_url):
	"""
	Loads one of the pickled datasets.
	Only pass in one of the viable options.
	Arguments:
	-----------
	- path_url: url to the hosted .pkl file.
		- Can also specify one of the following
		prefixes:
			- 'cifar' which loads mini_cifar.pkl
			- 'cell' which loads mini_cell_segmentation.pkl
	"""

	if path_url == "cifar":
		path_url = "https://stanford.edu/~rsikand/assets/datasets/mini_cifar.pkl"
	if path_url == "cell":
		path_url = "https://stanford.edu/~rsikand/assets/datasets/mini_cell_segmentation.pkl"

	loaded_object = cp.load(urlopen(path_url))

	return loaded_object

Note: still have the separate repo to hold the hosted files.


Allow creating archives without using a template file

Currently, a template volume (vol) or CLM file is required when creating a new archive.

Two possible solutions are:

Make The Packing functions in ClmFile and VolFile static so they do not rely on opening an existing ClmFile before packing a new one. Need to check/experiment if C++ supports static virtual functions for this solution.

Delete the virtual class ArchiveFile. This would require separating ClmFile and VolFile into 2 separate classes, ClmPacker, ClmUnPacker. Could use ClmReader/Writer instead as well.

I think there are pros and cons to both solutions.

I suggest dropping the virtual and just making the functions static. Packing archives doesn't really need a long lived archive object with access to memory variables. Besides, when you create the archives, you know what type of archive you are creating.
The existing code is probably closer in spirit to the first option.
I think the pro of the first option is it'd be the simplest change to make, and it would address the exact issue with have concerning object instantiation and requiring template files.
The pro of the second option would be if you needed to support long lived archive write operations, like a database that supports interleaved reading and writing operations. The downside is much more complicated code and likely much higher memory usage. I don't think we need the benefits of the second option, nor should we pay the costs of it.

I plan on resolving this issue next. OP2Archive currently creates archives by calling CreateArchive off of ArchiveFile.
To make CreateArchive static, we will need to remove the virtual base function. I don't think this will be too big of a deal, but it will have to be dealt with in OP2Archive by selecting the right type of Archive to be created based on the type.


menu name

Problem
Is there any way to access the menu name inside a menu position template?
Many thanks for your help
Technical Details

Pagekit version: 1.0.5

Yes, every node has a menu property.
<?php foreach ($root->getChildren() as $node) : ?>
    <?php echo $node->menu ?>
<?php endforeach; ?>


docs: added config sections and api codes

:tada: This PR is included in version 1.0.18 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Switch Rpi boards to Honister

Hi @jakogut , I've included the -Os flag for the Pi3-64 and Pi4-64 so that Honister can fit in their current rootfs space, as per your investigation in https://github.com/balena-os/meta-balena/pull/2411.  What I'm not sure is whether CONFIG_SLOB is needed and for what exactly. I saw you included it in your PR so maybe you recall what was the reason for using it and whether we should include it for these Pis too. Thanks

Hi @jakogut , I've included the -Os flag for the Pi3-64 and Pi4-64 so that Honister can fit in their current rootfs space, as per your investigation in balena-os/meta-balena#2411. What I'm not sure is whether CONFIG_SLOB is needed and for what exactly. I saw you included it in your PR so maybe you recall what was the reason for using it and whether we should include it for these Pis too. Thanks

@acostach, From the Kconfig description for CONFIG_SLOB:

SLOB replaces the stock allocator with a drastically simpler
allocator. SLOB is generally more space efficient but
does not perform as well on large systems.

So this is unnecessary and probably even detrimental on larger systems, but can slightly reduce the kernel size at the cost of performance.

So this is unnecessary and probably even detrimental on larger systems, but can slightly reduce the kernel size at the cost of performance.

Thank you Joseph, I was suspecting it helped reduce the kernel size but wanted to confirm this with you. In this case we can skip including it for now.

@resin-jenkins test this please

@balena-ci rebase

@resin-jenkins test this please

@resin-jenkins test this please

@balena-ci rebase

@resin-jenkins test this please

@resin-jenkins test this please


Make docker builds faster

Currently our Makefile which builds the images (hub, cull, user) whenever there's a push takes on the order of 20 minutes. Brainstorm ideas to make this faster.
https://github.com/travis-ci/travis-ci/issues/5358

This may work, https://github.com/travis-ci/travis-ci/issues/5358#issuecomment-186593265. Basically you push every layer to Docker hub (with a tag based on a git hash or some such) and new builds just need to fetch the last, unchanged layer. But then the FROM will wander as you chase the last unchanged layer, which isn't so good.


Add Network Utility Methods

These are utility functions to interact with Networks.

Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a JIRA issue filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue.

[x] Run mvn clean verify to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
[x] If this contribution is large, please file an Apache Individual Contributor License Agreement.

retest this please

Tests flaking due to Dataflow quota issues.


Move Timezone to Event Settings Preference

Chore
Description
The timezone spinner present in the Create Event can be moved to the Event Settings Preferences.
Would you like to work on the issue?
Yes

I think this can be closed as it is not a priority right now


Respect --control-plane-instance-type for AWS cluster templating instead of using default

What does this PR do?
Fix passing on this flag so it ends up in the manifest. Previously, the flag was ignored in favor of the default m5.xlarge.
What is the effect of this change to users?
The flag --control-plane-instance-type becomes effective for AWS cluster templating.
Should this change be mentioned in the release notes?

[x] CHANGELOG.md has been updated

Is this a breaking change?
I'll let you decide to add the label or not. I believe it can be surprising if someone used the flag and thought it was effective.

@AndiDog before merging do you mind double checking that the flag value is also being used correctly for CAPG/GCP clusters?

Working for GCP cluster templating.
For CAPG, do I need some experimental branch for testing? →
Error: Invalid flag: --provider must be one of: aws, azure, capa, gcp, openstack, vsphere

CAPG is GCP. I just wasn't sure which name we used here.

CAPG is GCP. I just wasn't sure which name we used here.

@AverageMarcus That's a little confusing, but you are right that --provider gcp means CAPG:
// PureCAPIProviders is the list of all providers which are purely based on or fully migrated to CAPI
func PureCAPIProviders() []string {
	return []string{
		ProviderCAPA,
		ProviderGCP,
		ProviderVSphere,
		ProviderOpenStack,
	}
}

Is that historical? For example, we never had a vintage Google integration, but only CAPG?
Anyway, what you asked is working (= CAPG), so no more obvious CLI flag parsing issues from my point of view.

Is that historical? For example, we never had a vintage Google integration, but only CAPG?

Correct. Not ideal but we needed a way to distinguish new vs old AWS clusters.

Rebase to solve conflict. Let me know if you rather use merges in such case to ease review 😉


How are read operations counted

How is one select counted against CRUD request limit if statement returns 1.000 records?
Is same answer for joined read operations (FetchXML with 2, 3 joined tables)?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 02f2c989-54cf-d81a-aafb-2f4cd6fef108
Version Independent ID: 69811228-6a61-a078-86b3-ed11a6d9c84d
Content: Requests limits and allocations - Power Platform
Content Source: power-platform/admin/api-request-limits-allocations.md
Service: power-platform
GitHub Login: @dileepsinghmicrosoft
Microsoft Alias: dileeps

@matejskubic
Sorry for the much delayed response. Clearing a backlog, we noticed this hadn't be resolved. Here's what the product team reported:
Customer generated actions/ API calls are what gets counted and these limits on APIs don’t depend on the number of records returned. We recommend that you refer the capacity entitlement report which will be made available in Power Platform Admin center:  https://docs.microsoft.com/en-us/power-platform/admin/capacity-storage.  There will be a new API consumption report available soon on this page and until then you can refer to https://docs.microsoft.com/en-us/power-platform/admin/analytics-common-data-service as a close approximation.


Fix typo in README.md

Mostly made this pull request to see the automated testing you spoke about in NYC Open JS. Doesn't appear to be working though.

haha thanks @dmvaldman ! I took the rig apart and brought in part of it to have with me on stage today, so yeah not quite actively working today :wink:


Start running the test suite on Resin CI

At least to run it every time we send a PR on this same repo, as a way to replace Circle CI

Closing stale issues


Update readme

https://github.com/ray-project/rayfed/issues/51
https://github.com/ray-project/rayfed/issues/59

@NKcqx Comments were addressed.


A simple example using vuejs?

interested

I'm coming from the laravel ecosystem, so this is what I cobbled together. So far so good!
https://github.com/open-function-computers-llc/go-inertia-starter

I'm coming from the laravel ecosystem, so this is what I cobbled together. So far so good! https://github.com/open-function-computers-llc/go-inertia-starter

Pretty interesting, I'll take a look. Does it use Goravel?

ha, i had no idea that goravel existed. I like to keep my go apps lean and mean, with as few dependencies as possible, so no. It's using this inertia wrapper package, logrus for logging, and then mostly just the std lib. I'll be looking into goravel though. Could be nice to have a rapid app cycle in go and i'm not really a fan of Buffalo.

Unfortunately I don't have time to make more examples, but the Example Apps are based on Vue.
@lapubell Thanks for sharing!

@petaki for sure! I'm actually loving this go/inertia combo, especially with the new route http method updates in go 1.22


Live update the stream privacy icon.

This live updates the stream privacy icon in the general stream
settings after changing the privacy setting of the stream.

@sahil839 i combined the functions that update the privacy icon and update the title

We call the text present at the top in right column "title", so I do not think it is right to update the icon besides the name in that function. Instead I think we can create a function called something like update_stream_privacy_icon_in_settings. And call that to update the icon at both places.

We call the text present at the top in right column "title", so I do not think it is right to update the icon besides the stream name in General section of stream settings in that function. Instead I think we can create a function called something like update_stream_privacy_icon_in_settings to handle update of the privacy icons at all places.

@sahil839
Like this?
I am also not sure how to resolve the changes you requested that block the merge (if i try to view the review, it is not found, i guess it is outdated). Could you help me with that?

I left a small comment. Otherwise LGTM, cannot figure a clean way to write common code for icons in both the places.

@sahil839 I also think that its not really doable to have some common code since the two icons are rendered in two different templates.

@juliaBichler01 can you squash the commits? I can do the review (probably final one from my side) then.

@sahil839 i squashed my commits, could you do another review?

Also, just want to point out here that the icons do not look aligned with the stream name, which is not due to changes in the PR but in general may be due to some other recent change. The lock icon still looks good, but the hash icon seems to be somewhat below the text and the globe icon is above the text.

Yeah, we noticed this recently, and @amanagr is working on this in #21703.

Merged, thanks @juliaBichler01! And thanks @sahil839 for catching all the code patterns issues in your review :)


Upgrading to go version 1.18

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
Upgrading go version to 1.18
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Release note:
none

/retest

@mkumatag can you review this PR?


Cannot change src of image url in Lexical Richtext Editor

Link to reproduction
No response
Payload Version
3.0.0-beta.68
Node Version
18.19.0
Next.js Version
15.0.0-canary.53
Describe the Bug
I have a media collection, of which media is stored in azure blob storage url. I have a field name called 'fileUrl' which is a blob url of the image. I'm using Lexical Richtext Editor, in that when I upload in the Thumbnail I've used that fileUrl i.e. is a blob url of a image. When I save the collection in the generated html string I see the local path of the img  like 'api/media/file/car.jpg' in src attribute but what i want it should use the fileUrl instead of local path of image. As in frontend, I've a call get the collections data and I'm directly using the generated Html string to display the data of lexical richtext editor.
Also, how we can override/change the url path in the media collection to fileUrl i.e. blob url.

Here are some images for references of the issue-

The 'fileUrl' in above img is a blob url of a image. I want this image url in the lexical rich text editor as a src.

In the above img you can see src is a local path, in that place it should be a fileUrl.
How we can change the src path of the image?
Reproduction Steps
Create a Media collection
Create a collection called 'Pages'
In that add Lexical Richtext Editor and the HTML converter which converts the lexical content into html in a string form as shown in ss "generatedHtml" field.
Create a new record in that in lexical editor select upload type and upload a media.
Save the record and open the API tab scroll to down and see the generatedHtml string in the img src path it is a local path url instead of blob url. So, instead of the local path I want to add that 'fileUrl' i.e. blob Url path.
Adapters and Plugins
@payloadcms/richtext-lexical": "3.0.0-beta.68"

Hey @GermanJablo — if you'd like to pair on this one when you get started, let me know!


Provision grafana-zabbix datasource via yaml file (Grafana 5.X new functionality)

I'm submitting a ...

[ ] Bug report
[ ] Feature requestdd
[X] Question / Support request

What did you do?
Hello!
i want try to use the new Grafana DataSource Provisioning functionalities in order to "deploy" the zabbix-grafana datasource using yaml files.
Is there any documentation on how to edits the yaml file to provision the Grafana-Zabbix datasource?
Like in the Grafana Documentation
--> http://docs.grafana.org/administration/provisioning/#datasources <--

Yes, this should be in docs, agree.

OOKi thx u so much Alex :)

+1 for this as well!
Would love to be able to activate Zabbix plugin when provisioning with k8s as well as the datasource using that plugin

+1 It will be very usefull.


Access to homeID

When a client connects  to the ZWave JS server, the first message it receives includes the homeID that identify the ZWave network.
https://github.com/zwave-js/zwave-js-server#api
Could we add access to this value in version 4?
Or is there any other way to get this value using the ZWaveJS.NET lib?
Thanks

Hi @spudwebb,
This should already be available as Driver.Controller.homeId?
If it's not - sure, it's trivial to add.

Ooops I missed that property, sorry.
It works thanks!


Port 10260 is unaccounted for in k3s v1.27.3+k3s1

Environmental Info:
K3s Version: 1.27.3+k3s1
Node(s) CPU architecture, OS, and Version:
Redhat 8.8, AMD64
Cluster Configuration:
Single node K3s cluster with traefik and helm-controller disabled.
Describe the bug:
There is port listening on k3s main process 10260 for which there is no information in documentation or anywhere in the internet. This is has been verified in multiple times. The following command shows.
$ sudo netstat -nlp
tcp6       0      0 :::10260                :::*                    LISTEN      7555/k3s server
Steps To Reproduce:
Install k3s 1.27.3+k3s1 with traefik and helm-controller disabled. check the open ports using
sudo netstat -nlp

This is the default port used by the upstream Kubernetes cloud controller manager webhook. Ref: https://github.com/kubernetes/cloud-provider/blob/master/options/webhook.go#L36
Webhook serving flags:
      --webhook-bind-address ip
                The IP address on which to listen for the --webhook-secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If set to an unspecified address
                (<IP_ADDRESS> or ::), all interfaces will be used. If unset, defaults to <IP_ADDRESS>. (default <IP_ADDRESS>)
      --webhook-cert-dir string
                The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.
      --webhook-secure-port int
                Secure port to serve cloud provider webhooks. If unset, defaults to 10260. (default 10260)
      --webhook-tls-cert-file string
                File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided,
                a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.
      --webhook-tls-private-key-file string
                File containing the default x509 private key matching --tls-cert-file.

If you want to turn it off, you can start k3s with --kube-cloud-controller-manager-arg=webhook-secure-port=0


I  deployed YOLOv7 and SAM on a CPU.

I  deployed YOLOv7 and SAM on a CPU. I would like to inquire, in a sincere manner, about the steps and settings required to transition their deployment to a GPU.

https://opencv.github.io/cvat/docs/administration/advanced/installation_automatic_annotation/#gpu-support
I guess just follow here the instructions, can do this.

Dublicate for #6793
@SpecLad
Have you tested this model running on GPU?
It looks like installation pipeline is correct, but for the user it does not work on GPU

You probably need to install onnxruntime-gpu instead of onnxruntime in
https://github.com/opencv/cvat/blob/develop/serverless/onnx/WongKinYiu/yolov7/nuclio/function-gpu.yaml#L110

hank you very much to all the scientists at CVAT for the help provided, the problem has been perfectly solved!CVAT is an exceptional product! :)

@seasenzzr000 change the issue name to a proper name to your problem, that you have already deployed on cpu is not your intension to ask

@seasenzzr000 change the issue name to a proper name to your problem, that you have already deployed on cpu is not your intension to ask

Sure, the title has been updated


Add support for intercept_v1.dial_option.identity

This enables the Py client to address specific terminators by identity name, correct?

Yes


fix compatibility with nvim < 0.8

see comments on #1213
cc @briandipalma @appelgriebsch

Ph. I didn't know how to nvim's version checking. Thank you!

Why this bug was not fixed for nvim 0.8?

What do you mean?
nvim v0.8 has breaking change. This patch is checking nvim version.


Fix issue 2119: make live photo identifier more generic

Motivation
Issue 2119
We are running into the problem outlined in issue 2119 where there is a "Could not find image file..." error on iOS when selecting images that are on iCloud but not yet fully synced to the user's device.
I found by updating the live photo identifier to be more generic (rather than it explicitly being "public.jpeg") this resolved the issue.
Please note: I am a React Native / Javascript developer, and not by any means a native iOS / Objective C developer. To reach the solution in this PR I referenced the code in this PR, and added the least amount I could to resolve the issue. Someone with more native iOS experience may be able to suggest a more robust solution to the issue.
Test Plan
(Requires testing with two devices logged into the same iCloud account)

Take a photo on device A
Use the example app on device B logged into the same iCloud account
Select the image synced over with iCloud (without making any edits)
Without this change you'll get the "Could not find image file..." warning. With this change, you'll retrieve the image as expected.

Hi folks, any updates on this one?


Fix EntityResponseGenerator

We got a failing test in our test suite after trying to upgrade http4s. After some digging it appeared to be introduced in https://github.com/http4s/http4s/pull/2446
This PR fixes the failing test for us.
Take body: F[A] as parameter instead of G[A] which enables following the code to be structured as it was here https://github.com/http4s/http4s/pull/2446/files#diff-09e729dc8ff0315c384e774351a8720fL42
If 👍 I'm happy to make analogous change to other branches.

I'm surprised that this is binary compatible, but type erasure saves us here

Unless there are some constraints on G, I'm pretty sure this is binary incompatible just in a way that mima won't catch.

The constraint is EntityEncoder[G, A].
For most people, F is G. And this will be mostly applications calling it. But, eh, maybe this would cause a ClassCastException in the encoder for the unlucky few? 🤔

Could also cause a ClassCastException in the Monad[F].flatMap. I do agree that for most people, those two would probably be unified. I guess it depends on how fussed you want to be about it. In theory, the old function was just… broken… so even breaking that code is kind of justified? I dunno.

This broke the DirectivesDsl.
I suppose we will have to live with:
BadRequest(Directive.liftF(IO("Output")))

If we instead change this to
  def apply[A](body: G[A])(implicit nat: G ~> F ,F: Monad[F], w: EntityEncoder[G, A]): F[Response[G]] =
    F.flatMap(nat[A](body))(apply[A](_))

then it works in both cases

Sorry. I thought it was a mistake, but I remember that Http4sDsl2 was added for the directives DSL, and should have tried that or pinged you.
Since it broke working code, and is of questionable binary compatibility, I think we should revert for 0.20.

As for 0.21, I don't think of ~> as something we usually pass around as implicit evidence.  I wonder about just giving the F version a different name than apply.

We might then need a different way of plugging in the lifting of G to F in 0.21 ? I can try to come up with something.

Yeah, my first priority here is that we not break http4s-directives. If we can come up with something clean for both that and the use case that motivated this original PR, that's the best thing for 0.21... and if it's compatible, for 0.20.


Planning work staggering

Originally created by Tom Smyth at 2018-07-19 08:28, migrated from redmine-#8660

Status changed from New to Closed

By Tom Smyth at 2018-08-08 22:43


Treat body as IO in param_parser

Waiting for Crystal 0.20.0 to be merged 👍

Never mind, this also works in Crystal 0.19.4 🎉


Harfbuzz version too old (1.2.7)

When trying to run version 2.0.0 x64 on Fedora 32, I get the following output:
me@localhost ~/S/Pine64 Installer> ./pine64-installer-2.0.0-beta.3-linux-x64.AppImage
/home/me/.cache/thumbnails/normal/53a10613671955bad5fd6e2d458157e8.png is missing. Probably not running ./bin//pine64-installer.wrapper from within an AppImage.
Hence falling back to using .DirIcon
Gtk-Message: 17:55:12.126: Failed to load module "appmenu-gtk-module"

(pine64-installer:60452): Pango-ERROR **: 17:55:12.202: Harfbuzz version too old (1.2.7)

fish: “./pine64-installer-2.0.0-beta.3…” terminated by signal SIGTRAP (Trace or breakpoint trap)

My kernel is 5.11.11-100.fc32.x86_64, and pango-1.44.7-2.fc32.x86_64 (currently the latest version according to dnf) is installed.

Harfbuzz version too old (1.2.7)

Also occurring on Ubuntu 20.10 with libharfbuzz0b at version 2.6.4-1ubuntu5


Changing Fake User Agent

Per #11349, adjusting the fake user-agent string to something Core is hopefully ok with.

Pinging @ndr-qef @phinze @dwaite.

Looks good. Could you also update the user agent test at spec/cask/download_strategy_spec.rb?

Test updated. Thanks for pointing that out. I'm still learning and frequently forget to look for tests.

Not to worry; at present, our test suite is rather untidily strewn about.
Thanks to you, and to @dwaite for his report on the most common user agents.


[FEATURE] Solution to https://leetcode.com/problems/binary-tree-maximum-path-sum/

Feel free to add as much solutions as you want, maybe you took a different approach which is valuable :)
Please be sure to not delete any current solution, or if the file has to be created follow this-type-of-naming.
Finally, create a new branch for the problem(s) you'll submit, and make sure to fulfill all the PR's requirements.

I added both of you because it is possible that you two have different approaches

If not, then a single solution is fine.


How to user "replay" option when subscribe something on netopeer2-cli.

I don't found out the example of this option.

It is all standardized and explained in the official RFC.


Fail to run cmake and protobuf targets from Unix playbook

Attempting to run the specific tags for either ccache or protobuf fail to be found.
This change appears to be what caused the issue;
https://github.com/AdoptOpenJDK/openjdk-infrastructure/pull/584

Hmmm yes that's potentially an issue. What's the use case for wanting to run individual ones like that in the playbook?

I have a machine that is missing protobuf and I don't want to waste time running the entire playbook to get one role.

Doesn't the playbook 'skip' everything that's already in place?

@karianna It does but takes a bit longer than is ideal (An issue I'd quite like to look at...)

The cmake, ccache and protobuf tasks, in their respective roles, are all tagged with cmake, ccache and protobuf respectively. So these specific roles can be run using --tags "cmake,ccache,protobuf" in the ansible-playbook command. However this will not run the protobuf tasks as the role is not included in the AdoptOpenJDK_Unix_Playbook/main.yml, as of the pr mentioned in the first comment.
Are there any plans to add protobuf back into main.yml?

Up until a week ago it wasn't required by AdoptOpenJDK but since the JITserver support went into the extensions repository a few days ago it is (Linux/x64 only for now) so yeah I'll chuck in a PR to re-enable it ...

The local_srcinstall role has been removed as per https://github.com/AdoptOpenJDK/openjdk-infrastructure/pull/1102.
Closing this issue.


Completing the hydration story with the remaining models

Describe changes
Secrets stores still remain. This PR is a WIP.
Pre-requisites
Please ensure you have done the following:

[X] I have read the CONTRIBUTING.md document.
[ ] If my change requires a change to docs, I have updated the documentation accordingly.
[X] If I have added an integration, I have updated the integrations table and the corresponding website section.
[ ] I have added tests to cover my changes.
[X] I have based my new branch on develop and the open PR is targeting develop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[X] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Other (add details above)

@coderabbitai review

@stefannica any chance you could add some breaking changes notes to this PR description to help me prepare the release notes?


encoding/asn1: Unmarshal rejects PrintableString containing underscore

in my program，import a customer X509 digital certificate， But ,one of this  get a error, and I found in
KeyBlock,Bytes has a  code  130 0x82 is not  printable,so get reject, but I use C program and java all can load success! Please help me if thiis is a go mistake?
Cert, err2 := x509.ParseCertificate(KeyBlock.Bytes)
28     if err2 != nil {
29         return nil, e.New(1004, err2, "ReadX509Cert x509.ParseCertificate error;")
30     }

Can you include the certificate? This sounds like it's just a case where Go is being stricter about the rules.
(Note: you can still run a TLS server with a certificate that doesn't parse if you wish.)

-----BEGIN CERTIFICATE-----
MIIELjCCAxagAwIBAgIFIABlIQQwDQYJKoZIhvcNAQEFBQAwWTELMAkGA1UEBhMC
Q04xMDAuBgNVBAoTJ0NoaW5hIEZpbmFuY2lhbCBDZXJ0aWZpY2F0aW9uIEF1dGhv
cml0eTEYMBYGA1UEAxMPQ0ZDQSBURVNUIE9DQTExMB4XDTEzMTIxODA5MjcwN1oX
DTE1MTIxODA5MjcwN1owazELMAkGA1UEBhMCQ04xDjAMBgNVBAoTBXl1emhpMRIw
EAYDVQQLEwlDTUJDX0RDTVMxFDASBgNVBAsTC0luZGl2aWR1YWwxMSIwIAYDVQQD
FBljbj0wMDA0MDAwMDpTSUdOQDAwMDAwMDAyMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAtTBhimpRdnIMC7SBfl3xeSJR8XF0Jqf9XItroLMoffxyNfRB
khKGf87oLDaGfZa6s/+vct2iktHNMcjS8gMmlBj/RjXflAdEVkeQAfSiZFN8Bn9q
14+dEXtVbvrosaFHmfTmHWchMMeXMJlg9K6akM+lVwmPjLfT4bcDk67VVUCy4uLj
xe0YgrBFfqAxcj4/vlHAdKY+3n+keUQ9PrBRuYXQcGgXaBEGlhnCJIrnVnnZdRUS
T3qt6iUUUu7/2rIvvrv+/aLN3li432LuhJc3DWANhwucC7DDpXoFScoqrgpQbofc
vPdbJdsRrLvZ/XYDmmcdYOMsQ7AnZIbVVUws3wIDAQABo4HqMIHnMB8GA1UdIwQY
MBaAFPwLvESaDjGhg6mBhyceBULGv1b4MEgGA1UdIARBMD8wPQYIYIEchu8qAQIw
MTAvBggrBgEFBQcCARYjaHR0cDovL3d3dy5jZmNhLmNvbS5jbi91cy91cy0xNS5o
dG0wOQYDVR0fBDIwMDAuoCygKoYoaHR0cDovLzIxMC43NC40MS44Ny9vY2ExMS9S
U0EvY3JsNjM5LmNybDALBgNVHQ8EBAMCA+gwHQYDVR0OBBYEFLBo5Ut5b4js+bsL
yVe73j4xNcDOMBMGA1UdJQQMMAoGCCsGAQUFBwMCMA0GCSqGSIb3DQEBBQUAA4IB
AQBQbuGIjDNd0wT1dGcjsImtXAmHZaytqUn3mY2VdAEZhR1rLEGjkG+j30zIV7K8
BrUWOf0Pny6tQ+OTevNGCV1hiO2ZQryB50R/eBGSk70dBqIDzpOk4cAj6OBaO+uh
Uqw2jdDOt1rV3UeptMD4Mbs3FhdQy3sDApkZs10px/5PHMohjktnwSuFmXaNwYsX
Sv282c6tjiDWYcQrTHZVfDhhrooAjLSQ6wDWW/QnjhbcNbY2lnf9hReYoDBloK8G
O0AH1nfVkMVY0Ja6OdwnJD+NFIL0yPqz1gH7qzrmcGOZefiUPC+LGU0D+qYU1f22
afnVaYT41MCjuM1nXmMcEmLt
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
MIIELjCCAxagAwIBAgIFIABlIQQwDQYJKoZIhvcNAQEFBQAwWTELMAkGA1UEBhMC
Q04xMDAuBgNVBAoTJ0NoaW5hIEZpbmFuY2lhbCBDZXJ0aWZpY2F0aW9uIEF1dGhv
cml0eTEYMBYGA1UEAxMPQ0ZDQSBURVNUIE9DQTExMB4XDTEzMTIxODA5MjcwN1oX
DTE1MTIxODA5MjcwN1owazELMAkGA1UEBhMCQ04xDjAMBgNVBAoTBXl1emhpMRIw
EAYDVQQLEwlDTUJDX0RDTVMxFDASBgNVBAsTC0luZGl2aWR1YWwxMSIwIAYDVQQD
FBljbj0wMDA0MDAwMDpTSUdOQDAwMDAwMDAyMIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEAtTBhimpRdnIMC7SBfl3xeSJR8XF0Jqf9XItroLMoffxyNfRB
khKGf87oLDaGfZa6s/+vct2iktHNMcjS8gMmlBj/RjXflAdEVkeQAfSiZFN8Bn9q
14+dEXtVbvrosaFHmfTmHWchMMeXMJlg9K6akM+lVwmPjLfT4bcDk67VVUCy4uLj
xe0YgrBFfqAxcj4/vlHAdKY+3n+keUQ9PrBRuYXQcGgXaBEGlhnCJIrnVnnZdRUS
T3qt6iUUUu7/2rIvvrv+/aLN3li432LuhJc3DWANhwucC7DDpXoFScoqrgpQbofc
vPdbJdsRrLvZ/XYDmmcdYOMsQ7AnZIbVVUws3wIDAQABo4HqMIHnMB8GA1UdIwQY
MBaAFPwLvESaDjGhg6mBhyceBULGv1b4MEgGA1UdIARBMD8wPQYIYIEchu8qAQIw
MTAvBggrBgEFBQcCARYjaHR0cDovL3d3dy5jZmNhLmNvbS5jbi91cy91cy0xNS5o
dG0wOQYDVR0fBDIwMDAuoCygKoYoaHR0cDovLzIxMC43NC40MS44Ny9vY2ExMS9S
U0EvY3JsNjM5LmNybDALBgNVHQ8EBAMCA+gwHQYDVR0OBBYEFLBo5Ut5b4js+bsL
yVe73j4xNcDOMBMGA1UdJQQMMAoGCCsGAQUFBwMCMA0GCSqGSIb3DQEBBQUAA4IB
AQBQbuGIjDNd0wT1dGcjsImtXAmHZaytqUn3mY2VdAEZhR1rLEGjkG+j30zIV7K8
BrUWOf0Pny6tQ+OTevNGCV1hiO2ZQryB50R/eBGSk70dBqIDzpOk4cAj6OBaO+uh
Uqw2jdDOt1rV3UeptMD4Mbs3FhdQy3sDApkZs10px/5PHMohjktnwSuFmXaNwYsX
Sv282c6tjiDWYcQrTHZVfDhhrooAjLSQ6wDWW/QnjhbcNbY2lnf9hReYoDBloK8G
O0AH1nfVkMVY0Ja6OdwnJD+NFIL0yPqz1gH7qzrmcGOZefiUPC+LGU0D+qYU1f22
afnVaYT41MCjuM1nXmMcEmLt
-----END CERTIFICATE-----

this is the load fail x509 cert please check .thanks

@agl @ianlancetaylor I have upload the certificate.... please help me thanks

@laybay I did some digging, and I think there is a bug in the asn1 package.
Your certificate looks like this :
$ openssl asn1parse -in githubtestcert.pem
    0:d=0  hl=4 l=1070 cons: SEQUENCE
    4:d=1  hl=4 l= 790 cons: SEQUENCE
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=   5 prim: INTEGER           :2000652104
   20:d=2  hl=2 l=  13 cons: SEQUENCE
   22:d=3  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
   33:d=3  hl=2 l=   0 prim: NULL
   35:d=2  hl=2 l=  89 cons: SEQUENCE
   37:d=3  hl=2 l=  11 cons: SET
   39:d=4  hl=2 l=   9 cons: SEQUENCE
   41:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   46:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :CN
   50:d=3  hl=2 l=  48 cons: SET
   52:d=4  hl=2 l=  46 cons: SEQUENCE
   54:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   59:d=5  hl=2 l=  39 prim: PRINTABLESTRING   :China Financial Certification Authority
  100:d=3  hl=2 l=  24 cons: SET
  102:d=4  hl=2 l=  22 cons: SEQUENCE
  104:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  109:d=5  hl=2 l=  15 prim: PRINTABLESTRING   :CFCA TEST OCA11
  126:d=2  hl=2 l=  30 cons: SEQUENCE
  128:d=3  hl=2 l=  13 prim: UTCTIME           :131218092707Z
  143:d=3  hl=2 l=  13 prim: UTCTIME           :151218092707Z
  158:d=2  hl=2 l= 107 cons: SEQUENCE
  160:d=3  hl=2 l=  11 cons: SET
  162:d=4  hl=2 l=   9 cons: SEQUENCE
  164:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  169:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :CN
  173:d=3  hl=2 l=  14 cons: SET
  175:d=4  hl=2 l=  12 cons: SEQUENCE
  177:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
  182:d=5  hl=2 l=   5 prim: PRINTABLESTRING   :yuzhi
  189:d=3  hl=2 l=  18 cons: SET
  191:d=4  hl=2 l=  16 cons: SEQUENCE
  193:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  198:d=5  hl=2 l=   9 prim: PRINTABLESTRING   :CMBC_DCMS
  209:d=3  hl=2 l=  20 cons: SET
  211:d=4  hl=2 l=  18 cons: SEQUENCE
  213:d=5  hl=2 l=   3 prim: OBJECT            :organizationalUnitName
  218:d=5  hl=2 l=  11 prim: PRINTABLESTRING   :Individual1
  231:d=3  hl=2 l=  34 cons: SET
  233:d=4  hl=2 l=  32 cons: SEQUENCE
  235:d=5  hl=2 l=   3 prim: OBJECT            :commonName
  240:d=5  hl=2 l=  25 prim: T61STRING         :cn=00040000:SIGN@00000002
  267:d=2  hl=4 l= 290 cons: SEQUENCE
  271:d=3  hl=2 l=  13 cons: SEQUENCE
  273:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
  284:d=4  hl=2 l=   0 prim: NULL
  286:d=3  hl=4 l= 271 prim: BIT STRING
  561:d=2  hl=3 l= 234 cons: cont [ 3 ]
  564:d=3  hl=3 l= 231 cons: SEQUENCE
  567:d=4  hl=2 l=  31 cons: SEQUENCE
  569:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Authority Key Identifier
  574:d=5  hl=2 l=  24 prim: OCTET STRING      [HEX DUMP]:30168014FC0BBC449A0E31A183A98187271E0542C6BF56F8
  600:d=4  hl=2 l=  72 cons: SEQUENCE
  602:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Certificate Policies
  607:d=5  hl=2 l=  65 prim: OCTET STRING      [HEX DUMP]:303F303D060860811C86EF2A01023031302F06082B060105050702011623687474703A2F2F7777772E636663612E636F6D2E636E2F75732F75732D31352E68746D
  674:d=4  hl=2 l=  57 cons: SEQUENCE
  676:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 CRL Distribution Points
  681:d=5  hl=2 l=  50 prim: OCTET STRING      [HEX DUMP]:3030302EA02CA02A8628687474703A2F2F3231302E37342E34312E38372F6F636131312F5253412F63726C3633392E63726C
  733:d=4  hl=2 l=  11 cons: SEQUENCE
  735:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Key Usage
  740:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:030203E8
  746:d=4  hl=2 l=  29 cons: SEQUENCE
  748:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Key Identifier
  753:d=5  hl=2 l=  22 prim: OCTET STRING      [HEX DUMP]:0414B068E54B796F88ECF9BB0BC957BBDE3E3135C0CE
  777:d=4  hl=2 l=  19 cons: SEQUENCE
  779:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Extended Key Usage
  784:d=5  hl=2 l=  12 prim: OCTET STRING      [HEX DUMP]:300A06082B06010505070302
  798:d=1  hl=2 l=  13 cons: SEQUENCE
  800:d=2  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
  811:d=2  hl=2 l=   0 prim: NULL
  813:d=1  hl=4 l= 257 prim: BIT STRING

At the 240 byte offset you have
 240:d=5  hl=2 l=  25 prim: T61STRING         :cn=00040000:SIGN@00000002 string.
A T61STRING is treated by golang as any other printable string, and parsed in the parsePrintableString method in the encoding/asn1/asn1.go. I copied the method below:
// parsePrintableString parses a ASN.1 PrintableString from the given byte
// array and returns it.
func parsePrintableString(bytes []byte) (ret string, err error) {
	for _, b := range bytes {
		if !isPrintable(b) {
			err = SyntaxError{"PrintableString contains invalid character"}
			return
		}
	}
	ret = string(bytes)
	return
}

// isPrintable reports whether the given b is in the ASN.1 PrintableString set.
func isPrintable(b byte) bool {
	return 'a' <= b && b <= 'z' ||
		'A' <= b && b <= 'Z' ||
		'0' <= b && b <= '9' ||
		'\'' <= b && b <= ')' ||
		'+' <= b && b <= '/' ||
		b == ' ' ||
		b == ':' ||
		b == '=' ||
		b == '?' ||
		// This is technically not allowed in a PrintableString.
		// However, x509 certificates with wildcard strings don't
		// always use the correct string type so we permit it.
		b == '*'
}

Now, if the T61STRING is not a part of a sequence like in your case, the parsePrintableString wouldn't be called, because it is the parseSequenceOf that casts all the string type to the printableString.
I will do some more digging ( I might be wrong on this, and I don't know the asn1 package by heart yet ).

@laybay  I have just checked how openssl handle PRINTABLESTRING. This is a CSR but certificate should be handled the same way :
# openssl asn1parse -in testCSR.csr -i
    0:d=0  hl=4 l= 414 cons: SEQUENCE
    4:d=1  hl=4 l= 263 cons:  SEQUENCE
    8:d=2  hl=2 l=   1 prim:   INTEGER           :00
   11:d=2  hl=2 l=  94 cons:   SEQUENCE
   13:d=3  hl=2 l=  11 cons:    SET
   15:d=4  hl=2 l=   9 cons:     SEQUENCE
   17:d=5  hl=2 l=   3 prim:      OBJECT            :countryName
   22:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :AU
   26:d=3  hl=2 l=  13 cons:    SET
   28:d=4  hl=2 l=  11 cons:     SEQUENCE
   30:d=5  hl=2 l=   3 prim:      OBJECT            :stateOrProvinceName
   35:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :test
   41:d=3  hl=2 l=  13 cons:    SET
   43:d=4  hl=2 l=  11 cons:     SEQUENCE
   45:d=5  hl=2 l=   3 prim:      OBJECT            :localityName
   50:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :city
   56:d=3  hl=2 l=  21 cons:    SET
   58:d=4  hl=2 l=  19 cons:     SEQUENCE
   60:d=5  hl=2 l=   3 prim:      OBJECT            :organizationName
   65:d=5  hl=2 l=  12 prim:      T61STRING         :!@#$%^&*(()_
   79:d=3  hl=2 l=  11 cons:    SET
   81:d=4  hl=2 l=   9 cons:     SEQUENCE
   83:d=5  hl=2 l=   3 prim:      OBJECT            :organizationalUnitName
   88:d=5  hl=2 l=   2 prim:      PRINTABLESTRING   :OU
   92:d=3  hl=2 l=  13 cons:    SET
   94:d=4  hl=2 l=  11 cons:     SEQUENCE
   96:d=5  hl=2 l=   3 prim:      OBJECT            :commonName
  101:d=5  hl=2 l=   4 prim:      PRINTABLESTRING   :AAA
  107:d=2  hl=3 l= 159 cons:   SEQUENCE
  110:d=3  hl=2 l=  13 cons:    SEQUENCE
  112:d=4  hl=2 l=   9 prim:     OBJECT            :rsaEncryption
  123:d=4  hl=2 l=   0 prim:     NULL
  125:d=3  hl=3 l= 141 prim:    BIT STRING
  269:d=2  hl=2 l=   0 cons:   cont [ 0 ]
  271:d=1  hl=2 l=  13 cons:  SEQUENCE
  273:d=2  hl=2 l=   9 prim:   OBJECT            :sha256WithRSAEncryption
  284:d=2  hl=2 l=   0 prim:   NULL
  286:d=1  hl=3 l= 129 prim:  BIT STRING

As you can see if the RDN can be represented as a PRINTABLESTRING (localityName) PRINTABLESTRING is used. If the RDN contains characters forbidden in the PRINTABLESTRING, (organizationName) then T61STRING is used instead.
I guess the certificate wasn't generated with openssl.

@agl, should we accept _ in PrintableString? Seems like probably yes? What restrictions does Chrome apply to PrintableString? Thanks.

Per discussion with @agl, this is the first time we've seen a certificate that is malformed in this particular way, so we're not going to rush into relaxing the requirements. If you can give us more context about what software (erroneously) generated that certificate and how common the problem might be, that would help us decide what to do for Go 1.7. Adam is thinking about scanning through the Certificate Transparency logs to see if there are more there. Thanks.

I scanned 4.5M of the 12M certificates in the Certificate Transparency "pilot" log (I ran out of disk space downloading it all) and below are the counts for the invalid characters found in PRINTABLESTRING values:
  1  ´
  1  !
  1  #
  1  å
  1  ^K
  1  ñ
  1  ø
  1  ^Y
  2  "
  2  á
  2  ç
  2  í
  3  ó
  3  (tab)
  4  @
  4  à
  4  Ý
  5  ü
121  &

That only includes public certificates but it doesn't suggest that underscore is sufficiently widely misplaced in these strings to bend the standard for. (Ampersand is used in about one in 40K certificates and might be something that we have to make an exception for in the future.)

Also dealing with this issue: a certificate with an underscore, in my case a client certificate.  Looking for advice about how I might still use this certificate without using tls.LoadX509KeyPair.

Decode it manually, verify manually (if you wish), extract the public key and there you go.  What exactly do you wan to do with it ?


Add puppetlabs-java to requirements.

We should probably put the puppetlabs-java module in the requirements section of the README.
I've not added it to metadata.json as it's optional.
Inspired by https://github.com/elastic/puppet-elasticsearch/issues/586

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then  say 'jenkins, test it'.


KongPlugin error is causing entire Kong to be down when in different namespace

Summary
Hi, I am encountering an issue when I tried to run KongPlugin for Bot Detection in a different namespace. Note that I have a main Kong Ingress Controller running in Kong namespace and I have created another Kong Ingress Controller in Kong-Test namespace to test out if my Bot Detection Plugin is working. I noticed that Kong have shared resources across cluster wide, as when I tried to run my own CRD in Kong-Test namespace, it will tell me it is configured and not created. This is when the issue comes. After running my plugin yaml, the entire Kong; including the main Kong in Kong namespace (note that I have been working on Kong-Test namespace from the start till the end) crashed and threw an error

E1022 07:11:28.949563       1 controller.go:128] unexpected failure updating Kong configuration:
posting new config to /config: 400 Bad Request {"fields":{"plugins":[null,null,null,null,null,null,null,{"config":{"deny":"expected an array","allow":"expected an array"}},{"config":{"deny":"expected an array","allow":"expected an array"}}]},"name":"invalid declarative configuration","cod
e":14,"message":"declarative config is invalid: {plugins={[8]={config={allow="expected an array",deny="expected an array"}},[9]={config={allow="expected an array",deny="expected an array"}}}}"}
W1022 07:11:28.949586       1 queue.go:112] requeuing rachel-test/ingress-svc-rachel, err posting new config to /config: 400 Bad Request {"fields":{"plugins":[null,null,null,null,null,null,null,{"config":{"deny":"expected an array","allow":"expected an array"}},{"config":{"deny":"expected
an array","allow":"expected an array"}}]},"name":"invalid declarative configuration","code":14,"message":"declarative config is invalid: {plugins={[8]={config={allow="expected an array",deny="expected an array"}},[9]={config={allow="expected an array",deny="expected an array"}}}}
"}

Here is my Bot Detection yaml

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: bot-detection-example
config:
allow:
User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
deny:
User-Agent: Mozilla/5.0 (Window NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36; 360Spider
plugin: bot-detection

Steps To Reproduce

Create a new Kong Ingress Controller in another namespace apart from Kong (the main namespace)
Create a Bot Detection KongPlugin inside your own namespace/the same as where your new Kong Ingress Controller is
Create a new Ingress and Service object inside your own namespace/the same as where your new Kong Ingress Controller is
Curl your host

Additional Details & Logs

Kong version 2.1.3
Operating system mac OS
Kong Ingress Controller Image kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.1

So I created a response-transformer plugin and I found this inside my kong logs on loop
time="2021-03-25T05:09:42Z" level=error msg="failed to sync: posting new config to /config: 400 Bad Request {\"fields\":{\"plugins\":[{\"config\":{\"add\":{\"headers\":[\"expected a string\",\"expected a string\",\"expected a string\",\"expected a string\",\"expected a string\"]}}}]},\"name\":\"invalid declarative configuration\",\"code\":14,\"message\":\"declarative config is invalid: {plugins={{config={add={headers={\\\"expected a string\\\",\\\"expected a string\\\",\\\"expected a string\\\",\\\"expected a string\\\",\\\"expected a string\\\"}}}}}}\"}" component=sync-queue

What my yaml look like for it
apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: global-response-transformer
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: "true"
config:
  add:
    headers:
    - X-Frame-Options: deny
    - Content-Security-Policy: script-src 'self';
    - Strict-Transport-Security: max-age=16070400; includeSubDomains
    - X-Content-Type-Options: nosniff
    - Set-Cookie: HttpOnly; Secure;
plugin: response-transformer

Not long after, my infra node went down. Went into cloudwatch to check what went wrong and saw that one of my nodes had a spike of 100% cpu memory, the other one have a spike of 50% cpu memory. Hence, one of my node went down

@rachelang09 How large is your Kong configset (an estimation count of service/route/plugins/consumers will be good)? Could you also use tools like cAdvisor to pin point which container on the node is taking CPU?

@fffonion est to be about 20 each or more as it will continue to grow. I will check on cAdvisor and check which container on the node is taking CPU. Thanks!

We also face the same issue, what we see that need to use Kong validation webhook to handle these kinds of errors in configuration.
But also, very weird the whole kong ingress controller goes down.
This will be a deal breaker on production environments


 incorrectly requires both aria-label and aria-labelledby

What package(s) are you using?

[ ] Carbon for IBM Products (or Carbon for IBM Cloud & Cognitive)
[x] Legacy/CDAI
[ ] Legacy/Security

Detailed description

Describe in detail the issue you're having.

<IdeNavigation> has the following prop:
https://github.com/carbon-design-system/ibm-cloud-cognitive/blob/f3eb2a8ccda451aa4f7cb9a76925bdb4d0fc4e64/packages/cdai/src/components/IdeNavigation/Navigation/IdeNavigation.js#L96-L102
This requires both aria-label and aria-labelledby to be specified as strings. We have had an accessibility issue raised where in fact it is incorrect to have both of these. Either aria-label provides the actual label string, or aria-labelledby provides the selector for the element which provides the actual label string - having both makes the correct label ambiguous.
In practice, when both are present, aria-labelledby takes priority. In our case, we use a direct aria-label, so we have no choice but to specify aria-labelledby=''. Screen readers take aria-labelledby='' to be authoritative, and so the screen readers conclude that there is no element to provide the label, and therefore no label at all. It is not possible to satisfy both screen readers and the prop type requirements at once.

What version of the @carbon/ibm-products (or @carbon/ibm-cloud-cognitive)
package are you using?

<EMAIL_ADDRESS>
What offering/product do you work on? Any pressing ship or release dates we
should be aware of?

IBM App Connect Enterprise. If this could be fixed well before mid-September 2022 that would be nice.

https://github.com/carbon-design-system/ibm-cloud-cognitive/pull/2243 was merged on 1 September 2022. Now awaiting a new release of @carbon/ibm-cloud-cognitive-cdai with this fix in.

@s100 Some changes to our auth in the Carbon Design org caused our recent publishes to fail, but we should have fixed it and your fixes should be out on the 13th! Thanks for contributing.

Fixed in<EMAIL_ADDRESS>


notify-user-bluetooth

https://github.com/covid19risk/covidwatch-android/issues/26
Added timer task in IntentService that launches a foreground notification if a user turns off their bluetooth. Not sure how to observe the user turning on bluetooth in a battery efficient manner.  Open to suggestions.

Closed until decision where this logic lives (client or library) is reached.

Closed until decision where this logic lives (client or library) is reached.


Bump go version to 1.15

What does this PR do?
Bump go version to 1.15
What ticket does this PR close?
Resolves #278
Checklists
Change log

[ ] The CHANGELOG has been updated, or
[X] This PR does not include user-facing changes and doesn't require a CHANGELOG update

Test coverage

[ ] This PR includes new unit and integration tests to go with the code changes, or
[X] The changes in this PR do not require tests

Documentation

[ ] Docs (e.g. READMEs) were updated in this PR, and/or there is a follow-on issue to update docs, or
[X] This PR does not require updating any documentation

@BradleyBoutcher what's the motivation for this change?

@orenbm we're standardizing all of our Go projects on the latest Go version 1.12 and 1.13 are already technically end of life.

@orenbm There's a ticket you filed, #278

@orenbm There's a ticket you filed, #278

I always trust Oren of the Past :)
Thanks!

@BradleyBoutcher I am just seeing that the Dockerfiles have go versions that are still stuck at 1.13 - can you please put up a follow-up PR to resolve this?


Fix for local_signature_view for native functions.

This is a patch that fixes a crash that happens if the stackless bytecode generator is run on a module with a native function.

@bors-libra r=sblackshear

@cbarrettfb: :key: Insufficient privileges: Collaborator required

@bors-libra r+

:pushpin: Commit b744a68 has been approved by sblackshear

:hourglass: Testing commit b744a68ac4f73d036721cd8ab0f268298e19dad8 with merge e664ca3e892c11520d9227947ca90a641e44c53d...

:sunny: Test successful - checks-circle_commit_workflow
Approved by: sblackshear
Pushing e664ca3e892c11520d9227947ca90a641e44c53d to master...


Markdown Input behaves weirdly when content contains double spaces

Package
filament/filament
Package Version
v2.16.19
Laravel Version
v9.32.0
Livewire Version
v2.10.7
PHP Version
PHP 8.1.8
Problem description
When a markdown input contains a double space, the markdown input behaves very weirdly:

Expected behavior
Markdown field to behave as expected

Steps to reproduce
Checkout reproduction repo, add user, then seed. View created user in edit mode.
Reproduction repository
https://github.com/chris-ware/filament-markdown-issue.git
Relevant log output
No response

I tested from your reproduction repository. This issue is because you are using \r\n, whereas if you try to save via form, you will get \n. Why not change your seeder?
Do we have to replace \r\n to \n via component @danharrin?

The purpose of the seeder is to show the issue, so replacing it does nothing.

LF and CR are control characters or bytecode that can be used to mark a line break in a text file.

LF = Line Feed (\n, 0x0A in hexadecimal, 10 in decimal) — moves the cursor down to the next line without returning to the beginning of the line.
CR = Carriage Return (\r, 0x0D in hexadecimal, 13 in decimal) — moves the cursor to the beginning of the line without advancing to the next line.

Did you see the context? I tried with demo.
LF
process.stdout.write('long message first\n')
process.stdout.write('short message\n')

long message first
short message

CF
process.stdout.write('long message first\r')
process.stdout.write('short message\n')

short messagefirst

Fixed by #4360


[packager] JReleaser error without details

Discussed in https://github.com/jreleaser/jreleaser/discussions/1233

Originally posted by gotson March  3, 2023
Hello,
my JReleaser release on CI failed with no details whatsoever: https://github.com/gotson/komga/actions/runs/4322026365/jobs/7544068802#step:11:838
INFO]  JReleaser 1.5.0
[INFO]    - basedir set to /home/runner/work/komga/komga/komga
[INFO]    - outputdir set to /home/runner/work/komga/komga/komga/build/jreleaser
[INFO]  git-root-search set to true
[INFO]  Loading variables from /home/runner/.jreleaser/config.properties
[WARN]  Variables source /home/runner/.jreleaser/config.properties does not exist
[INFO]  Validating configuration
[INFO]  Strict mode set to false
[INFO]    [validation] Adding registry docker.io to distribution.komga.docker with username '***'
[INFO]  Project version set to 0.162.0
[INFO]  Release is not snapshot
[INFO]  Timestamp is 2023-03-03T09:31:51.906Z
[INFO]  HEAD is at cb332a6
[INFO]  Platform is linux-x86_64
[INFO]  dry-run set to false
[INFO]  Generating changelog
[INFO]  Extracting issues from changelog
[INFO]  Storing issues: build/jreleaser/release/issues.txt
[INFO]  Storing changelog: build/jreleaser/release/CHANGELOG.md
[INFO]  Storing changelog: ../CHANGELOG.md
[INFO]  Cataloging artifacts
[INFO]    [sbom] Cataloging is not enabled. Skipping
[INFO]  Calculating checksums for distributions and files
[INFO]    [checksum] build/libs/komga-0.162.0.jar.sha256
[INFO]  Signing distributions and files
[INFO]    [sign] Signing is not enabled. Skipping
[INFO]  Deploying Maven artifacts
[INFO]    [maven] Deploying is not enabled. Skipping
[INFO]  Uploading distributions and files
[INFO]    [upload] Uploading is not enabled. Skipping
[INFO]  Releasing to https://github.com/***/komga
[INFO]   - uploading checksums_sha256.txt
[INFO]   - uploading komga-0.162.0.jar
[INFO]   - uploading komga-0.162.0.jar.sha256
[WARN]  Could not update release v0.162.0 with discussion category Announcements
[INFO]  Marking 2 issue(s) as released
[INFO]  Preparing distributions
[INFO]    - Preparing komga distribution
[INFO]      [docker] preparing komga distribution
[INFO]  Packaging distributions
[INFO]    - Packaging komga distribution
[INFO]      [docker] packaging komga distribution
[INFO]      [docker]  - docker.io/***/komga:latest
[INFO]      [docker]  - docker.io/***/komga:0.162.0
[INFO]      [docker]  - docker.io/***/komga:0.x
[INFO]      [docker]  - ghcr.io/***/komga:latest
[INFO]      [docker]  - ghcr.io/***/komga:0.162.0
[INFO]      [docker]  - ghcr.io/***/komga:0.x
[INFO]  Publishing distributions
[INFO]    - Publishing komga distribution
[INFO]      [docker] publishing komga distribution
[INFO]      [docker] setting up repository ***/komga-docker
[INFO]    Announcing release
[INFO]      [announce] Announcing is not enabled. Skipping
[INFO]  Writing output properties to build/jreleaser/output.properties
[ERROR] JReleaser failed after 01:00 m

Any idea what happened ?
The release was created on Github, the files were attached.
However the Docker images were not pushed to the registries, and the closed issues were not commented upon,

The value of packagers.docker.repository.active is not propagated to distributions.<name>.docker.registry.active.

🎉 This issue has been resolved in v1.5.1 (Release Notes)


Remove code: No need to execute smart contract before tx admission into mempool

🌟 Feature Request
📝 Description
A transaction will be executed twice in the current implementation: 1) before mempool admission, 2) block validation (i.e., when it is to be included in the next block).
I don't find it necessary to execute the smart contract twice, and reducing it to one contract execution should help improve the sequencer's performance.
I suggest removing the code of pre-mempool contract execution.
I did remove the related code and found the zkSync sequencer was still functioning.
🤔 Rationale
Benefits

Reducing overall computation overhead and improving performance

📋 Additional Context
The code for executing a contract before mempool is in the following link. It is also the code that has been suggested to be removed.
https://github.com/matter-labs/zksync-era/blob/761bda19844fb3935f8a57c47df39010f88ef9dc/core/node/api_server/src/tx_sender/mod.rs#L378

Jmm


Entity Concerns

for the entity of each model to work, it's required that the properties of each value be defined. to me it's of a huge concern, cus if I have a database table with a large amount of columns, I have to define all. Also which I do a joining, I also have to define all in the entity. also the make's the entity a kind of tedious task.
there should be a lighter mode of calling all properties without having to define everything.

In order for the Entity to be saved back to the database properly, it has to know what fields are part of it. I was trying to think of a solution, and the model has $fillable, which I imagine you'll also have a problem with :) but it doesn't include all of the fields that might need to be saved so I'm not 100% sure that will work. Will need to look back through the code this evening.
However, the pragmatic solution is, instead of trying to change things to save 5 minutes of typing, you spend a couple of hours to automate it for the future. I've already started that process for anyone using CI4.

looking at Vulcan,  it still does not solve it.  especially when doing table joining (the basic extent have tried implementing to).  like you said,  a code review would be better to proffer a better work around.

I'm not able to come up with a good solution, and I don't believe there is one.

The model's $allowedFields variable is there to protect against mass assignment errors, so that won't be going anywhere, so you'll already be typing the fields out.
There has to be some way to save the entity back to the database, so it would have to know what fields to try and save. If we were to remove those fields, and rely on the $fillable array, we would potentially miss some data on the way back in.
While most of the time, we could rely on $allowedFields to get the fillable names, it is possible to turn off the check that removes fields not in the $allowedFields array, and if the Entity itself doesn't know about it there's no way to save that to the model.

The Entity class is an optional class on top of the database layer, anyway, so no one's forcing you to use it. You could always create your own base entity class if you wanted, tailored to your workflow.
I looked at how the largest database system that I know of that uses a similar pattern as this system does things (Doctrine) and the entity class has to define each field type as well.

Ok, @lonnieezell , i would not give up on it. would make some research on it


Add generic parsing to selfhost

Now allows us to parse functions like:
function b <U> (input: U) {

}

Hello!
One or more of the commit messages in this PR do not match the Jakt code submission policy, please check the lint_commits CI job for more details on which commits were flagged and why.
Please do not close this PR and open another, instead modify your commit message(s) with git commit --amend and force push those changes to update this PR.

Wait what the heck happened here? It says that I merged an unrelated commit 1addd7b into the main branch, but the actual code from this PR disappeared?

Also, I didn't merge 1addd7b, JT did. :thonk:


cargo test --all does not compile

$ cargo test --all
   Compiling num-rational v0.1.40
   Compiling rayon-futures v0.1.0 (file:///Volumes/VM2/rayon/rayon-futures)
error[E0432]: unresolved import `rayon_core::internal`
  --> rayon-futures/src/lib.rs:14:17
   |
14 | use rayon_core::internal::task::{Task as RayonTask, ScopeHandle, ToScopeHandle};
   |                 ^^^^^^^^ Could not find `internal` in `rayon_core`

I'm using latest master fea384b4b9b5365741cb9804ca65d418c54945f2 and rustc 1.23.0-nightly (fa26421f5 2017-11-15).

Yes, rayon-futures uses unstable features in rayon-core, which we've intentionally made hard. See this section in the README.  It should work with RUSTFLAGS as described there.


MNT: Re-rendered with conda-build 3.17.8, conda-smithy 3.2.13, and co…

…nda-forge-pinning 2019.02.11
This adds a needed pinning for librsvg.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


+args.txt => implement comment function to disable arguments

Provide us a use case of the feature
How could the user invoke the new function?
add the # before the arguments similar to text editor comment addition.
Which problem would it solve?
prevent every arguments provided in the args.txt file, this function needed for session case, if needed can be executed or not silent the argument.
Describe the solution you'd like
add the # before the arguments similar to text editor comment addition. I'm requesting this because I can add the arguments in one go and can use comment case to disable certain argument for particular run.
If the feature request is accepted, would you be willing to submit a pull request?
Yes, but need an example to work as I'm not an programmer.

Yes, this would be a nice feature!
We use fromfile_prefix_chars for the +args.txt feature, hence it may suffice to override ArgumentParser.convert_arg_line_to_args() to implement this feature.

As mentioned in #1625, besides allowing lines to be marked as comments, please also allow blank lines (currently, blank lines will cause an error). Comments and blank lines will make the config files much more readable and editable.

Can you elaborate on the feature your requested @bachig26.
Is anyone working on this feature or should I start?

Can you elaborate on the feature your requested @bachig26.

for example, in python you use "#"and in js and swift you use "//" to indicate the line as comment.
this function is not available for executing the argument file (args.txt), because currently even if you add # or // to a line in args.txt, while reading the file, the # or // is also read as a character like a profile name or whatsoever... it doesn't think as --fast-update, because fast update is coded to read.
In this, I want to have a comment condition or function added for argument file, so I can hide the commands or profile from executing while it reads for execution.
please let me know if this clarifies you.

Is anyone working on this feature or should I start?

I think no one is working on it, as I'm also not good at programming to work this....
I think help wanted label needed to be added.

Yes, it is all clear what exactly you are requesting.
I will start working on it.

@Hassibayub If you're working on this, an additional important feature, as mentioned above, is to also allow blank lines (currently, blank lines will cause an error).

Thank you for pointing this out, @EveryDirection.
One thing you guys can help me with is sharing your dummy args.txt file so I can get a better idea of the input files.
@bachig26

One thing you guys can help me with is sharing your dummy args.txt file so I can get a better idea of the input files.

this is my two args.txt file that I use, one with --fast-update (args_bachfu.txt) and other without --fast-update (args_bach.txt):

args_bach.txt
args_bachfu.txt

Requested feature added and pending pull request.
This kind of file now works.
<Blankline>
# foo
cristiano
--no-video-thumbnails
--no-metadata-json
--no-compress-json # this is test comment
--no-profile-pic # this is another comment
--dirname-pattern=cristiano
--count=10

thank you for this, this is what i needed too


Prince

https://about.sainsburys.co.uk/

@mapimp84 Your comment is meaningless to us. Closing the issue.


Help with duplicated channels

Hi Elbowz,
I have some duplicate channels in my list (same channel but different quality), and I'd like to get epg for all of them, but your script right now is only keeping the ultimate channel of those duplicated in the .config.
Can you help me?
Thanks and regards

HI @jorges4lu2 and sorry for the delay, but I had advised you :)
The new release add the possibility to append new xml element/tag.
So, for example, you can add to your channel the tag:
<display-name lang="en">Latino: Discovery Kids HQ</display-name>

Adding your json config:
"add": [
        {
          "xpath": ".",
          "xml": "<display-name lang=\"en\">Latino: Discovery Kids HQ</display-name>"
        }

See it.config.json for more example.
I hope it works like expected...

Hi @elbowz,
This is simply AWESOME!!!!!!!
What a great surprise I got this morning on my email. You made my day bro.
I just tested the new version and is working perfectly.  This I exactly what I needed.
Thank you so much for all your work. Greatly appreciated.

ahahah...glad to know you still need it...and It works :)


gaxios throw error in onSync of requestsync endpoint

I've deployed to firebase on step 3, but see a web interface with no device.
Viewing logs of requestsync in firebase see next:
requestsync
{ Error: Requested entity was not found.
    at Gaxios._request (/workspace/node_modules/googleapis/node_modules/gaxios/build/src/gaxios.js:85:23)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  response:
   { config:
      { url: 'https://homegraph.googleapis.com/v1/devices:requestSync',
        method: 'POST',...

Any ideas ho to solve this?

Receiving the same issue


Could you release 1.8.0 on github

Hi, I'm the maintainer of this nushell plugin on nixos.
Thank you so much for making this!
On nixos we rely on building from source, and it looks like 1.8.0 is not tagged on github.
When you have a moment if you could tag it, I could update this in nixpkgs.
Thanks again for making this!

Of course! Thank you for the kind words. Sorry I forget to tag it. Should be tagged and pushed now.


Sorting children elements

Hi
My XML document has a set of nodes in it that are based on date. So each element is named like this:
WYYYYNNDD
Eg:
W20151016
Each node has it's data inside. As users use my application and read / save a variety of data, the ordering of the content gets out of synch. This is not a real issue as such but if you were to manually view the XML file, it would be difficult to follow.
So could we please add a generic SortChildren(bAtoZ) etc that would sort all the top level children of the node based on their names?
Thanks for considering.
Andrew

It's a good idea, but that sort of utility is well out of the "tiny" direction of the API surface.


Add AgX tone mapping

This adds AgX tone mapping as an option, mostly for comparison and because it's now easy. It has some great improvements over ACES, but for e-commerce it's still far too desaturated. I've also added it as an option in our editor, and made Commerce the editor's default, in preparation for it becoming MV's default in our next major release (which won't be this one). I also added an example to make comparison easy. FYI @donmccurdy @mrdoob.

Two notes (feel free to integrate or ignore them).

ACES is a film industry standard that is widely used...

I think this might be unnecessarily generous to ACES Filmic. While various ACES standards are commonly used in the film industry for working with imagery, I do not believe the ACES Filmic Tone Mapping is one of them. How it became prevalent in realtime, I'm not sure.

[AgX] has the same film-centric drawback of significant desaturation.

Note that introduction of Looks providing (among other things) contrast choices will be a high priority, and I consider it an essential part of supporting AgX. Not sure if Looks are factored into your evaluation?

Thanks for the info! I'll try to clean up the language a bit. I don't know anything about Looks - what can they do? A tone mapper is already an arbitrary function, so if AgX encompasses another arbitrary function called a Look, then I'm not sure what AgX specifies. Any info you have would be appreciated.

@donmccurdy In particular, if you know of any way to construct a "Look" that would make AgX similar to my Commerce tone mapper, that would be very interesting.

I'll borrow some phrasing from Romain —

... a bigger problem I think is that the tone mapper is often used as a look. Instead the tone mapper should do as little as possible (compress the dynamic range, hopefully with knobs) but the look should come from color grading.

I think AgX and Commercial are alike in trying to do less, and that's a good thing.
Troy has described the default Look from AgX this way:

... the default is an attempt to pack high picture signal density, which means that folks will probably want to push the “contrast” ever so slightly; it’s impossible to go the other way.

In the case of AgX, Looks provide some simple color grading presets. I'm unsure, so far, what the pros/cons of having such presets are, compared to exposing a full color grading API.
As for constructing a Look with less desaturation in AgX, I suspect that the "Punchy" or higher contrast looks would move in that direction, but we haven't added them to three.js yet.

Gotcha, so more of a semantic distinction. Is there a "standard" way to apply contrast, or is that curve another black art like tone mapping?

There exists, at least, an "easy" way:
https://github.com/mrdoob/three.js/blob/208f5e48adb750e15c4fb5167f492b414d8bd5bf/examples/jsm/shaders/BrightnessContrastShader.js#L46-L50
AgX does this a bit differently, see the approximation in https://github.com/google/filament/pull/7236.
Whatever the color grading operation, the decision about when to apply it is important. From Real-time Rendering, 4th Ed:

When real-time applications first adopted color grading, the display-referred approach was predominant. However, the scene-referred approach has since been gaining traction due to its higher visual quality.

With AgX, the Look is applied to a log encoding, before encoding to the output color space. I don't understand the practice of tone mapping and color grading design much beyond that, I'm afraid.

Yeah, that's looking pretty black art 🥲. I wonder if applying the Punchy look would be a good default, or at least if you know your output is sRGB? Looks simple enough.

Would need criteria to define "a good default", I think. I'm inclined toward using 'base contrast', as Blender does, as the starting point for new projects. But perhaps knowing the default would be replacing ACES Filmic in existing projects could change the situation. And I'm less in touch with the e-commerce discussions.