public.ecr.aws/datadog/lambda-extension only has ARM images for versions >=11

We currently use the datadog-lambda-extension to monitor our containerized x86 architected lambdas (base image is FROM amazon/aws-lambda-python:3.8)
We have been successfully following installation instructions to import the lambda extension into our Docker image from public.ecr.aws/datadog/lambda-extension
COPY --from=public.ecr.aws/datadog/lambda-extension:<TAG> /opt/extensions/ /opt/extensions

However, trying to use any lambda-extension version >= 11 currently results in the lambda failing with the below error:
Error: fork/exec /opt/extensions/datadog-agent: exec format error Extension.LaunchError
Running docker inspect on the lambda-extension images in public.ecr.aws/datadog/lambda-extension reveals that lambda-extension versions 1-10 are amd64 (x86) while versions >=11 has changed to arm64. Guessing this architecture incompatibility between the extension and our lambda is causing the error.
Is there a ecr repo where we can pull x86 compatible lambda-extension images? Also recommend fixing the architecture tags for the public ecr repo.

Whoops, this is user error, my bad. images >= 11 have both architectures bundled, the default is just arm for >=11.
For others benefits - to make sure the correct public.ecr.aws/datadog/lambda-extension image is pulled just make sure to specify your base image's arch when building
ex.
docker build --platform "linux/amd64" .

I could have confirmed this my pulling the manifest file from the public image
docker manifest inspect --verbose public.ecr.aws/datadog/lambda-extension:18

Feel free to close this - but might still be worth clarify this gotcha  in the container install documentation on https://docs.datadoghq.com/serverless/installation for engineers like myself who normally look to tagging to take care of different architecture installs.  Did a draft of what this might look like for just the python instructions here - https://github.com/DataDog/documentation/pull/12972

This issue was discussed here:
https://github.com/DataDog/documentation/pull/12972
The issue was that when a Docker image is built on an ARM-based machine (e.g. a Mac with an Apple Silicon chip), Docker will default to building an ARM image, which won't work properly for a x86_64 Lambda function. The solution was to override the default when building the image.

I would like to add a comment for those who might find this issue searching for help.
I currently build & deploy my lambda containers using Terraform which does not allow me to specify the platform I build the container easily. In order to build the container and pull the appropriate datadog-layer platform, I did the following in the Dockerfile
# install datadog lambda extension
FROM --platform=linux/x86_64 public.ecr.aws/datadog/lambda-extension:latest AS dd

# base lambda image
FROM --platform=linux/x86_64 public.ecr.aws/lambda/python:3.9

# copy over the datadog extension into our lambda
COPY --from=dd /opt/extensions/ /opt/extensions

# ensure libs are up to date
RUN : \
    && yum -y update --security \
    && yum clean all \
    && rm -rf /var/cache/yum \
    :

# copy over the lambda code
WORKDIR ${LAMBDA_TASK_ROOT}
COPY ./*.py ${LAMBDA_TASK_ROOT}
COPY requirements.txt ${LAMBDA_TASK_ROOT}

# install the lambda code requirements
RUN : \
    && python3.9 -m pip install -r requirements.txt --target "${LAMBDA_TASK_ROOT}" \
    && python3.9 -m pip install datadog-lambda --target "${LAMBDA_TASK_ROOT}" \
    &&:

# set dd extension
CMD ["datadog_lambda.handler.handler"]

I'm doing this on a Mac M1 and works great for deploying x86_64 arch lambdas


What is full_sparse_dict_feature_with_label.tfrecords

I want to change the dataset to myself dataset ,so I want to  what full_sparse_dict_feature_with_label.tfrecords is.
Thanks

Hi @MargerA-ops ,
Thanks for reaching out.
The "full_sparse_dict_feature_with_label.tfrecords" file is the subset of training data used for speeding up training.
For more information, please refer to section 4.2 in our supplementary material.
Best,
Dat

Thanks for lot
dathuynh @.***> 於 2022年1月11日 週二 13:00 寫道：

Hi @MargerA-ops https://github.com/MargerA-ops ,
Thanks for reaching out.
The "full_sparse_dict_feature_with_label.tfrecords" file is the subset of
training data used for speeding up training.
For more information, please refer to section 4.2 in our supplementary
material https://hbdat.github.io/pubs/suppmat_ssmll_final.pdf.
Best,
Dat
—
Reply to this email directly, view it on GitHub
https://github.com/hbdat/cvpr20_IMCL/issues/5#issuecomment-1009600609,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASJOWBZJ627UISTTPYZNGWTUVO2OTANCNFSM5LN7WCKA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>

Thank for your help
YOU XIAN LIN @.***> 於 2022年1月15日 週六 14:12 寫道：

Thanks for lot
dathuynh @.***> 於 2022年1月11日 週二 13:00 寫道：

Hi @MargerA-ops https://github.com/MargerA-ops ,
Thanks for reaching out.
The "full_sparse_dict_feature_with_label.tfrecords" file is the subset of
training data used for speeding up training.
For more information, please refer to section 4.2 in our supplementary
material https://hbdat.github.io/pubs/suppmat_ssmll_final.pdf.
Best,
Dat
—
Reply to this email directly, view it on GitHub
https://github.com/hbdat/cvpr20_IMCL/issues/5#issuecomment-1009600609,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ASJOWBZJ627UISTTPYZNGWTUVO2OTANCNFSM5LN7WCKA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>


[WIP] Migration cleanup

This fixes #734 by adding all migrated files to a list that the project will clean on load. We can't just delete the xprojs and project.json files after running dotnet migrate because other projects might depend on those files.
Tagging @dotnet/project-system @jinujoseph for review. /cc @piotrpMSFT.
@MattGertz, I'm not sure if this scenario meets the bar for escrow. The scenario is that after migration, we currently leave the old project.json and .xproj files on disk, as well as backing them up. This adds deletion of those files after migration is complete. If you think that meets the bar, I'll retarget to dev15-rc2 and add the full escrow template.

I don't think there's a compelling reason to push this into RC.2 -- RC.3 would be fine.

@dotnet-bot retest please
Jenkins was having issues when this was submitted.

@dotnet-bot retest this please

If I understand what you've implemented, this is what happens:

One or more projects are upgraded. Projects that are upgraded, have their project.json and xproj files added to a list of files to be cleaned up.

On the first load of any C# CPS project, any files that are in this list are cleaned up.

Repeated for the life of the Visual Studio process.

This logic does not seem right to me - while at first glance it appears that 1 and 2 are related - they aren't. There's no guarantee that 2 actually follows 1, or that a project loaded in 2 even occurred due to 1.
Given these are observable changes to the user (these files will end up in their unstaged changes), I think you need to think about these situations:

What if I don't upgrade all projects at once? I can selectively choose to only upgrade 1 project in a solution, if these files are needed by other projects - what happens then?

What if I upgrade via the command-line where a CPS project won't ever be loaded?

What if after an upgrade, a CPS project is never successfully loaded. While that might be hard to imagine now, in a future version - the support for these projects might be not installed by default, causing these projects to be converted but remain unloaded.

Maybe we can Skype tomorrow afternoon your time and discuss the problem you're trying to solve, and see if there's an a approach we can take that doesn't require 2.

Once transitive refs are supported by migration, we'll no longer need to go this roundabout method of removing the files, and we'll be able to just remove them after migration. Closing this.


Controller configuration & port loopback error

Hi,
I am trying to benchmark switchML on a Tofino switch(APS Networks BF2556X-1T-A1F) and I'm having some difficulty running the controller. I think I have managed to compile the P4 program and deploy it on the switch. Then when I'm trying to execute the controller, I'm getting the following output from the python switchml.py script:
$ python switchml.py
Subscribe attempt #1
Subscribe response received 0
Binding with p4_name SwitchML
Binding with p4_name SwitchML successful!!
Received SwitchML on GetForwarding on client 0, device 0
Setting 49 front panel ports in loopback mode
Unexpected error. Stopping controller.
Terminated

and the bf_switchd throws the following errors:
2022-03-09 16:33:56.086991 BF_PLTFM ERROR - bf_pm_pre_port_add_cfg_set: Unsupported speed dev : 0 : front port : 41/0 port-cfg-nlanes:4 invalid channel:1
2022-03-09 16:33:56.087064 BF_PM ERROR - bf_pm_port_add_with_lanes:4897 Error in setting pltfm pre-port-add cfg for dev : 0 : front port : 41/0 (128) :  cfg-speed:100g n-lanes:4 err: Invalid arguments (3)
2022-03-09 16:33:56.087094 BF_PAL ERROR - Unable to add port for dev : 0 : front port : 41/0 (128) : Invalid arguments (3)
2022-03-09 16:33:56.087122 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.087210 BF_PLTFM ERROR - bf_pm_pre_port_add_cfg_set: Unsupported speed dev : 0 : front port : 45/0 port-cfg-nlanes:4 invalid channel:1
2022-03-09 16:33:56.087237 BF_PM ERROR - bf_pm_port_add_with_lanes:4897 Error in setting pltfm pre-port-add cfg for dev : 0 : front port : 45/0 (132) :  cfg-speed:100g n-lanes:4 err: Invalid arguments (3)
2022-03-09 16:33:56.087262 BF_PAL ERROR - Unable to add port for dev : 0 : front port : 45/0 (132) : Invalid arguments (3)
2022-03-09 16:33:56.087286 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.087364 BF_PLTFM ERROR - bf_pm_pre_port_add_cfg_set: Unsupported speed dev : 0 : front port : 37/0 port-cfg-nlanes:4 invalid channel:1
2022-03-09 16:33:56.087389 BF_PM ERROR - bf_pm_port_add_with_lanes:4897 Error in setting pltfm pre-port-add cfg for dev : 0 : front port : 37/0 (136) :  cfg-speed:100g n-lanes:4 err: Invalid arguments (3)
2022-03-09 16:33:56.087412 BF_PAL ERROR - Unable to add port for dev : 0 : front port : 37/0 (136) : Invalid arguments (3)
2022-03-09 16:33:56.087434 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.343103 BF_PLTFM ERROR - bf_pm_pre_port_add_cfg_set: Unsupported speed dev : 0 : front port : 33/0 port-cfg-nlanes:4 invalid channel:1
2022-03-09 16:33:56.343144 BF_PM ERROR - bf_pm_port_add_with_lanes:4897 Error in setting pltfm pre-port-add cfg for dev : 0 : front port : 33/0 (144) :  cfg-speed:100g n-lanes:4 err: Invalid arguments (3)
2022-03-09 16:33:56.343174 BF_PAL ERROR - Unable to add port for dev : 0 : front port : 33/0 (144) : Invalid arguments (3)
2022-03-09 16:33:56.343201 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.343252 BF_PAL ERROR - Unable to get the front port handle for dev : 0 : dev_port : 148 : Invalid arguments (3)
2022-03-09 16:33:56.343283 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.343341 BF_PAL ERROR - Unable to get the front port handle for dev : 0 : dev_port : 152 : Invalid arguments (3)
2022-03-09 16:33:56.343370 BF_BFRT ERROR - BF_RT_SERVER:write_entry:3111 Table Add failed table:$PORT Invalid arguments
2022-03-09 16:33:56.599107 BF_PLTFM ERROR - bf_pm_pre_port_add_cfg_set: Unsupported speed dev : 0 : front port : 29/0 port-cfg-nlanes:4 invalid channel:1
2022-03-09 16:33:56.599154 BF_PM ERROR - bf_pm_port_add_with_lanes:4897 Error in setting pltfm pre-port-add cfg for dev : 0 : front port : 29/0 (160) :  cfg-speed:100g n-lanes:4 err: Invalid arguments (3)

....

(Continues similarly for the other ports)

I expect this to have something to do with the exact configuration of my switch maybe? I have also included the switchml.log file.
Controller Log:
switchml.log
Environment:

Tofino Switch: APS Networks BF2556X-1T-A1F
SDE: 9.6.0
Python: 3.8.10

thank you,

Hello, thanks for engaging with SwitchML.
This issue is due to your switch having a Tofino chip with only 2 pipes instead of 4.
In a single pass through tofino,  we can process up to 256B of payload data per packet. With a 4 pipes switch we have an additional mode called folded-pipe, that sets all the ports in pipes 1,2,3 (plus some internal special ports) in loopback mode. In this way you can only use front panel ports connected to pipe 0 to connect your servers, and every packet is recirculated in order to go through pipe 0->1->2->3 in sequence. In this way a packet goes 4 times through tofino and we can process 1024B per packet.
Your tofino chip has only 2 pipes and cannot support the folded pipe mode, which was enabled by default in the controller (which is causing the error you saw).
Please pull the last commit I just pushed. Now the controller does not enable the folded pipeline by default. This allows to use 2-pipes switches.  The folded pipe mode is now optionally enabled using the enable-folded-pipe argument when starting the controller.
Let me know if this solves your issue.

Hi, thank you for going over my issue and pushing the changes, I highly appreciate it.
The changes did indeed solved the issue and I was able to run the controller successfully and program the ports.


Build from source

With this PR we can build k8s binaries from a branch in any repository.
We also make sure we remove files that do not need to be in the snap package.

I dont understand why you wouldn't want to use the binaries we already build from source on launchpad? You can just easily use stage-snaps at that point and be pulling from the same builds CDK runs.

You can just easily use stage-snaps at that point and be pulling from the same builds CDK runs

That would be fantastic! I am +1.
However based on a roadmap item we picked up these days we may want to push some patches to MicroK8s only. Let's sync up tomorrow.

That would be fantastic! I am +1.

Great!

However based on a roadmap item we picked up these days we may want to push some patches to MicroK8s only. Let's sync up tomorrow.

Sure! I've got a PoC for including upstream patches into our builds, I think we can expand that to allow microk8s specific patches as well.


Allow matching without creation

Critical Changes
Changes
Issues Closed

@Scarrien,
I fixed the Summit_Event__c object for meta data getting whipped out. I was able to build this branch and register. I think we should have someone besides me to test for functionality. Like @jimhubert or @walt0019.
Thanks,
Thad


How to use it for inference ?

i already do
finetune_forecast_trainer.evaluate(test_dataset)
but how i use it to make inferention ?

Try: finetune_forecast_trainer.predict(test_dataset)
I believe the first element of the result should be the predictions.

Try: finetune_forecast_trainer.predict(test_dataset)
I believe the first element of the result should be the predictions.

Thanks

@CrasCris The basic process is as follows:
# define train and test data
train_data = # pandas dataframe
test_data = # pandas dataframe

# define a preprocessor
tsp = TimeSeriesPreprocessor(
        timestamp_column=...
        scaling=True,
    )

# train the preprocessor
tsp.train(train_data)

# define a forecasting pipeline
forecast_pipeline = TimeSeriesForecastingPipeline(
    model=model,
    timestamp_column=...
    feature_extractor=tsp, # note, needed if we want to inverse scale internally
    inverse_scale_outputs=True,
)

# forecast using the pipeline on the scaled test data
forecasts = forecast_pipeline(tsp.preprocess(test_data))

You can find a similar example in the tests: https://github.com/ibm-granite/granite-tsfm/blob/main/tests/toolkit/test_time_series_forecasting_pipeline.py#L113

The difference between this approach using TimeSeriesForecastingPipeline and the method outlined in the ttm tutorial notebook  is confusing.  What's the difference?
The approachin in the Zeroshot evaluation section of the ttm notebook ran immediately whereas this one has been running for 5 minutes on my machine.

@matsuobasho In the notebook you are passing in a pytorch dataset and a tensor is returned. TimeSeriesForecastingPipeline is meant to work with a pandas dataframe, embed more of the steps, and return a pandas dataframe.
If you can share your code for zeroshot evaluation using the pipeline, I will take a look.

@wgifford , thanks for the offer.
# Forecasting parameters
context_length = 512
forecast_length = 96
fewshot_fraction = 0.05

timestamp_column = "Timestamp"
id_columns = []
target_columns = ["y"]
split_config = {
                'train': 0.7,
                'test': 0.2
                }

column_specifiers = {
    "timestamp_column": timestamp_column,
    "id_columns": id_columns,
    "target_columns": target_columns,
    "control_columns": [],
}

tsp = TimeSeriesPreprocessor(
    **column_specifiers,
    context_length=context_length,
    prediction_length=forecast_length,
    scaling=True,
    encode_categorical=False,
    scaler_type="standard",
)

train_dataset, valid_dataset, test_dataset = tsp.get_datasets(
    df, split_config, fewshot_fraction=fewshot_fraction, fewshot_location="first"
)

TTM_MODEL_REVISION = "main"
zeroshot_model = TinyTimeMixerForPrediction.from_pretrained("ibm/TTM", revision=TTM_MODEL_REVISION)

from tsfm_public.toolkit.time_series_forecasting_pipeline import TimeSeriesForecastingPipeline
tsp.train(train_dataset)

# define a forecasting pipeline
forecast_pipeline = TimeSeriesForecastingPipeline(
        model=zeroshot_model,
        timestamp_column=timestamp_column,
        id_columns=id_columns,
        target_columns=target_columns,
        feature_extractor=tsp,
        explode_forecasts=False,
        inverse_scale_outputs=True,
    )

# forecast using the pipeline on the scaled test data
forecasts = forecast_pipeline(tsp.preprocess(test_dataset))

I now get an error on the tsp.train step.
Also, you wrote that TimeSeriesForecastingPipeline works with dataframes, but that's not what I see from the test  you referenced.  It appears it also uses the ForecastDFDataset type.  Let me know if I'm misinterpreting.  Thanks.

@matsuobasho Sorry for the very slow response.
The issue was that tsp.train(train_dataset) as well as the forecast_pipeline() require pandas dataframes as input -- no a torch dataset. Here is a working example is here:

# Forecasting parameters
context_length = 512
forecast_length = 96
fewshot_fraction = 0.05

timestamp_column = "Timestamp"
id_columns = []
target_columns = ["y"]
split_config = {"train": 0.7, "test": 0.2}

column_specifiers = {
    "timestamp_column": timestamp_column,
    "id_columns": id_columns,
    "target_columns": target_columns,
    "control_columns": [],
}

tsp = TimeSeriesPreprocessor(
    **column_specifiers,
    context_length=context_length,
    prediction_length=forecast_length,
    scaling=True,
    encode_categorical=False,
    scaler_type="standard",
)

from tsfm_public.toolkit.time_series_preprocessor import get_datasets, prepare_data_splits

train_df, valid_df, test_df = prepare_data_splits(df, context_length=context_length, split_config=split_config)

# this call also trains the time series preprocessor (tsp)
train_dataset, valid_dataset, test_dataset = get_datasets(
    tsp, df, split_config, fewshot_fraction=fewshot_fraction, fewshot_location="first"
)

TTM_MODEL_REVISION = "main"
zeroshot_model = TinyTimeMixerForPrediction.from_pretrained("ibm/TTM", revision=TTM_MODEL_REVISION)

from tsfm_public.toolkit.time_series_forecasting_pipeline import TimeSeriesForecastingPipeline

# define a forecasting pipeline
forecast_pipeline = TimeSeriesForecastingPipeline(
    model=zeroshot_model,
    timestamp_column=timestamp_column,
    id_columns=id_columns,
    target_columns=target_columns,
    feature_extractor=tsp,
    explode_forecasts=False,
    inverse_scale_outputs=True,
)

# forecast using the pipeline on the scaled test data
forecasts = forecast_pipeline(tsp.preprocess(test_df))

@CrasCris The basic process is as follows:
# define train and test data
train_data = # pandas dataframe
test_data = # pandas dataframe

# define a preprocessor
tsp = TimeSeriesPreprocessor(
        timestamp_column=...
        scaling=True,
    )

# train the preprocessor
tsp.train(train_data)

# define a forecasting pipeline
forecast_pipeline = TimeSeriesForecastingPipeline(
    model=model,
    timestamp_column=...
    feature_extractor=tsp, # note, needed if we want to inverse scale internally
    inverse_scale_outputs=True,
)

# forecast using the pipeline on the scaled test data
forecasts = forecast_pipeline(tsp.preprocess(test_data))

You can find a similar example in the tests: https://github.com/ibm-granite/granite-tsfm/blob/main/tests/toolkit/test_time_series_forecasting_pipeline.py#L113

When I use this to process my multi-channel dataframe, I encounter a KeyError: 'column name of the first channel' on the line forecasts = forecast_pipeline(tsp.preprocess(test_data)), but I can ensure that this column name exists in both train_data and test_data.

@NING-CSU
Thank you for the interest. Do the train / test dataframes have identical columns? Could you share a small example which reproduces the error?

Yes. The train / test dataframes were split from the same dataframe.
My code is as follows:
from tsfm_public.toolkit.time_series_preprocessor import TimeSeriesPreprocessor

# define train and test data
train_data = data.iloc[:-(672+512),:]# pandas dataframe
test_data = data.iloc[-(672+512):,:]# pandas dataframe

column_specifiers = {
    "timestamp_column": timestamp_column,
    "id_columns": [],
    "target_columns": target_column,
    "control_columns": [],
}

tsp = TimeSeriesPreprocessor(
    **column_specifiers,
    context_length=512,
    prediction_length=96,
    scaling=True,
    encode_categorical=False,
    scaler_type="standard",
)

# train the preprocessor
tsp.train(train_data)

# define a forecasting pipeline
forecast_pipeline = TimeSeriesForecastingPipeline(
    model=zeroshot_model,
    timestamp_column=timestamp_column,
    target_columns=target_column,
    explode_forecasts=True,
    freq="15T",
    id_columns=[],
    prediction_length=prediction_length,
    feature_extractor=tsp,  # note, needed if we want to inverse scale internally
    inverse_scale_outputs=True,
)

# forecast using the pipeline on the scaled test data
forecasts = forecast_pipeline(tsp.preprocess(test_data))
forecasts.head()

The error message is as follows:
KeyError                                  Traceback (most recent call last)
File ...\Lib\site-packages\pandas\core\indexes\base.py:3805, in Index.get_loc(self, key)
   3804 try:
-> 3805     return self._engine.get_loc(casted_key)
   3806 except KeyError as err:

File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc()

File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc()

File pandas\\_libs\\hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas\\_libs\\hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: '5036_lane_1'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[95], line 40
     27 forecast_pipeline = TimeSeriesForecastingPipeline(
     28     model=zeroshot_model,
     29     timestamp_column=timestamp_column,
   (...)
     36     inverse_scale_outputs=True,
     37 )
     39 # forecast using the pipeline on the scaled test data
---> 40 forecasts = forecast_pipeline(tsp.preprocess(test_data))
     41 forecasts.head()

File ...\granite-tsfm\tsfm_public\toolkit\time_series_forecasting_pipeline.py:318, in TimeSeriesForecastingPipeline.__call__(self, time_series, **kwargs)
    249 def __call__(
    250     self,
    251     time_series: Union["pd.DataFrame", str],
    252     **kwargs,
    253 ):
    254     """Main method of the forecasting pipeline. Takes the input time series data (in tabular format) and
    255     produces predictions.
    256
   (...)
    315
    316     """
--> 318     return super().__call__(time_series, **kwargs)

File ...\Lib\site-packages\transformers\pipelines\base.py:1302, in Pipeline.__call__(self, inputs, num_workers, batch_size, *args, **kwargs)
   1294     return next(
   1295         iter(
   1296             self.get_iterator(
   (...)
   1299         )
   1300     )
   1301 else:
-> 1302     return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)

File ...\granite-tsfm\tsfm_public\toolkit\time_series_forecasting_pipeline.py:100, in TimeSeriesPipeline.run_single(self, inputs, preprocess_params, forward_params, postprocess_params)
     97 model_outputs["prediction_outputs"] = torch.cat(accumulator, axis=0)
     99 # call postprocess
--> 100 outputs = self.postprocess(model_outputs, **postprocess_params)
    102 return outputs

File ...\granite-tsfm\tsfm_public\toolkit\time_series_forecasting_pipeline.py:479, in TimeSeriesForecastingPipeline.postprocess(self, input, **kwargs)
    477 # inverse scale if we have a feature extractor
    478 if self.feature_extractor is not None and kwargs["inverse_scale_outputs"]:
--> 479     out = self.feature_extractor.inverse_scale_targets(out)
    480     if kwargs["add_known_ground_truth"]:
    481         out = self.feature_extractor.inverse_scale_targets(out, suffix="_prediction")

File ...\granite-tsfm\tsfm_public\toolkit\time_series_preprocessor.py:596, in TimeSeriesPreprocessor.inverse_scale_targets(self, dataset, suffix)
    593 if suffix is not None:
    594     cols_to_scale = [f"{c}{suffix}" for c in cols_to_scale]
--> 596 col_has_list = [df[c].dtype == np.dtype("O") for c in cols_to_scale]
    598 def explode_row(df_row, name, columns):
    599     df = pd.DataFrame(df_row[columns].to_dict())

File ...\granite-tsfm\tsfm_public\toolkit\time_series_preprocessor.py:596, in <listcomp>(.0)
    593 if suffix is not None:
    594     cols_to_scale = [f"{c}{suffix}" for c in cols_to_scale]
--> 596 col_has_list = [df[c].dtype == np.dtype("O") for c in cols_to_scale]
    598 def explode_row(df_row, name, columns):
    599     df = pd.DataFrame(df_row[columns].to_dict())

File ...\Lib\site-packages\pandas\core\frame.py:4102, in DataFrame.__getitem__(self, key)
   4100 if self.columns.nlevels > 1:
   4101     return self._getitem_multilevel(key)
-> 4102 indexer = self.columns.get_loc(key)
   4103 if is_integer(indexer):
   4104     indexer = [indexer]

File ...\Lib\site-packages\pandas\core\indexes\base.py:3812, in Index.get_loc(self, key)
   3807     if isinstance(casted_key, slice) or (
   3808         isinstance(casted_key, abc.Iterable)
   3809         and any(isinstance(x, slice) for x in casted_key)
   3810     ):
   3811         raise InvalidIndexError(key)
-> 3812     raise KeyError(key) from err
   3813 except TypeError:
   3814     # If we have a listlike key, _check_indexing_error will raise
   3815     #  InvalidIndexError. Otherwise we fall through and re-raise
   3816     #  the TypeError.
   3817     self._check_indexing_error(key)

KeyError: '5036_lane_1'


Docs: Correct description of streaming cursor

#13476

the documentation recommended not using the stream options for long queries.

That is an outdated comment and was targeted at the MMFiles storage engine, which is no longer supported. Unless you're using exclusive locks or so, you can safely ignore it and use streaming queries also for long running queries.

Remove MMFiles leftovers

As far as I understand, one should still not use streaming cursors with the RocksDB engine if a query sets OPTIONS { exclusive: true } for a modification query, because it will block all other queries involving the same collection. Are there additional cases?

[when stream is set to false, then] query results are either returned right away (if the result set is small enough), ...

Does that mean the query result is calculated on-the-fly as long as it fits within one cursor batch?

The description leaves open how this works in practice:

query statistics, warnings and profiling data will only be available after the query is finished.

Is this information contained in the last batch of results before the cursor is exhausted?

Needs to be backported to 3.7. Should also adjust 3.6 docs and distinguish between MMFiles and RocksDB.

As a synopsis, the main difference is as follows:
A query with stream: false will be immediately entirely executed on the server, and its result stored in memory (on the contacted coordinator, if in a cluster). All other resources will immediately be freed (like locks or RocksDB snapshots).
A query with stream: true will suspend execution of the query and return a batch of results as soon as batchSize many results were calculated. Depending on the query, this can mean that the first result will be available much faster, and that we don't need to store the entirety of the result in memory at any point in time. Though certain queries won't benefit from either; for example, a query that SORTs all its results (without using an index) and has no LIMIT will still collect everything in memory in order to sort it, before returning it peu à peu. But on the other hands, the query is not ended but just suspended, and still holds resources. For example, a RocksDB snapshot, which will prevent compactions to some degree. Writes to RocksDB will also be in memory until the query is committed. And writes will make RocksDB hold write locks on those documents, which will add some overhead to other (write) transactions and make other queries that try to write the same documents fail with conflicts until it has ended.
One other consequence of this behavior is that if a query fails, e.g. with a write conflict, a query with stream: false will report that error before returning any results. A streaming query may return some results up to the point where it encounters a conflict, which may render the already returned results meaningless as the transaction was aborted.

As far as I understand, one should still not use streaming cursors with the RocksDB engine if a query sets OPTIONS { exclusive: true } for a modification query, because it will block all other queries involving the same collection. Are there additional cases?

So the question is, when should we recommend or discourage the use of streaming cursors.
For queries returning at most a batchSize of results, it doesn't make a difference.
Especially read-only queries can benefit greatly from streaming cursors, unless they have to gather everything in memory anyways, e.g. due to a SORT, or COLLECT. And the cost for streaming cursors is low in that case.
For queries that write lots of documents and produce a lot of results, it may not be advisable to use a streaming cursor, though it may also have advantages.

[when stream is set to false, then] query results are either returned right away (if the result set is small enough), ...

Does that mean the query result is calculated on-the-fly as long as it fits within one cursor batch?

That is actually true in both cases. The difference is:

with stream: false, the result is completely calculated and the query ended before the first request is answered. If the result fits inside one batch, everything else is also cleaned up immediately. If the result doesn't fit in one batch, it is stored in memory attached to the cursor and can be fetched later, but the actual query is already finished and cleaned up.
with stream: true, the query is executed only as long as necessary to produce the first batch of results, and no more. If that means the query is exhausted, it and everything else is also cleaned up immediately. If it isn't, its execution is suspended until the next request arrives.

The description leaves open how this works in practice:

query statistics, warnings and profiling data will only be available after the query is finished.

Is this information contained in the last batch of results before the cursor is exhausted?

Yes, the final response, i.e. the one returning hasMore: false, contains this data.

@mchacki Please have a look at my response, whether I've layed out everything correctly.

Thank you @goedderz!

with stream: true, the query is executed only as long as necessary to produce the first batch of results, and no more. If that means the query is exhausted, it and everything else is also cleaned up immediately. If it isn't, its execution is suspended until the next request arrives.

Until the next batch is ready, or until the client requests the next batch from the cursor?
Is "first batch" defined as the first batchSize (1000) elements/documents?

Until the next batch is ready, or until the client requests the next batch from the cursor?

Until the client requests the next batch. Currently, there is no asynchronous execution while arangod waits for the next client request. Though that would be sleek.

Is "first batch" defined as the first batchSize (1000) elements/documents?

Yes. To clarify that:
When the request to create the (streaming) cursor arrives, the query is executed until it produced exactly batchSize many results, and/or the query is done. These are returned. If the query is done, that's it - no persistent cursor is created, and everything is cleaned up immediately. If the query can produce more results, its execution is suspended, and the batchSize many results we produced are returned, together with a cursor id to continue execution and retrieve more results later.

* > [when stream is set to false, then] query results are either returned right away (if the result set is small enough), ...

  Does that mean the query result is calculated on-the-fly as long as it fits within one cursor batch?

* The description leaves open how this works in practice:
  > query statistics, warnings and profiling data will only be available after the query is finished.

  Is this information contained in the last batch of results before the cursor is exhausted?

stream: false (which is the default) means that the entire query is executed on the server, and the results will be built up in memory. The query responds with its first batch once the entire query is finished and results have been stored in memory.
Everything will be returned in a single batch if the number of query results is less equal to the batch size (default: 1000). If there are more results, a cursor is automatically created on the server, and the client application can fetch more data by sending follow-up requests to the cursor API using the cursor's id (using the "id" attribute of the response).
Query statistics and warnings are available with the first response batch already, as the query was completely executed once the first batch becomes available.
Example:
curl -X POST http://<IP_ADDRESS>:8529/_api/cursor --data '{"query":"FOR i IN 9..0 RETURN 1 / i","batchSize":5,"profile":true}' --dump - --user "root:"; echo
HTTP/1.1 201 Created
X-Content-Type-Options: nosniff
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 404

{"result":[0.1111111111111111,0.125,0.14285714285714285,0.16666666666666666,0.2],"hasMore":true,"id":"27169809","extra":{"stats":{"writesExecuted":0,"writesIgnored":0,"scannedFull":0,"scannedIndex":0,"filtered":0,"httpRequests":0,"executionTime":2.593730023363605e-4,"peakMemoryUsage":38912},"warnings":[{"code":1562,"message":"in operator /: division by zero"}]},"cached":false,"error":false,"code":201}

stream: true means that the query is started on the server and can return its first batch as soon as it is available. Once there is data to fill a response batch, the server will respond it, and the query will be paused on the server until the next follow-up request comes in to fetch more data. This can go on as long as the query returns more data.
When it comes to query statistics etc., these are not known until the query is fully finished. With a streaming query this means that the statistics are returned with the batch that finishes the query. It can be the first batch if the query does not return more than this fits in a batch, or any of the follow-up batches. In any case, statistics will contained be the last batch in a streaming query.
Now an example for a streaming query:
 curl -X POST http://<IP_ADDRESS>:8529/_api/cursor --data '{"query":"FOR i IN 9..0 RETURN 1 / i","batchSize":5,"profile":true,"stream":true}' --dump - --user "root:"; echo
HTTP/1.1 201 Created
X-Content-Type-Options: nosniff
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 152

{"result":[0.1111111111111111,0.125,0.14285714285714285,0.16666666666666666,0.2],"hasMore":true,"id":"27169890","cached":false,"error":false,"code":201}

No statistics nor warnings available in the first batch. However, we got an "id" here, so we can query more data from the cursor:
curl -X PUT http://<IP_ADDRESS>:8529/_api/cursor/27169890 --dump - --user "root:"; echo
HTTP/1.1 200 OK
X-Content-Type-Options: nosniff
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 353

{"result":[0.25,0.3333333333333333,0.5,1,null],"hasMore":false,"cached":false,"extra":{"stats":{"writesExecuted":0,"writesIgnored":0,"scannedFull":0,"scannedIndex":0,"filtered":0,"httpRequests":0,"executionTime":19.554455741999845,"peakMemoryUsage":39072},"warnings":[{"code":1562,"message":"in operator /: division by zero"}]},"error":false,"code":200}

Now the query is done ("hasMore" is false), so the statistics and warnings are returned.
@Simran-B


Method to use subdirectories for the views

I created um method in my application for improve the organization of my views, I don't know if the code it's interesting for be used as internal resource of bottle library.
import bottle
from os import listdir
from os.path import dirname, realpath, join, isdir

def config_views_directory(all_subdirs=False):
    """ config the path of views """
    abs_app_dir_path = dirname(realpath(__file__))
    abs_views_path = join(abs_app_dir_path, 'views')
    index = 0

    # insert the path of views
    bottle.TEMPLATE_PATH.insert(index, abs_views_path)

    # verify the directory
    def process_directory(index, path):
        """ process the directory """
        for file_name in listdir(path):
            next_path = join(path, file_name)

            # verify the directory
            if isdir(next_path):
                index += 1

                # insert the path of views
                bottle.TEMPLATE_PATH.insert(index, next_path)

                # checks to process all subdirectories
                if all_subdirs:
                    index = process_directory(index, next_path)

        return index

    process_directory(index, abs_views_path)

One thing that I forgot comment it's that when I create new views files I take care to not create files with same names, even if they are in different subdirectories.
views
|----backend
----|----login.tpl
|----fronend
----|----front_login.tpl
@view('login')
def backend_login():
    pass

@view('front_login')
def frontend_login():
    pass


[TEST] TF broadcast tests taking 25+ mins on CI runs

Context
CI is taking a while (see #8552 and build#1384), and I am tracking short-term fixes to alleviate this.
Summary of the issue
The following test cases are located in tests/python/frontend/tensorflow/test_forward.py
test_forward_broadcast_args takes over 15 minutes to run 3 cases.
The parameters passed into _test_broadcast_args do not seem to be minimal:
    _test_broadcast_args((4, 1, 32, 32), [4, 8, 32, 32])
    _test_broadcast_args((6, 32, 32, 1), [6, 32, 32, 16])
    _test_broadcast_args((32, 32, 16), [6, 32, 32, 16])

test_forward_broadcast_to takes over 10 minutes to run 3 cases.
The parameters passed into _test_broadcast_to do not seem to be minimal:
    _test_broadcast_to((4, 1, 32, 32), [4, 8, 32, 32])
    _test_broadcast_to((6, 32, 32, 1), [6, 32, 32, 16])
    _test_broadcast_to_from_tensor((1))

Suggested actions

Consider using smaller input shapes for these tests.

Closing based on completion of https://github.com/octoml/tvm-roadmap/issues/5


chore(all): copyright headers in test lambda functions

The Lambda function source files for our unit and integration tests were missing some copyright/license headers - this PR addresses that.

AWS CodeBuild CI Report

CodeBuild project: githubautobuild-for-cdk-v2
Commit ID: 86cd279e3fa841dac3d203182a26616fd0b3e461
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Add CHANGELOG

@adamreichold

I've rebased and updated the changelog in preparation for v0.6. I browsed the history and I think have captured the newsworthy items not in @AndrewGaspar's prior work here. Comments certainly welcome.


IMPLICIT_GROUP_BY_FROM_ORDER_BY 'false' doesn't affect groupBy generation

IMPLICIT_GROUP_BY_FROM_ORDER_BY 'false' doesn't affect groupBy generation.

I tried to set IMPLICIT_GROUP_BY_FROM_ORDER_BY property to false, but already see extra columns in group by of generated SQL yet.
I setted this property by CriteriaBuilderConfiguration:
@Bean
public CriteriaBuilderFactory createCriteriaBuilderFactory() {
        CriteriaBuilderConfiguration config = Criteria.getDefault()
        .setProperty(ConfigurationProperties.IMPLICIT_GROUP_BY_FROM_ORDER_BY, "false");

        return config.createCriteriaBuilderFactory(entityManagerFactory);
}

and by criteriaBuilder.setProperty(String propertyName, String propertyValue):
public Page<Review> findAll() {
        CriteriaBuilder<Review> criteriaBuilder = criteriaBuilderFactory.create(entityManager, Review.class, ReviewMetaModel.ALIAS)
            .innerJoinFetch(ReviewMetaModel.getFromAlias(ReviewMetaModel.WEB_RESOURCE), WebResourceMetamodel.ALIAS)
            .leftJoinFetch(ReviewMetaModel.getFromAlias(ReviewMetaModel.LIKES), ReviewLikeMetamodel.ALIAS)
            .innerJoinFetch(ReviewMetaModel.getFromAlias(ReviewMetaModel.USER), UserMetaModel.ALIAS)
            .groupBy(
                ReviewMetaModel.ID,
                ReviewLikeMetamodel.getFromAlias(ReviewLikeMetamodel.REVIEW_ID),
                ReviewLikeMetamodel.getFromAlias(ReviewLikeMetamodel.ID)
            )
            .orderByDesc("SUM(case when " + ReviewLikeMetamodel.getFromAlias(ReviewLikeMetamodel.POSITIVE) + " = true then 1 else 0 end)");

        criteriaBuilder.setProperty(ConfigurationProperties.IMPLICIT_GROUP_BY_FROM_ORDER_BY, "false");

        List<Review> reviews = criteriaBuilder.getResultList();

        return new PageImpl<>(reviews);
    }

But in both cases I see generated SQL:
...
group by r1_0.id, l1_0.review_id, l1_0.id, r1_0.creation_date, r1_0.rating, r1_0.resource_id, r1_0.review_id, r1_0.text, r1_0.update_date, u1_0.id, r1_0.user_id, w1_0.id

Expected behavior

group by section in result SQL is: group by r1_0.id, l1_0.review_id, l1_0.id
Actual behavior

group by section in result SQL includes extra columns: group by r1_0.id, l1_0.review_id, l1_0.id, r1_0.creation_date, r1_0.rating, r1_0.resource_id, r1_0.review_id, r1_0.text, r1_0.update_date, u1_0.id, r1_0.user_id, w1_0.id
Steps to reproduce

Set com.blazebit.persistence.implicit_group_by_from_order_by property to false for CriteriaBuilderConfiguration or for CriteriaBuilder instance.
Specify group by expressions.
Invoke criteria for getting result.
Check generated SQL string.
Repeat steps without this setting and compare generated SQL.

Environment

Version: 1.6.9
JPA-Provider: Hibernate 6.2.5
DBMS: PostgreSQL 14.1
Application Server: Embedded Tomcat 10.1.10 (Spring Boot 3.1.1)

@beikov, thanks for the recomendation, I will try this. I don't have any results for a week with this filtration =). In this reason I found blaze-persistence (p.s. Realy perfect work, guys!).
Also I will check IMPLICIT_GROUP_BY_FROM_SELECT today evning.

@beikov, I just tried set IMPLICIT_GROUP_BY_FROM_SELECT property to CriteriaBuilderConfiguration and it works. Also I reread documentation and decide, that my bad.
I close this issue and thanks for your recommendation about my business query - it solved my question =)


Arguments of the layout reader

While working on #11473, it was not that straightforward to add support for Path in the I/O roundtrip for layouts because of the strange arguments from read_layout.
kind is supposed to be the name of the layout. According to the docstring, without the suffix.
path is supposed to be the folder in which the layout file is.
In reality, kind can be provided with or without suffix, and it can be the path to a layout file in which case the value provided in path is ignored. https://github.com/mne-tools/mne-python/blob/d740e47c8bbce81e6c0beda304748e606f20fc6c/mne/channels/tests/test_layout.py#L60-L61

I propose to add a new argument fname:
fname : path-like | str
    Either the path to a .lout or .lay file or the name of a built-in layout.
    c.f. Notes for a list of the available built-in layouts.

And make the list of built-in layouts include the suffix to reduce ambiguity.
And to deprecate both kind and path. It's definitely backward incompatible, but it would simplify this reader.

I think it's useful to be able to just say 'standard-1020' it whatever so that users don't have to know where the MNE built-in layouts are stored. Is there a way to keep that but otherwise do as you suggest?

And make the list of built-in layouts include the suffix to reduce ambiguity.

This point can be omitted, in which case fname is either the path to a layout or the name of a layout.

SGTM.


Foreget Password Option

Make interface to implement foreget password functionallity
when user clicks on foreget password, user will be prompted to The dialog box to enter registred email id. Ones user enters email id then validate it end make api call to our server as documented in api documentation.

i will try to do it


New package: ITerm2Images v1.0.0

Registering package: ITerm2Images
Repository: https://github.com/eschnett/ITerm2Images.jl
Created by: @eschnett
Version: v1.0.0
Commit: 7cb739b18e133aa9d1772e452c4196dd252c8ffe
Git reference: HEAD
Description: Inline graphics in the REPL for iTerm2

[noblock] This looks pretty exciting! Can you elaborate on “It also supports the tmux terminal multiplexer.” in the README?

[noblock] @goerz
# The tmux terminal multiplexer rewrites some escape sequences. See
# <https://iterm2.com/utilities/imgcat> for details.

In essence, you need to detect whether you are running under tmux, and if so, send a certain prefix ("\ePtmux;"), escape all escape characters, and send a certain postfix ("\e\\").


Increased RPC communication timeout to 20 secs

We have observed that in some cases DRM initialization takes longer than 10 seconds causing OpenCDM update() to fail with a timeout. Increasing the timeout to 20 seconds reduces the frequency of these errors.
Signed-off-by: Gurdal Oruklu<EMAIL_ADDRESS>

https://jenkins.code.rdkcentral.com/job/github-triggered/337/ started

That is a bit scary. It means that the communication channel (since it is a synchronous invoke) is blocked for potentially 20S. It looks like the initialization of the OCDM should be decoupled and report back after some time that it initialized correctly.
I assume this happens during the loading of the OCDM plugin ?

we are seeing this when creating a new DRM session with Playready 2.5. I'll see if we can increase the timeout value only in that particular case instead of increasing the generic timeout value.

If it is on crteating a session, the anser is probably even easier :-) Currently I know that the callback to report the License message is part of the Creation of the plugin (different from WideVine where this is really decoupled) I think this is a good time and reason to also in PlayReady decouple it. I guess this will drop the invoke time more than enough.

will try to find another way to handle the slow DRM_initialize() cases.


Sorting

Is sorting implemented by default? Seems like {{t.head}} has sort enabled by default, but nothing seems to happen. Here's my setup:
{{#light-table table class='table table-striped table-hover' as |t|}}
  {{t.head multiColumnSort=true}}
  {{t.body onRowClick=(action 'onclick')}}
{{/light-table}}

Looking at the demo app, it looks like you implement sorting yourself by sending the params to the server..

Yeah all data handling is up to you. The component just sets up everything for you so it's as easy as possible.

On this topic, is it best practice to sort / filter at the route or within the component containing the light-table?

The controller/route (little more hassle on the route).

Hmm..
I'm trying the below on the controller, but it sorts it when referencing sortedPortfolios outside the component (ie when I do an {{#each}} in the template, but the sort order will not update even though I am passing sortedPortfolios in as `{{pg-tables/portfolios model=sortedPortfolios onSort=(action "sortPortfoliosBy")}} ?
Can you see anything I'm doing wrong?

    sortedPortfolios: computed.sort('portfolios', 'sortDefinition'),
    sortBy:           'title', // default sort by title
    sortAsc:    true, // default sort in ascending order
    sortDefinition:   computed('sortBy', 'sortAsc', function () {
        let sortOrder = get(this, 'sortAsc') ? 'asc' : 'desc';
        console.log('sort recomputing');
        return [ `${get(this, 'sortBy')}:${sortOrder}` ];
    }),

  actions: {

        sortPortfoliosBy(property, ascending) {
            console.log('about to sort by ' + property + ' : ' + ascending);
            set(this, 'sortBy', property);
            set(this, 'sortAscending', ascending);
        },

  }

Nevermind, figured it out, needed to add an observer
    modelChanged: observer('model', function () {
        let table = get(this, 'table');
        if (table !== null) {
            table.setRows(get(this, 'model'));
        }
    }),


Support thread local storage - Partially Fixes #291

This patch adds a pico_tls module, which adds support for GCC's __thread, C11's _Thread_local, and C++'s thread_local storage class specifiers. For example:
__thread int foo;

Different implementations of pico_tls may be selected by means of the pico_set_tls_implementation cmake macro, similar to some other SDK modules. For example, a different implementation could be selected if using an RTOS.
In this implementation, a thread corresponds to a core. No TLS storage is allocated for exceptions handlers. So there are two threads total.
Issue #291 has two parts: making C++ thread_local work and making C++ exceptions work. This patch adds support for thread_local but not C++ exceptions.
Tested by building SDK and running modified kitchen_sink in debug and release, stepping through tls_init and __wrap___emutls_get_address in GDB.

Nice; just glanced at this so far.
A couple o-things

We should probably not make core_thread the default (as it pulls in new extra code), by which i actually mean we should make it the default, but not unilaterally add pico_tls as a dependency. (i.e. add that if you want TLS)
Can you add our copyright (congratulations on perhaps being the first that i recall to add new code!) ... you can include your name as well if you want.

Copyright comments added to tls.c and tls.S but not CMakeLists.txt, which I think is consistent with other sources. I removed pico_tls from pico_runtime's dependencies, which I think is what you want.

moving to 1.7.0 so we can fix this on clang too


fix(explain): add livenessProbe constraints in the samples

:tada: This PR is included in version 7.5.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


math::mat4f::translate() -> math::mat4f::translation()?

This perfectly reasonable looking bit of code:
void my_fun(math::float3 position, math::quatf rotation) {
  math::mat4f view_matrix =
       math::mat4f(rotation).translate(position);
  ...

Doesn't do what you'd expect - translate() is a static method that returns a new matrix, not a member function that adds translation to an existing matrix.
Would it be possible to nounify (i.e. translation() instead of translate()) static methods like this?  My eye would catch math::mat4f(rotation).translation(position);
This would just be for readability - the more idiomatic version doesn't need this method anyway:
void my_fun(math::float3 position, math::quatf rotation) {
  math::mat3f rotation_matrix{rotation};
  math::mat4f view_matrix{rotation_matrix, position};
  ...

@pixelflinger
That's be fine with me. We're uses to it because our math library doesn't rely on member functions really.

I think the idea was that you could write something like:
auto m = mat4::rotate(...) * mat4::translate(...);

I think I would go further than this proposal, and rename to mat4::makeTranslation(), etc...

By the way we currently have a variant of translate that takes a scalar t and does a translation by t,t,t which I think is really weird, maybe we should it remove it, seems like copy-pasta from its "scale" counterpart.


Update http4s-blaze-server, http4s-circe, ... to 0.21.19

Updates

org.http4s:http4s-blaze-server
org.http4s:http4s-circe
org.http4s:http4s-dsl

from 0.21.1 to 0.21.19.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Applied Migrations

dependency:v0_21@org.http4s:http4s-scalafix:0.21.5

Documentation:

https://github.com/http4s/http4s/releases/tag/v0.21.5

Files still referring to the old version number
The following files still refer to the old version number (0.21.1).
You might want to review and update them manually.
build.sbt

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s" } ]

labels: library-update, semver-patch, scalafix-migrations, old-version-remains

Superseded by #270.


update chart for managed-serviceaccount

Description

update the managedServiceAccount CRD
update the chart generate script to update chart version before rendering templates, this can make the generated addontemplate name with ACM version prefix.

Related Issue
https://issues.redhat.com/browse/ACM-7072
Changes Made
Provide a clear and concise overview of the changes made in this pull request.
Screenshots (if applicable)
Add screenshots or GIFs that demonstrate the changes visually, if relevant.
Checklist

[ ] I have tested the changes locally and they are functioning as expected.
[ ] I have updated the documentation (if necessary) to reflect the changes.
[ ] I have added/updated relevant unit tests (if applicable).
[ ] I have ensured that my code follows the project's coding standards.
[ ] I have checked for any potential security issues and addressed them.
[ ] I have added necessary comments to the code, especially in complex or unclear sections.
[ ] I have rebased my branch on top of the latest main/master branch.

Additional Notes
Add any additional notes, context, or information that might be helpful for reviewers.
Reviewers
Tag the appropriate reviewers who should review this pull request. To add reviewers, please add the following line: /cc @reviewer1 @reviewer2
Definition of Done

[ ] Code is reviewed.
[ ] Code is tested.
[ ] Documentation is updated.
[ ] All checks and tests pass.
[ ] Approved by at least one reviewer.
[ ] Merged into the main/master branch.

cc @dislbenn

/lgtm


Dask Array Chunking does not match example in docs

To reproduce:
import numpy as np
npa = np.array([
   [1, 2, 3, 4, 5, 6],
   [7, 8, 9, 0, 1, 2],
   [3, 4, 5, 6, 7, 8],
   [9, 0, 1, 2, 3, 4],
   [5, 6, 7, 8, 9, 0],
   [1, 2, 3, 4, 5, 6]
])
a = da.from_array(npa, chunks=3)
a.blocks[0]
# dask.array<blocks, shape=(3, 6), dtype=int64, chunksize=(3, 3), chunktype=numpy.ndarray>
a.blocks[0].compute()
# array([[1, 2, 3, 4, 5, 6],
#       [7, 8, 9, 0, 1, 2],
#       [3, 4, 5, 6, 7, 8]])
a.blocks[2]
# IndexError: Index is not smaller than dimension 2 >= 2

What happened:

Calling compute on the block at index 0 of the dask array produced a numpy array of shape (3,6)
Got an IndexError when reading the block at index 2.

What you expected to happen:

Calling compute on block at index 0 of the dask array produces a numpy array of shape (3,3)
I should be able to read the block at index 2 (and 3).

Anything else we need to know?:
I was copying the example from the documentation.
Environment:

Dask version: 2.21.0
Python version: 3.7.4
Operating System: MacOS
Install method (conda, pip, source): pip

If this is a bug, I am happy to work on it. I originally posted on SO and was referred to file an issue.

Hey @joshreback -- sorry for the delay in getting to this.
I don't think this is a bug.
Your chunks/blocks are in two dimensions, so you can index them in two dimensions:
In [4]: a.blocks[0, 0].compute()
Out[4]:
array([[1, 2, 3],
       [7, 8, 9],
       [3, 4, 5]])

In [5]: a.blocks[0, 1].compute()
Out[5]:
array([[4, 5, 6],
       [0, 1, 2],
       [6, 7, 8]])

In [6]: a.blocks[0].compute()
Out[6]:
array([[1, 2, 3, 4, 5, 6],
       [7, 8, 9, 0, 1, 2],
       [3, 4, 5, 6, 7, 8]])

This tracks with numpys slicing, but it operates block-by-block
In [8]: npa[0]
Out[8]: array([1, 2, 3, 4, 5, 6])

In [9]: npa[1]
Out[9]: array([7, 8, 9, 0, 1, 2])

I see - makes sense! Thanks, @gforsyth


Prepare for 1.1.0

Blocked by #106 and #107

@Zegnat Could you also review the diff? I want to have your opinion of they optimizations made in #102

LGTM. Am I right that this will also fix #94 in that is changes the type-hint? Worth adding?

Yes, please fix #94. =)

Oh, sorry @Zegnat, I read this too quickly.
Yes, it includes a fix for #94. I've also created #109 to further explain whats happening.


Importing a gist does not populate the content in editor

Fixed in https://github.com/maheshj01/pastelog/commit/d19c38b62a1d52d08c1a75528b2b632a8e7fb811


feat: Focus search input when double-tapping bottom navigation search item

Description
This PR adds common app behavior and focuses on the search input (and opens the keyboard) when double-tapping the bottom navigation search item.
PR Checklist

[ ] I have tested my changes on iOS and Android.
[ ] I hid my changes behind a feature flag, or they don't need one.
[ ] I have included screenshots or videos, or I have not changed the UI.
[ ] I have added tests, or my changes don't require any.
[ ] I added an app state migration, or my changes do not require one.
[ ] I have documented any follow-up work that this PR will require, or it does not require any.
[ ] I have added a changelog entry below, or my changes do not require one.

To the reviewers 👀

[ ] I would like at least one of the reviewers to run this PR on the simulator or device.

Changelog updates
Changelog updates

Cross-platform user-facing changes

iOS user-facing changes

Android user-facing changes

Dev changes

Need help with something? Have a look at our docs, or get in touch with us.

Warnings

:warning:

❌ An error occurred while validating your changelog, please make sure you provided a valid changelog.

  Generated by :no_entry_sign: dangerJS against 53d94a965be7baee027f55f49fd856851f9ed489

Nicee


Rule request: required-chains (e.g. no it() without describe())

As the inverse of #340, some teams prefer to always wrap its with parent describes. For example, when testing a class, it can be preferable to always wrap that class' its with a describe("ClassName", ...).
We could generalize this rule to be:
{
    "required-chains": [
        true,
        [
            {
                "chain": ["describe", "it"],
                "message": "Wrap your tests with a describe() block referring to the class or function they test."
            }
        ]
    ]
}

💀 It's time! 💀
TSLint is being deprecated; therefore, it and tslint-microsoft-contrib are no longer accepting pull requests for major new changes or features. See #4534. 😱
If you'd like to see this change implemented, you have two choices:

Recommended: Check if this is available in ESLint + typescript-eslint ✅
Not Recommended: Fork TSLint or tslint-microsoft-contrib locally 🤷♂

👋 It was a pleasure open sourcing with you!
If you believe this message was posted here in error, please comment so we can re-open the issue!


Request for Instance ID service support

Is it possible to have a client for Instance ID Service ?
https://developers.google.com/instance-id/reference/server#manage_relationship_maps_for_multiple_app_instances

Looking into it. It won't be anytime soon, unfortunately.

This API does not provide a public discovery document (see https://www.googleapis.com/discovery/v1/apis for a complete list). The code generator for the clients in this repo only supports APIs that do so.


cron.daily error regarding mdma

Required information

Distribution: Ubuntu
Distribution version: 16.04
The output of "lxc info" or if that fails:

environment:
  architectures:
  - x86_64
  - i686
  driver: lxc
  driverversion: 2.0.5
  kernel: Linux
  kernelarchitecture: x86_64
  kernelversion: 4.4.0-38-generic
  server: lxd
  serverpid: 15681
  serverversion: 2.0.5
  storage: zfs
  storageversion: "5"

Issue description
Not sure if the right place for this Issue. We get on our lxc launch ubuntu: installations a run-parts: /etc/cron.daily/mdadm exited because of uncaught signal 11.
The output of the script mdadm --monitor --scan --oneshot:
Segmentation fault (core dumped)
Steps to reproduce
Create new container, wait for email (-,

Hm, works fine here.
lxc exec sid -- bash
root@sid:~# mdadm --monitor --scan --oneshot
root@sid:~# echo $?
0

Okay, I double checked with a new machine. On the host it works. In the container segfault.
Install: Ubuntu 16.04.1 LTS
I will try to find out what makes mdadm segfaulting.
lg Michael

Hmm, so mdadm was configured specifically not to start in containers, I suspect the cron job should be configured not to run when the daemon isn't running, or possibly just not run in containers too.
It's very unlikely that there's anything we should do about this in LXD itself as we very much don't want a LXD user to be able to alter RAID configuration so having mdadm fail is fine, it should just be made to fail better and not spam you :)
Once you figure out exactly what it is that mdadm is doing, I'd recommend you file a bug at https://launchpad.net/ubuntu/+source/mdadm/+filebug and then post the link here so I can subscribe us to it for tracking.

Has anyone found a solution yet (maybe you @l00mi)?
This error message is badgering me for quite a while now :/

nah sorry, its badgering still along on our servers too :/

Did one of you file a bug report at the URL I gave a few months ago?

I can't access the affected servers right now, but maybe this has already been reported? https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1691763
We should move the discussion over there.
I will strace it myself tomorrow and also try the "workaround" provided in the bug report:
As others have observed[^1], the workaround is to remove mdadm from inside lxd containers.
[^1] https://blog.sleeplessbeastie.eu/2017/05/04/how-to-fix-mdadm-segfault-inside-lxc-container/


Add zsh completion for 'docker search --limit'

Ref. #23107
Thx @thaJeztah for the ping.
Signed-off-by: Steve Durrheimer<EMAIL_ADDRESS>

LGTM 🐮

Thanks!
LGTM


Learner's score is not recorded using  IE10 Windows 7.

Summary
Learner's score is not recorded using IE10 Windows 7, but when the learner use Google Chrome version 49.0, the score is recorded properly
Branch or installer
Did issue occur on the current develop branch or on installation source (please specify OS and version)?

Installer: Windows
Version: 0.16.x using build 208

or:

Branch: develop

Screenshots

How to reproduce

Login as learner.
View a video.
After viewing, refresh the page. The score is not recorded.

Closing this as it's opened in the source repo.


UL element

UL element to wrap around <ListItem/>
Would be great to have a setting like this
<ListItems
    shouldShowAlternativeColors={true}
    items={[]}
/>

Where shouldShowAlternativeColors will give odds and evens different colors

Also what about a maxHeight={400} prop. So when you set this the list is scrollable inside?


Edit Django to include help & embed

Edited django.js to (hopefully) make it use an embed, as well as provide users with the proper link to the setup guide to aid in fixing.

All looks fine, still needs to be tested though.

👍


Testing CLA

Overview:
Was requested to test CLA

Initial CLA Test Failed and cannot fix in this PR.  Time for round 2.


get "RuntimeError: Unexpected error in an Op's C code." when using GPU

Hi, I get "RuntimeError: Unexpected error in an Op's C code." when I run "tests/test_ctc.py" by using GPU. However,  no error occurs when I run the test example by using CPU. Would you please help me to solve this problem?  Thank you very much!
P.S.  My envirment is "Ubuntu14.04 + GTX1070 + theano_ctc"
Error details when using GPU:
▶ python test_ctc.py
Using gpu device 0: GeForce GTX 1070 (CNMeM is disabled, cuDNN 5005)
Activations
[[[  0.   0.   0.   0.   0.]
[  1.   2.   3.   4.   5.]
[ -5.  -4.  -3.  -2.  -1.]]
[[  0.   0.   0.   0.   0.]
[  6.   7.   8.   9.  10.]
[-10.  -9.  -8.  -7.  -6.]]
[[  0.   0.   0.   0.   0.]
[ 11.  12.  13.  14.  15.]
[-15. -14. -13. -12. -11.]]]
Softmax outputs
[[[ 0.2         0.2         0.2         0.2         0.2       ]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]]
[[ 0.2         0.2         0.2         0.2         0.2       ]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]]
[[ 0.2         0.2         0.2         0.2         0.2       ]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]
[ 0.01165623  0.03168492  0.08612854  0.23412167  0.63640863]]]
Theano function to calculate costs and gradient of mean(costs):
<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>.0 [id A] 'ctc_cost'   0
|acts [id B]
|actT [id C]
|labels [id D]
HostFromGpu [id E] '(dmean/dacts)'   9
|GpuElemwise{Mul}[(0, 1)] [id F] ''   8
|GpuReshape{3} [id G] ''   7
| |GpuAlloc [id H] ''   6
| | |GpuElemwise{Inv}[(0, 0)] [id I] ''   5
| | | |GpuFromHost [id J] ''   4
| | |   |Elemwise{Cast{float32}} [id K] ''   3
| | |     |InplaceDimShuffle{x} [id L] ''   2
| | |       |Shape_i{0} [id M] ''   1
| | |         |<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>.0 [id A] 'ctc_cost'   0
| | |Shape_i{0} [id M] ''   1
| |TensorConstant{[ 1 -1  1]} [id N]
|<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>.1 [id A] 'ctc_grad'   0
Theano function to calculate costs:
<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b4b6cc10> [id A] 'ctc_cost'   0
|acts [id B]
|actT [id C]
|labels [id D]
warpctc.compute_ctc_loss() exited with status 3
Traceback (most recent call last):
File "test_ctc.py", line 62, in
cost, grad = train()
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 871, in call
storage_map=getattr(self.fn, 'storage_map', None))
File "/usr/local/lib/python2.7/dist-packages/theano/gof/link.py", line 314, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "/usr/local/lib/python2.7/dist-packages/theano/compile/function_module.py", line 859, in call
outputs = self.fn()
RuntimeError: Unexpected error in an Op's C code. No Python exception was set.
Apply node that caused the error: <theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>(acts, actT, labels)
Toposort index: 0
Inputs types: [CudaNdarrayType(float32, 3D), TensorType(int32, vector), TensorType(int32, matrix)]
Inputs shapes: [(3, 3, 5), (3,), (3, 2)]
Inputs strides: [(15, 5, 1), (4,), (8, 4)]
Inputs values: ['not shown', array([1, 3, 3], dtype=int32), 'not shown']
Outputs clients: [[Shape_i{0}(ctc_cost), 'output'], [GpuElemwise{Mul}[(0, 1)](GpuReshape{3}.0, ctc_grad)]]
Backtrace when the node is created(use Theano flag traceback.limit=N to make it longer):
File "test_ctc.py", line 45, in
tCost = ctc_cost(tsActs, tsLabels, tsActT)
File "/usr/local/lib/python2.7/dist-packages/theano_ctc/init.py", line 40, in ctc_cost
return GpuCtc()(acts, labels, input_lengths)
File "/usr/local/lib/python2.7/dist-packages/theano_ctc/gpu_ctc.py", line 17, in init
self.costs = T.fvector(name="ctc_cost")
Debugprint of the apply node:
<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>.0 [id A] <TensorType(float32, vector)> 'ctc_cost'
<theano_ctc.gpu_ctc.GpuCtc object at 0x7f22b5848710>.1 [id A] <CudaNdarrayType(float32, 3D)> 'ctc_grad'
Storage map footprint:

acts, Shared Input, Shape: (3, 3, 5), ElemSize: 4 Byte(s), TotalSize: 180 Byte(s)
labels, Shared Input, Shape: (3, 2), ElemSize: 4 Byte(s), TotalSize: 24 Byte(s)
TensorConstant{[ 1 -1  1]}, Shape: (3,), ElemSize: 8 Byte(s), TotalSize: 24 Byte(s)
actT, Shared Input, Shape: (3,), ElemSize: 4 Byte(s), TotalSize: 12 Byte(s)
TotalSize: 240 Byte(s) 0.000 GB
TotalSize inputs: 240 Byte(s) 0.000 GB

I just find that my warp-ctc is not correctly builded, thus causing the error "warpctc.compute_ctc_loss() exited with status 3" occurs.
code in https://github.com/ganji15/warp-ctc/blob/master/src/ctc_entrypoint.cpp
"""
else if (info.loc == CTC_GPU) {
#ifdef CUDACC
GpuCTC ctc(alphabet_size, minibatch, workspace, info.stream);
    if (gradients != NULL)
        return ctc.cost_and_grad(activations, gradients, costs,
                                 flat_labels, label_lengths,
                                 input_lengths);
    else
        return ctc.score_forward(activations, costs, flat_labels,
                                 label_lengths, input_lengths);

#else
std::cerr << "GPU execution requested, but not compiled with GPU support" << std::endl;
return CTC_STATUS_EXECUTION_FAILED;
#endif
"""


Suggestion: Infinite scroll block scrolling up

When I load new ellements with scroll up, the scroll is set to top, and then, the new elements are shown and I lose reference of the element I was watching at the moment the new elements loaded.
Would be awesome to have an option to mantain scroll position in order to keep seeing the '0' element when '-1', '-2' are loaded.
status pre load:
0 --> on screen
1
2
status after load:
-2 --> on screen
-1
0
1
2
status after load desired:
-2
-1
0 --> on screen
1
2

hi @Perezmarc
let me know if i understood correctly:
you want to be able to prevent the user from scrolling up?

Nope, I want to prevent the screen, after load the new elements, to go up to -2 element, to stick with the element 0 that was the top of the screen before loading the elements.
The user has 0 1 2 loaded, and if he scrolls up, he will be seeing the element 0, and because he scrolled up, we trigger the function to load elements before 0, when that's done, as the scroll position was top, it is still top, but now, we have -2 and -1 before the element 0, the one we were looking at. Therefore, we loaded new elements, but those are not seen until we scroll up again, having loaded the elements.
In fact, what I want is exactly the same behavior as scrolling down. I am in position 2, and after elements 3 4 are loaded, I'm in the same position I was before they load, position 2, but now I can scroll down to view the elements 3 and 4.
I hope I made my self clear now :)

@Perezmarc
I think i understood....
to solve this, I see it as a matter of logics that can be applied to the array of the rendered list in response to the relevant event. I think you can apply this logic by listening to the scrollUp/scrollDown event. However, it sounds complicated :)
I'm not sure this kind of logics should be a part of this directive though.

I believe it's a matter of scrolling, not array... the scroll position goes from top to bottom, if it is in 0 position (top), after load -2 -1 , it will still be in scroll 0, but now, it will be showing -2 element.
Resolving this in array... would be weird, I think that if people want to use scrollup function to load elements they will all end up with this issue.
Thanks anyway, I'll try to implement it through array manipulation

I am having the same issue. I have updated the example plunkr, to demonstarte the issue.
https://plnkr.co/edit/n7iqb2Ugv6tQv1DF2SX2?p=preview
When scrolling up, I am basically adding the items to the array, by using the unshift command.
[0, 1, 2]
array.unshift(-1)
[-1, 0, 1, 2]
array.unshift(-2)
[-2, 0, 1, 2]
etc.
It looks like when you do it like this, the scroll jumps automatically to the first element in the array. I am not sure if this is related to this module or the the implementation of ngFor.

Suggested fix: (angular2-infinite-scroll   lines 89-106)
InfiniteScroll.prototype.onScrollDown = function (data) {
var _this = this;
if (data === void 0) {
data = { currentScrollPosition: 0 };
}
this.zone.run(function () {
return _this.scrolled.emit(data);
});
};
InfiniteScroll.prototype.onScrollUp = function (data) {
var _this = this;
if (data === void 0) {
data = { currentScrollPosition: 0 };
}
this.zone.run(function () {
return _this.scrolledUp.emit(data);
});
};
When scrolling down, I believe currentScrollPosition (for instance might be) 80 of 100.  Then new data is added, now currentScrollPosition is 80 of 120.  The scroll bar is placed in the right position.
When scrolling up, currentScrollPostion might be 20 of 100.  Then new data is added, and current ScrollPosition is still 20 of 120 when it should be 40 of 120 to maintain it's position.
Note:  I added this comment into item #114 as well.

I was not liking the action of my down scroll so I changed the 'scroll distance' from 2 to 3.  When I did this, the up scroll started working as it is supposed to.  This was a big win for me because I had nightmares of how I was going to fix it.  Try this adjustment to see if it will help your scrolling up.
Thanks

Is there any updates on this issue?

@mackelito not on the lib side.

@Perezmarc this lib now supports scroll up - please try again (you may look at the /example directory).
please reopen if this issue is different - or rather open an new issue linked to this one.


Get file uploader working

Overview
What is the change?

Removed privacy page and replaced with uploader

Simple drag and drop or click to upload form added

File uploads to back end and verifies it can be parsed

[ ] Bug Fix

[x] New Feature

[ ] Refactoring

[ ] Other

Trello card link?
https://trello.com/b/w5ByjDl2/story-points-sprint-1
Other Notes/Attachments
Meme

Looks good, nice job.


Sometimes posting large dataset twice causes datamart to throw exception

Tried posting twice the FSIall_AN dataset in performance_test branch test/test_data/FSI_all_Annotated.xlsx file.
The exception occurs when datamart tries to delete the previous post 50,000 edge ids at a time.
[2020-10-14 10:51:10,916] ERROR in app: Exception on /datasets/FSIall_AN/annotated [POST]
Traceback (most recent call last):
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context
    cursor, statement, parameters, context
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
psycopg2.OperationalError: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/home/ktyao/dev/dsbox/datamart-api/api/annotated/main.py", line 8, in post
    return post_data.process(dataset)
  File "/home/ktyao/dev/dsbox/datamart-api/api/annotated/post.py", line 109, in process
    import_kgtk_dataframe(kgtk_exploded_df, is_file_exploded=True)
  File "/home/ktyao/dev/dsbox/datamart-api/db/sql/kgtk.py", line 175, in import_kgtk_dataframe
    import_kgtk_tsv(tsv_path, config=config)
  File "/home/ktyao/dev/dsbox/datamart-api/db/sql/kgtk.py", line 153, in import_kgtk_tsv
    session.execute(delete_q)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1292, in execute
    clause, params or {}
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1020, in execute
    return meth(self, multiparams, params)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_clauseelement
    distilled_params,
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1324, in _execute_context
    e, statement, parameters, cursor, context
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1518, in _handle_dbapi_exception
    sqlalchemy_exception, with_traceback=exc_info[2], from_=e
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
    raise exception
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1284, in _execute_context
    cursor, statement, parameters, context
  File "/home/ktyao/anaconda3/envs/datamart/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
(Background on this error at: http://sqlalche.me/e/e3q8)
Exception ignored in: <module 'threading' from '/home/ktyao/anaconda3/envs/datamart/lib/python3.7/threading.py'>

Here is part of the sql statement from the exception. It's too long include the whole thing.

@zmbq if you do a PUT instead of POST the existing data for the variables will be first deleted.
But the issue posted by @kyao is not about that.
What is the purpose of this line ?
https://github.com/usc-isi-i2/datamart-api/blob/development/db/sql/kgtk.py#L153


Fix spelling mistakes

Merge after #55

Thanks! Is the vscode Hunspell settings file necessary/useful?

@Fevol settings file is not necessary but it is useful as it will help to distinguish possible future spelling mistakes from the whitelisted ones that spell checker didn't know

As I am personally using Webstorm, the config wouldn't be very useful for me. If it's alright, I think it should be an user-config, rather than a part of the repository.

@Fevol I can modify all of my PRs removing this config if you don't want to keep it in the repository.
I understand that Webstorm can't take advantage of the config directly but I thought it's a good idea to list all the spell checker exception whitelist words in the plain text format, so you can take those words and add to your tool of interest.
We can move them into some tool agnostic file like spellcheck-exceptions.txt, if you choose to.
Please let me know how do you want to proceed.

I did some digging and it seems like Jetbrains IDE's primarily make use of .dic files -- setting those up properly seems to be too much of a hassle (both in terms of integrating them into the IDE configuration, and actually creating and updating them), so I'm fine with just merging the PR as-is.


Change symbolic syntax.

Instead of the previous syntax
%{ #1 := #2.lock }(x, y)

We now have the rather more readable
%{ [|x|] := [|y|].lock }

This has required a fair amount of surgery to Starling, as the
symbolic representation has changed from sentence/arguments to a
list of inline strings and expressions.
All of the currently passing GRASShopper examples have been
converted over.

I've paged through this and it all looks fairly uncontroversial. Happy to dig in if there are more detailed areas that need review. Otherwise :shipit:

@septract I've fixed and tested the bug you found.  If this passes CI should we ship?

Yeah, :shipit:
On 1 February 2017 at 10:14, Matt Windsor<EMAIL_ADDRESS>wrote:

@septract https://github.com/septract I've fixed and tested the bug you
found. If this passes CI should we ship?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/septract/starling-tool/pull/134#issuecomment-276620267,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANd9ti1IuzEM6UWtY3Q6fXhhefIYSuhks5rYFrtgaJpZM4Lyv-e
.

--
http://www-users.cs.york.ac.uk/~miked/


add schematron rule for MIDI ppq

From<EMAIL_ADDRESS>on July 20, 2012 11:52:44
When the values of @ppq differ across the initial staffDef elements, @ppq should be required on the containing scoreDef element.  In the following example, @-ppq should be required on scoreDef. Furthermore, each value of staffDef/@ppq should be a factor of scoreDef/@ppq. 48 is divisible (without remainder) by 2, 4, and 24.
<scoreDef meter.count="4" meter.unit="4" meter.sym="common" key.sig="1f" ppq="48">
<staffGrp>
<staffDef n="1" xml:id="midi.P1" lines="5" clef.line="2" clef.shape="G" key.sig="1f" ppq="2"/>
<staffDef n="2" xml:id="midi.P2" lines="5" clef.line="4" clef.shape="F" key.sig="1f" ppq="4"/>
<staffDef n="3" xml:id="midi.P3" lines="5" clef.line="4" clef.shape="F" key.sig="1f" ppq="24"/>
</staffGrp>
</scoreDef>
Original issue: http://code.google.com/p/music-encoding/issues/detail?id=91

From<EMAIL_ADDRESS>on October 08, 2012 11:24:59
Actually, the lowest common multiple of 2, 4, and 24 is 24, not 48.

From<EMAIL_ADDRESS>on October 08, 2012 13:28:17
<constraintSpec ident="Check_staff_ppq_ancestor" scheme="isoschematron">
<constraint>
<sch:pattern>
<sch:rule context="mei:staffDef[@ppq][ancestor::mei:scoreDef[@ppq]]">
<sch:let name="staffPPQ" value="@ppq"/>
<sch:let name="scorePPQ" value="ancestor::mei:scoreDef[@ppq][1]/@ppq"/>
<sch:assert test="($scorePPQ mod $staffPPQ) = 0">The value of ppq on staffDef must be a factor of the value of ppq on an ancestor scoreDef.</sch:assert>
</sch:rule>
</sch:pattern>
</constraint>
</constraintSpec>
<constraintSpec ident="Check_staff_ppq_preceding" scheme="isoschematron">
<constraint>
<sch:pattern>
<sch:rule context="mei:staffDef[@ppq][preceding::mei:scoreDef[@ppq]]">
<sch:let name="staffPPQ" value="@ppq"/>
<sch:let name="scorePPQ" value="preceding::mei:scoreDef[@ppq][1]/@ppq"/>
<sch:assert test="($scorePPQ mod $staffPPQ) = 0">The value of ppq on staffDef must be a factor of the value of ppq on a preceding scoreDef.</sch:assert>
</sch:rule>
</sch:pattern>
</constraint>
</constraintSpec>

From<EMAIL_ADDRESS>on October 12, 2012 11:07:42
This issue was closed by revision r710 .
Status: Fixed


[bug]: run dekstop failed on windows system

Environment
Windows 10
Steps to reproduce
After executing yarn run start.
The following error occurs:
C:\Users\WL\Documents\GitHub\flat\node_modules\electron\dist\electron.exe exited with signal SIGINT

(node:1656) ExtensionLoadWarning: Warnings loading extension at C:\Users\WL\AppData\Roaming\Electron\extensions\fmkadmapgofadopljbjfkapdkoienihi: Unrecognized manifest key 'browser_action'. Unrecognized manifest key 'minimum_chrome_version'. Unrecognized manifest key 'update_url'. Cannot load extension with file or directory name _metadata. Filenames starting with "_" are reserved for use by the system.

Checking for update
Update for version 11.3.0 is not available (latest version: 1.1.0-beta.1, downgrade is disallowed).

Checking for update
Update for version 11.3.0 is not available (latest version: 1.1.0, downgrade is disallowed).

The app launches successfully with errors and I can neither create nor join a room:

this is a known problem, but one that we currently do not have the ability to solve. We can only wait for agora-electron-sdk to fix it
the root cause is: on windows, you have to use 32-bit nodejs. One temporary solution is to switch your nodejs to 32-bit and re-execute yarn run bootstrap
Related Discussions: https://github.com/netless-io/flat/issues/996#issuecomment-943208480


Add methods with Resource to GrpcServer and ManagedChannelInterpreter

What this does?
Add a method to GrpcServer and ManagedChannelInterpreter returning a Resource for shutting down the server/channel.
Checklist

[ ] Reviewed the diff to look for typos, println and format errors.
[ ] Updated the docs accordingly.

I haven't checked, but I think this might fix #826 🤞


Android 13 package cannot be parsed

I don't know entirely what causes this issue, but trying to install the APK results in the parsing error. I'm guessing it's something simple like a malformed manifest, but I can't be sure

I have been using it on Android 13 for over a year and it's never had an issue. Redownload the APK and try it with a session-based installer like SAI or APKmirror.


PublishTestResults throws exception if searchFolder is relative path

Required Information
Question, Bug, or Feature?
Type: Bug
Enter Task Name: PublishTestResults
Environment

Server - Azure Pipelines

If using Azure Pipelines, provide the account name, team project name, build definition name/build number: https://dev.azure.com/azure-sdk/public/_build/results?buildId=203814

Agent - Hosted or Private: Hosted

If using Hosted agent, provide agent queue name: Linux and Mac

Issue Description
If PublishTestResults.searchFolder is set to a relative path, the task fails with an exception.  It fails on Linux and Mac but succeeds on Windows.  The root cause appears to be that the JS portion of the task resolves the relative path correctly (relative to repo root), but the dotnet portion of the task resolves the relative path against a different root, so the resolved path is no longer valid.
A workaround is to use an absolute path (e.g. $(System.DefaultWorkingDirectory)/sdk rather than just sdk).  However, other settings like CopyFiles.sourceFolder support relative paths, so I think PublishTestResults.searchFolder should support them as well.
Task logs
log_147_203814.zip
Error logs
##[debug]Reading test results from file 'sdk/appconfiguration/app-configuration/test-results.xml'.
Failed to parse result files: System.IO.DirectoryNotFoundException: Could not find a part of the path '/home/vsts/agents/2.160.1/bin/sdk/appconfiguration/app-configuration/test-results.xml'.
   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
   at System.Xml.XmlDownloadManager.GetStream(Uri uri, ICredentials credentials, IWebProxy proxy, RequestCachePolicy cachePolicy)
   at System.Xml.XmlUrlResolver.GetEntity(Uri absoluteUri, String role, Type ofObjectToReturn)
   at System.Xml.XmlTextReaderImpl.FinishInitUriString()
   at System.Xml.XmlTextReaderImpl..ctor(String uriStr, XmlReaderSettings settings, XmlParserContext context, XmlResolver uriResolver)
   at System.Xml.XmlReaderSettings.CreateReader(String inputUri, XmlParserContext inputContext)
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.JUnitResultParser.ReadFile(String filePath, XmlDocument doc)
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.JUnitResultParser.ParseTestResultFile(TestRunContext runContext, String filePath)
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.JUnitResultParser.<>c__DisplayClass1_0.<ParseTestResultFiles>b__0(String file)
   at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Microsoft.TeamFoundation.TestClient.PublishTestResults.JUnitResultParser.ParseTestResultFiles(TestRunContext runContext, IList`1 resultFilePaths)
   at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.Parser.ParseFiles(IExecutionContext executionContext, TestRunContext testRunContext, List`1 testResultsFiles, ITestResultParser testResultParser)
##[debug]Processed: ##vso[results.publish type=JUnit;mergeResults=false;runTitle=Linux - NodeJS - Unit Tests - [Node 12.x%5D;publishRunAttachments=true;resultFiles=sdk/appconfiguration/app-configuration/test-results.xml,sdk/core/abort-controller/test-results.xml,sdk/core/core-amqp/test-results.xml,sdk/core/core-arm/test-results.xml,sdk/core/core-auth/test-results.xml,sdk/core/core-http/test-results.xml,sdk/core/core-lro/test-results.xml,sdk/core/core-tracing/test-results.xml,sdk/core/logger/test-results.xml,sdk/identity/identity/test-results.xml,sdk/keyvault/keyvault-certificates/test-results.xml,sdk/keyvault/keyvault-keys/test-results.xml,sdk/keyvault/keyvault-secrets/test-results.xml,sdk/storage/storage-blob/test-results.xml,sdk/storage/storage-file-datalake/test-results.xml,sdk/storage/storage-file-share/test-results.xml,sdk/storage/storage-queue/test-results.xml,sdk/template/template/test-results.xml;failTaskOnFailedTests=false;testRunSystem=VSTS - PTR;]
##[debug]task result: Succeeded

Hey,
This is a bug on our side. We'll fix this soon. We'll keep you updated.

This has been fixed in the latest version of the task.
Please reopen the issue if you are still blocked.


Pin version require for pandas

Older version of pandas change the index when applying the .agg. method to a pandas.DataFrame, breaking the calc_stats method in the Bandit class. Investigate version compatibility.

Fixed in #6 .


Legacy tester tokens may no longer be created

In the examples you're referring to legacy tokens, however these cannot be created anymore. I there an alternative?

@bar9 at the bottom of the page here: https://api.slack.com/rtm#classic -- you can create a "classic" slack app, which will allow you to create a bot user which will contain a token that can be used to auth with the slack API.
Though, you may still run into issues with this, I created a workaround here: https://github.com/slack-rs/slack-rs-api/pull/105 -- but depending on how you're pulling in this library you may need to use the other forks I created to get this working:

https://github.com/silverjam/yobot
https://github.com/silverjam/slack-rs-api
https://github.com/silverjam/slack-rs

No idea if this workaround works for other parts of the Slack API.


Resolve possible missing reference error when enriching vulnerabilities

Vulnerability was missing a generate_id(), which is necessary for non-SCOs in order to get a consistent ID.


For loops with dynamic ranges

[WIP] Add support for for loops with dynamic ranges and steps.
@sa2257 @sampsyo @tedbauer comments for step syntax. Right now, it is:
for (let i = 0..10 step 2) ...

Add support for loops with arbitrary expressions in place of start and end ranges.
These loops cannot be unrolled.
I'm worried that adding these is moving us towards the Vivado problem where certain magic constructs in the language can be unrolled whiles others can't be. Maybe it would be better to just use while loops as the un-unrollable loops and treating this as syntactic sugar.

I think your suggested syntax is good because it's simple and clear.
Some random thoughts:
for i in [ 0..10 | step 2 ] { ...

for (let i = step 2: 0..10) { ...

Hm, I'm not happy with the current proposal. It conflicts with the current syntax making parsing gnarly and it isn't clean when a for loop can be unrolled and when it can't.

I don't mind the step syntax all by itself! Reminds me of Apple BASIC. :smiley:
https://rosettacode.org/wiki/Loops/For_with_a_specified_step#Applesoft_BASIC
However, this is an instance where we could consider embracing the all-consuming everything is a view philosophy. That is, to iterate over this array, you could take a new kind of view that strides over an underlying array. You can imagine the hardware gadget that would do the index adjustment: something along the lines of i => i*s where s is the step size. This would be cheap as long as s divides the banking factor—and it also seems straightforward to compute the new memory type of the strided view. Then, the loop would just iterate over the view instead of the underlying array.


Testsunit1

Multiple Corrections + New UT

Correcting vCharts bug
Erasing the print_info parameter, it is now possible to use the set_display_parameters method of the vdf to choose either to print or not to print the information.
Erasing properly the temp relations
Sorting correctly the vDataFrame
Adding multiple tests for the plot test


array() -> ArrayObject ?

Hey.
Since I am coding on a portable PHP application, I can not relay on the PECL extension and have come along to find this very nice and functional bit of PHP.
My goal is to use objects instead of arrays in most places - especially ArrayObjects since these are a "hybrid" of both.
Which places in the class would I have to re-write a bit to turn the whole thing into an ArrayObject instead of regular array? Thanks!
Regards, Ingwie.

Oi, this issue be old. But time to revisit the past! (not like we had the BTTF day this year, right? :) )
So, I just revisited this issue because I want to move from INI to Yaml, since it is naturally laid out to be more structured. However, PHP arrays are ok, but if almost the entire code uses Objects now and a library passes by that only returns an Array, that leads me to the question: how to convert? And ontop of that, Symfony's own Yaml parser is broken, as the "enable object support" flag is simply ignored.
Here is what I would suggest: Add a flag.
if($objectSupportEnabled) {
    $output = new ArrayObject;
} else {
    $output = array();
}

I am not a hundred percent sure of Spyc's source, since I get confused easily by parsers and the like. But it would allow for Spyc to keep it's default, but support the flexibility to return an object as well.

And there I just realized, @ryanuber had suggested a similar idea. Oops! (: Sorry. But, you get what I was wanting to say.


Fix asm.rs test

add 1, %rax in AT&T syntax gets interpreted as add rax, qword ptr [1]. Use $ to make it explicit that it is a constant rather than an address.

I don't know how this didn't result in a crash for cg_gcc (maybe GCC adds the $ for you?), but both LLVM and GAS (as used by Cranelift) produce code that crashes.

Thanks for finding this. I fixed it in #330.
cg_gcc added $ manually.


Logger.SetLevel

It appears that we can't (safely) configure the loglevels of loggers except for the standard logger.
For the standard logger, we can use the logrus.SetLevel method, which protects the level with a mutex, but this mutex is unexported. So if we set var log = logrus.New() in our application (which is recommended in the docs), we actually can't configure the log level.
The only other thing we can do is to call SetLevel on the standard logger and then assigning it:

var log *logrus.Logger

func init(){
    logrus.SetLevel(logrus.DebugLevel)
    log = logrus.StandardLogger()
}

but that's a bit clumsier, and it also means we're restricted to just using the standard logger.
Am I missing an alternative?

:+1:  Had a similar situation.
Perhaps instead logrus.New() try instantiating logger on your own:
&log.Logger{
    Out: os.Stderr,
    Formatter: new(log.TextFormatter),
    Hooks: make(log.LevelHooks),
    Level: log.DebugLevel,
}

Unfortunately that doesn't work for us, because we need to be able to set the log level based on the configuration file that's passed. The only way to do that and also preserve the var log *logrus.Logger declaration as a drop-in replacement for the standard library log is to have SetLevel. Otherwise, it still introduces a race condition.
@sirupsen I'm happy to put together a PR for this if it's something you're interested in adding.

Yes, that's still a limitation - can't change the log level after it has been set,.
Would be interested in testing out your PR too.

This makes sense, feel free to propose a PR upstream.

@ChimeraCoder @sirupsen
because #512  had been merged - maybe it will make sense to change from func (logger *Logger) setLevel(level Level) to func (logger *Logger) SetLevel(level Level) { to make this method publicly available? Any downsides?

This has shipped 🎉

🎉