GCP ApiConfig creation always returns 404

What happened?
When creating gcp api config we get the following error:
    error: 1 error occurred:
        * Error creating ApiConfig: googleapi: got HTTP response code 404 with body: <!DOCTYPE html>

      <meta charset=utf-8>
      <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
      <title>Error 404 (Not Found)!!1</title>

404. That’s an error.

The requested URL /v1beta/projects/889354801458/locations/global/apis/projects/vital-dev-305313/locations/global/apis/vital-consumer-api-eu/configs?alt=json&apiConfigId=projects%2Fvital-dev-305313%2Flocations%2Fglobal%2Fapis%2Fvital-consumer-api-eu-config was not found on this server.  That’s all we know.

The URL looks wrong.
We are on the latest published versions of GCP and Pulumi
Steps to reproduce
export const DEFAULT_YAML: string = "../modules/api_spec/open_api_spec.yaml";

export const getYamlData = (cloudRunUrl: string, yamlLocation: string) => {
  const doc = yaml.load(fs.readFileSync(yamlLocation, "utf8"));
  // @ts-ignore
  doc["x-google-backend"] = {
    address: cloudRunUrl,
    disable_auth: true,
  };
  const data = yaml.dump(doc);
  return data;
};

 const gatewayApi = new gcp.apigateway.Api(
    `${baseName}-${getVitalRegion(region)}`,
    {
      apiId: `${baseName}-${getVitalRegion(region)}`,
    }
  );

    const apiConfig = cloudRunUrl.apply((cloudRunUrl) => {
      const yamlSpecForEnvironment = getYamlData(cloudRunUrl, DEFAULT_YAML);
      return gatewayApi.id.apply((apiId) => {
        return new gcp.apigateway.ApiConfig(`${apiId}-config`, {
          api: apiId,
          openapiDocuments: [
            {
              document: {
                path: "open_api_spec.yaml",
                contents: Buffer.from(yamlSpecForEnvironment).toString(
                  "base64"
                ),
              },
            },
          ],
        });
      });
    });

Expected Behavior
To create a config resource
Actual Behavior
404
Output of pulumi about
No response
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

Thank you for reporting this issue! Would it be possible to include a more complete repro? Perhaps a minimal version of how API was constructed to derive apiId, yamlSpecForEnvironment value, and this.opts if relevant. Thank you!

@t0yv0 I've updated the example

Sorry I still cannot repro as your code is not a complete runnable program. Here is something I tried that seems to work for me:
import * as pulumi from "@pulumi/pulumi";
import * as fs from "fs";
import * as gcp from "@pulumi/gcp";

const apiCfgApi = new gcp.apigateway.Api("apiCfgApi", {apiId: "api-cfg"});

const spec: string = btoa(JSON.stringify({
  "swagger": "2.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore",
    "description": "A sample API that uses a petstore as an example to demonstrate features in the swagger-2.0 specification",
    "termsOfService": "http://swagger.io/terms/",
    "contact": {
      "name": "Swagger API Team"
    },
    "license": {
      "name": "MIT"
    }
  },
  "host": "petstore.swagger.io",
  "basePath": "/api",
  "schemes": [
    "http"
  ],
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/pets": {
      "get": {
          "x-google-backend": {
              "address": "http://publicIP/api/v1/app/catalogos/direcciones"
          },
        "operationId": "petsget",
        "description": "Returns all pets from the system that the user has access to",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "A list of pets.",
            "schema": {
              "type": "array",
              "items": {
                "$ref": "#/definitions/Pet"
              }
            }
          }
        }
      }
    }
  },
  "definitions": {
    "Pet": {
      "type": "object",
      "required": [
        "id",
        "name"
      ],
      "properties": {
        "id": {
          "type": "integer",
          "format": "int64"
        },
        "name": {
          "type": "string"
        },
        "tag": {
          "type": "string"
        }
      }
    }
  }
}));

const apiCfgApiConfig = new gcp.apigateway.ApiConfig("apiCfgApiConfig", {
    api: apiCfgApi.apiId,
    apiConfigId: "cfg",
    openapiDocuments: [
        {
            document: {
                path: "openapi.json",
                contents: spec,
            },
        }
    ],
});

Output of pulumi about:
CLI
Version      3.48.0
Go Version   go1.19.2
Go Compiler  gc

Plugins
NAME    VERSION
gcp     6.43.0
nodejs  unknown

Host
OS       darwin
Version  13.0.1
Arch     x86_64

This project is written in nodejs: executable='/Users/t0yv0/.nix-profile/bin/node' version='v16.16.0'

Current Stack: dev

TYPE                                URN
pulumi:pulumi:Stack                 urn:pulumi:dev::i951::pulumi:pulumi:Stack::i951-dev
pulumi:providers:gcp                urn:pulumi:dev::i951::pulumi:providers:gcp::default_6_43_0
gcp:apigateway/api:Api              urn:pulumi:dev::i951::gcp:apigateway/api:Api::apiCfgApi
gcp:apigateway/apiConfig:ApiConfig  urn:pulumi:dev::i951::gcp:apigateway/apiConfig:ApiConfig::apiCfgApiConfig

Found no pending operations associated with dev

Backend
Name           pulumi.com
URL            https://app.pulumi.com/t0yv0
User           t0yv0
Organizations  t0yv0, pulumi

Dependencies:
NAME            VERSION
@pulumi/gcp     6.43.0
@pulumi/pulumi  3.48.0
@types/node     14.18.33

Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/ by default

A very similar thing is happening to me.
Commands:
export GOOGLE_APPLICATION_CREDENTIALS=key.json
export PULUMI_CONFIG_PASSPHRASE=""
export GITHUB_TOKEN=${PULUMI_TOKEN_GITHUB}
pulumi login ${PULUMI_STATE_BUCKET}
pulumi stack select ${CI_COMMIT_BRANCH}
pulumi config set gcp:project REDACTED
pulumi config set gcp:region REDACTED
pulumi preview

Pulumi.yaml
name: Vega
runtime: yaml
resources:
  pulumi-data-uk:
    type: google-native:storage/v1:Bucket
    properties:
      location: EU
  pulumi-api-gw-api:
    type: google-native:apigateway/v1:Api
    properties:
      apiId: api-gw-dev-id
      displayName: api-gw-dev-id
      location: global
  pulumi-api-gw-service-account:
    type: google-native:iam/v1:ServiceAccount
    properties:
      accountId: pulumi-api-gw-service-account
      displayName: pulumi-api-gw-service-account
  pulumi-api-gw-config-native:
    type: google-native:apigateway/v1:Config
    properties:
      apiConfigId: pulumi-api-gw-config-native
      apiId: ${pulumi-api-gw-api.id}
      location: global
      openapiDocuments:
        - document:
            path: "spec.yaml"
            contents: "<REDACTED>"
  pulumi-api-gw-config-classic:
    type: gcp:apigateway:ApiConfig
    properties:
      api: ${pulumi-api-gw-api.id}
      apiConfigId: pulumi-api-gw-config-classic
      openapiDocuments:
        - document:
            path: "spec.yaml"
            contents: "<REDACTED>"

I used both gcp:apigateway:ApiConfig and google-native:apigateway/v1:Config to see which one works but they both cause similar error.
Pulumi/GCP is looking for a config that is creating and it can't find it.
configs/pulumi-api-gw-config-native' was not found

apiConfigId=pulumi-api-gw-config-classic was not found on this server

➜  pulumi up
Previewing update (dev):
     Type                                   Name                          Plan
     pulumi:pulumi:Stack                    Vega-dev
 +   ├─ google-native:apigateway/v1:Config  pulumi-api-gw-config-native   create
 +   └─ gcp:apigateway:ApiConfig            pulumi-api-gw-config-classic  create

Resources:
    + 2 to create
    4 unchanged

Do you want to perform this update? yes
Updating (dev):
     Type                                   Name                          Status                  Info
     pulumi:pulumi:Stack                    Vega-dev                      **failed**              1 error
 +   ├─ google-native:apigateway/v1:Config  pulumi-api-gw-config-native   **creating failed**     1 error
 +   └─ gcp:apigateway:ApiConfig            pulumi-api-gw-config-classic  **creating failed**     1 error

Diagnostics:
  google-native:apigateway/v1:Config (pulumi-api-gw-config-native):
    error: error sending request: googleapi: Error 404: Resource 'parent resource not found for projects/REDACTED/locations/global/apis/v1%2Fprojects%REDACTED%2Flocations%2Fglobal%2Fapis%2Fapi-gw-dev-id/configs/pulumi-api-gw-config-native' was not found
    Details:
    [
      {
        "@type": "type.googleapis.com/google.rpc.ResourceInfo",
        "resourceName": "parent resource not found for projects/REDACTED/locations/global/apis/v1%2Fprojects%REDACTED%2Flocations%2Fglobal%2Fapis%2Fapi-gw-dev-id/configs/pulumi-api-gw-config-native"
      }
    ]: "https://apigateway.googleapis.com/v1/projects/REDACTED/locations/global/apis/v1%2Fprojects%REDACTED%2Flocations%2Fglobal%2Fapis%2Fapi-gw-dev-id/configs?apiConfigId=pulumi-api-gw-config-native" map[apiConfigId:pulumi-api-gw-config-native apiId:v1/projects/REDACTED/locations/global/apis/api-gw-dev-id location:global openapiDocuments:[map[document:map[contents:REDACTED path:spec.yaml]]] project:REDACTED]

  gcp:apigateway:ApiConfig (pulumi-api-gw-config-classic):
    error: 1 error occurred:
        * Error creating ApiConfig: googleapi: got HTTP response code 404 with body: <!DOCTYPE html>

      <meta charset=utf-8>
      <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
      <title>Error 404 (Not Found)!!1</title>

404. That’s an error.

The requested URL /v1beta/projects/REDACTED/locations/global/apis/v1/projects/REDACTED/locations/global/apis/api-gw-dev-id/configs?alt=json&apiConfigId=pulumi-api-gw-config-classic was not found on this server.  That’s all we know.

  pulumi:pulumi:Stack (Vega-dev):
    error: update failed

Resources:
    4 unchanged

Duration: 5s

I have the same problem creating the API config using Python
# API Gateway -----------------------------
# Create the API Gateway
api = apigateway.Api.get("linkedai-integration", id="linkedai-integration")

# Assume the configuration file is base64-encoded, replace with actual content
config_path = os.path.join(Path(__file__).parent,"api_definition.yaml")
config_contents = base64.b64encode(open(config_path).read().encode()).decode()

# Create an ApiConfig resource for the API
api_config = apigateway.ApiConfig(
    "api-config",
    api_config_id="",
    api=api.name,
    openapi_documents=[
        gcp.apigateway.ApiConfigOpenapiDocumentArgs(
            document=gcp.apigateway.ApiConfigOpenapiDocumentDocumentArgs(
                path=config_path,
                contents=config_contents,
            ),
        )
    ],
)

# # Create a Gateway for the API
gateway = apigateway.Gateway("linkedai-webhooks",
    region="us-central1",
    api_config=api_config.id,
    gateway_id="linkedia-webhooks",
)

I got this error
The requested URL /v1beta/projects/autonomy-286821/locations/global/apis/projects/autonomy-286821/locations/global/apis/linkedai-integration/configs?alt=json&apiConfigId=terraform-20230608225806753600000001 was not found on this server.  That’s all we know.

It looks like the issue is y'all are using api_config.id instead of api_config.api_id.
The .id includes the prefix /projects/<project>/locations/global/apis/ so you end up with that twice in the request URL. The .api_id is just the api id without that prefix.

import pulumi_gcp as gcp

def encode_file_b64():
    import base64

    with open("openapi.yaml", "rb") as binary_file:
        binary_file_data = binary_file.read()
        base64_encoded_data = base64.b64encode(binary_file_data)
        base64_output = base64_encoded_data.decode("utf-8")

        return base64_output

def create_api():
    api = gcp.apigateway.Api("carechat", api_id="carechat")

    api_config_resource = gcp.apigateway.ApiConfig(
        "carechat_api_config",
        api=api.id,
        api_config_id="carechat_api_config",
        openapi_documents=[
            {
                "document": {
                    "contents": encode_file_b64(),
                    "path": "openapi.yaml",
                },
            }
        ],
    )

Still didnt figure out the problem, I am getting 404.

404. That’s an error.

The requested URL /v1beta/projects/primary-care-378721/locations/global/apis/projects/primary-care-378721/locations/global/apis/carechat/configs?alt=json&apiConfigId=carechat_api_config was not found on this server.  That’s all we know.

Hi, i have the same issue just creating the config.
Like @mythral stated, there seems to be an issue with the resource url that is being checked. But the origin is unclear to me, is it relates to the actual resource i am trying to create.
i get the same error output: Error creating ApiConfig: googleapi: got HTTP response code 404
For me the api path reads (line breaks to show the doubling of /projects/<project_id>/locations/global/apis):
/v1beta
/projects/<project_id>/locations/global/apis
/projects/<project_id>/locations/global/apis
/<api_id>/configs?alt=json&apiConfigId=<config_id>
Any hint is highly appreciated!


[WIP] Refactor worker spawners

This PR moves worker spawners (most of the code that runs in the worker master) into their own package, as well as abstracting out many common behaviors into library functions.
Still a lot of loose ends, but I will update this once there is a more comprehensive list of changes.
This is based off of a branch where I did nothing but move the code (which obviously doesn't compiler, but it makes the diff easier to read).  I ended up finding several functions later that needed to move, so I will update the base branch once this is more complete.

Made another pass, I think this is mostly ready to go, the only important thing left to resolve is the channel code in forLatestCommit. There's also the minor naming stuff with FAILED vs FAILURE.


A question about Tutorial 6 - Transformers

Hello, I am trying to implement your Tutorial 6 for sequences with different lengths. I am little confused about getting non zero values for the padded positions (where we don't have any 'non-padding' tokens) in the 'value' matrix.
This is the example I use;
num_heads = 2
seq_length=4
batch_size=2
head_dim = embed_dim // num_heads

embed = torch.nn.Embedding(5, embed_dim, 0)
s =  torch.tensor([[1,2,0, 0], [1,2,3,4] ]) # here 0 is the padding token
e = embed(s)

qkv_proj = nn.Linear(embed_dim, 3*embed_dim)
qkv = qkv_proj(e)
qkv = qkv.reshape(batch_size, seq_length, num_heads, 3*head_dim)
qkv = qkv.permute(0, 2, 1, 3) # [batch_size, num_heads, seq_length, heads]
q, k, v = qkv.chunk(3, dim=-1)

d_k = q.size()[-1]
attn_logits = torch.matmul(q, k.transpose(-2, -1))

Now regarding the mask, I created it using two ways. First method;
mask = torch.tensor([[0,0,1,1],[0,0,0,0]], dtype=torch.bool)
mask = mask.unsqueeze(1).unsqueeze(2).to(torch.bool)

attn_logits=attn_logits.masked_fill(mask, float('-inf') )

attention = F.softmax(attn_logits, dim=-1)
values = torch.matmul(attention, v)

values = values.permute(0, 2, 1, 3) # [Batch, SeqLen, Head, Dims]
values = values.reshape(batch_size, seq_length, embed_dim)

With the above, I get following for values matrix;
tensor([[[-0.4799, -0.1019],
         [-0.4799, -0.0683],
         [-0.4800, -0.0904],
         [-0.4800, -0.0904]],

        [[-0.5304,  0.0575],
         [-0.5303,  0.1040],
         [-0.5277,  0.1278],
         [-0.5298,  0.0873]]], grad_fn=<ReshapeAliasBackward0>)

So we have got non zero values for positions where we don't have actual values. That is, we have got         [-0.4800, -0.0904], for positions where we had the padding token zero. Is this Ok?

Now I can also define the mask as follows,
mask = torch.tensor([
  [[0,0,1,1], [0,0,1,1], [1,1,1,1], [1,1,1,1]],

  [[0,0,0,0], [0,0,0,0],[0,0,0,0], [0,0,0,0]]
], dtype=torch.bool)

mask = expand_mask(mask) # I used your expand_mask function

If I use the above mask, the values matrix I get looks like this,
tensor([[[-0.4799, -0.1019],
        [-0.4799, -0.0683],
        [    nan,     nan],
        [    nan,     nan]],

       [[-0.5304,  0.0575],
        [-0.5303,  0.1040],
        [-0.5277,  0.1278],
        [-0.5298,  0.0873]]], grad_fn=<ReshapeAliasBackward0>)

So, here, I have got nan for positions where we have zero paddings. We can replace these nans by zeros.
What do you think is the correct approach?

Hi, a padding token is generally masked such that unmasked tokens are not influenced by it. However, since these are tokens we do not need and do not influence the content tokens, we usually don't care about their values.
In your first example, you restrict all tokens in the first batch sequence to only look at the first two values. This includes the padding tokens, meaning that their new value depends on the first two tokens and thus are not zero. In the second case, you mask out everything for the padding tokens, which results in the division by zero.
Generally, which of the two options you choose is up to you, since they don't make a difference in the content tokens. If you prefer a clearer indication of where the padding tokens are throughout the network, you can go with option 2 and prevent NaNs. Otherwise, option 1 is the simpler version to implement.


Allow a easy override of the user lookup functionality.

I have the use case where I need to do a lookup for a user by the ID, and if that does not exist by E-mail address. I think this is useful for other people as well.

Oops, I forgot to describe what this PR changes. This change make it easy to override the functionality which lookup and/or create the user.


Give examples for moving data between a GCS bucket and Blob storage with azcopy or the azure cli

There are no examples and I'm not sure what to do with the JSON that is pasted into the docs.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 521d3279-cc0e-d002-c0d6-2cd367322fde
Version Independent ID: 06f8d134-ce4f-1667-bf37-bf9359a09ad0
Content: Copy data from Google Cloud Storage by using Azure Data Factory - Azure Data Factory
Content Source: articles/data-factory/connector-google-cloud-storage.md
Service: data-factory
GitHub Login: @linda33wj
Microsoft Alias: jingwang

@rbavery Thanks for your feedback! We will investigate and update as appropriate.

@rbavery You can use the json content while creating the linked service

Once you hit create button you can see the linked service on UI.

@rbavery We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Publish the latest branch to npm

I see that the last publication happened 2 months back. Since then a lot of chains have been added that we need for our project. I request to please publish the latest changes to npm.

It would be helpful here too @rmeissner

Hello @swetshaw @akshatcx
The new version 1.17.0 was published last week<EMAIL_ADDRESS>


Where to download and how to install?

Some guidelines in the README regarding where to download and how to install this mod would be great. Because I have no idea how to do either. Do I have to compile and build it myself? The releases only contain the source code.

The compiled mod is on https://thunderstore.io/package/KingEnderBrine/ProperSave/, you can use mod managers to install it and any other mod from the site


support %06d strings in mfem root files

-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.
Ticket number: 2117
Status: Resolved
Project: VisIt
Tracker: Feature
Priority: Normal
Subject: support %06d strings in mfem root files
Assigned to: Cyrus Harrison
Category: -
Target version: 2.9
Author: Cyrus Harrison
Start: 01/16/2015
Due date:
% Done: 0%
Estimated time:
Created: 01/16/2015 11:44 am
Updated: 01/21/2015 07:40 pm
Likelihood:
Severity:
Found in version: 2.12.3
Impact: 3 - Medium
Expected Use: 3 - Occasional
OS: All
Support Group: Any
Description:
Comments:
Hi Everyone,Updated the MFEM reader to support %06d format strings for domain identifiers. This style string will be written into mfem_root files directly from MFEM lib in the future.RC:Sending databases/MFEM/JSONRoot.CTransmitting file data .Committed revision r25422.Trunk:Sending databases/MFEM/JSONRoot.CTransmitting file data .Committed revision r25424.-Cyrus

2123_smime.p7s.zip


support ivfpq and cagra in benchmark

Inherites the default parameters from cuml/cuvs. They may not give the best performance.

build


My vs code stoped working..

VS Code recently stop working and showing the error
Error: could not open `C:\user\owner name '
I have tried every thing including reinstalling my vs code but still nothing, so far.
The language I am using is java and I have installed the all the basic extensions which are provided by the microsoft.
My programs are working on command prompt but not in the terminal of the vs code and i have also replaced my jdk due to this and tried to reinstall the vs code but the error is still their

/needsMoreInfo

I have basically the same issue.
Microsoft Windows [Version 10.0.19041.264]
(c) 2020 Microsoft Corporation. All rights reserved.
C:\Users\Deepak Kumar\Desktop\Java> "c:\Users\Deepak Kumar.vscode\extensions\vscjava.vscode-java-debug-0.26.0\scripts\launcher.bat" "C:\Program
Files\Java\jdk-14.0.1\bin\java.exe" --enable-preview -Dfile.encoding=UTF-8 @C:\Users\Deepak Kumar\AppData\Local\Temp\cp_cqgq09d9thyv5bgmq5xga7hr.argfile Addition
Error: could not open `C:\Users\Deepak'
C:\Users\Deepak Kumar\Desktop\Java>

This is the complete error showing here if i am trying to run my program

The issue appears to be spaces in the file path leading to the code.cmd; There may have been a breaking change in powershell.

C:\Users\Chris\AppData\Local\Programs\Microsoft : The term 'C:\Users\Chris\AppData\Local\Programs\Microsoft' is not
recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:1 char:1
+ C:\Users\Chris\AppData\Local\Programs\Microsoft VS Code\bin\code.cmd
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (C:\Users\Chris\...grams\Microsoft:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

Ah, I found the issue for me at least:
echo $env:ComSpec

had been set to powershell.exe. It appears to need to be C:\Windows\System32\cmd.exe

@ketanAggarwal58 does this https://github.com/microsoft/vscode/issues/98957#issuecomment-637062952 fix your issue as well?

@ketanAggarwal58 does this #98957 (comment) fix your issue as well?

I have checked my path it didn't have any spaces in them and it is still showing the error


Missing icon/font ?

Hi :)
Your (marvelous ;)) plugin has a pb on my setup with one font on "TEST" ....

What can be wrong ? VScode v1.68.1 ?
Thanks :)

Good evening,
I'm facing the same issue as you on my Arch Linux system. To address this problem, I've installed the "Noto Fonts" typeface from Google.
I hope this information proves helpful to you.

Thanks @leknoppix 🙂
I forgot this issue, moreover i do not face it anymore in 1.73 !


How do I engage or request something from a public sector org?

As a developer working in the public sector, how do I engage with a public sector org to request something?  For example, I'd love to request from BC Gov to share their .www2.gov.bc.ca LESS/CSS stylesheets so I can write apps for BC Government without having to reverse engineer www2.gov.bc.ca.
Similar to:
https://wet-boew.github.io/v4.0-ci/index-en.html

@GregTurner There is no central repository (i.e. skin) but we do have this Developers Guide to make it easier to build stuff. It has all the rules, colour codes, etc. Check it out and let us know if you have questions.
http://www2.gov.bc.ca/gov/content/about-gov-bc-ca/web-presence/developers-guide

Thanks @lmullane, more generally speaking, how would one request a resource be made available?

Given the nature of your request I think what you did (opening an issue here) works best. In the future we may look at other channels, but GitHub issues can work for now.
Any suggestions?

Could these github issues be surfaced on bcdevexchange.org?  It's nice to see a list of currently available API/code.  And even better for ones that people are requesting and means to do so.  It would help make visible the demand side to able to motivate government to make available resources.


Kafka benchmark support cases not available

Two common cases seems not possible with this benchmark:
Case1:
I checked the workload samples https://github.com/openmessaging/benchmark/tree/master/workloads and https://openmessaging.cloud/docs/benchmarks/ it seems not have way to set topic name when benchmarking
As we know the kafka logs dir have topic name as prefix, if not have name set for workload, it may not quite easy to find the related log directory.
Case 2:
Also, for the workload, it seems only support random case or same payload message over and over.
Sometimes I have some payload, like production logs 1GB, how can make it 10Kb message iterate the 1GB logs.
It seems right now, you can only pass 10Kb payload file, and the benchmark can only send 10Kb payload over and over. Not very common case.

Hi HackToday, would useRandomizedPayloads: true option be helpful for your case?

hi @plazma-prizma  I did use useRandomizedPayloads, it could not solve the above two cases. Thanks


pipenv install not respecting lock file since 2024.3.1 release.

Issue description
When running pipenv install using pipenv 2024.3.0, my lockfile appears to be respected and the correct dependency versions from it are installed. When doing the same with 2024.3.1, far newer versions of the dependencies are installed.
Expected result
The versions specified in the lock file should be used.
Actual result
The latest version of dependencies are installed instead.
Steps to replicate
To reproduce I've made this Dockerfile.
FROM python:3.9
WORKDIR /tmp/
RUN pip install pipenv==2024.3.1
RUN cat <<EOF > Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
sh = "*"
EOF
RUN cat <<EOF > Pipfile.lock
{
    "_meta": {
        "hash": {
            "sha256": "f9adf532d46f4787b6afe331abe415d5698ef7523cd6225605328b61f361d427"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "sh": {
            "hashes": [
                "sha256:39aa9af22f6558a0c5d132881cf43e34828ca03e4ae11114852ca6a55c7c1d8e",
                "sha256:75e86a836f47de095d4531718fe8489e6f7446c75ddfa5596f632727b919ffae"
            ],
            "index": "pypi",
            "version": "==1.14.1"
        }
    },
    "develop": {}
}
EOF
RUN pipenv install
RUN (pipenv graph | grep 1.14.1) || (pipenv graph && false)

When building this Dockerfile the build fails on the last stage as pipenv has not installed sh 1.14.1 and instead used sh 2.1.0.
The problem does not reproduce if the pipenv version is changed to 2024.3.0, or the --deploy flag is added to the install command.

$ pipenv --support
Pipenv version: '2024.3.1'
Pipenv location: '/usr/local/lib/python3.9/site-packages/pipenv'
Python location: '/usr/local/bin/python3.9'
OS Name: 'posix'
User pip version: '24.2'
user Python installations found:
PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.9.20',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '6.8.0-47-generic',
 'platform_system': 'Linux',
 'platform_version': '#47~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Oct  2 '
                     '16:16:55 UTC 2',
 'python_full_version': '3.9.20',
 'python_version': '3.9',
 'sys_platform': 'linux'}

System environment variables:

PATH
HOSTNAME
TERM
LANG
GPG_KEY
PYTHON_VERSION
PYTHON_SHA256
HOME
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:
Debug–specific environment variables:

PATH: /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG: C.UTF-8

Contents of Pipfile ('/tmp/Pipfile'):
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
sh = "*"

Contents of Pipfile.lock ('/tmp/Pipfile.lock'):
{
    "_meta": {
        "hash": {
            "sha256": "f9adf532d46f4787b6afe331abe415d5698ef7523cd6225605328b61f361d427"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "sh": {
            "hashes": [
                "sha256:7e27301c574bec8ca5bf6f211851357526455ee97cd27a7c4c6cc5e2375399cb",
                "sha256:bf5e44178dd96a542126c2774e9b7ab1d89bfe0e2ef84d92e6d0ed7358d63d01"
            ],
            "index": "pypi",
            "markers": "python_full_version >= '3.8.1' and python_version < '4.0'",
            "version": "==2.1.0"
        }
    },
    "develop": {}
}

@chrisgavin PR opened with additional test case if you wouldn't mind having a look.   CC @oz123

Thanks @matteius! That was quick. I can confirm this fix seems to do the job on the project I was having the issue on.


Sudden increase in disk space usage after/while running compaction (filling entire disk)

Bug Report
What did you do?
Ran two Prometheus "master/parent" instances with identical config, federating metrics from four other "child" Prometheus instances + scraping two Cloudwatch exporters
What did you expect to see?
Free disk space slowly decreasing
What did you see instead? Under which circumstances?
All disk space being used up in 30 mins on both instances at the same time, twice.
Four weeks ago from slow and steady increase up to 30 GiB and then within 30 minutes both instances had used up an additional 65 GiB for a total of 95 GiB.
Again during the past weekend, slow and steady up to 60 GiB and suddenly within 30 minutes both had used an additional 50 GiB for a total of 110 GiB.
Environment

System information:

AWS EC2 instance, Linux 4.9.0-7-amd64 x86_64

Prometheus version:

All six instances:
prometheus, version 2.10.0 (branch: HEAD, revision: d20e84d0fb64aff2f62a977adc8cfb656da4e286)
build user:       root@a49185acd9b0
build date:       20190525-12:28:13
go version:       go1.12.5

Prometheus configuration file:

global:
  scrape_interval: 10s
  scrape_timeout: 10s
  evaluation_interval: 10s
  external_labels:
    kubernetes_master_cluster_name: eu-west-1-ci
alerting:
  alertmanagers:
  - kubernetes_sd_configs:
    - role: pod
    bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
    tls_config:
      ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      insecure_skip_verify: false
    scheme: http
    timeout: 10s
    relabel_configs:
    - source_labels: [__meta_kubernetes_namespace]
      separator: ;
      regex: monitoring
      replacement: $1
      action: keep
    - source_labels: [__meta_kubernetes_pod_label_app]
      separator: ;
      regex: prometheus
      replacement: $1
      action: keep
    - source_labels: [__meta_kubernetes_pod_label_component]
      separator: ;
      regex: alertmanager
      replacement: $1
      action: keep
    - source_labels: [__meta_kubernetes_pod_container_port_number]
      separator: ;
      regex: null
      replacement: $1
      action: drop
rule_files:
- /etc/config/rules
- /etc/config/alerts
scrape_configs:
- job_name: federate
  honor_labels: true
  honor_timestamps: true
  params:
    match[]:
    - '{__name__="kube_namespace_labels"}'
    - '{__name__="container_spec_cpu_quota"}'
    - '{__name__="container_spec_cpu_period"}'
    - '{__name__="container_cpu_usage_seconds_total"}'
    - '{__name__="container_memory_usage_bytes"}'
    - '{__name__="kube_pod_container_resource_limits_memory_bytes"}'
    - '{__name__="container_network_receive_bytes_total"}'
    - '{__name__="container_network_transmit_bytes_total"}'
    - '{__name__="kubelet_volume_stats_capacity_bytes"}'
    - '{__name__="kubelet_volume_stats_available_bytes"}'
    - '{__name__="kube_pod_container_info"}'
    - '{__name__="kube_pod_container_status_ready"}'
    - '{__name__="node_memory_MemAvailable_bytes"}'
    - '{__name__="node_memory_MemTotal_bytes"}'
    - '{__name__="kube_node_labels"}'
    - '{__name__="node_filesystem_free_bytes"}'
    - '{__name__="node_filesystem_size_bytes"}'
    - '{__name__="node_cpu_seconds_total"}'
    - '{__name__="kube_cronjob_next_schedule_time"}'
    - '{__name__="kube_pod_container_status_restarts_total"}'
    - '{__name__="kube_deployment_spec_replicas"}'
    - '{__name__="kube_deployment_status_replicas_available"}'
    - '{__name__="kube_node_status_condition"}'
    - '{__name__="elasticsearch_cluster_health_status"}'
    - '{__name__="container_memory_working_set_bytes"}'
    - '{__name__="kube_pod_container_resource_limits_memory_bytes"}'

  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /federate
  scheme: http
  static_configs:
  - targets:
    - monitoring-prometheus-server.monitoring.svc.cluster.local:80
- job_name: inter-cluster-federate
  honor_labels: true
  honor_timestamps: true
  params:
    match[]:
    - '{__name__="kube_namespace_labels"}'
    - '{__name__="container_spec_cpu_quota"}'
    - '{__name__="container_spec_cpu_period"}'
    - '{__name__="container_cpu_usage_seconds_total"}'
    - '{__name__="container_memory_usage_bytes"}'
    - '{__name__="kube_pod_container_resource_limits_memory_bytes"}'
    - '{__name__="container_network_receive_bytes_total"}'
    - '{__name__="container_network_transmit_bytes_total"}'
    - '{__name__="kubelet_volume_stats_capacity_bytes"}'
    - '{__name__="kubelet_volume_stats_available_bytes"}'
    - '{__name__="kube_pod_container_info"}'
    - '{__name__="kube_pod_container_status_ready"}'
    - '{__name__="node_memory_MemAvailable_bytes"}'
    - '{__name__="node_memory_MemTotal_bytes"}'
    - '{__name__="kube_node_labels"}'
    - '{__name__="node_filesystem_free_bytes"}'
    - '{__name__="node_filesystem_size_bytes"}'
    - '{__name__="node_cpu_seconds_total"}'
    - '{__name__="kube_cronjob_next_schedule_time"}'
    - '{__name__="kube_pod_container_status_restarts_total"}'
    - '{__name__="kube_deployment_spec_replicas"}'
    - '{__name__="kube_deployment_status_replicas_available"}'
    - '{__name__="kube_node_status_condition"}'
    - '{__name__="elasticsearch_cluster_health_status"}'
    - '{__name__="container_memory_working_set_bytes"}'
    - '{__name__="kube_pod_container_resource_limits_memory_bytes"}'
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /federate
  scheme: https
  static_configs:
  - targets:
    - prometheus.example1.com
    - prometheus.example2.com
    - prometheus.example3.com
- job_name: cloudwatch-exporters
  honor_labels: true
  honor_timestamps: true
  scrape_interval: 1m
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  static_configs:
  - targets:
    - monitoring-cloudwatch-exporter-eu-west-1-prometheus-cloudwatch.monitoring.svc.cluster.local:9106
    - monitoring-cloudwatch-exporter-us-east-1-prometheus-cloudwatch.monitoring.svc.cluster.local:9106

Logs:
Logs for both instances when the issue occurred this weekend.
Why / how can it occur at the same time on both instances, twice, within a month?
Can I do anything to prevent it from happening again?
Both times this has happened I have increased the disk space after a while and then everything has worked fine except for big .tmp files laying around.
I am grateful for any help. If more info could help anyone understand I would be glad to provide it.
I was adviced here: https://groups.google.com/d/msg/prometheus-users/aYFrqsP__X4/lj6TDnw7AgAJ to create an issue.
Kind regards,
Jonatan

Can you share your logs?

Can you share your logs?

They were almost 200 lines to I attached them as files, you should see two links in the beginning of the logs section. Are you able to view them?

There's no obvious problems in the logs. Did you change your retention period at some point?

No, the two instances have had the same retention since being created around the end of april.

One thing I forgot to mention here is that all instances ran 2.8.0 four weeks ago during the first incident.

@bullboa was Prometheus still running after the compaction failure or was it killed?

We are currently running a bug scrub with the team.
We have a theory what happened here, but with newer Prometheus servers, it should not happen in the same way. We are closing this. If you still run into this issue with recent versions of Prometheus, please follow up.


Dark "Enter message..." in some pages

When visiting some pages, "Enter message..." text is shown in dark.

This has been fixed in a recent commit. Next release will include the fix.

@drwasho Reopen issue. It is still not fixed on v1.1.2

@jjeffryes Please reopen this issue.

@rhcastilhos I don't know how you are getting that result. I can't replicate it on my client.

Light colored pages have this problem

Try a page with the white theme

Here's a white theme. Maybe it's OS specific?

Aha. If you have a light theme, and you have advanced graphics on, and you have a low-contrast screen, it's hard to make out. I'll adjust it.


Android doesn't display the brandLogo

"@stripe/stripe-identity-react-native": "^0.2.5",
"react": "18.2.0",
"react-native": "0.72.5",

The built and released Android app doesn’t show the brandLogo:

The virtual device / Android Studio simulator shows the logo - but it’s too big:

The built and released iOS shows the logo in the correct size:

@ccen-stripe Can you reproduce this problem?

@ccen-stripe Can you reproduce this problem?

Hi @googol7 same here: https://github.com/stripe/stripe-identity-react-native/issues/139

hi @googol7 and @nicolasdevienne could you verify if your local environment is similar to our example app here and logo is loaded here?
You can try clone the repo and run yarn android under the example folder to test the android integration, this was just recorded on a real pixel 5 device

@ccen-stripe We do it exactly like you do it in the example app:
brandLogo: Image.resolveAssetSource(logo),
IMPORTANT: It's not sufficient to run it locally! You need to build the app and then run the bundle (e.g. by releasing it to closed testing). Can you try that please? Can you confirm the problem then?

fixed in https://github.com/stripe/stripe-identity-react-native/releases/tag/v0.2.10


Shape inference for QLinearConv, QLinearMatMul, & GlobalAveragePool

Hello,
I was trying to get onnx-mlir --EmitONNXIR resnet50v2.quantized.onnx to work and it required adding shape inference for QLinearConv & QLinearMatMul & GlobalAveragePool. It also required updating the TransformValueToONNXData template for int8_t and uint8_t because they use int32_t for the underlying data storage.
resnet50v2.quantized.onnx.zip

Can one of the admins verify this patch?

Another PR #339 has a template for Global pooling ops' shape inference. Could we remove GlobalAveragePool here to avoid duplication and focus on QLinear ops only. Thanks!

@tungld @thebyrd it is unfortunate that we ran into situations where engineering efforts are duplicated; especially when it seems like everyone is properly following the rules/best practices. @thebyrd let me do this: let me merge this patch right away, so that your contribution will get recorded & reflected in the project history, even though #339 may overwrite a part of your code contribution later.
I will follow up with fixes to reduce code duplication unless you feel strongly about doing it yourself.

@jenkins-droid test this please.


Move problem code to its own directory.

I should move the problem code to its own directory. Right now, the problem-evaluation logic lives in rules, which isn't ideal.  There's:

flattenMovetree, in treepath,
lots of problem code in problems.js

A better home would probably just be src/problems

After thinking about this, I don't think this is a necessary cleanup.


Is there any option to enable SMTP support?

Hi.
Is there any flag to build static curl with SMTP support?
Thank you!

Currently there is not support for this, but a feature flag to add this would be welcome!

I'm going to check the curl-sys sources to understand it to provide some patch ...


angle_metric vs val_angle_metric

What is the difference between the angle_metric and the val_angle_metric? I ask because I realize the goal is the make the angle_metric as high as possible, approaching 1, but it seems that the val_angle_metric is really the value you want to approach towards 1 and that's the way you guage whether your training is working. What's confusing to me is what's the difference between the two?
When I'm running a training, like I am now, I can see the angle_metric growing (usually) at each epoch, albeit growing slowly, but then at the end of the epoch it calculates the val_angle_metric. I'm just wondering why/how?

Likewise what's the difference with direction_metric and val_direction_metric?

There there two datasets, one for training and one for validation/testing. The training dataset should be about 80% of the data and the test dataset about 20%. Now during training, you validate the angle and direction metrics to see how well the network is learning. These are updated at every training step. However, there is a risk that you will overfit to this data. When you actually deploy the robot the images may look slightly different. This is where the test set come in. At the end of every epoch (one run through the training dataset), the current policy is validated against the complete test dataset. This is exactly what the validation metrics show. So, if the training metrics are high, it means the model fits the dataset well. If the validation metrics are high, it means it generalizes to slightly different data. So you want the training metrics as good as possible and the validation metrics as close as possible to them.


Need licensing info

I would like to use the template. But the repository doesn't seem to have license info. Can you please add them?

Hi Sivachandran,
I'm really sorry for the delay in getting back to you! I've just added the MIT License to the repository, so you can now use the template as you need.


difference is unmaintained

https://rustsec.org/advisories/RUSTSEC-2020-0095

I believe this is coming through https://github.com/assert-rs/predicates-rs/issues/94

I believe this is coming through https://github.com/assert-rs/predicates-rs/issues/94


Tabular: Fixed defect where model val_score was not updated properly

Issue #, if available:
Description of changes:
Issue caused when time_limit specified and repeated bagging occurs. Model will not update val_score beyond its initial score achieved after a single bagged fit. This fix makes it so the val_score is updated each time it is updated as a model in trainer. Before this fix, .leaderboard() would not show correct val_score for these models, however the best_model would still be chosen correctly.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-400-1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-400/1/index.html


issue while evaluating MMA expression in Python

Hi All,
This is a follow up to the question posted here
I've the following MMA code
edges = {1 <-> 2, 1 <-> 3, 1 <-> 4, 2 <-> 5, 2 <-> 6, 5 <-> 6,
   3 <-> 4, 3 <-> 7, 6 <-> 7, 7 <-> 8, 2 <-> 9};

vd = {{75., 25., 0}, {115., 45., 0}, {10., 5., 0}, {45., 0, 0},
  {90., 60., 0}, {45., 55., 0}, {0, 25., 0}, {10., 50., 0}, {115.,  25.,0}};

vl = Range[Length@vd];

vcoords = MapIndexed[#2[[1]] -> # &, vd];
ew = {1 \[UndirectedEdge] 2 -> 49.6, 1 \[UndirectedEdge] 3 -> 74.4,
 1 \[UndirectedEdge] 4 -> 49.6, 2 \[UndirectedEdge] 5 -> 37.2,
 2 \[UndirectedEdge] 6 -> 74.4, 5 \[UndirectedEdge] 6 -> 49.6,
 3 \[UndirectedEdge] 4 -> 37.2, 3 \[UndirectedEdge] 7 -> 24.8,
 6 \[UndirectedEdge] 7 -> 62, 7 \[UndirectedEdge] 8 -> 37.2,
 2 \[UndirectedEdge] 9 -> 24.8}

g3d = Graph3D[vl, edges, VertexCoordinates -> vcoords,
  EdgeWeight -> ew, VertexLabels -> Placed["Name", Center],
  EdgeLabels -> {e_ :> Placed["EdgeWeight", Center]},
  VertexSize -> .3, VertexStyle -> Red]
vars3d = Array[Through[{x, y, z}@#] &, Length @ vd];

λ = 1/100.;

obj3d = Total[(Norm[vars3d[[First@#]] - vars3d[[Last@#]]] - # /. ew)^2 & /@
  EdgeList[g3d]] +  λ Total[Norm /@ (vars3d - vd)];

lbnd = 0;
ubnd = 500;

solution3d = Last@Minimize[{obj3d, And @@ Thread[lbnd <= Join @@ vars3d <= ubnd]},
    Join @@ vars3d];

edgeLengths3d = # -> Norm[vars3d[[First@#]] - vars3d[[Last@#]]] /.
     solution3d & /@ EdgeList[g3d];

Grid[Prepend[{#, # /. ew, # /. edgeLengths3d} & /@
   EdgeList[g3d], {"edge", "EdgeWeight", "Edge Length"}],
 Dividers -> All]

Following the suggestions offered in the link shared above I installed wolframclient and tried the following

from wolframclient.evaluation import WolframLanguageSession
session = WolframLanguageSession()
from wolframclient.language import wlexpr

session.evaluate(wlexpr(
'edges = {1 <-> 2, 1 <-> 3, 1 <-> 4, 2 <-> 5, 2 <-> 6, 5 <-> 6, 3 <-> 4, 3 <-> 7, 6 <-> 7, 7 <-> 8, 2 <-> 9};'

'vd = {{75., 25., 0}, {115., 45., 0}, {10., 5., 0}, {45., 0, 0},{90., 60., 0}, {45., 55., 0}, {0, 25., 0}, {10., 50., 0}, {115.,  25.,0}};'

'vl = Range[Length@vd];'
'vcoords = MapIndexed[#2[[1]] -> # &, vd];'
'ew = {1 \[UndirectedEdge] 2 -> 49.6, 1 \[UndirectedEdge] 3 -> 74.4,'
'1 \[UndirectedEdge] 4 -> 49.6, 2 \[UndirectedEdge] 5 -> 37.2,'
'2 \[UndirectedEdge] 6 -> 74.4, 5 \[UndirectedEdge] 6 -> 49.6,'
'3 \[UndirectedEdge] 4 -> 37.2, 3 \[UndirectedEdge] 7 -> 24.8,'
'6 \[UndirectedEdge] 7 -> 62, 7 \[UndirectedEdge] 8 -> 37.2,'
'2 \[UndirectedEdge] 9 -> 24.8};'

'g3d = Graph3D[vl, edges, VertexCoordinates -> vcoords,EdgeWeight -> ew, VertexLabels -> Placed["Name", Center],'
'EdgeLabels -> {e_ :> Placed["EdgeWeight", Center]},'
'VertexSize -> .3, VertexStyle -> Red];'

'vars3d = Array[Through[{x, y, z}@#] &, Length @ vd];'
'λ = 1/100.;'
'obj3d = Total[(Norm[vars3d[[First@#]] - vars3d[[Last@#]]] - # /. ew)^2 & /@EdgeList[g3d]] +  λ Total[Norm /@ (vars3d - vd)];'
'lbnd = 0;'
'ubnd = 500;'

'solution3d = Last@Minimize[{obj3d, And @@ Thread[lbnd <= Join @@ vars3d <= ubnd]},Join @@ vars3d];'

'edgeLengths3d = # -> Norm[vars3d[[First@#]] - vars3d[[Last@#]]] /.solution3d & /@ EdgeList[g3d];'

'Grid[Prepend[{#, # /. ew, # /. edgeLengths3d} & /@EdgeList[g3d], {"edge", "EdgeWeight", "Edge Length"}],Dividers -> All];'

'z1 = Values[solution3d] // Partition[#, 3] &;'

'theFile = File["op.txt"];'

'Export[theFile, z1, "Table"];'
    ))

but I get the following error
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Undefined message MessageName[Message, msgl] with arguments {$MessageList}
The problem may be unbounded. Specifying a value for MaxIterations greater than 5000 may improve the solution.
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Undefined message MessageName[Message, msgl] with arguments {$MessageList}
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Further output of MessageName[StringForm, string] will be suppressed during this calculation.
The problem may be unbounded. Specifying a value for MaxIterations greater than 5000 may improve the solution.
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Undefined message MessageName[Message, msgl] with arguments {$MessageList}
The problem may be unbounded. Specifying a value for MaxIterations greater than 5000 may improve the solution.
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Undefined message MessageName[Message, msgl] with arguments {$MessageList}
String expected at position 1 in StringForm[MessageName[General, msgl], $MessageList].
Further output of MessageName[StringForm, string] will be suppressed during this calculation.
The problem may be unbounded. Specifying a value for MaxIterations greater than 5000 may improve the solution.

Suggestions on how to resolve this error will be really helpful.

This is the same WolframLanguage bug as witnessed in #22. It'll be fixed in 12.2 (I can reproduce on a 12.1 build, but not with a recent prototype build).
Messages that should be Quiet-ed, are incorrectly issued in some cases.

@DbxDev Thanks, could you please suggest how to get this working until the issue is fixed?

@DbxDev Thanks, could you please explain how to pass the input arguments from python to .m file and loas it back as MMA expression in python {edges,vl,ew,vd}=ImportString[#,"PythonExpression"]&/@ {"[(1,2),(1,3),(1,4),(2,5),(2,6),(5,6),(3,4),(3,7),(6,7),(7,8),(2,9)]", "[1,2,3,4,5,6,7,8,9]","{(1,2):49.6,(1,3):74.4,(1,4):49.6,(2,5):37.2,(2,6):74.4,(5,6):49.6,(3,4):37.2,(3,7):24.8,(6,7):62,(7,8):37.2,(2,9):24.8}", "{1:[75.,25.,0],2:[115.,45.,0],3:[10.,5.,0],4:[45.,0,0],5:[90.,60.,0],6:[45.,55.,0],7:[0,25.,0],8:[10.,50.,0],9:[115.,25.,0]}"}; ?

@DbxDev
I tried
   from wolframclient.evaluation import WolframLanguageSession
   from wolframclient.language import wl, wlexpr, Global

    session = WolframLanguageSession()

    edges = [(1, 2), (1, 3), (1, 4), (2, 5), (2, 6), (5, 6), (3, 4), (3, 7), (6, 7), (7, 8), (2, 9)]
    vl = [1, 2, 3, 4, 5, 6, 7, 8, 9]
    ew = {(1, 2): 49.6, (1, 3): 74.4, (1, 4): 49.6, (2, 5): 37.2, (2, 6): 74.4, (5, 6): 49.6, (3, 4): 37.2, (3, 7): 24.8, (6, 7): 62, (7, 8): 37.2, (2, 9): 24.8}
    vd = {1: [75., 25., 0], 2: [115., 45., 0], 3: [10., 5., 0], 4: [45., 0, 0], 5: [90., 60., 0], 6: [45., 55., 0], 7: [0, 25., 0], 8: [10., 50., 0], 9: [115., 25., 0]}
    with WolframLanguageSession() as s:
        solution3d = wl.Get('test.m')
        s.evaluate(Global.solution3d(edges, vd, vl, ew))

test.m includes
solution3d[edges_, vd_, vl_, ew_] := (

    {edges,vl,ew,vd}=ImportString[#,"PythonExpression"]&/@{edges,vl,ew,vd};
    edges = UndirectedEdge @@@ edges; vcoords = List @@ vd;
    ew = Normal @ KeyMap[UndirectedEdge @@ # &,ew];

    g3d = Graph3D[vl, edges, VertexCoordinates -> vcoords, EdgeWeight->ew, VertexLabels->Placed["Name",Center],
    EdgeLabels->{e_:>Placed["EdgeWeight",Center]}, VertexSize->.5, BaseStyle->16];

    edgeLengths3d = # -> Norm[vars3d[[First@#]] - vars3d[[Last@#]]] /.solution3d & /@ EdgeList[g3d];

    vars3d = Array[Through[{x, y, z}@#] &, Length@vd];
    \[Lambda] = 1/100.; lbnd = 0; ubnd = 500;

    obj3d = Total[(Norm[vars3d[[First@#]] - vars3d[[Last@#]]] - # /. (Rule @@@ ew))^2 & /@ EdgeList[g3d]] + \[Lambda] * Total[Norm /@ (vars3d - Values@vd)];
    solution3d = Last @ Minimize[{obj3d, And @@ Thread[lbnd <= Join @@ vars3d <= ubnd]}, Join @@ vars3d];

    edgeLengths3d = # -> Norm[vars3d[[First@#]] - vars3d[[Last@#]]] /.solution3d & /@ EdgeList[g3d];
    ResourceFunction["PrettyGrid"][{#, # /. ew, # /. edgeLengths3d} & /@EdgeList[g3d],  "ColumnHeadings" -> {"edge", "EdgeWeight", "Edge Length"}];

    z1 = Values[solution3d] // Partition[#, 3] &;

    theFile = File["result.txt"];

    Export[theFile, z1, "Table"];
)

I'm not sure if
 solution3d = wl.Get('test.m')
 s.evaluate(Global.solution3d(edges, vd, vl, ew))

this right, nothing is written to the txt file, unfortunately. Could you please have  a look?

@DbxDev This is a kind reminder

This goes beyond the scope of this ticket and should rather be discussed on stackoverflow.
You're confusing Wolfram Language representation in Python and actual code evaluation. The python code solution3d = wl.Get('test.m') represents a WL expression that is never evaluated and should be replaced with s.evaluate(wl.Get('test.m')). It'll declare the WL function Global`solution3d for the given session that you're using afterward.
Remember that WL code declared in Python (using wl.xxx syntax) must be evaluated in a session using (session.evaluate(...)) otherwise it stays inert!

@DbxDev Thank you. Unfortunately, I couldn't run it successfully after making the change that has been suggested. I have opened a new thread on stackoverflow as you suggested.


feat(ios): detect if app is running on a macOS desktop with Apple Silicon

Platforms affected
iOS 14 and up
Motivation and Context
When running on iOS14, on a Mac that has an Apple Silicon processor (ARM64 instead of Intel), this change allows you to determine this condition using cordova-plugin-device.
This minor change exposes the iOS isiOSAppOnMac variable, which is the definitive way to detect that the Cordova iOS app is running on the Mac desktop on a non-Intel processor.
On Cordova apps running on the Apple Silicon desktop, the hardware menu (wifi, clock, cell strength, battery) goes away, and other container changes need to be compensated for, so access to this variable is essential.  This plugin seems like the best place to expose this variable.
Description
in the iOS NSProcessInfo under iOS 14+, a new system variable has been defined, this change exposes that variable.
[NSProcessInfo processInfo].isiOSAppOnMac

Testing
This change works on iOS 14 on macOS Big Sur Version 11.0 Beta 8 (20A5374i) running on an Apple A12Z processor.
The change causes no problem (returns false) on macOS Catalina Version 10.15.6 running on an Intel processor inside an iOS simulator (SE 2nd Gen, 8 Plus 13.3, 11 Pro 14.0, 11 13.3, iPadPro 9.7" 14.0, iPadPro 12" 4th gen 14.0), and on a physical
iPhone Xs Max running iOS 13.7.
Also no problem on an Android simulator for a Galaxy Tab S5e, and a Galaxy S9+ at API 30, and a Moto G5 plus (Android 8.1.10) physical phone.
Checklist

[ ] I've run the tests to see all new and existing tests pass
[ ] I added automated test coverage as appropriate for this change
[x] Commit is prefixed with (platform) if this change only applies to one platform (e.g. (android))
[ ] If this Pull Request resolves an issue, I linked to the issue in the text above (and used the correct keyword to close issues using keywords)
[x] I've updated the documentation if necessary

Thinking about this again I am confused again. Why would I want to know if the app runs on Apple ARM chips? It should not make a difference if it runs on Mac using Catalyst or natively with ARM, shouldn't it? I may need to know if it runs on desktop or mobile to adjust my styling etc?
Shouldn't the platform return something different if it runs on macOS? Technically it still runs the iOS platform but it's runnig on a Mac using Catalyst or Big Sur.
Other opinions?

I personally don't see what is the need of this new property. I feel that using both the device.model and device.platform should be enough to identify what is running on what device.
To recap from the documents printed in the README of this repo,
device.platform should return either iOS or Mac OS X. But since we are talking about building an iOS app with the Cordova iOS platform, iOS will always be returned. This value is hardcoded.
device.model should return one of the following two cases:
// iOS:     for the iPad Mini, returns iPad2,5; iPhone 5 is iPhone 5,1. See http://theiphonewiki.com/wiki/index.php?title=Models
// OSX:                        returns "x86_64"

From the Cordova iOS related source code, I am expecting to see either a simulator's model identifier or the hardware machine which is one of the two items above. When I did a quick test of the source code on a macOS, x86_64 is returned.
Basically, for an iOS app on a macOS, I am expecting to see the iOS platform on a x86_64 model. I suspect on a Apple Silicon, it would say instead iOS platform on an ARM64.
I personally think this might be enough, but can anyone explain why it might not be?

This is the object returned by device, when running on Apple Silicon.  (With the addition in this PR).
2020-10-19 09:28:00.803495-0700 We Vote[1530:17084] Dump Object window.device available: true
2020-10-19 09:28:00.803554-0700 We Vote[1530:17084] Dump Object window.device platform: iOS
2020-10-19 09:28:00.803605-0700 We Vote[1530:17084] Dump Object window.device version: 14.0
2020-10-19 09:28:00.803698-0700 We Vote[1530:17084] Dump Object window.device uuid: 4C9A03D3-4457-5253-8384-B4581A3F7396
2020-10-19 09:28:00.803778-0700 We Vote[1530:17084] Dump Object window.device cordova: 6.1.1
2020-10-19 09:28:00.804903-0700 We Vote[1530:17084] Dump Object window.device model: iPad8,9
2020-10-19 09:28:00.805041-0700 We Vote[1530:17084] Dump Object window.device manufacturer: Apple
2020-10-19 09:28:00.805150-0700 We Vote[1530:17084] Dump Object window.device isVirtual: false
2020-10-19 09:28:00.805631-0700 We Vote[1530:17084] Dump Object window.device serial: unknown
2020-10-19 09:28:00.806016-0700 We Vote[1530:17084] Dump Object window.device isiOSAppOnMac: true

Unfortunately 'iPad8,9' matches a real physical device, so it is not usable to detect isiOSAppOnMac condition.

I need access to isiOSAppOnMac so that I can tell that the app covers the entire window, and that the hardware menu is not present (Wifi signal, Cell signal, battery, etc).

@erisu I initially used cordova-plugins-diagnostic to find out that the architecture is ARM64.  Then I realized that both the phones and the Mac will be using ARM64 at some point, so the architecture can't be used to determine the equivalent of isiOSAppOnMac.
By the way, I have no attachment to the variable name isiOSAppOnMac, we can name it whatever we want, but I just defaulted to the name Apple chose.

@erisu I  adopted your suggestion, rebased, tested on a device simulator, and the desktop and it works fine.  The CI check is failing on a README.md merge problem, but the markdown looks good.  Maybe the ci needs to be forced to run again?

Closing/reopening PR to restart appveyor.

That didn't work..
Upon closer inspection, looks like your fork is out-dated from apace's master.
You're going to want to add apache's remote to your fork:
git remote add apache https://github.com/apache/cordova-plugin-device.git
Then fetch it:
git fetch apache
Then rebase apache's master into your master:
git rebase apache/master
You can do all of this on your own fork's master. During the rebase, you should hit the conflict that it's complaining about and solve it.

I should be able to merge this PR as soon as it is rebased and the conflicts are resolved.

I'm sorry, I use git all day, but only in a simple and repetitive way.  I followed breauteks instructions, and they looked successful.  I had to fix the problem in the README.md.   But it now looks like my PR contains all the changes since I forked -- is that the way it should look?
The good news, is that when I diff my branch with apache/master the changes look good -- ie the diff only shows the changes I want to have merged.

I'm sorry, I use git all day, but only in a simple and repetitive way.

No worries, I wasn't used to rebasing until I started contributing to Cordova/other open source projects myself. For the longest time I was afraid of rebasing (because it can be dangerous)
But the extra commits is still problematic. Not sure why the PR believes these are new commits. While the changes are okay, having all of these commits in this PR will be problematic if we ever have to revert this PR in the future for whatever reason.
It might be easier to create a new branch off of current master and cherry-pick the commits that is actually related to this PR.
I've gone ahead and created a PR against your own fork. You won't be able to merge it due to merge conflicts because you're fork's master is in a diverged state. But if the changes looks like they are matching what should be provided here, then there are instructions in the PR that should be enough to correct it the diverged state problem and once you force push your master, it should reflect here with your single commit encapsulating all your changes.

If you have any questions, feel free to ping me in slack

Thanks!  I'll do that on Thursday.
On 4/5/22 1:09 PM, Norman Breau wrote:

I'm sorry, I use git all day, but only in a simple and repetitive way.

No worries, I wasn't used to rebasing until I started contributing to
Cordova/other open source projects myself. For the longest time I was
afraid of rebasing (because it can be dangerous)
But the extra commits is still problematic. Not sure why the PR
believes these are new commits. While the changes are okay, having all
of these commits in this PR will be problematic if we ever have to
revert this PR in the future for whatever reason.
It might be easier to create a new branch off of current master and
cherry-pick the commits that is actually related to this PR.
I've gone ahead and created a PR
https://github.com/SailingSteve/cordova-plugin-device/pull/1 against
your own fork. You won't be able to merge it due to merge conflicts
because you're fork's master is in a diverged state. But if the
changes looks like they are matching what should be provided here,
then there are instructions in the PR that should be enough to correct
it the diverged state problem and once you force push your master, it
should reflect here with your single commit encapsulating all your
changes.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/cordova-plugin-device/pull/133#issuecomment-1089278084,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABU2LBJFX3YB5JGYEVPIACLVDSMYDANCNFSM4SSQQEGA.
You are receiving this because you authored the thread.Message ID:
@.***>


A minor syntax highlighting issue in Typescript/React documents

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.71.0
OS Version: Macos 12.0.1

Steps to Reproduce:

Create the React component below
See the syntax highlighting for the function keyword doesn't work

export const SvgDocument = forwardRef<SVGSVGElement, {
  width: number;
  height: number;
  nodes: NodeData[];
  edges: EdgeData[];
}>(function SvgDocument({ width, height, nodes, edges }, ref) {
// function body

It works fine when you move the type as a separate type definition
type Props = {
  width: number;
  height: number;
  nodes: NodeData[];
  edges: EdgeData[];
};
export const SvgDocument = forwardRef<SVGSVGElement, Props>(function SvgDocument({ width, height, nodes, edges }, ref) {
// function body

I don't know if both they use same library for syntax highligting (feels like they are same) but it seems to be working fine on github but has a minor glitch on vscode.

This is limitation of the tmLanguage grammar which doesnt let us determine if these are type arguments or expression followed by next variable declaration/binding pattern


Add possibility to set the statement class

Summary of the changes / Why this is an improvement
I currently have no way to test this but we need something to allow to set the statement class from Doctrine DBAL, see also https://github.com/crate/crate-dbal/pull/113#pullrequestreview-646391056.
Checklist

[x] CLA is signed

Did you know we added support for invoking the test suite easily by using Docker for running CrateDB [1] just the other day?
[1] https://github.com/crate/crate-pdo/blob/main/DEVELOP.rst#using-docker

@amotl  thx for the link, works great with docker. Seems like this is green now :)


Add an in-memory mode

For slow codecs, the exact details of IO aren't that important, as long as you aren't doing physical disk IO (i.e., as long as the reads are in the page cache, and the writes are effectively hidden by the page cache and delayed writeback).
For fast codecs, say those around 1 GiB/s or faster, the exact details of how the IO is performed can matter a lot. Especially on the write side, there are a lot of gremlins and non-determinism in how a modern OS like Linux will handle IO. For example, something like the write() syscall typically writes into the page cache, but not to the disk. Later, the OS will flush to the disk, but exactly when that happens depends on many factors such as how much time has elapsed, how much pressure there is for free pages, how many pages are dirty, etc. Sometimes writes may block on actual IO if you have enough dirty pages in the cache - so a typical pattern is that the first N MBs of data written to a quiet system is very fast, followed by a crash in speed down to close to the underlying disk speed once you hit some number of dirty pages. Sometimes the crash is well below the actual IO speed, depending on how the queued IO is handled, and then it covers back to the physical speed.
Well the exact details don't matter, but safe to say that getting repeatable IO performance can be tricky.
In light of this, it might make sense to have an enhancement which allows an "in memory" mode, at least for the purpose of benchmarking (which IMO is one of the really valuable things about squash). The write side can be purely in memory, as you can just "sink" the output of the codec to nowhere (perhaps you want to checksum it?). On the read side, it may not be feasible to buffer the entire file in memory, but perhaps that's fine for smaller files?

I agree that I/O is complicated, but if people want a purely in-memory mode all they have to do is not use the I/O or splicing APIs.  Actually, what you're suggesting can't really be done with the I/O or splicing APIs since they take FILE*s, but for a purely in-memory test you would need to provide input and output buffers—i.e. use squash_compress and squash_decompress.
I'm closing this issue, but if you want you can open up an issue for the benchmark.  My response will be:
First off, most of the issues around exactly how I/O is handled by the kernel are glossed over by using CPU time when benchmarking.  The current issue with a performance regression when using memory-mapped files on x86 (which wasn't there a couple months ago) aside, I think CPU time does a pretty good job of eliminating I/O from the equation.
I would be willing to add a flag to the benchmark to cause it to read the entire file into a malloc()ed buffer, malloc() another buffer for the output, then simply time squash_compress/squash_decompress operations, but I can't make that the default.  Some of the larger files will fail with some codecs on machines with less memory.  Until I added the mmap code to the old file-to-file API it would do that, and the OOM-killer would routinely strike it down.
Basically, this could be used for people wanting to run this benchmark code themselves.  I feel like that is a bigger audience than it probably should be… If you are running a benchmark yourself you should probably be doing it in your application, with your data.  Squash makes this really easy—all you have to do to change the codec is change a single string.  The only real exception, IMHO, is codec developers who are trying to optimize their implementation.

Ugh, I had totally forgotten than the benchmark is a separate project when I wrote that. You are completely correctly the the request doesn't make sense in the context of the squash API. I really meant for it to be a feature of the benchmark as you pointed out.
The problem of IO-incurred CPU time is only glossed over for codecs that aren't "extremely fast". For example, perhaps some IO strategy incurs 100 ms of CPU time per 1 GB read. Effectively, the IO itself can run at 10 GB/s (where time is measured CPU time, not wall time). For a codec running at 100 MB/s, or even 1 GB/s, that's just a small adjustment to the timing (about 1% error in the former case and about 10% in the latter). For a codec otherwise running at 20 GB/s, however, the benchmark would only report ~6.67 GBs, or about one third of the true value.
Such fast codecs are not likely to be used with actual typical IO devices, but rather in-memory or over some other very fast interface, so the true number is useful.
So perhaps I can re-open this against the benchmark at some point. There are a few options beyond the entire-file-in-buffer + time full buffer API approach. For example, you could still call the streaming API but measure the sum of time taken in each API call, or the IO could be run on a separate thread or process and only the thread/process running the plugin could be measured.

The problem of IO-incurred CPU time is only glossed over for codecs that aren't "extremely fast". For example, perhaps some IO strategy incurs 100 ms of CPU time per 1 GB read. Effectively, the IO itself can run at 10 GB/s (where time is measured CPU time, not wall time). For a codec running at 100 MB/s, or even 1 GB/s, that's just a small adjustment to the timing (about 1% error in the former case and about 10% in the latter). For a codec otherwise running at 20 GB/s, however, the benchmark would only report ~6.67 GBs, or about one third of the true value.

I have some doubts about this, but testing would be needed (surprise!).  AFAIK the kernel tends to be pretty good about keeping stuff cached, so my guess would be that even for very fast codecs the time required is insignificant.  Of course this breaks down when the machine doesn't have enough memory to keep the file around, but trying to do everything in-memory will not work well there anyways—you'll end up thrashing and it would be much worse than if you just mmaped the file.
This does bring up an interesting point that the first plugin to run may have a small disadvantage.  Perhaps for each file we should run the copy codec once first (and discard the results) to try to get the kernel to cache stuff.  Filed as quixdb/squash-benchmark#27.

So perhaps I can re-open this against the benchmark at some point. There are a few options beyond the entire-file-in-buffer + time full buffer API approach. For example, you could still call the streaming API but measure the sum of time taken in each API call,

Perhaps, but codecs which only support the all-at-once API will basically degrade to placing the entire file in a buffer, they would just also have the overhead of an extra memcpy and potentially some reallocs to get to that point.

or the IO could be run on a separate thread or process and only the thread/process running the plugin could be measured.

I'm already planning on using the POSIX AIO API in the future (see #127).  I'm not sure how helpful it will be for CPU time but wall-clock should be a big win, and it should be much more elegant than a thread.
Anyways, definitely feel free to open this back up in the benchmark.  It wouldn't be very difficult to implement.

Right, in the mmap read case, you might get "free" IO (actually still some loss in some scenarios due to no huge pages in the page cache) - but that doesn't apply to read() or write() syscalls (those definitely incur CPU time per byte) and in some cases not to mmap writes (that case is relatively complex).
Agree it's all just speculation until there are some real numbers and I'll try to get some (I don't have the wide variety of hardware you have to test on but I can certainly try on Intel/Linux and OSX.

On Oct 2, 2015, at 5:19 PM, Evan Nemerson<EMAIL_ADDRESS>wrote:
The problem of IO-incurred CPU time is only glossed over for codecs that aren't "extremely fast". For example, perhaps some IO strategy incurs 100 ms of CPU time per 1 GB read. Effectively, the IO itself can run at 10 GB/s (where time is measured CPU time, not wall time). For a codec running at 100 MB/s, or even 1 GB/s, that's just a small adjustment to the timing (about 1% error in the former case and about 10% in the latter). For a codec otherwise running at 20 GB/s, however, the benchmark would only report ~6.67 GBs, or about one third of the true value.
I have some doubts about this, but testing would be needed (surprise!). AFAIK the kernel tends to be pretty good about keeping stuff cached, so my guess would be that even for very fast codecs the time required is insignificant. Of course this breaks down when the machine doesn't have enough memory to keep the file around, but trying to do everything in-memory will not work well there anyways—you'll end up thrashing and it would be much worse than if you just mmaped the file.
This does bring up an interesting point that the first plugin to run may have a small disadvantage. Perhaps for each file we should run the copy codec once first (and discard the results) to try to get the kernel to cache stuff. Filed as quixdb/squash-benchmark#27.
So perhaps I can re-open this against the benchmark at some point. There are a few options beyond the entire-file-in-buffer + time full buffer API approach. For example, you could still call the streaming API but measure the sum of time taken in each API call,
Perhaps, but codecs which only support the all-at-once API will basically degrade to placing the entire file in a buffer, they would just also have the overhead of an extra memcpy and potentially some reallocs to get to that point.
or the IO could be run on a separate thread or process and only the thread/process running the plugin could be measured.
I'm already planning on using the POSIX AIO API in the future (see #127). I'm not sure how helpful it will be for CPU time but wall-clock should be a big win, and it should be much more elegant than a thread.
Anyways, definitely feel free to open this back up in the benchmark. It wouldn't be very difficult to implement.
—
Reply to this email directly or view it on GitHub.


JEQB-134/Fetch_PSK_from_psk_file_in_P2P_transport

Geting pre-shared key from file in P2P transport.

Where exactly is the part that fetches key from file? Is it handled generically by network params?

params.network_config.pre_shared_key.into_pre_share_key()?


[RabbitMQ] Fix dependency issue

Purpose

$title

Fixes #19920
PR to 1.0.x #19964
Check List

[x] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #19965 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #19965   +/-   ##
=======================================
  Coverage    19.9%    19.9%
=======================================
  Files          55       55
  Lines        1437     1437
  Branches      218      218
=======================================
  Hits          286      286
  Misses       1138     1138
  Partials       13       13

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc5d654...79ec694. Read the comment docs.


drop net_raw on rp related containers

Which issue this PR addresses:
Fixes https://msazure.visualstudio.com/AzureRedHatOpenShift/_workitems/edit/13005012/
What this PR does / why we need it:
drops net_raw to fix https://access.redhat.com/security/cve/cve-2020-13401
Test plan for issue:
e2e, deploy to int, and maybe even deploy to a single canary region to do some more manual testing to confirm this doesn't accidentally break anything.

Is there any documentation that needs to be updated for this PR?
n/a cve fix

/azp run ci

/azp run e2e

/azp run e2e


Towards enabling ninja to be built as a library

I wish to use ninja's code as a library inside of another application.
I'm well aware that ninja does not attempt to be a library, and am not asking for the Ninja project to support any kind of stable API.
I would like to remove the use of "using namespace std" in headerfiles, however, so that I don't have to deal with namespace pollution issues in my code that uses the ninja code as a library.
If I submit a pull request that does this, will it be accepted (assuming high enough quality) ?

Let's continue the "using namespace std"-discussion in #1614.
And for the library discussion #1499.


Improve temporary token security

Make the following changes to the linking process to prevent link
hijacking. Link hijacking could occur if User A logs out while a link
process token is still active. If User B then logs in in the same
browser (even if it's been closed) User B could link User A's identities
to their account.

/link/start now requires a login token, either in a cookie or posted
in the form.
/link/start sets a temporary token in the database identifying the
user (currently lasts for 30m). This token is returned to the user as a
session cookie, so that if the browser is closed it's no longer
available.
/link/complete now (effectively) adds the remote identities to the
temporary token and sets the token lifetime to 10m. The token is still
keyed to the specific user.
/link/choice and /link/pick will now throw an error if the user
identified by the login token does not match that of the link process
token.
delete temporary tokens from the DB when a login or link is complete.

Token refactoring:
Made a builder for temporary session data to prevent another refactor
like this in the future.
Temp session data is now assigned an operation and that operation is
checked when accessing the token. This prevents using a stolen link
token to log in or vice versa, or rewriting the identities associated
with a link token.
Step 2: /logout should clear all link in process tokens for the user
(since login in process tokens don't belong to a specific user, they
cannot be cleared).

@eapearson

Codecov Report

Merging #223 into develop will increase coverage by 0.08%.
The diff coverage is 98.62%.

@@              Coverage Diff              @@
##             develop     #223      +/-   ##
=============================================
+ Coverage      88.31%   88.39%   +0.08%
+ Complexity      1890     1884       -6
=============================================
  Files            119      118       -1
  Lines           6649     6697      +48
  Branches        1080     1079       -1
=============================================
+ Hits            5872     5920      +48
+ Misses           714      713       -1
- Partials          63       64       +1


Vite.config ts issue, Type 'Plugin_2[][]' is not assignable to type '() => PluginOption[]'

Hi,
I have an issue setting up this plugin with comlink.
Following the docs, I've added the plugin to the plugins array and to worker.plugins but got an error about incompatible types.
No overload matches this call.
  The last overload gave the following error.
    Type 'Plugin_2[][]' is not assignable to type '() => PluginOption[]'.
      Type 'Plugin_2[][]' provides no match for the signature '(): PluginOption[]'.ts(2769)
reporters-1evA5lom.d.ts(1789, 18): The last overload is declared here.

"vite": "^4.5.2",
"vite-plugin-comlink": "^4.0.2",
"typescript": "^4.9.3",

Would appreciate any hints on what's going on.

Please provide a reproduction for this = at least your full vite config

Here you go Stackblitz Vite Starter

Use
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { comlink } from 'vite-plugin-comlink';

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [comlink(), react()],
  worker: {
    plugins: () => [comlink()],
  },
});

As your config this is a change in vite 5 I didn't see...

Oh, right... The double array in the type confused me but it unfolds...  Thanks, Sebastian!


[W5][W11-2]Ren Ying

Good effort. You can improve upon:

Nice that you have updated tests.
It is good that you have incremental changes. You can try to have commits done in a batch for each enhancement.
Header comment missing for class TypicalPersonsVariedCase. Make sure you don't miss it for other classes as well.
After reading the comments, please close the PR.


Field name cannot be a number

I am trying to set a Field's name prop as a number (e.g. <Field name="123" component="input" />).
The problem is that when the fields key in state is created, instead of creating an object with the key "123", it creates an array with a length of 124, filled with nulls except the last index.
The issue I believe is that the setIn function, which is what creates the state, does not differentiate between dot and bracket syntax. If it sees a number, it creates an array. So even though it is being called as setIn(result, 'fields.123.visited', true), it is being considered as setIn(result, 'fields[123].visited', true).
The solution would probably be to have this function account for whether or not dot notation was used when encountering a number. The other solution would be to just not use numbers as names, but I feel that since this is valid in the DOM, it should be able to be done here as well.
I already wrote a failing test, but don't have a working solution yet.
I'm curious to know if I am understanding the problem correctly, and if this functionality is something that should be supported.
Thanks

First of all: please send a PR even though all you have is a failing test 👍
Second: it's better to use a permalink when linking code in GitHub (eg a commit or a tag), because the master branch moves and the code we're looking at now may not be the same in the future :)
Now... the syntax for these object paths usually follow the JS syntax, so you can't have something like a.1 because you can't have a variable that starts with a number.
The behavior that I saw in the code is that if the preceding path already exists, then the numeric key will be set in the object.
This is probably because the container of field values doesn't exist yet, so in your example 123 creates an array instead of an object.

TIL what a permalink is 😉
So basically this "bug" is a result of the initial state being empty (resulting fields not existing yet) instead of defaulting to the top level structure with field and values being empty objects.
Was this design intentional? Why can't the initial state be something like:
form-name: {
  registeredFields: {},
  fields: {},
  values: {}
}

😆 😆  What a funny side effect of this silly language we write in and setIn.

So what's the plan with this?
Should we add logic to setIn to detect dot vs bracket notation? It will mean that the function will no longer be obeying true JS syntax, because foo.123 will be considered valid.
I don't mind giving this a shot, just want to make sure everyone is on board.

I just spent an hour investigating this.
In v5, I had quite a bit of code that was parsing the dot-vs-bracket notation, and it was a veritable mess, and a source of very many bugs. It was a very conscious decision to switch to use _.toPath() in v6 to avoid that headache.
So I think the answer is that numeric field names are not supported. Prefix it with something. e.g. 'field_123'.
If you wrote a very thoroughly tested implementation of setIn, for both plain and immutable structures, I might consider a change of this policy.

Thank you so much for taking the time to look into this.
I will see if I can come up with something worth the headache. Totally understand that it's not trivial.
Thanks again for an amazing lib!


build: upgrade to TypeScript 4.0

This PR upgrades:

TypeScript to 4.0
Node.js to 14
Relevant packages to the latest versions
rollup-plugin-typescript was deprecated, so I've replaced it with rollup-plugin-typescript2. Same goes for rollup-* plugins

Notes:

Due to improved typechecking inference, HTMLCanvasElement, HTMLImageElement, CanvasRenderingContext2D were needed to be replaced by their platform-specific Canvas, Image, Context types.
Due to prettier being upgraded , diff contains some minor formatting changes

@joolean could you please take a look?

Hello, thanks for wanting to contribute! Unfortunately we no longer actively maintain this project. I would suggest creating your own fork of it.


graphql in gitpod

need to get graphql server to run in gitpod

todo

[x] make sure all deps are in the gitpod_docker file
[x] have a install_graphql.sh file which installs
[ ] have a run_... file which checks if compile was done, if not do it, and then run postgresql+ other components
[ ] have a test script to test (vlang?)

The install_graphql.sh file installs the graphql stack in a docker-compose and runs it as well.
After it starts, you need to make port 4000 public and then you can browse to that port + add /graphql at the end of the url


AzureCliCredential interrupts credential chain if Azure CLI is not logged in

Package Name: azure-identity
Package Version: 1.4.0b3
Operating System: Windows
Python Version: 3.7

Describe the bug
On Windows (haven't tried other OSes, but would expect the same), if the Azure CLI client is logged out, then the AzureCliCredential class will raise ClientAuthenticationError, which causes the credential chain to stop. A common need is to include InteractiveBrowserCredential as the final link in a chained credential, and this bug prevents it from being used.
To Reproduce
Steps to reproduce the behavior:

Use the following credential, which behind the scenes creates a chain:

        credential = identity.DefaultAzureCredential(
            exclude_shared_token_cache_credential=False,
            exclude_visual_studio_code_credential=False,
            exclude_cli_credential=False,
            exclude_interactive_browser_credential=False
        )

Make sure Visual Studio Code is logged out of Azure and that the Azure CLI is also logged out.
Use the credential for some purpose that would cause it to retrieve a token.

Expected behavior
I believe if the AzureCliCredential class were to raise CredentialUnavailableError instead, then the chain would continue.
Additional context
Found in the course of attempting to work around https://github.com/Azure/azure-sdk-for-net/issues/8934.

@mpderbec thanks for reporting this, @chlowell could you take a look at this

Thanks @mpderbec, this will be fixed in our next release.


posixToDayjs test not handling UTC properly

Why:
Tests fail because posixToDayjs test is not handling the UTC date properly
This commit:
Changed the test to take into consideration the UTC date.
Screenshots:

How to test:
yarn test

@johnschoeman using Date.parse is the same thing
> new Date(Date.parse("2020-07-01"))
2020-07-01T00:00:00.000Z
> new Date("2020/07/01")
2020-06-30T21:00:00.000Z

@johnschoeman using Date.parse is the same thing

yup, I misspoke prior to my comments edit, i think the issue is related to the Date parsing "2020-07-01" and "2020/07/01" to slightly different times?

Yeah it seems to be an inconsistency due to formating.
> new Date("2020/07/01 21:00")
2020-07-01T18:00:00.000Z
> new Date("2020-07-01 21:00")
2020-07-01T18:00:00.000Z
>
> new Date("2020/07/01")
2020-06-30T21:00:00.000Z
> new Date("2020-07-01")
2020-07-01T00:00:00.000Z


[Bug] Error in vllm_adapter when enable_chunked_prefill

Describe the error
If I enable chunked prefill without LMCache, there is no error. If I enable chunked prefill with LMCache, the error message is as follows:
Traceback (most recent call last):
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/vllm/worker/model_runner_base.py", line 112, in _wrapper
    return func(*args, **kwargs)
  File "/local/shaotingf/lmcache1/lmcache-vllm/lmcache_vllm/vllm_injection.py", line 45, in new_execute_model
    model_input, is_skip = lmcache_retrieve_kv(
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/nvtx/nvtx.py", line 116, in inner
    result = func(*args, **kwargs)
  File "/local/shaotingf/lmcache1/lmcache-vllm/lmcache_vllm/vllm_adapter.py", line 615, in lmcache_retrieve_kv
    rebuilt_model_input = build_partial_prefill_input(
  File "/local/shaotingf/lmcache1/lmcache-vllm/lmcache_vllm/vllm_adapter.py", line 669, in build_partial_prefill_input
    is_prefill = is_prefill_list[idx]
IndexError: list index out of range

How to reproduce
python3 main.py tests/tests.py -f test_chunk_prefill -o outputs/

I have pulled the latest LMCache and lmcache-vllm. The error message is different now. 8K workload is fine but 16K and 24K gives the following error.
Error message:
Traceback (most recent call last):
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/vllm/worker/model_runner_base.py", line 112, in _wrapper
    return func(*args, **kwargs)
  File "/local/shaotingf/lmcache1/lmcache-vllm/lmcache_vllm/vllm_injection.py", line 173, in new_execute_model
    output: SamplerOutput = self.model.sample(
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/vllm/model_executor/models/llama.py", line 466, in sample
    next_tokens = self.sampler(logits, sampling_metadata)
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/vllm/model_executor/layers/sampler.py", line 249, in forward
    logits = _apply_min_tokens_penalty(logits, sampling_metadata)
  File "/local/shaotingf/anaconda3/envs/lmcache1/lib/python3.10/site-packages/vllm/model_executor/layers/sampler.py", line 391, in _apply_min_tokens_penalty
    assert logits_applied == logits.shape[0]
AssertionError

How to reproduce

In lmcache-tests repo, git checkout vary-length
In lmcache-tests repo, run python3 main.py tests/tests.py -f test_chunk_prefill -o outputs/
With workload context = 16K / 24K, check the stderr log


New release?

Hi! I’m a fan of commit 3503922. Is there a chance that you could release a version 0.3.1 to PyPI?
Thanks and best!

Done and done

Maybe add a tag as well. 0.3.0 was previously tagged.

Wow, thank you. That was fast!


Fix override of Write(AsyncLogEventInfo)

Instead of RabbitMQTarget overriding the following method:
protected override void Write(AsyncLogEventInfo logEvent)
Then it should just move the code into this method, and change the try-catch to re-throw:
protected override void Write(LogEventInfo logEvent)
Then the handling of AsyncLogEventInfo.Continuation will be done automatically by NLog.

Have you got a pull request for this change?

@wonea Created #8 to resolve this issue.


[6.4.x] DROOLS-1286: Guided Decision Table: Unable to specify field type with Condition BRL fragment

Cherry picked from https://github.com/droolsjbpm/drools/pull/922
See https://issues.jboss.org/browse/DROOLS-1286

Jenkins please retest this.

Jenkins please retest this.

Jenkins please retest this.

Jenkins please retest this.


Page 2: Description of critical systems looses reader

In the second paragraph, some kind of critical systems fail due to "overload of critical systems". However, the text does not lead the reader and she remains unknown of what kind of systems she should have in mind. This costs important attention so early in the course.

The issue is also addressed in https://github.com/NFDI4Energy/EFZN_rdm/issues/11.


Parsing markdown code blocks as real code

Example markdown file:
-- start of .md file --
Hello code
this is nice piece of coce
defmodule Test do
  def render(assigns) do
    ~L"""

<%= @current %>

    """
  end

-- end of .md file ---
and the warning:
warning: assign @current not available in EEx template. Please ensure all assigns are given as options. Available assigns: [:author, :author_image, :date, :dependency_chain, :env, :extension, :image, :input_file, :permalink, :preprocessor, :tags, :title]

Looks like elixir trying to parse code in code block of md file like it's read eex.

Yeah, that's the issue! I wonder if there's a way to "escape" that elixir, or do I have to execute the elixir in the parser.


Need a way to know if Model properties have changed

There are several use-cases where I need to know if a property has been changed. Either the user editing a two-way binding, they navgiated to a new page, or changed a selection.  In the Observable world I would have used onValueChanged, but I don't see an equivalent for this.

You can define a $requestChange method on your model. Note that, if this method is defined, you need to return true to confirm that the change should go through. This also won't work for arrays being updated.
class {
    $requestChange(key, value) {
        return true;
    }
}

What context are we talking about? In JavaScript?
Are we talking about the model knowing about changes on itself?
The intended design is that the model is not internally reactive. The model is only reactive when seen from the outside. The world interacts with the model through methods, and if the model needs to notify itself internally about changes, this must be solved by explicit logic.
The weird corner case is two-way bindings, for which there are two patterns to "respond" to changes:

Using getters. Getters are automatically recomputed when relevant data visible to the class is changed, such as the content of an array. This should be a pure functional response, and is the preferred way.
In weird corner cases, using things like $requestChange is possible. However, I would consider this an antipattern, and we should look for other options to achieve the goal.

@mortoray I would like to understand your exact use cases. I am not out of the gate buying the general premise that "there are use cases" that need to be solved in an explicit reactive model.

I have a few cases:

remote lookups attached to entry criteria, such as interactive search results. These cannot be reactively bound as there needs to be timing logic, and a control over requesting and cancelling results
persisting data when it's changed
costly filters, such as in my Netrunner card browser. This is related to the first one for interactive results. I can't rely on having a derived property that updates continuously, it's too expensive
mapping to new UI components. For example, as you select items new editors appear on the screen. Ignoring the expensive, it's important that  the objects are preserved and new ones not created. The JS needs controls of this creation.

We may not need this yet. I'll run some more tests to see if I can get away without it (some of the other High model issues being fixed might alleviate some of the problems here)


chore(docs): update node.md

🚀 What's changed?

The import name needs to be corrected. I'm just changing the name to the correct one.

🐛 Fixes

[x] Rename from 'CustomNodeof import fromOverwriteCustomNode.vuetoOverwriteCustomNode`.

Thanks for the PR, I'll merge it asap :)

Can you do me a favor and edit the commit message to chore(docs): update node.md?

Sure, I will send a commit message soon.

@bcakmakoglu I have already committed message to chore(docs): update node.md


Fix tree traversal C++ code

node has no member children_size, using children.size();
create_node is wrong, using create_tree

I approve of the PR, but this will be the first PR we are merging from you. Would you mind adding your name to the CONTRIBUTORS.md file?

I don't mind. You can, of course.

Alright, then please add the name you prefer to the CONTRIBUTORS.md file in this PR


⚠ Bofrost.de has degraded performance

In 246b76b, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 896 ms

Resolved: Bofrost.de performance has improved in ea510b4 after 13 minutes.


add hadoop user to base accumulo image

What I am finding is that when working on HDFS instance that has permissions enabled, like EMR, it's important to be the correct user. Currently if you were to try to accumulo init on EMR you would get something like:
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=root, access=WRITE, inode="/accumulo/version/7":hdfs:hadoop:drwxr-xr-x

Weirdly the user in question is not the owner of the container process but the user inside the container. Adding this user so one can:
docker run --rm -u hadoop ....

solves the issue.
Is there a more graceful way to handle this case? While hadoop is the only user, its probably not the only possible option in the wild.

That's definitely a nice solution (workaround) to a problem. Already were some cases with that, and that was one of motivation points to have hadoop user inside dev images.

Thanks, nice catch!