Axios Safe JSON parsing

JSON.parse will throw on any input that isn't a string. Interceptors may change the data to return as a valid JS object/array/etc even if the client instance was controlled enough to ensure that every returned response data point is a JSON parseable string.
While not perfect, it will simply return the object it has in the case that it throws an error. This update also adds in the jsonParse function call for HandleReferences in case an override wants to be provided though code.
One mention of the issue here. https://github.com/RicoSuter/NSwag/commit/5d548d7b88f49c073bdd9e5296ee069cfdd3dba2#r37786449

@RicoSuter Any idea on when this can get looked at and implemented?

@thijscrombeen @JyrkiHei @nunonux can you have a look?

Found the template directory functionality so I'm not hurting for this, and not wed to the implementation. Thanks for anyone that digs in. :)

I've checked your branch and was able to successfully generate a typescript file with your changes.
In my projects however I use the Class TypeStyle so I haven't actually run the generated code.
protected processSomeMethod(response: AxiosResponse): Promise<boolean> {
    const status = response.status;
    ...
    if (status === 200) {
      const _responseText = response.data;
      let result200: any = null;
      let resultData200 = _responseText;
      try {
        result200 = _responseText === '' ? null : <boolean>JSON.parse(resultData200, this.jsonParseReviver);
      } catch (err) {
        result200 = _responseText;
      }
      return result200;
    }
    ...

While above code doesn't give any compiler errors, it is not really correct.
'result200' is of type any and set to null while the method should actually return only boolean.
The method can in this case return null but with this PR anything at all based on whatever is in '_responseText'
We could change the generated code to explain that it can return either boolean or null but then it no longer works with your change.
protected processSomeMethod(response: AxiosResponse): Promise<boolean | null> {
    const status = response.status;
    ...
    if (status === 200) {
      const _responseText = response.data;
      let result200: boolean | null = null;
      let resultData200 = _responseText;
      try {
        result200 = _responseText === '' ? null : <boolean>JSON.parse(resultData200, this.jsonParseReviver);
      } catch (err) {
        result200 = _responseText;
      }
      return result200;
    }
    ...

In which cases do you use interceptors that would make the JSON.parse fail?

So, given these parameters:
/TypeScriptVersion:3.8 /PromiseType:Promise /Template:Axios /GenerateClientClasses:true /GenerateClientInterfaces:true /TypeStyle:Interface /GenerateOptionalParameters:true /OperationGenerationMode:MultipleClientsFromFirstTagAndOperationId /MarkOptionalProperties:true /WrapResponses:true /ResponseClass:ThingClientResponse
Here's an example generation based on a pretty simple "get by id" type endpoint client:
export class ThingClient implements IThingClient {
    private instance: AxiosInstance;
    private baseUrl: string;
    protected jsonParseReviver: ((key: string, value: any) => any) | undefined = undefined;

    constructor(baseUrl?: string, instance?: AxiosInstance) {
        this.instance = instance ? instance : axios.create();
        this.baseUrl = baseUrl ? baseUrl : "";
    }

    /**
     * @return Success
     */
    get_api_thing_id(id: number): Promise<ThingClientResponse<ThingDto>> {
        let url_ = this.baseUrl + "/api/Thing/{id}";
        if (id === undefined || id === null)
            throw new Error("The parameter 'id' must be defined.");
        url_ = url_.replace("{id}", encodeURIComponent("" + id));
        url_ = url_.replace(/[?&]$/, "");

        let options_ = <AxiosRequestConfig>{
            validateStatus: () => true,
            method: "GET",
            url: url_,
            headers: {
                "Accept": "application/json"
            }
        };

        return this.instance.request(options_).then((_response: AxiosResponse) => {
            return this.processGet_api_thing_id(_response);
        });
    }

    protected processGet_api_thing_id(response: AxiosResponse): Promise<ThingClientResponse<ThingDto>> {
        const status = response.status;
        let _headers: any = {};
        if (response.headers && typeof response.headers === "object") {
            for (let k in response.headers) {
                if (response.headers.hasOwnProperty(k)) {
                    _headers[k] = response.headers[k];
                }
            }
        }
        if (status === 200) {
            const _responseText = response.data;
            let result200: any = null;
            let resultData200  = _responseText;
            if (typeof resultData200 !== "string") {
                result200 = resultData200;
            }
            else {
                try { result200 = (_responseText === "" ? null : <ThingDto>JSON.parse(resultData200, this.jsonParseReviver)); }
                catch(err) { result200 = resultData200; }
            }
            return Promise.resolve<ThingClientResponse<ThingDto>>(new ThingClientResponse<ThingDto>(status, _headers, result200));
        } else if (status === 404) {
            const _responseText = response.data;
            let result404: any = null;
            let resultData404  = _responseText;
            if (typeof resultData404 !== "string") {
                result404 = resultData404;
            }
            else {
                try { result404 = (_responseText === "" ? null : <ProblemDetails>JSON.parse(resultData404, this.jsonParseReviver)); }
                catch(err) { result404 = resultData404; }
            }
            return throwException("Not Found", status, _responseText, _headers, result404);
        } else if (status !== 200 && status !== 204) {
            const _responseText = response.data;
            return throwException("An unexpected server error occurred.", status, _responseText, _headers);
        }
        return Promise.resolve<ThingClientResponse<ThingDto>>(new ThingClientResponse(status, _headers, <any>null));
    }
}

Note, this stack overflow answer that basically says that Axios automatically parses json results, unless it specifically is fed transformers (or even, a lack of them, because it will remove the default one). https://stackoverflow.com/questions/41013082/disable-json-parsing-in-axios
Note this important bit of code from above:
return this.instance.request(options_).then((_response: AxiosResponse) => {             return this.processGet_api_thing_id(_response);         });
It's specifically modifying the response from the request above. The only valid thing that can come through as _response.data is a string representation of JSON. Currently there isn't anything that, for example, removes the transformers so that is the case all the time, and that seems kind of arbitrarily limiting doesn't it?
This is a super specific thing that it needs in order for it to function, and I'm halfway convinced it probably shouldn't even be trying to parse this data.

I just checked the Axios code and the default implementation of transformResponse is as follows:
transformResponse: [function transformResponse(data) {
    /*eslint no-param-reassign:0*/
    if (typeof data === 'string') {
      try {
        data = JSON.parse(data);
      } catch (e) { /* Ignore */ }
    }
    return data;
  }],

Would it be best to by default overwrite this so that axios itself does not do any parsing?
That doesn't fix the issue I mentioned above where we don't really type correctly currently.

@thijscrombeen I'd think we just don't try to parse it. Seems like it's trying to solve for a non-problem within this specific generation context. I think that would solve for all of it yeah?

At the moment (nswag <IP_ADDRESS>), when /template:Axios and /typeStyle:Interface, then the output of the response process throws Json exception
SyntaxError: Unexpected token o in JSON at position 1
in this line (when trying to parse JSON)
let resultData200  = _responseText;
result200 = JSON.parse(resultData200);
guess this PR gonna fix this, right?

SyntaxError: Unexpected token o in JSON at position 1

I have exactly the same issue as the one reported by @mohsenZaim when toggling Interfaces with Axios.
Last week, I opened #2966 to suggest a fix for that issue (then closed it as I saw the current PR would be even better) but I believe that the current PR covers more edge cases.

SyntaxError: Unexpected token o in JSON at position 1

I have exactly the same issue as the one reported by @mohsenZaim when toggling Interfaces with Axios.
Last week, I opened #2966 to suggest a fix for that issue (then closed it as I saw the current PR would be even better) but I believe that the current PR covers more edge cases.

Is there any way to use the .liquid based templates locally and generate a correct output for the axios template? in swagger i used to modify .mustach files and use those instead of default templates to generate my code. here for this nswag, i havent found a way yet?
@RicoSuter do we have such feature to use locally modified .liquid templates to generate files, so we can use this axios based template ? or the only solution at the moment would be to wait for this bug to be fixed? :)

You can test the changed tpl with an override:
https://github.com/RicoSuter/NSwag/wiki/Templates
As soon as i have the ok that its fine ill merge the pr.

You can test the changed tpl with an override:
https://github.com/RicoSuter/NSwag/wiki/Templates
As soon as i have the ok that its fine ill merge the pr.

Thanks @RicoSuter :)

@mohsenZaim @dubzzz
What kind of data are you returning where this throws the JSON error ?
Could you provide a swagger/open api json document and maybe the api controller implementation if the api is in c#?

@thijscrombeen The failure happens whenever I try to return a simple string in the end-point

@mohsenZaim @dubzzz
What kind of data are you returning where this throws the JSON error ?
Could you provide a swagger/open api json document and maybe the api controller implementation if the api is in c#?

Arrays for example.
Here is swagger contract sample:
"/api/Locations": {
"get": {
"tags": [
"Location"
],
"operationId": "Location_List",
"responses": {
"200": {
"x-nullable": false,
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/LocationListItem"
}
}
here is the output when use axios + type interface inside the process method.
the part which throws error is when status 200 oreturn for example
the data suppose to be array but it is trying to Json.parse it, therefore it throw error SyntaxError: Unexpected token o in JSON at position 1.
if (status === 200) {
const _responseText = response.data;
let result200: any = null;
let resultData200  = _responseText;
result200 = JSON.parse(resultData200);
return result200;
}
Not sure why the template is JSON parsing here ? and why not just return the response.body. or at least check if it is parsable.
and here is sample C# in the backend
    [ProducesResponseType(StatusCodes.Status404NotFound)]
    [ProducesResponseType(StatusCodes.Status200OK, Type = typeof(List<ApiModel.LocationListItem>))]
    [HttpGet]
    [Route("[controller]s", Name = "ListLocations")]
    public ActionResult<List<ApiModel.LocationListItem>> List()
    {
        // Call business logic
        var output = _locationService.List();

        if (output == null)
        {
            return NotFound();
        }

        // Convert output
        var mappedOutput = Mapper.Map<List<ApiModel.LocationListItem>>(output);

        return Ok(mappedOutput);
    }
}

As explained above Axios is also doing a JSON.parse. When using interface typestyle NSwag is also doing a JSON.parse which happens after the one from Axios. At that time it's no longer a string but it's a json object and it fails.
This can be overridden by setting the following on the axios client:
transformResponse: [function transformResponse(data) {
return data;
}],
For example in the constructor of the generated client:
constructor(baseUrl?: string, instance?: AxiosInstance) {
this.instance = instance
? instance
: axios.create({
transformResponse: [
function transformResponse(data: any) {
return data;
},
],
});
this.baseUrl = baseUrl ? baseUrl : "";
}
If needed I can make a seperate PR for this change to add the transformresponse in the case of typestyle interface.

If needed I can make a seperate PR for this change to add the transformresponse in the case of typestyle interface.

Would be great if you can add this feature in a seperate PR that can be merged and release a new version, so we use that instead of overriding the template on our side.

@thijscrombeen But why override that functionality to stop parsing, just to do it ourselves anyway down the line? Doesn't make sense to me

Based on the discussion above, I recently tried to use the following instance of axios in my codebase and it worked like charm with interface mode on:
axios.create({
      transformResponse: [
        (data) => {
          // The output of this function is a string that can be passed to JSON.parse
          try {
            // types such as strings, numbers are already parsed so JSON.parse will fail on them
            // complex objects on the contrary come as strings
            JSON.parse(data);
            return data;
          } catch (err) {
            return JSON.stringify(data);
          }
        },
      ],
    });

This issue is still here. I am not sure why the template is trying to parse the response as Axios already does that for us.
Generating a client from the pet store: https://petstore.swagger.io/v2/swagger.json with axios and interface DTOs will create a client that has this behavior.
Here is an example config:
{
  "runtime": "Net50",
  "defaultVariables": null,
  "documentGenerator": {
    "fromDocument": {
      "url": "https://petstore.swagger.io/v2/swagger.json",
      "output": null,
      "newLineBehavior": "Auto"
    }
  },
  "codeGenerators": {
    "openApiToTypeScriptClient": {
      "className": "{controller}Client",
      "moduleName": "",
      "namespace": "",
      "typeScriptVersion": 2.7,
      "template": "Axios",
      "promiseType": "Promise",
      "httpClass": "HttpClient",
      "withCredentials": false,
      "useSingletonProvider": false,
      "injectionTokenType": "OpaqueToken",
      "rxJsVersion": 6.0,
      "dateTimeType": "String",
      "nullValue": "Undefined",
      "generateClientClasses": true,
      "generateClientInterfaces": true,
      "generateOptionalParameters": false,
      "exportTypes": true,
      "wrapDtoExceptions": false,
      "exceptionClass": "ApiException",
      "clientBaseClass": null,
      "wrapResponses": false,
      "wrapResponseMethods": [],
      "generateResponseClasses": true,
      "responseClass": "SwaggerResponse",
      "protectedMethods": [],
      "configurationClass": null,
      "useTransformOptionsMethod": false,
      "useTransformResultMethod": false,
      "generateDtoTypes": true,
      "operationGenerationMode": "MultipleClientsFromFirstTagAndOperationId",
      "markOptionalProperties": true,
      "generateCloneMethod": false,
      "typeStyle": "Interface",
      "enumStyle": "Enum",
      "useLeafType": false,
      "classTypes": [],
      "extendedClasses": [],
      "extensionCode": null,
      "generateDefaultValues": true,
      "excludedTypeNames": [],
      "excludedParameterNames": [],
      "handleReferences": false,
      "generateConstructorInterface": true,
      "convertConstructorInterfaceData": false,
      "importRequiredTypes": true,
      "useGetBaseUrlMethod": false,
      "baseUrlTokenName": "API_BASE_URL",
      "queryNullValue": "",
      "useAbortSignal": false,
      "inlineNamedDictionaries": false,
      "inlineNamedAny": false,
      "templateDirectory": null,
      "typeNameGeneratorType": null,
      "propertyNameGeneratorType": null,
      "enumNameGeneratorType": null,
      "serviceHost": null,
      "serviceSchemes": null,
      "output": "index.ts",
      "newLineBehavior": "Auto"
    }
  }
}

But why override that functionality to stop parsing, just to do it ourselves anyway down the line? Doesn't make sense to me

Because some properties (eg dates) need to be serialized differently, and sometimes the property names on the wire (in JSON) do not match the property names in the generated TypeScript classes (eg on the wire we have 'first-name' which cannot be used as a TS/JS property name and is renamed to "firstName").

I just removed JSON.parse(...). It seems that it is not needed at all in the context of axios if I am not wrong

In the case of $ref resolution it probably would be needed as axios does not pass the special reviver function in...

I do not use Axios and I have no clue whether this here is fine or not. What's the current state?

@thijscrombeen @RicoSuter is there any update on this?
I've uses nswag successfully with dotnet, but now when trying to use it in our typescript service and axios, I'm facing an error when our endpoint returns [] for empty result.
The suggested axios transformResponse: works, but is really not a performant solution.
I would love to take the nswag generated clients into proper use in our microservice environment, but I need it work well.

i also want to use axios with nswag to generate TS client
with fetch it works fine and it adds this when you check the "handle $values" option

but with axios it doesnt include that

even though jsonParse is in the generated code (with axios)
but that function is never referenced.

Is this PR still valid?

@RicoSuter Yes, I believe the PR is still valid and would fix several open issues as described here: https://github.com/RicoSuter/NSwag/issues/5003


Functions cannot be passed directly to Client Components unless you explicitly expose it by marking it with "use server".

"next": "^13.4.7"
"next-translate": "^2.4.2"
"next-translate-plugin": "^2.4.2"

node v18.15.0
macOS Ventura

I use next.js app folder.
This is my configuration file:
// i18n.js
module.exports = {
  locales: ["it", "en"],
  defaultLocale: "it",
  loadLocaleFrom: (lang, namespace) =>
    import(`./locales/${lang}/${namespace}.json`).then((m) => m.default),
  localeDetection: false,
  pages: {
    "*": ["common"],
    "/": ["common"],
    "/account": ["account", "common", "errors"],
    // others
  },
};

I get the error:

Functions cannot be passed directly to Client Components unless you explicitly expose it by marking it with "use server".
If I just remove the loadLocaleFrom function from the configuration, translation are loaded fine, but, when on backend code I use getT (for example to translate page title, or validation errors) the label key is rendered instead of the actual translation.

Yes, but this is how Client components work, is not related with next-translate. All the values show be serializable (json)

Yes, I understand that - the fact that when you cross the server/client boundary you can share only serializable values; but i'm still confused.
I am not using loadLocaleFrom directly in my code; so if this is passed to a client component this is something happening out of my control at the moment.
Also when I remove it (and this would fix the immediate issue with client components), then getT stops working, even if my translations file are in (what i suppose is) the default location (./locales/${lang}/${namespace}.json)... and this to me looks like a next-translate problem (sorry, if i'm wrong and i'm missing something).
I understand you're busy, and I appreciate the effort you put in this project.
I can help you providing a better repro for the issue - please let me know if it's needed.

I'm going to take a look

The problem is that we are passing the config, but we don't remove the not serializable data. I'm going to fix it because we didn't need it.

@brunoscopelliti try:

Next-translate-plugin 2.4.3
Next-translate 2.4.3

@aralroca
Thank you for the quick fix.
I confirm that version 2.4.3 fixes the problem with loadLocaleFrom.
Anyway for some reason, titles are still not rendered properly in my application.
export const generateMetadata = async ({ searchParams }: PageParams) => {
  const t = await getT(searchParams.lang, ["common"]);
  return {
    title: t("common:page_title"),
  };
};

Will investigate further, and if turns out is a problem will open a different issue with proper repro.
Thank you!


[DataFactory] validation activity

Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ x] I have reviewed the documentation for the workflow.
[x ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[x ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

Automation for azure-sdk-for-ruby

Automation for azure-sdk-for-python

Can one of the admins verify this patch?

Automation for azure-sdk-for-java

Automation for azure-sdk-for-net
A PR has been created for you:
https://github.com/azure/azure-sdk-for-net/pull/-1
.NET SDK Commits:
https://github.com/adxsdknet/azure-sdk-for-net/pull/-1/commits/68b520e5de7483dbe90fa8ca3673fcf63abb2ce9

Automation for azure-sdk-for-js

Automation for azure-sdk-for-net
A PR has been created for you:
https://github.com/azure/azure-sdk-for-net/pull/-1
.NET SDK Commits:
https://github.com/adxsdknet/azure-sdk-for-net/pull/-1/commits/68b520e5de7483dbe90fa8ca3673fcf63abb2ce9

startbuild

startbuild

startbuild

startbuild

startbuild

startbuild

startbuild

Automation for azure-sdk-for-net
A PR has been created for you:
https://github.com/azure/azure-sdk-for-net/pull/5607
.NET SDK Commits:
https://github.com/adxsdknet/azure-sdk-for-net/pull/5607/commits/dd80df1392aaff91701348abdb677b31edd4a7b3

@sarangan12, can you please review this PR? We need it in our next release.

@cesher @hvermis The PR Looks fine to me. But, where exactly this new Model "ValidationActivity" being used? It is not being used anywhere in the swagger file. So, what is the use of this model?

@sarangan12 This type is derived from Activity type, which is used in Pipeline.
Here's the property in PIpelines that holds all activities:
https://github.com/Azure/azure-rest-api-specs/blob/22e4440bc8d496f16dfa56d7074bb0f1d70ecaa5/specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/entityTypes/Pipeline.json#L17

@sarangan12 Can you please merge this PR? We are trying to release our SDK today and this is needed.

Automation for azure-sdk-for-go

@AutorestCI regenerate azure-sdk-for-go


Memory leak/OOM with "Received update for IP range I own" messages in log

What you expected to happen?
Memory usage of the weave process is expected to be stable and not grow unbounded over time.
What happened?
I had a stable 2.5.0 weave network in my Kubernetes 1.9 cluster of about 100 nodes.  The weave was initially installed by kops and had a memory limit of 200mb set.  There were no occurrences of "Received update for IP range I own" in the log files and memory usage for weave pods in the cluster had been very stable over time for weeks.
As part of refactoring some services, about 30 nodes were removed from the cluster (bringing the cluster size down to 71 nodes).  After this action, the memory usage of the weave pods started growing until it exceeded the memory limit, at which time the pod was OOM killed and restarted.  These restarts result in brief disruption for the node on which the restart occurs.  At this time the "Received update for IP range I own" message started appearing in the logs (although not from all pods, this nuance was not discovered until later).
After looking at some related tickets and such here (#3650, #3600, #2797), the following actions were taken:

The "status ipam" output was checked and seen to have a lot of "unreachable" peers listed in it
The unreachable nodes listed by "status ipam" were removed with rmpeer on one node, though this did not fix all the unreachables on all the nodes, the process of listing and removing unreachables was done on a couple of other systems before all systems were showing all 71 nodes in the list and all as reachable.
updated to 2.5.2 as there were some related looking tickets mentioned in that release
increased the memory limit so that OOM killing might happen less frequently (from 200mb to 1gb)

Weave pods continue to grow in memory usage, the new 2.5.2 pods have not hit their 1g limit yet but look to be heading that way.  The "update for IP range I own" messages are still being seen - however on closer inspection these messages are only coming from 3 of the 71 pods.
How to reproduce it?
Have a working kubernetes cluster and delete some nodes out of it.
Anything else we need to know?
Versions:

        Version: 2.5.2 (up to date; next check at 2019/07/12 18:43:12)

        Service: router
       Protocol: weave 1..2
           Name: ea:38:6f:58:7b:81(ip-10-32-124-236.us-west-2.compute.internal)
     Encryption: disabled
  PeerDiscovery: enabled
        Targets: 71
    Connections: 71 (70 established, 1 failed)
          Peers: 71 (with 4966 established, 4 pending connections)
 TrustedSubnets: none

        Service: ipam
         Status: ready
          Range: <IP_ADDRESS>/11
  DefaultSubnet: <IP_ADDRESS>/11

admin@ip-10-32-92-49:~$ docker version
Client:
 Version:      17.03.2-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   f5ec1e2
 Built:        Tue Jun 27 02:09:56 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.2-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   f5ec1e2
 Built:        Tue Jun 27 02:09:56 2017
 OS/Arch:      linux/amd64
 Experimental: false

Linux ip-10-32-92-49 4.4.121-k8s #1 SMP Sun Mar 11 19:39:47 UTC 2018 x86_64 GNU/Linux

Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.8", GitCommit:"c138b85178156011dc934c2c9f4837476876fb07", GitTreeState:"clean", BuildDate:"2018-05-21T18:53:18Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}```

Logs:
This is the logs from one of the weave pods that is showing the "Received update for IP range I own" messages: weave-net-q56hl.log
This is the pprof/heap output for the above node
weave-net-q56hl.heap.gz
This is status ipam from the above node
weave-net-q56hl.ipam.txt
This is status peers from the above node
weave-net-q56hl.peers.txt
This is the logs from one of the weave pods not showing that message:
weave-net-9t7d8.log
This is the pprof/heap output for the above node weave-net-9t7d8.heap.gz
And here's a picture showing the history of memory usage form these pods

Here are the log files from the other two weave pods producing the "Received update for IP range I own" log messages - only 3 of the 71 pods are producing this message (the other is in the original comment)
weave-net-67hrw.log
weave-net-cqfc9.log

@sferrett thanks for reporting the issue and sharing the logs. pprof heap output file seems to be corrected. I can not usethem.
Looking at the logs, there is continuous activity connections getting established to the peers and then subsequently shutting down the established connected with the peers due to conflict in the IPAM entry. There might be resource leak during the connection shuttingdown. Please share the heap output i will investigate further.
cat weave-net-q56hl.log| grep "connection fully established\|connection shutting down due to" | more
INFO: 2019/07/12 14:48:45.096844 ->[<IP_ADDRESS>:6783|1a:81:c0:7d:65:8d(ip-10-32-118-152.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to 1a:81:c0:7d:65:8d(ip-10-32-118-152.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:48:45.102204 ->[<IP_ADDRESS>:47487|1a:81:c0:7d:65:8d(ip-10-32-118-152.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:48:45.104220 ->[<IP_ADDRESS>:38935|c2:57:fe:eb:f5:1b(ip-10-32-116-209.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:48:45.105973 ->[<IP_ADDRESS>:11183|ee:b4:d4:3e:29:08(ip-10-32-122-86.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to ee:b4:d4:3e:29:08(ip-10-32-122-86.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:48:45.106002 ->[<IP_ADDRESS>:6783|66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to 66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:48:45.114811 ->[<IP_ADDRESS>:22137|66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:48:45.115645 ->[<IP_ADDRESS>:6783|ee:b4:d4:3e:29:08(ip-10-32-122-86.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:48:51.961639 ->[<IP_ADDRESS>:10584|36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11
INFO: 2019/07/12 14:48:52.063684 ->[<IP_ADDRESS>:6783|36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to 36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:48:52.071075 ->[<IP_ADDRESS>:6783|36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to 36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:48:52.147730 ->[<IP_ADDRESS>:56584|36:0e:50:ea:19:dd(ip-10-32-122-167.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:48:59.451552 ->[<IP_ADDRESS>:6783|d2:3b:d6:be:05:46(ip-10-32-122-154.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11
INFO: 2019/07/12 14:48:59.535330 ->[<IP_ADDRESS>:56632|d2:3b:d6:be:05:46(ip-10-32-122-154.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:49:04.569976 ->[<IP_ADDRESS>:22137|66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11
INFO: 2019/07/12 14:49:04.578200 ->[<IP_ADDRESS>:6783|66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal)]: connection shutting down due to error: Multiple connections to 66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal) added to 9e:53:50:e6:1e:31(ip-10-32-117-193.us-west-2.compute.internal)
INFO: 2019/07/12 14:49:04.625605 ->[<IP_ADDRESS>:64725|66:d8:18:5e:d8:28(ip-10-32-121-63.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:49:09.368974 ->[<IP_ADDRESS>:54777|42:fe:cc:22:06:be(ip-10-32-121-110.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11
INFO: 2019/07/12 14:49:09.373978 ->[<IP_ADDRESS>:6783|ee:b4:d4:3e:29:08(ip-10-32-122-86.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11
INFO: 2019/07/12 14:49:09.501779 ->[<IP_ADDRESS>:48605|ee:b4:d4:3e:29:08(ip-10-32-122-86.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:49:09.876308 ->[<IP_ADDRESS>:6783|42:fe:cc:22:06:be(ip-10-32-121-110.us-west-2.compute.internal)]: connection fully established
INFO: 2019/07/12 14:49:11.879385 ->[<IP_ADDRESS>:6783|f2:84:e6:76:02:c8(ip-10-32-125-220.us-west-2.compute.internal)]: connection shutting down due to error: Received update for IP range I own at <IP_ADDRESS> v10: incoming message says owner e2:54:77:2d:03:f5 v11

This happened to us last week on our production cluster ... 👇

We noticed that there were two groups of IPAM lists with overlapping entries so it wasn't a complete split-brain. We bit the bullet and terminated all the nodes in one group to recover. It was brutal.
After looking at the memory profile, 200MB as a limit makes sense for weave-npc container but not for the weave container. We decided to increase the memory limit to 300MB.

Here's how I'm generating the heap profile: curl 'http://localhost:6784/debug/pprof/heap' > weave-net-6bgmx.dump
Then I gzip the file - here's another attached, it seems different in size and the method I used to copy it from the system that generated it was slightly different, so please let me know if this is a good one or not?  It's from one of the more-rapidly growing instances that got a restart over the weekend
weave-net-6bgmx.heap.gz

@itskingori - interesting.  I ran the following on my prd cluster, which is having this symptom, and I also see there are two distinct set of "status ipam" hosts returned from the cluster, each comprising about 50% of the nodes:
{noformat}
Generate a sorted list of hosts-only from ipam output per weave node
$ for s in kubectl get pod | grep weave-net | awk '{ print $1 }'; do kubectl exec -ti $s -c weave -- ./weave --local status ipam | awk '{ print $1 }' | sort > $s.ipam; done
See how many different variants of that list there are - this shows there are two different checksums for the files, and there are 38 of one and 33 of the other.
$ sum *.ipam | awk '{ print $1 }' | sort | uniq -c
38 02398
33 57603
{noformat}
Here's the "02398" list
weave-ipam-02398.txt
Here's the "57603" list
weave-ipam-57603.txt
The lists are identical except that the "57603" list has an additional entry in it:
{noformat}
$ diff /host-user/weave-ipam-57603.txt /host-user/weave-ipam-02398.txt
56d55
< be:f4:ea:cd:a5:9a(ip-10-32-127-95.us-west-2.compute.internal)
{noformat}
I wonder if I can do something with that one host that's in one list but not the other?  Or if I do indeed need to restart all the systems in one of the lists....
When you say you terminated the hosts in one of the groups, did you do anything to the remaining systems or just terminate the nodes in one of the lists?  Also did the nodes that were terminated have anything special done on them prior (such as running weave reset or similar?) . I'm also curious if you did them one at a time, and somehow observe that they came up with a consistent ipam list?  Basically I think I need to do the same thing here, just want to make sure that deleting those nodes won't just increase the amount of breakage as it seems like node removal is what precipitated all of this in the first place.
Thanks!

OK so I have this condition happening in two different clusters, so I guess that's good news I can try things in one and compare?
Anyhow - in one of the clusters I did the following:

Identified weave pods into two groups based on what their 'ipam status' output was showing.  There were generally two "views" of the ipam data and nodes either had one or the other of the views.
I chose one "view" as being the correct one (the one that had two of the three kube masters in it) and on the nodes that were in the "wrong" list the following was done:
** The weave db file was removed
** The pod was deleted/restarted
After the above, the pods cpu usage dropped and it looks like the memory growth for all of the pods (even the ones that were not restarted) has stopped.  Also all the pods now return the same details for  their 'ipam status' output

So - this sounds like somewhat of a success
However - I am not sure what the ramifications are of deleting the weave db file on those nodes and just restarting weave and not the entire node.   I am somewhat concerned about any ill-effect on the pods still running on these nodes where weave was restarted and if there's any potential issue that has been set up due to not having them restart (ie: full node restart).
If anyone knows what negative side-effect might arise from deleting that db file and restarting weave but not restarting other pods on the system, I'd appreciate the insight.
Pending negative feedback from the above, I will be looking to do the same action in the other cluster soon

@itskingori - interesting. I ran the following on my prd cluster, which is having this symptom, and I also see there are two distinct set of "status ipam" hosts returned from the cluster, each comprising about 50% of the nodes:
# Generate a sorted list of hosts-only from ipam output per weave node
$ for s in `kubectl get pod | grep weave-net | awk '{ print $1 }'`; do kubectl exec -ti $s -c weave -- ./weave --local status ipam | awk '{ print $1 }' | sort > $s.ipam; done

# See how many different variants of that list there are - this shows there are two different checksums for the files, and there are 38 of one and 33 of the other.
$ sum *.ipam | awk '{ print $1 }' | sort | uniq -c
     38 02398
     33 57603

Yes, this is more or less the position we were in. I'll borrow your commands are they seem similar to (if not better than) what I have to look through weave 👇
#!/bin/bash

set -u

context=$1
command=$2
pods=$(kubectl get pods -n=kube-system -lname=weave-net --context=${context} | tail -n+2 | awk '{print $1}')

for pod in ${pods[@]}
do
  kubectl exec -n=kube-system --context=${environment} ${pod} -- /home/weave/weave --local ${command}
done

I use something like ./script.sh <cluster> "<status ipam>" to loop through all the weave pods.

I wonder if I can do something with that one host that's in one list but not the other? Or if I do indeed need to restart all the systems in one of the lists....

Weave works by sharing state by consensus, the fact that there are two states breaks weave and you need to bite the bullet and get rid of a group or any weave pods that have inconsistent state. There are two ways to do this:

Terminate all the nodes with bad state at the same time.
Terminate weave pods with bad state while deleting their database (that stays on the host) at the same time.

I ran with no. 2 because this was production so recovery was critical. I didn't have time to write a script for no. 2. I pretty much figured out which group to terminate and terminated them all at the same time. I'm guessing, it's better to terminate weave while deleting the database on the host so that the new pod starts from a clean slate and gets its state from other correct weave pods.

When you say you terminated the hosts in one of the groups, did you do anything to the remaining systems or just terminate the nodes in one of the lists?

I didn't do anything to the remaining group as their state was similar and correct. Once you get rid of the bad weave pods, everything goes back to normal and the state is now consistent among those remaining ... the problem goes away, and the cluster heals.

Also did the nodes that were terminated have anything special done on them prior (such as running weave reset or similar?) .

I didn't do anything to the ones remaining. The fact that they had similar state was all I needed.

I'm also curious if you did them one at a time, and somehow observe that they came up with a consistent ipam list?

I terminated them all at the same time because I could not risk them sharing state any longer. I figured doing it one by one might not work because new pod might get it state from another bad one. I wanted all the 'bad' ones gone and only the 'good' ones left to share state.

Basically I think I need to do the same thing here, just want to make sure that deleting those nodes won't just increase the amount of breakage as it seems like node removal is what precipitated all of this in the first place.

It didn't cause more breakage for me. Other than the time when I had to wait for the autoscaling group to replace the nodes that I just terminated. And the disruption it caused to the apps on the nodes I just terminated as new pods come up.

so please let me know if this is a good one or not? It's from one of the more-rapidly growing instances that got a restart over the weekend

@sferrett I am able to use the recent heap output. It does look like continous establishing and shutting down the connections is causing the memeroy use to grow up between the go GC runs or there is a memory leak. Will investigate further.
(pprof) top
Showing nodes accounting for 137.61MB, 90.98% of 151.24MB total
Dropped 54 nodes (cum <= 0.76MB)
Showing top 10 nodes out of 64
      flat  flat%   sum%        cum   cum%
   63.01MB 41.66% 41.66%    73.61MB 48.67%  github.com/weaveworks/weave/router.(*OverlaySwitch).PrepareConnection
   30.51MB 20.17% 61.83%    30.51MB 20.17%  runtime.malg
   14.50MB  9.59% 71.42%    19.02MB 12.58%  github.com/weaveworks/weave/vendor/github.com/weaveworks/mesh.(*gossipSender).run
   10.11MB  6.68% 78.10%    10.11MB  6.68%  github.com/weaveworks/weave/router.NewNonEncryptor (inline)

However - I am not sure what the ramifications are of deleting the weave db file on those nodes and just restarting weave and not the entire node. I am somewhat concerned about any ill-effect on the pods still running on these nodes where weave was restarted and if there's any potential issue that has been set up due to not having them restart (ie: full node restart). If anyone knows what negative side-effect might arise from deleting that db file and restarting weave but not restarting other pods on the system, I'd appreciate the insight.

No ramifications from my understanding. Anyone from weave can correct me if I'm wrong but when you delete the database and terminate weave, it just recreates it. In my opinion, this is the right approach and I took the other approach out of the need for a quick win. If you have a script that you can share for terminating weave pods and clearing the databases please share. 😅

Yes, there are no ramifications. For now these manual steps will reconcile from state where there were IPAM conflicts. In 2.6 release IPAM conflicts are automatically resolved (https://github.com/weaveworks/weave/pull/3637)

@sferrett Now that you're on 2.5.2 and are able to get the cluster stable, could you continue monitoring the IPAM lists to see if they remain consistent (assuming you're scaling up and down)? I really struggled remaining stable on that version for a couple of days (it would eventually get borked after scaling up and down) and so reverted to 2.5.1 as an experiment and I'm since stable. I could be drawing the wrong conclusions or that there's something unique about my setup (highly unlikely since I use kops) 🤷♂ ... would be good to have another data point.

@murali-reddy here's another dump from one of the processes that is just about to surpass it's 1Gb heap limit, in case that's also useful.
weave-net-fllrr.heap.gz

@itskingori - I will keep an eye on it for sure.  We're not doing a huge amount of scaling at the moment, however we did just do some refactoring of instance types hence we had a fairly large amount of hosts created and deleted which is what set the current condition into motion.   There is some more similar adjustments still outstanding so I will check and let you know how things look as that happens.
Also, thanks very much for your feedback and detail - sounds like your process to recover normal operation was very similar to mine.  The only difference seems to be that you rebooted/terminated the nodes whereas I just recycled the weave pod on the node and left the other pods and the node itself alone.  We'll be doing this same process in our prd cluster today so if I cook up a noteworthy script I will share it here.  Cheers -

OK so since the weave nodes became consistent, things have been stable with no memory growth or OOM issues.  I have done a minor amount of scaling (perhaps up/down of 10 nodes or so) and things have remained consistent throughout that with 2.5.2.
I wrote a quick script
verify-weave.sh which will go through all weave pods in the cluster, compute a checksum on the 'status ipam' list and tell you if there are any weave pods that disagree on their peer list.   We used this to identify the different groups of pods in the cluster and decide which we wanted to preserve and which ones we wanted to reset/restart.  Another quick script
bump-weave.sh was then used to remove the db file and restart for those weave pods we wished to reset.
Both of those scripts were written quickly to address a particular condition here and so are not intended as portable or good examples of coding, but may be useful to someone so here they are.
@itskingori , @murali-reddy  - thank you both for your help on this.

@sferrett Thanks for the update! Will try out 2.5.2 again and check out yours scripts!

Note we found a leak which matches the OOM symptom here: #3807


Create root level folder for volume content

Some Docker containers (e.g. the official postgres:9.6) need the volume mount point to be empty. This is not the case when the mount point is the file system root with it's lost+found directory.

@j-griffith I cleaned up my commits ... Best regards!


Update severity-levels.md

@PatrickFarley
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

We can't merge this PR because it breaks the tables. I will create a new PR to fix all the severity level examples


Wish for the add_to_app samples using SwiftUI to match the new updated docs

Use case
We've updated https://docs.flutter.dev/add-to-app/ios/add-flutter-screen which is great!
Apparently, the add to app samples over in https://github.com/flutter/samples/tree/main/add_to_app have not yet been updated.
Proposal
We should update the SwiftUI based samples in https://github.com/flutter/samples/tree/main/add_to_app to match the updates we've made in https://docs.flutter.dev/add-to-app/ios/add-flutter-screen

cc @hellohuanlin as we were just talking about this. No coding action needs to be taken right now. However, would you be willing to confirm that I articulated the "problem statement" correctly?


Update intro.json

The phrase has been updated
from
This project will cover concepts like the map() method, find() method, parseInt(), the includes() method."
to
This project will cover concepts like the map(), find(), parseInt(), and includes() methods."
Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Closes #XXXXX

The bug has been resolved.
Updated Phrase:
This project will cover concepts like the map(), find(), parseInt(), and includes() methods."


Can't use type aliases or conditional types resolving to Promise for async/await return types

TypeScript Version:  3.1

Search Terms:

async promise alias
async promise conditional

Code
type P<T> = Promise<T>

async function foo<T>(x: T): P<T> {
    return x;
}

async function bar<T>(x: T): T extends number ? Promise<number> : Promise<any> {
    return x;
}

Expected behavior:
No error.
Actual behavior:
Type 'P' is not a valid async function return type in ES5/ES3 because it does not refer to a Promise-compatible constructor value.
Type 'T extends number ? Promise : Promise' is not a valid async function return type in ES5/ES3 because it does not refer to a Promise-compatible constructor value.
Playground Link:
Link
Related Issues:

Yes, that works for my use case. Should have thought of it myself, thanks!

just had same issue, this looks to be regression in latest TS version (3.1.3) in prev versions this results in compilation error (as far as I can see).
Work around indeed works, but it would be better if this will break compilation, so will know immediately this is not supported

I just had this problem, and I did not think to place the whole conditional Promise<inside of here>.
Here's my playground example.
Thanks for the work around, I'll use that.

It looks like this is still a problem, and furthermore that the workaround suggested by @.weswigham doesn't work as of Typescript 3.9.2:
async function test<T>(x: T): Promise<T extends any ? string : number> {
    return 'test';
}

Expected behavior:
No error
Actual behavior:
Type '"test"' is not assignable to type 'T extends any ? string : number'.

Even when I change the return type to T extends any ? any : any, I still get Type '"test"' is not assignable to type 'T extends any ? any : any'.
Let me know if:

I did something wrong
I should report this as a separate issue

Nah - it's a flaw in how we relation conditional types, which'd need #27932 or #30639 to fix.

How are we expected to work around this issue in the case where we're NOT using conditional type aliases? In my example, I have a module that makes API requests, and I'd love to refactor the return type as a Promise that resolves to either an Error or the Response type.
export type GRPCResult<T> = Promise<FP.Either<Error, T>>;
async function postFoo(call: CF.API.PostFooCall): GRPCResult<PostFooResponse> {
  // ...
}

Looks like the workaround is to create a variable that aliases the Promise constructor at runtime in order to appease the TS compiler. My previous example now compiles with this:
const GRPCResult = Promise; // <---- workaround
// eslint-disable-next-line @typescript-eslint/no-redeclare
export type GRPCResult<T> = Promise<FP.Either<Error, T>>;

async function postFoo(call: CF.API.PostFooCall): GRPCResult<PostFooResponse> {
  // ...
}

It's annoying and concerning that this triggers a lint rule for redeclaring a variable, but I'm willing to live with this for now until this issue is fixed properly...

@akatechis it should work without workaround when you set your target to ES2015 or later


Process events to/from sub/sibling machines

Hi Hi ^_^
This pull requests makes it possible to use process on events of sub-machines, and for sub-machines to use process on events of parent sub-machines - or any machine in the hierarchy. It includes a subset of stuff from 44d8f57681b96c327e7a61718902467f9c3f7017, new tests that wouldn't compile before the changes. It addresses the following issues: #283, #173, #76, #121,  #181.

Nice one @drorspei, thank you very muc h :+1:


Support GRPO?

Hi thanks for the library! It seems that GRPO is used for many mathematical and reasoning related scenarios and has good results, thus it would be great if it can be supported in this library.

duplicate: https://github.com/OpenRLHF/OpenRLHF/issues/535

@hijkzzz Thank you! Btw the notion link does not seem to work (see below).

https://hijkzzz.notion.site/unraveling-rlhf-and-its-variants-engineering-insights?pvs=97#147d9a33ecc9806090f3d5c749d31f05

Thank you!

@hijkzzz Btw, do you have some ideas why GRPO seems to be adopted in math tasks? For example, is it possible that GRPO works well on and only on math


Minimize, Restore and Close buttons ..

Hello grandmaster, I have just a few caveats about functional design:
1 - I would like the functionality of hovering the mouse over it to affect only one button, that is, only the button I hovered the mouse over, when you hover the mouse over it and press the 3 buttons at the same time, it stays strange visually speaking. I'm using the Monterey theme.
Now it's like this:

I would like it to be like this for each button:

2 - The effect when you place the mouse over the Yellow(Minimize) and Red(Close) buttons is working perfect, it gives that expand effect .. Now the green(Restore) button does not work, the button does not expand, could you kindly check, I'm using the Monterey theme.
Yellow Button(expand effect) OK:

Red Button(expand effect) OK:

Green Button(expand effect) Failed: The button just stays the same, even with hover.
I don't know if this is on purpose, as it is a button that we hardly use, it has no expand functionality, while the other 2 buttons at the end have the expand function turned on. Please check if you can.
The Firefox theme is too beautiful, thank you for the great work and effort <3

YEEAAH! it is perfect! <3 Simple and intuitive!


Cannot find name 'GooglePlaceDirective'. angular 6

I added google Directive but always following error occurs.
Cannot find name 'GooglePlaceDirective'.

+1

I added google Directive but always following error occurs.
Cannot find name 'GooglePlaceDirective'.

Hi @anuraggupta816  you can find the GooglePlaceDirective as below  :
import { GooglePlaceDirective } from  "node_modules/ngx-google-places-autocomplete/ngx-google-places-autocomplete.directive";
this fixed my issue


Raymondluo1 containerregistry microsoft.container registry 2023 11 01 preview

ARM (Control Plane) API Specification Update Pull Request
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

[1] ARM review queue (for merge queues, see [4])
The PRs are processed by time opened, ascending. Your PR may show up on 2nd or later page.
If you addressed Step 1 from the diagram and your PR is not showing up in the queue, ensure the label ARMChangesRequested
is removed from your PR. This should cause the label WaitForARMFeedback to be added.
[2] https://aka.ms/azsdk/support/specreview-channel
[3] List of SDK breaking changes approvers in pinned Teams announcement
[4] public repo merge queue, private repo merge queue (for ARM review queue, [1])
If you need further help with anything, see Getting help section below.
Purpose of this PR
What's the purpose of this PR? Check all that apply. This is mandatory!

[x] New API version. (Such PR should have been generated with OpenAPI Hub, per this wiki doc.)
[ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix swagger quality issues in S360.
[x] Other, please clarify:

Added a new property into the RegistryProperties and RegistryPropertiesUpdateParameters in the new API version to support Metadata Search Enablement.

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[x] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[x] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to Step 2, "ARM Review", for this PR.

Breaking changes review (Step 1)

If the automation determines you have breaking changes, i.e. Step 1 from the diagram applies to you,
you must follow the breaking changes process.
IMPORTANT This applies even if:

The tool fails while it shouldn't, e.g. due to runtime exception, or incorrect detection of breaking changes.
You believe there is no need for you to request breaking change approval, for any reason.
Such claims must be reviewed, and the process is the same.

ARM API changes review (Step 2)

If this PR is in purview of ARM review then automation will add the ARMReview label.
If you want to force ARM review, add the label yourself.
Proceed according to the diagram at the top of this comment.

Viewing API changes
For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.
Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.
Getting help

For general PR approval workflow, see the diagram at the top of this comment.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

    }

ARM recommends enums over booleans for future proof APIs.
Standard guidance is: replace boolean/switch properties with a more meaningful enum whenever possible.
A boolean will forever have two valid values (true or false). A string enum type is always preferred. Also, properties should always provide better values just than True and False. For example two switches "isTypeA" and "isTypeB" should be replaced with one enum "type": [A, B, DefaultType]. Enums are always a more flexible and future proof option because they allow additional values to be added in the future in a non-breaking way, e.g. [Enabled, Disabled, Suspended, Deallocated].
Note: do NOT define a 'boolean enum' with two values 'True and False'. This might be easier to 'extend' in terms of types, but semantically its cofusing, and no better than a boolean. Properties with type:object that don't reference a model definition are not allowed. ARM doesn't allow generic type definitions as this leads to bad customer experience.

Refers to: specification/containerregistry/resource-manager/Microsoft.ContainerRegistry/preview/2023-11-01-preview/containerregistry.json:6199 in 213abee. [](commit_id = 213abeecdad2c7c947d6ad8e577235ddf6f27466, deletion_comment = False)

    "metadataSearchEnabled": {

Same comment here. You can define the enum once elsewhere and use it in both places via reference

Refers to: specification/containerregistry/resource-manager/Microsoft.ContainerRegistry/preview/2023-11-01-preview/containerregistry.json:6617 in 213abee. [](commit_id = 213abeecdad2c7c947d6ad8e577235ddf6f27466, deletion_comment = False)

    }

ARM recommends enums over booleans for future proof APIs.
Standard guidance is: replace boolean/switch properties with a more meaningful enum whenever possible. A boolean will forever have two valid values (true or false). A string enum type is always preferred. Also, properties should always provide better values just than True and False. For example two switches "isTypeA" and "isTypeB" should be replaced with one enum "type": [A, B, DefaultType]. Enums are always a more flexible and future proof option because they allow additional values to be added in the future in a non-breaking way, e.g. [Enabled, Disabled, Suspended, Deallocated]. Note: do NOT define a 'boolean enum' with two values 'True and False'. This might be easier to 'extend' in terms of types, but semantically its cofusing, and no better than a boolean. Properties with type:object that don't reference a model definition are not allowed. ARM doesn't allow generic type definitions as this leads to bad customer experience.
Refers to: specification/containerregistry/resource-manager/Microsoft.ContainerRegistry/preview/2023-11-01-preview/containerregistry.json:6199 in 213abee. [](commit_id = 213abee, deletion_comment = False)

Thanks for the review. I've made the accommodated changes. Please let me know if there is anything else. Thanks!

/pr RequestMerge

/pr RequestMerge


danger_disable_hostname_verification doesn't seem to work

let mut https_connector = ::hyper_tls::HttpsConnector::new(4, &core.handle()).unwrap();
https_connector.danger_disable_hostname_verification(true);
let client = ::hyper::Client::configure()
    .connector(https_connector)
    .build(&core.handle());

my Cargo.toml contains
hyper = "0.11"
hyper-tls = "0.1.2"

If I try to use this client to connect to https://in-house-server-with-dubious-self-signed-cert/yadda/yadda, I get

Io(Error { repr: Custom(Custom { kind: Other, error: Error { repr: Os { code: -2146762487, message: "A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider." } } }) })

Just in case, I tried calling danger_disable_hostname_verification(false) but that didn't change anything.
Am I missing something? Is the intent of danger_disable_hostname_verification actually to ignore the certificate/trust process, or is it some other part of the SSL process that I'm misunderstanding?

That method is used to disable verifying that the certificate is issued for the domain requested. It still will validate that the certificate is trusted by a root authority.
You may wish to look at the reqwest crate, which makes it easier to add root certificates.

Thanks for the pointer, I'll check that out.


Specify process for naming future architectures

Not opposed but I am guessing you meant for this to be on the arch line, not the OS line?

Oops, fixed...
On 04/15/16 02:35, Brandon Philips wrote:

Not opposed but I am guessing you meant for this to be on the arch
line, not the OS line?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/opencontainers/image-spec/pull/29#issuecomment-210223454

I just realized that the manifest spec already specifies the use of the official list for $GOARCH. https://golang.org/doc/install/source#environment

You're correct. The problem is, the GOARCH values are not specific enough by themselves (e.g. for ARM, necessitating the additional GOARM variable), which is part of the reason we ended up avoiding using them directly in appc. I don't think we got it entirely right there, but I think there are some learnings we can take from it: e.g. armv7l vs armv7b, arm64 vs aarch64)
The manifest spec (which as you noted already uses the GOARCH values) attempts to take this into consideration with the additional variant property, but this just raises the point that there's a mismatch between here and there.

@jonboulle is the debian ports list specific enough? Pardon me, that I have no actual knowledge of the Arms race or whatever we're going to call this conundrum.

@timthelion well, from the brief exchange we had with ARM engineers (see below), I had thought that endianness was a requirement, but I also am not extremely well versed in this area

On ARM64 Linux, "uname -m" reports "aarch64". When you run a 32-bit on arm64, it still reports "aarch64" unless you run it with a 32-bit personality. So "linux32 uname -m" reports "armv8l" for backwards compatibility (independently of whether uname is a 32 or 64-bit application). On big-endian, Linux reports "aarch64_be" and "armv8b" respectively.

For the older architectures, using the output of ‘uname –m’ would also make sense and negate the armel/armhf/armv7l confusion. Doing this would mean ‘armv7l' would be the correct “tag” to use for little-endian, and ‘armv7b’ for big endian.

Perhaps this would be another possible source for standard names: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch

On Fri, Apr 15, 2016 at 05:56:00AM -0700, Timothy Hobbs wrote:

Perhaps this would be another possible source for standard names:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch

Not unless you want to dig down in there to find obscure architectures
like x86_64 1 ;).

Maybe just reference 1 here, and move this conversation over to that
repository?  I don't think we want separate versions of this
conversation in each repository, and that's the lower-level spec.

@jonboulle We studied this problem early on in OCI and during multi-arch considerations for the V2.2 manifest. Indeed, GOARCH is insufficient, but it is a good starting point.
The intent with these field sets was to define a class of machines on which the image would run. The real work here is defining the classes and aliasing in such way to make these fields useful in multi-arch scenarios. In some case, this may actually be better left to implementations, leaving OCIs role to define the common architectures which must be compatible (amd64, etc.), along with the data model and process for expressing the os/architecture (platform) for an image.
That said, we can say that these are seeded by GOARCH, but not limited by.
Ultimately, this is going to have to be a real community effort and that will be as successful as the process we adopt.

This seems like something that could utilize annotations of a prescribed namespace.
This PR seems LGTM, and we can iron out particular hardware details as needed.

@vbatts I would rather this spec be authoritative on archs and OS strings than not. This will be an important namespace for the spec to own.
If we reference the GOARCH / GOOS spec I think it should be more along the lines of "New architectures and OSes will be added over time and through discussion on the OCI Image Project. A source for inspiration for new entries is the Go docs etc etcd"

@philips While agree we need to call these out here, it is unlikely that anyone will ever "own" the platform namespace ;).
Deferring to GOOS/GOARCH as the inspiration, but not the canonical source, seems to be the right balance.

@stevvooe So, are you saying you are OK merging as-is? Or you want it to be more along the lines of "future entries should be inspired by". I am not seeing what you want.

@philips I agree with "future entries should be inspired by" and https://github.com/opencontainers/image-spec/pull/29#issuecomment-215577479. We're going to have to curate these and I don't see a way around it.
I'm not sure about merging this PR. We can merge this but then it will be rolled back later.

@stevvooe Ack.
@timthelion Sorry, but would you mind making the language be more like what @stevvooe suggests? If our indecisiveness is getting too annoying LMK and I can fixup the language and merge on your behalf.

Closing in favor of: https://github.com/opencontainers/image-spec/pull/60


Use proxy-agent for micromamba download

Motivation
Currently, setup-micromamba fails behind corporate proxies as node-fetch does not automatically read the HTTP_PROXY and HTTPS_PROXY environment variables. This PR introduces the proxy-agent package whose ProxyAgent takes over the necessary configuration.

@borchero can you rebase onto main s.t. the CI can run?


[Feature Request]: Environment variables for Apollo Vampire Accelerators.

Could you please add the following variables to your next release, Vampire Core Revision and Vampire Version?
Information is here ... http://www.apollo-core.com/index.htm?page=coding&tl=2
Vampire Core Revision is at address 0xDFF3EA & Vampire Version is at address 0xDFF3FC.
I am still learning how to code and this would help.
Thank you.

Could you please add the following variables to your next release, Vampire Core Revision and Vampire Version? Information is here ... http://www.apollo-core.com/index.htm?page=coding&tl=2 Vampire Core Revision is at address 0xDFF3EA & Vampire Version is at address 0xDFF3FC.
I am still learning how to code and this would help.
Thank you.

I have glanced over the documentation you provided. Are you sure that Core Revision is actually implemented? The documentation says "Reserved", which usually indicates that it is not. Also, there is no further information behind it (in contrast to the other items).
The Vampire Version (0xDFF3FC) I can do, but someone with an actual Vampire would have to test it.

Thursday, 10 October 2024
8:06:30 AM
Hi Larson,
Yes the core revision is actually implemented.
I have peeked at address 0xDFF3EA (Core Revision) using the APEEK command (http://aminet.net/util/shell/APeek.lha) and the value returned is the current flashed firmware .jic file for my Amiga Manticore accelerator.
The vampire core version is not necessary.
Some beta test cores give a 0 or -1 result in the Core Revision.
So ideally the VCoreRev environment variable would have the 16-bit core revision or “N/A” if 0 or -1.
I can test this quite easily.
Thank you,
Alexander.
From: larsonmars @.>
Sent: Thursday, 10 October 2024 7:51 AM
To: larsonmars/sysvars @.>
Cc: Alexander Gent @.>; Author @.>
Subject: Re: [larsonmars/sysvars] [Feature Request]: Environment variables for Apollo Vampire Accelerators. (Issue #6)
Could you please add the following variables to your next release, Vampire Core Revision and Vampire Version? Information is here ... http://www.apollo-core.com/index.htm?page=coding http://www.apollo-core.com/index.htm?page=coding&tl=2 &tl=2 Vampire Core Revision is at address 0xDFF3EA & Vampire Version is at address 0xDFF3FC.
I am still learning how to code and this would help.
Thank you.
I have glanced over the documentation you provided. Are you sure that Core Revision is actually implemented? The documentation says "Reserved", which usually indicates that it is not. Also, there is no further information behind it (in contrast to the other items).
The Vampire Version (0xDFF3FC) I can do, but someone with an actual Vampire would have to test it.
—
Reply to this email directly, view it on GitHub https://github.com/larsonmars/sysvars/issues/6#issuecomment-2403404616 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BDQILXUFHMDTIUFM5RXSQX3Z2WJKTAVCNFSM6AAAAABPP5YFZWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBTGQYDINRRGY .
You are receiving this because you authored the thread.  https://github.com/notifications/beacon/BDQILXSF5CPGIVHBR2MSMBDZ2WJKTA5CNFSM6AAAAABPP5YFZWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUPIEFUQ.gif Message ID: @.*** @.***> >

So ideally the VCoreRev environment variable would have the 16-bit core revision or “N/A” if 0 or -1.

Sounds reasonable.
Currently, my time is quite sparse, but I will see what I can do. Thank you for taking the risk to be the volunteer tester 😄

Can you please test the appended version?
sysvars.zip

Didn't work.
The variables VampireCoreRev, VampireType, VampireClockMult were not created.
To check if the core revision was in address $dff3ea I ran this utility.
Quick_080_chk.zip
Regards,
Alex.

Are you sure you have the last version?

Here are the my results...
SysVars_Results.zip
Regards,
Alex.

Can you please peek at 0xdff3fc (Version)
Is the core rev correct? If not, what is expected and can you also give the peek?

The core revision displayed should be 10370.
Regards,
Alex.

Ok, didnt know the numbers get this big. My print routine only handles 4 chars atm. Can you give the Vampire Version peek?

Here are the results...
[00] $00DFF3EA: $2882  10370
[00] $00DFF3FC: $070D   1805
Regards,
Alex.

Ok I adapted the number printing routine (which was originally only meant for version strings) and fixed the offset for the constants. Can you check?
sysvars.zip

Works beautifully.
Here are the results from the cli SET command...
BSDSockLib
BSDSockLibRev
BSDSockLibVer
Chipset           SAGA
CPU               68080
FPU               68882
KickRev           111
KickVer           47
process           2
RC                0
Result2           0
UAEMajor
UAEMinor
UAERev
VampireClockMult  13
VampireCoreRev    10370
VampireType       V4_600
VFreq             60
VIEWER            MULTIVIEW
Thank you very much...
Regards,
Alexander.


Is it possible to do multi streaming ?

Hello,
Let's say I have a folder with multiple mp4 that I'd like to broadcast.
Let's say there are 10 files.
1/ Can I use one instance of ffmpeg to nginx rtmp, set the playlist as the input and get several m3u8 files (one for every files (10)) ? Like, if the key is "live", get "live1.m3u8" for the 1rst, "live2.m3u8" for the 2nd... etc until the 10th ? If it is possible, please, how to proceed ?
Right now, I am using OBS but it takes a lot more of the RAM/CPU and I can only run one instance of it at the time, so only one output.
Thank you !

You can run multiple instance of obs. This will be burden for your cpu but
you can. I have done that before.
8 Nis 2018 Pzr 10:15 tarihinde MrFrieza<EMAIL_ADDRESS>şunu
yazdı:

Hello,
Could anyone tell me if it is possible to stream a whole playlist from
ffmpeg to nginx rtmp server, and get several outputs ? (one instance of
ffmpeg sending multiple inputs and getting in nginx rtmp multiple outputs ?)
Or do I have to do it with several ffmpeg instances ? (and put different
keystream) ?
Cause for now I am using OBS but only one instance is allowed .......
Thank you !
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/arut/nginx-rtmp-module/issues/1244, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AXEwRK8ZDeEftCjdnPJHk89a4KM8751Bks5tmbj9gaJpZM4TLa_w
.

Yep, but I am using OBS to stream a chrome tab with audio.
The thing is that I want to stream multiple chrome tab, with multiple audio.
OBS only allows to capture "incoming audio" so if I open many tabs with multiple audio tracks, it will be noisy ..


Oobabooga Extension for lm-format-enforcer

I used lm-format-enforcer with with llama-cpp-python example and adapted it a bit to get it to work, but I was limited to just what llama-cpp-python can handle. I have switched to using the Oobabooga text-generation-webui and its API. I cannot figure out how to use lm-format-enforcer with Oobabooga, though.
Is there some way to make lm-format-enforcer into an Extension for Oobabooga? That way it could be used both within the WebUI and also from the API. There is a JSON format extension here on Github, but it seems to have been abandoned months ago and when I just tried it, it did not limit output format to JSON at all: oobabooga-jsonformer-plugin

This looks possible. Their example extension applies a logits processor, which is what LMFE is.
I do not have the time to implement it right now. If you want to give it a shot I'll be happy to help. In the meantime, I'll leave this issue open, flag it as an enhancement, and use it for voting to see how popular this request is.
Feel free to spread it around to get people to vote for it :)


use ReentrantLock for by-path promotions and fix sync in getGroupsContaining

ReentrantLock is mapped to target StoreKey to lock the target and allow only one promotion on that target at a time
MemoryStoreDataManager synchronization was using StoreKey in one case and this in other cases when dealing with group-member reverse mapping. Fixed to always synchronize on MemoryStoreDataManager instance in order to keep monitor effects localized

retest this please


DeepSparseModelForCustomTasks

from transformers.onnx.utils import get_preprocessor
from optimum.deepsparse import DeepSparseModelForCustomTasks
from transformers import pipeline
model_id = "bert-base-cased"
onnx_model = DeepSparseModelForCustomTasks.from_pretrained(model_id, export=True)
tokenizer = get_preprocessor(model_id)
pipe = pipeline("feature-extraction", model=onnx_model, tokenizer=tokenizer)
text = "My Name is Philipp and i live in Germany."
outputs = pipe(text)

Looks good but please update the PR example and resolve conflicts

Resolved conflicts. What changes would like to see on the PR example?


⚠ Bofrost.be has degraded performance

In 047d6b1, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1932 ms

Resolved: Bofrost.be performance has improved in 74cf878.


(re-4883) Update to ezbake 0.3.10 for service account upgrades

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://jenkins.puppetlabs.com/job/platform_puppetdb_intn-sys_pr/1522/


Feature | "X" drop all selected items

Make all selected list box elements removed on "X" button

Done with attached dependency property "Selected Items"


BUG: Lua error when playing movies

The error has something to do with the way the SubtitlesFrame is being re-anchored. Something about a anchor family connection.

Seems to be resolved? I didn't change anything though ¯_(ツ)_/¯


Update administrative appeals tribunal decisions sub-categories values

Update administrative appeals tribunal decisions sub-categories values.
Fix numeric sorting errors.
Add a few new sub-categories.

@robmckinnon - this seems clear enough - but it needs a rebase before we can merge.


Automatically drain the body on close?

I just started using the module, and was a little surprised by the drain + close behavior.
I completely understand the need to always require res.Body.Close(), and that's pretty standard practice for Go. However, what seems a little unusual to me is that Close doesn't automatically do this for you.
I understand that there are some restrictions and that it's important to always read it, but it still looks a bit like a footgun. I want to remove this risk entirely from my codebase, since it's unlikely this be remembered at every call site (and frankly, it's too cumbersome). However, I do think it's more reasonable to expect defer resp.Body.Close() since that's consistent with every IO based library out there.
Two ways that first come to mind, would just need to make an io.Reader wrapper that can automatically drain itself on close:
type autodrainingReader struct {
	io.ReadCloser
}

func (a *autodrainingReader) Close() error {
	_ = io.Copy(ioutil.Discard, a.ReadCloser)
	return a.ReadCloser.Close()
}

// might not even be needed, looks like this is just for websockets
type autodrainingReadWriter struct {
	io.ReadWriteCloser
}

func (a *autodrainingReadWriter) Close() error {
	_ = io.Copy(ioutil.Discard, a.ReadWriteCloser)
	return a.ReadWriteCloser.Close()
}

func makeAutoDraining(r io.ReadCloser) io.ReadCloser {
	if rw, ok := r.(io.ReadWriteCloser); ok {
		return &autodrainingReadWriter{rw}
	}

	return &autodrainingReader{r}
}

I think the question then comes down to where this goes. Two options come to mind

Wrap the transport http.RoundTripper so that it returns requests with auto-draining bodies. I'll test this out client-side, at least that gives me a workaround
Wrap the requests themselves as they are performed (more error prone)

What seems strange to me is that I see nothing in net/http that suggests the same issue where bodies must be read before they are closed (wrong: corrected below). But yet looking in the go-elasticsearch code base, I don't see anything that would challenge that either, so I'm not sure what the disclaimer is referring to.
Update: discovered this https://github.com/golang/go/commit/ce8341554caa8be64aeafd9bf2077a98db462fda via https://github.com/elastic/go-elasticsearch/issues/123#issuecomment-573191784.

That's an interesting idea, thank you for raising this.
We could implement it as an option as this hugely depends on how you built your consumption of the body, if any.
Let me know how your experiments are going, there are several options and one of them being not in the client but as you pointed in the transport.


Import image as ReactComponent

Currently, I'm trying to use next-optimized-images with @svgr/webpack, but I noticed they don't play along nicely. With the latter I can import SVG images as react components and I'm wondering if this is feasible with NOI, so I can stop using one in favor of the other.
Is this possible?

The upcoming major version (v3, currently in the canary branch) provides an Svg component which I think is exactly what you want.
However, it uses a span tag as a wrapper currently but I plan to use svgr internally within this Svg component which would resolve that issue. The change to svgr should be in one of the next canary versions.
I can't say how long it will take until v3 is published on a non-canary tag, but if you want, you can already start using it using the @canary version. It should already be stable, just a few planned features need to be done.


Could we create a Twitter player card?

https://developer.twitter.com/en/docs/tweets/optimize-with-cards/overview/player-card

Probably not worth doing.


Error on Deno Deploy using serveFileWithTs

First of all, thank you so much for this great library! I've implemented it in my Simple Deno Website Boilerplate, but it's failing on Deno Deploy:
Uncaught SyntaxError: The requested module<EMAIL_ADDRESS>does not provide an export named 'ServeDirOptions'
    at https://deno.land/x/ts_serve@v1.1.0/mod.ts:1:20

It's quite weird because I can see the export does exist in std and has for a while, maybe it's a Deno Deploy runtime issue (I've asked in the Discord server and also created an issue at https://github.com/denoland/deploy_feedback/issues/252), but wanted to log here in case someone else sees it.

I was able to fix it for myself by implementing a simpler serving of the files but this will likely affect others trying to use this library.

Thanks for letting me know about this issue!
I have seen this error before and remember fixing it. Perhaps if I release a new version everything should be fine. wait a moment please.

This issue has been fixed in https://github.com/ayame113/ts-serve/pull/7/files#diff-e8c3622ccfbed67f4b3058cd4417bf41d066236f49c6f3e63b7c243b584c6734R1-R6 and I just released a new version (v1.2.0) with that fix so I think this can be closed.


[BP-1.13][FLINK-22462][tests] Fix JdbcExactlyOnceSinkE2eTest by pausing emission until checkpoint confirmed

Backport of #16328 to 1.13.

The last CI failure is caused by FLINK-22367.


Implement Saga

https://github.com/uber-go/cadence-client/issues/797

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Pull Request Test Coverage Report for Build 427

0 of 27 (0.0%)  changed or added relevant lines in 1 file are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 74.989%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

workflow/saga.go
0
27
0.0%

Files with Coverage Reduction
New Missed Lines
%

internal/internal_task_pollers.go
4
73.48%

Totals

Change from base Build 426:
-0.2%

Covered Lines:
8674

Relevant Lines:
11567

💛  - Coveralls

Hi @mingfang , since it's been a while I wonder do you have any plans to address the feedbacks and move forward on this PR?

Sorry I don't have time to make the necessary changes. Please feel free to take over this PR or close it.


Add formatting options (e.g. left pad)

The encoder should support common formatting needs such as left padding.
Proposed Spec
Formatting Options

default - No padding is applied to the value.
rightpad - The value is padded on the right to fill available space.
leftpad - The value is padded on the left to fill available space.

In all cases the value is written to the available space in a left-to-right manner. If the value length is greater than available space, the right most characters will be omitted.
Struct Tags
The struct tag schema will be updated to support and optional third option to specify formatting – fixed:"{startPos},{endPos},{format}".
Padding Characters

Types
Padding Character

int, int8, int16, int32, int64
0

uint, uint8, uint16, uint32, uint64
0

float32, float64
0

string, []byte
\u0020 (space)

Any type not listed will default to being padded with \u0020 (space).

It would also be advantageous to allow the user to define the character that will be used for padding.
This would change the struct tag schema to fixed:"{startPos},{endPos},{format},{padChar}". Both new parameters would be optional. If padChar is not defined it should fall back to the table defined above.
The only issue I see with this is defining space as the padding character. Using a literal space is har to read (e.g. fixed:"1,5,rightpad, "). I propose _ is used for space and __ is used for a literal underscore.
With this in place, it would be trivial to implement a right and left option that only control alignment. fixed:"1,5,right" would be an alias for fixed:"1,5,padleft,_". Similarly fixed:"1,5,left" would be an alias for  fixed:"1,5,padright,_"

It would also be advantageous to allow the user to define the character that will be used for padding.

tuning into this thread as well.

👍


There is a pixel-sized hole in the clown's pants and you can see through it.

The clown jumpsuit sprite is fine, by the way.

reverse dixel

Apparently might have something to do with how SS13 handles adjusting jumpsuits for female characters.

Somebody screwed up and removed femaleMask: UniformTop from the clown jumpsuit.


Moving Boilerplate example to correct section

You don't need to recreate a PR just make a new commit to the same branch


Add documentation to rabbitmq bindings for Priority Queue

Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:

[x] Read the contribution guide
[x] Commands include options for Linux, MacOS, and Windows within codetabs
[x] New file and folder names are globally unique
[x] Page references use shortcodes instead of markdown or URL links
[x] Images use HTML style and have alternative text
[x] Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:
Description
Documentation to RabbitMQ binging for Priority Queue

maxPriority param for queue
priority param for message

Issue reference

https://github.com/dapr/components-contrib/pull/606

Closes: #1101

@yaron2 Thank you for your support!

@yaron2 Thank you for your support!


2nd Party

Alice is Here! It’s the 2nd Party and everyone is invited 💃🥳.

What to expect

We'll be rebasing this branch into main once the CI for 2ndparty plugin support passes (see closed PR with ADR, only closed because of branch rewrite to remove images, will be reopened asap).

ETA: 2022-07-27

We'll then rewrite history again, splitting the plugins out into their respective 2ndparty maintenance locations (dffml or builtree org or possibly other option on stay with 2ndparty within intel org, we’ll see how it goes).

ETA: 2022-10-01

Finally, we’ll flip the switch to our web5 world, where git is only used as a proxy for commit data encoded via DIDs. We will then have herstory, from then on everything will be Alice. Alice will be the methodology by which we interpret those nodes, DIDs in the web5 case. Alice will also exist as the entity whose execution is based on the same methodology used for definition of the graph.

ETA: 2023-01-01
We need DWNs to be feature complete to be able to share our vuln and ML data using VCs for provenance.

https://developer.tbd.website/blog/what-is-web5

Code

Alice

https://github.com/intel/dffml/tree/alice/entities/alice

Documentation

Contributing

Writing Alice Overlays

Tutorials

Rolling Alice

ADRs

Manifest
Open Architecture
DID + HSM Supply Chain Security Mitigation Option

Tagged RFCs

RFCv1: https://github.com/intel/dffml/tree/291cfbe5153414932afe446aa4f6c2e298069914/docs/tutorials/rolling_alice

Began by exploring how we should write clean dataflow docs in https://github.com/intel/dffml/issues/1279
Converted to discussion in https://github.com/intel/dffml/discussions/1369
Issue converted to discussion converted to files within https://github.com/intel/dffml/blob/alice/docs/arch/alice/discussion/
Pulled out existing ADRs and tutorials in their current states into

Tutorials

Rolling Alice

ADRs

Manifest
Open Architecture
DID + HSM Supply Chain Security Mitigation Option

Cross linked tutorials with their usage examples within README within alice entity directory https://github.com/intel/dffml/tree/alice/entities/alice

TODO (extra todos: https://github.com/intel/dffml/pull/1401#issuecomment-1168023959)

[x] Clean up tutorial docs that currently exist

[x] Find home for them in tree

https://github.com/intel/dffml/blob/alice/docs/tutorials/rolling_alice/

[x] Tentative chapter name for Question and Answering model

Volume 1: Coach Alice: Chapter 5: Question Everything

https://github.com/programmer290399/IT-710-Project-Video-QnA-System

[ ] DataFlow.export should include $schema as should all .export() methods.

[ ] Later for operations the schema is the schema for the associated manifest.

[x] Split overlays into separate file locations

[x] Update Alice contributing docs with new paths instead of AliceGitRepo omport from alice.cli

[ ] Docs build with alice branch if working
[ ] Run auto formatter on every commit in alice branch
[ ] Cloud development environment options

Public

[ ] GitPod

https://gitpod.io/#github.com/intel/dffml/tree/alice
TODO

mv dffml/operations/python.py operations/innersource/dffml_operations_innersource/python_ast.py
Add Alice CONTRIBUTING setup to the .gitpod.yml auto start setup

code tutorial.ipynb when done

Self-Hosted

[ ] Coder

https://coder.com/docs/coder-oss/latest/install

[x] Alice contributing documentation

https://github.com/intel/dffml/blob/alice/entities/alice/CONTRIBUTING.rst

[x] How to extend recommended community standards command with overlays

Basic tutorial where we grab the name from a configuration file

[ ] Show me a security overlay.

Write section of our open source guide tutorial where we implement the SECURITY.md overlay
Latwr go back and write how we implemented the base flow and the initial set of overlays, and the readme overlay.

We can prototype the use of commit messages as docs and commit the whole file when we move it with docs for that overlay, rST in commit message. Later explore log of file to changelog in rST to sphinx docs.
Link up with herstory to ipynb creation and shell command saving. Auto generate commit messages (docs) based on herstorical shell commands ran (or if in vscode debug buttons or run buttons executed) with output. Diff system context herstory state with link in chain at last clean tree. Run timeline resolution if dirty tree for set of commits (multiple git add runs). First we automate writing the docs, then we automate reading.

[ ] How to write new commands
[ ] Non CLI interfaces

[ ] Commenting in issue while debugging, this is an overlay to herstory collection
[ ] Get tbDEX up and running for backing storage

[ ] Write an operation that inserts data into tbDEX format, either via API or flat file duplication of formatting via libraries like the Python peerdid library.

Alice enables granular identification and application of static or dynamic policy.

She does this through context aware overlays whose application process to upstream may be dynamic, even in part end user (attacker) flows, which can be executed or synthesized within an appropriate (optionally adaptive, we do dynamic and static and we understand time across so we can come re synthesize in your codebase on trigger) sandbox

Volume 1: Coach Alice: Preface
To time travel, an entity must first accelerate. The entity we now
turn our attention to we know well. Her name is Alice, and she's
falling down the rabbit hole as we speak. We begin our series
somewhere between the tick and the tock. As she nears the bottom
of the rabbit hole time slows for a moment, for her, as she enters
Wonderland. The pattern which is Alice is a spark in the mind.
We will coach Alice and she will coach us. From our point of view
if you can't teach it you don't know it. So it's time to each Alice
how to be an open source contributor, by teaching her how to teach
it. In this volume, volume 1, we will build Coach Alice, our open
source developer coach. It's developer boot camp for Alice this
volume as her boots make contact with the ground at the bottom of
the rabbit hole.

Misc notes

If we can teach Alice how to operate based on intent,
and how to have her intentions always be good. Where good is defined
the communities strategic principles and values.
and we validate the hell out
of here. We will step through the looking glass into a community of
the future where we can only trust ourselves. In that trust in
ourselves we will find trust in others, in measured, yet meaningful
ways.
Where we can work optimally in a hybrid workplace environment. Allowing us to
reconnect with the physical world. To embrace the world that exists.
What this means in reality is that Alice will be communicating for us,
we will begin to think of her as a messenger relaying a message
The most open, self reliant, confident humans. Ready to take on the world.

2022-07-18 Notes

TODO

[ ] @aliceoa, @pdxjohnny: Kick off OSS scans

Target collaboration with CRob on

[ ] @pdxjohnny: Finish Q3 plans (Gantt chart, meeting templates, etc.)

Generate template for auto creation to fill every meeting / fillable pre-meeting

Engineering Log Process

Alice, every day at 7 AM in Portland's timezone create a system context (the tick)

Merge with existing system context looked up from querying this thread if exists

In the future we will Alice will create and update system contexts.

We'll start with each day, then move to a week, then a fortnight, then 2 fortnights.
She'll parse the markdown document to rebuild the system context as if it's cached
right before it would be synthesized to markdown, we then run updates and trigger
update of the comment body. Eventually we won't use GitHub and just DID based stuff.
We'll treat these all as trains of thought / chains of system contexts / state of the
art fields.

Take a set of system contexts as training data
The system context which we visualize as a line dropped from the peak of a pyramid
where it falls through the base.
We use cross domain conceptual mapping to align system contexts in a similar direction
and drop ones which do are unhelpful, do not make the classification for "good"
What remains from our circular graph is a pyramid with the correct decisions
(per prioritizer)
This line represents

Inputs

date

Type: Union[str, Date]
Example: 2022-07-18
Default: did:oa:architype:current-date

Add yesterdays unfinished TODO items to this train of though with the
Create a document (docutils?)

Make the top level header date with "Notes" appended
Collect all previous days TODOs from within the individual entity comments within the thread for the days comment (the team summary for that day)

Drop any completed or struck through TODOs
Output a list item "TODO" with the underlying bullets with handle prepended and then the TODO contents

Create comments for individuals after this the current system context is posted and we have a live handle to it to reply with each individuals markdown document.

Synthesis the document to markdown (there is a python lib out there that can do docutils to md, can't remember the name right now)
Upsert comment in thread

https://medium.com/51nodes/decentralized-schema-registry-aa662b8db12b

Refactoring and Thinking About Locking of Repos for Contributions

Metadata

Date: 2022-07-27 13:00 UTC -7


Upgrade default version of helm to 3.1.1

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

New or Affected Resource(s)

variable "helm_version" {
description = "version of helm to install"
default     = "3.1.1"
type        = string
}
Potential Terraform Configuration

# Copy-paste any Terraform configurations for how the requested feature may be used.

References

Done in #139


Reaction on immunization

As a Doctor I need to have an opportunity to link the reaction on immunization with the corresponding immunization so that users can see (and consider) the reaction while studying the immunization
Implement logic according to spec

Add "reaction_on"  field to the observation in the composite, validate it is an active immunization that belongs to the current patient

On postprocessing composite: add the observation to the immunization as a reaction Spec

[ ] Make it Object, please

 "entry": "$.observations.[1].reaction_on",
                    "entry_type": "json_data_property",
                    "rules": [
                        {
                            "description": "type mismatch. Expected Array but got Object",