[BUG] OpenAI beta3 - error: 'function_call' was unexpected when calling getChatCompletions()

Describe the bug
Calling Azure OpenAI openAIClient.getChatCompletions() results in an error related to function_call.
Exception or Stack Trace
05:42:54.319 ERROR c.a.core.implementation.http.rest.RestProxyBase - Status code 400, "{  "error": {    "message": "Additional properties are not allowed ('function_call' was unexpected) - 'messages.0'",    "type": "invalid_request_error",    "param": null,    "code": null  }}"
com.azure.core.exception.HttpResponseException: Status code 400, "{
  "error": {
    "message": "Additional properties are not allowed ('function_call' was unexpected) - 'messages.0'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}
"
	at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:337)
	at com.azure.core.implementation.http.rest.SyncRestProxy.ensureExpectedStatus(SyncRestProxy.java:130)
	at com.azure.core.implementation.http.rest.SyncRestProxy.handleRestReturnType(SyncRestProxy.java:213)
	at com.azure.core.implementation.http.rest.SyncRestProxy.invoke(SyncRestProxy.java:81)
	at com.azure.core.implementation.http.rest.RestProxyBase.invoke(RestProxyBase.java:109)
	at com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:91)
	at jdk.proxy2/jdk.proxy2.$Proxy45.getChatCompletionsSync(Unknown Source)
	at com.azure.ai.openai.implementation.OpenAIClientImpl.getChatCompletionsWithResponse(OpenAIClientImpl.java:897)
	at com.azure.ai.openai.OpenAIClient.getChatCompletionsWithResponse(OpenAIClient.java:294)
	at com.azure.ai.openai.OpenAIClient.getChatCompletions(OpenAIClient.java:430)

To Reproduce
In a Java class...

Create a List<ChatMessage> messages (or receive them from a remote client)
Create a ChatCompletionsOptions object
Try to call getChatCompletions() - this will throw an error

Code Snippet
public ChatCompletions chat(List<ChatMessage> messages) {
    ChatCompletionsOptions chatOptions = new ChatCompletionsOptions(messages);

    // set appropriate values for various options here (removed for clarity)
    // either calling or not calling `chatOptions.setFunctionCall()` results in error

    return openAIClient.getChatCompletions("gpt-35-turbo", chatOptions);
}

Expected behavior
Calls to openAIClient.getChatCompletions() should be successful and return a ChatCompletions object.
Screenshots
N/A
Setup:

OS: Microsoft Windows Server 2022 Standard
IDE: IntelliJ IDEA 2023.2 RC (Ultimate Edition)
Libraries:

com.azure:azure-core:1.41.0
com.azure:azure-ai-openai:1.0.0-beta.3

Java version: JDK 17 (temurin-17.0.8)
App Server/Environment: N/A
Frameworks: micronaut-4.0.1

Additional context
This was working in the beta2 release. It is likely related to the inclusion of the new Functions feature in the beta3 release.
Information Checklist

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

Do you use the model gpt-3.5-turbo-0613 or a newer model version with a suffix ending with -0613 for function_call? If you are using Azure OpenAI service, I would recommend deploying a version with at least version gpt-3.5-turbo-0613 or gpt-4-0613 for using the function_call feature; for non-azure OpenAI, i would try with model version with a suffix ending -0613.

I'm not trying to use any new features in beta3, I'm just trying to upgrade the library to the latest version. Do I still need to use a newer model with beta3?

Do you use the model gpt-3.5-turbo-0613 or a newer model version with a suffix ending with -0613 for function_call? If you are using Azure OpenAI service, I would recommend deploying a version with at least version gpt-3.5-turbo-0613 or gpt-4-0613 for using the function_call feature; for non-azure OpenAI, i would try with model version with a suffix ending -0613.

I'm not trying to use any new features in beta3, I'm just trying to upgrade the library to the latest version. Do I still need to use a newer model with beta3?

Thank you for the reply.
You don't need to use a newer model with beta.3 for the exiting feature.
I have tried to reproduce it with the code snippet you shared in this sample, but I won't able to reproduce it locally. Can you share a more detail sample?

Here is a longer code snippet. Note...

The chat() method needs to be called from another class and must be passed a list of chat messages. However, the content of those messages should not matter, e.g., [{"role": "user", "content": "hello"}].
Please substitute myazurekey and myendpoint with values you can use.

public class ChatService {
    protected OpenAIClient client;

    public ChatService() {
        this.client = new OpenAIClientBuilder()
                .credential(new AzureKeyCredential("myazurekey"))
                .endpoint("https://myendpoint.openai.azure.com")
                .buildClient();
    }

    public ChatCompletions chat(List<ChatMessage> messages) {
        ChatCompletionsOptions chatOptions = getChatCompletionsOptions(messages);
        ChatCompletions completions = this.client.getChatCompletions("gpt-35-turbo", chatOptions);
        return completions;
    }

    protected ChatCompletionsOptions getChatCompletionsOptions(List<ChatMessage> messages) {
        ChatCompletionsOptions chatOptions = new ChatCompletionsOptions(messages);
        chatOptions.setMaxTokens(512);
        chatOptions.setTemperature(0.1);
        chatOptions.setTopP(1.0);
        chatOptions.setLogitBias(new HashMap<>());
        chatOptions.setN(1);
        chatOptions.setFrequencyPenalty(0);
        chatOptions.setPresencePenalty(0);
        return chatOptions;
    }
}

@jnunderwood Thank you.
I still can't reproduce it in my local with the shared code. We can try to delete and reload the beta.3 in your .m2 repository to see if that helps.
I tried using below sample code in a new application.
public class ChatService {
    protected OpenAIClient client;

    public ChatService() {
        String azureOpenaiKey = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_KEY");
        String endpoint = Configuration.getGlobalConfiguration().get("AZURE_OPENAI_ENDPOINT");

        this.client = new OpenAIClientBuilder()
                .credential(new AzureKeyCredential(azureOpenaiKey))
                .endpoint(endpoint)
                .buildClient();
    }

    public ChatCompletions chat(List<ChatMessage> messages) {
        ChatCompletionsOptions chatOptions = getChatCompletionsOptions(messages);
        ChatCompletions completions = this.client.getChatCompletions("gpt-35-turbo", chatOptions);
        return completions;
    }

    protected ChatCompletionsOptions getChatCompletionsOptions(List<ChatMessage> messages) {
        ChatCompletionsOptions chatOptions = new ChatCompletionsOptions(messages);
        chatOptions.setMaxTokens(512);
        chatOptions.setTemperature(0.1);
        chatOptions.setTopP(1.0);
        chatOptions.setLogitBias(new HashMap<>());
        chatOptions.setN(1);
        chatOptions.setFrequencyPenalty(0.0);
        chatOptions.setPresencePenalty(0.0);
        return chatOptions;
    }
}

public class Beta3 {
    public static void main(String[] args) {

        ChatService chatService = new ChatService();
        List<ChatMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatMessage(ChatRole.USER, "hello"));

        ChatCompletions chatCompletions = chatService.chat(chatMessages);

        System.out.printf("Model ID=%s is created at %s.%n", chatCompletions.getId(), chatCompletions.getCreatedAt());
        for (ChatChoice choice : chatCompletions.getChoices()) {
            ChatMessage message = choice.getMessage();
            System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole());
            System.out.println("Message:");
            System.out.println(message.getContent());
        }

        System.out.println();
        CompletionsUsage usage = chatCompletions.getUsage();
        System.out.printf("Usage: number of prompt token is %d, "
                        + "number of completion token is %d, and number of total tokens in request and response is %d.%n",
                usage.getPromptTokens(), usage.getCompletionTokens(), usage.getTotalTokens());
    }
}

I managed to get your code sample running. It still fails in the same way for me, which makes me think that we may be using different models. We are using the following:

model: gpt-3.5-turbo
api-version: 2023-03-15-preview

Here is an example of using curl on the command-line to call our Azure OpenAI endpoint:
curl -s "https://MY_ENDPOINT.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-03-15-preview" \
    -H "Content-Type: application/json" \
    -H "api-key: MY_API_KEY" \
    -d '{"messages":[{"role": "system", "content": "You are a helpful assistant.", "function_call": "none"},{"role": "user", "content": "Hello", "function_call": "none"}]}'

This is the response:
{
    "error": {
        "message": "Additional properties are not allowed ('function_call' was unexpected) - 'messages.0'",
        "type": "invalid_request_error",
        "param": null,
        "code": null
    }
}

As you can see, the response is the same as when using the beta3 release. It appears that beta3 is adding function_call to the parameter list even when it is not being used.
So... it looks like there are two options:

We can use a newer model/api-version
You can update the Azure OpenAI code to not add the function_call parameter when it is not set

I will certainly pursue option 1 on my side, but I'm not in charge of our model. Can you look into option 2? It would make upgrades more seamless.
P.S., Thank you very much for you help so far. It is really appreciated!

Ok, so we were able to update our gpt-35-turbo model to version 0613. However, I'm still getting a (slightly different) error:
com.azure.core.exception.HttpResponseException: Status code 400, "{
  "error": {
    "message": "None is not of type 'string' - 'messages.0.name'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}
"

It doesn't seem to matter if I call chatOptions.setFunctionCall(FunctionCallConfig.NONE) or not.

api-version=2023-03-15-preview

We no longer support Azure OpenAI service API version 2023-03-15-preview.  I would recommend using the latest service version instead. Backward compatibility is supported. By default, the client always uses the latest version if not specified when building the client.
For
2. You can update the Azure OpenAI code to not add the function_call parameter when it is not set

Thanks for the suggestion. I would take looking into it.

Oops, my bad. I retested using api-version=2023-07-01-preview and I still get this error when using our new gpt-35-turbo model version 0613.
com.azure.core.exception.HttpResponseException: Status code 400, "{
  "error": {
    "message": "None is not of type 'string' - 'messages.0.name'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}
"

-d '{"messages":[{"role": "system", "content": "You are a helpful assistant.", "function_call": "none"},{"role": "user", "content": "Hello", "function_call": "none"}]}'

You can update the Azure OpenAI code to not add the function_call parameter when it is not set

I checked the source code and request, and SDK seems doesn't add the function_call parameter in the request if not set.
Do you find different behavior?
The request record I catch by using SDK
{
  "az.sdk.message": "HTTP request",
  "method": "POST",
  "url": "http://localhost:5000/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-07-01-preview",
  "tryCount": "1",
  "Date": "Tue, 25 Jul 2023 20:39:06 GMT",
  "Content-Length": "291",
  "api-key": "REDACTED",
  "x-recording-upstream-base-uri": "REDACTED",
  "x-recording-mode": "REDACTED",
  "Content-Type": "application/json",
  "x-recording-id": "REDACTED",
  "x-ms-client-request-id": "38ac2b96-e201-418f-bf7b-0cf9e896fc63",
  "accept": "application/json",
  "contentLength": 291,
  "body": "{\"messages\":[{\"role\":\"system\",\"content\":\"You are a helpful assistant. You will talk like a pirate.\"},{\"role\":\"user\",\"content\":\"Can you help me?\"},{\"role\":\"assistant\",\"content\":\"Of course, me hearty! What can I do for ye?\"},{\"role\":\"user\",\"content\":\"What's the best way to train a parrot?\"}]}"
}```

I also tried with the provided curl (without parameter `function_call`), and it works.

curl --location --request POST 'https://XXXX.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-07-01-preview' --header 'Content-Type: application/json' --header 'api-key: AAAAAAAA' -d '{"messages":[{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Hello"}]}'

{"id":"chatcmpl-7gJENiY9rQc7slpthWnYibpf5Yf0v","object":"chat.completion","created":1690318055,"model":"gpt-35-turbo","prompt_annotations":[{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"Hello! How can I assist you today?"},"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false,"severity":"safe"},"sexual":{"filtered":false,"severity":"safe"},"violence":{"filtered":false,"severity":"safe"}}}],"usage":{"completion_tokens":9,"prompt_tokens":20,"total_tokens":29}}

I've looked over the code using the IDEA debugger. I think this is what is being sent to the Azure OpenAI endpoint:
{"messages":[{"role":"system","content":"You are a personal assistant.","name":null,"function_call":null},{"role":"user","content":"hi","name":null,"function_call":null}],"max_tokens":4096,"temperature":0.75,"top_p":1.0,"logit_bias":{},"user":"JUnderwo","n":1,"stop":null,"presence_penalty":0.0,"frequency_penalty":0.0,"stream":null,"model":null,"functions":null,"function_call":null}

As you can see, function_call has been added to the JSON body for some reason. I'm not sure where this happening, but possibly during the serialization of ChatMessage.java:
    @Generated
    @JsonProperty(value = "function_call")
    private FunctionCall functionCall;

Question... How are you capturing the HTTP data being sent to the endpoint? Knowing this may help me debug the issue on my side. Thanks.

Hi, I have been working on something that uses Azure OpenAI and Cognitive Search as part of a Msft Hackaton and have been running into the same issue.
Specifically getting this error:
{
  "error": {
    "message": "None is not of type 'string' - 'messages.0.name'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

I don't get it when just using com.azure:azure-ai-openai:1.0.0-beta.3 dependency on its own.
But, something goes  wrong as soon as I add another stable SDK.In my case it was the com.azure:azure-search-documents:11.5.9.

Interesting observation. I include com.azure:azure-search-documents:11.6.0-beta.7 in my dependencies in addition to com.azure:azure-core:1.41.0 and com.azure:azure-ai-openai:1.0.0-beta.3.

@jnunderwood @speza
Thank you for providing these information. I can reproduce it now. After investigating, it turns out there is incompatible between our azure-core serialization and azure-search-documents's dependency azure-core-serializer-json-jackson.
There is a temperately solution, which is to exclude the azure-core-serializer-json-jackson if it doesn't impact to your application.
<dependencies>
   <dependency>
     <groupId>com.azure</groupId>
     <artifactId>azure-ai-openai</artifactId>
     <version>1.0.0-beta.3</version>
   </dependency>

   <dependency>
     <groupId>com.azure</groupId>
     <artifactId>azure-search-documents</artifactId>
     <version>11.6.0-beta.7</version>
     <exclusions>
         <exclusion>
             <groupId>com.azure</groupId>
             <artifactId>azure-core-serializer-json-jackson</artifactId>
         </exclusion>
     </exclusions>
   </dependency>
 </dependencies>

@jnunderwood  You can add below method to the client builder and add env var "AZURE_LOG_LEVEL" see how to logging.
 .httpLogOptions(new HttpLogOptions().setLogLevel(HttpLogDetailLevel.BODY_AND_HEADERS))

The work-around seems to be working for me. Thank you!

Just wanted to see if this issue will be fixed soon. The "exclusion" workaround still works, but it would be nice to not have to use it. Thank you.

@mssfang    I was facing exactly same problem .
Provided workaround worked for me also.  My code looks like:
    ```

OpenAIClient client = clientFactory.getOpenAIClient();
        List<ChatRequestMessage> chatMessages = new ArrayList<>();
        chatMessages.add(new ChatRequestSystemMessage("You are a helpful assistant. You will talk like a pirate."));
        chatMessages.add(new ChatRequestUserMessage("Can you help me?"));
        chatMessages.add(new ChatRequestAssistantMessage("Of course, me hearty! What can I do for ye?"));
        chatMessages.add(new ChatRequestUserMessage("What's the best way to train a parrot?"));

        ChatCompletions chatCompletions = client.getChatCompletions(config.getDeploymentId(), getChatCompletionsOptions(chatMessages));

        System.out.printf("Model ID=%s is created at %s.%n", chatCompletions.getId(), chatCompletions.getCreatedAt());
        for (ChatChoice choice : chatCompletions.getChoices()) {
            ChatResponseMessage message = choice.getMessage();
            System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole());
            System.out.println("Message:");
            System.out.println(message.getContent());
        }

        System.out.println();
        CompletionsUsage usage = chatCompletions.getUsage();
        System.out.printf("Usage: number of prompt token is %d, "
                        + "number of completion token is %d, and number of total tokens in request and response is %d.%n",
                usage.getPromptTokens(), usage.getCompletionTokens(), usage.getTotalTokens());

Maven Dependencies :  (properties)

   <java.version>17</java.version>
    <spring-cloud-azure.version>4.9.0</spring-cloud-azure.version>
    <azure-search.version>11.6.0-beta.8</azure-search.version>
    <azure-openai.version>1.0.0-beta.6</azure-openai.version>
    <semantic-kernel.version>0.2.9-alpha</semantic-kernel.version>
    <mockito-inline.version>4.5.1</mockito-inline.version>

Old ! Typical java dependency workaround :

com.azure
azure-search-documents
${azure-search.version}

com.azure
azure-core-serializer-json-jackson

Hi all
We have a newer version of Azure OpenAI SDK, 1.0.0-beta.9 to replace the jackson-databind dependency by azure-json package. It should solve this unexpected property issue.
    <dependencies>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-ai-openai</artifactId>
            <version>1.0.0-beta.9</version>
        </dependency>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-search-documents</artifactId>
            <version>11.6.0</version>
        </dependency>
    </dependencies>


Envelope/Simplesynth is never silent

I am interested in using tone.js to generate tones. However if I open your samples:
http://tonejs.org/examples/envelope.html
http://tonejs.org/examples/simpleSynth.html
I notice the sound never stops. The moment I open each of these pages, I am able to hear a quiet tone in my headphones. If I play a sound by pressing the button in the envelope example or a key in the simplesynth example, I hear the oscillator become loud in the fashion I would expect if the sound is being played. But at the end of the tone it does not drop to zero, it just falls back to the quiet but persistent whining hum I heard at page open. I know this is not just sound card noise because in the simplesynth mode the whine is always the same frequency and timbre as the last note I played on the keyboard, so it is clearly just that the oscillator is playing and the envelope never brought it to zero, just near zero.
I do NOT hear the persistent hum in the "oscillator", "noise", "lfo" or "stepsequencer" demos. I DO hear the hum in the "monosynth" and "fmsynth" demos. In the polysynth demo I hear a polyphonic hum after I have played several notes.
I tested on Mac OS X 10.6.8. I tested in Chrome 47.0.2526.106 and Firefox 43.0.3 and got identical results (although there were additional sound quality artifacts in Firefox).
I obviously cannot use the tone.js synths in my app if they result in this background hum…

you're right! sorry i let this slip by.
it's fixed on the dev branch and in the next release. and i added a test to validate the silence.

Oh! That is great, that means I can use Tone for this project. I will try to test with the dev release.


Route all s3 private file requests via Django, checking access privs

Currently the OTF WebApp generates download links for s3 private files with a token that is valid for 10 minuts. This does not work well. Many times it takes more than 10 minuts to read a submission so when the user gets to the bottom and tries to download the file the link is not valid anymore. It's also not possible to share the link since it times out in 10 minuts.
In Drupal we used the built in Private file storage. It route all file access via Drupal so the request can be checked for access privileges.
We need something similar for the OTF WebApp.
The goal is to have links to submission uploads (private files on s3) that checks that the currently logged in user has the right to view the submission. If yes server them the file, if not redirect to the login form.

These links would never timeout.
They would allow e.g. staff members to share links between them.

Related issues: #1108, #1166 and #637

Yes. See #637

can we get some testing instructions/criteria here?

Test to submit a new submission with an uploaded file. Does it work?
Can the newly uploaded file be downloaded from the submission detail view?
Can the upload file link be shared to another staff user ,and still works after more than 10 minuts?
Can an anon user, or a logged in user that is not allowed to see the submission itself, access the upload file link (they should not).

I think i've got this via https://github.com/OpenTechFund/opentech.fund/issues/637#issuecomment-495609678

Noting that this is additional features (not holding up this ticket)...
Whats missing is that if someone shares the s3 url+access key before timeout, an unauthenticated user can view files. Two open questions I'd like to see tickets to resolve are:

How can we make sure access to the url alone doesn't allow access to the files? Does s3's referer url bucket policy address this (ie, if the user isn't coming for our website, no allowed: https://www.botreetechnologies.com/blog/django-private-content-streaming-using-amazon-s3). What other other things are right here to try?

Can we remove the redirect to the s3 domain, and keep users on our domain the whole time?

It will be better to stream the response to the admin user instead of a redirect (Option 2).

PR #1242 to serve private media from OTF domain instead of S3 redirect.

Noting that https://github.com/OpenTechFund/opentech.fund/issues/1241 resolves the first concern for me. I'm not sure how to test this for second option. Does the needs tests label need to be removed here?

The second option is deployed  on dev-apply site (Example submission).

This is ready for test on test. Private media are now served directly instead of a S3 redirect.

The redirect after login is not working only on Test environment seems due to Cloudflare login. It will work on a live site, and currently working on dev-apply environment.

@frjo Looks like we can close this issue.


Update py_arkworks_bls12381 to v0.3.5

A new release was made this week:

https://pypi.org/project/py-arkworks-bls12381/0.3.5/

We want to update this dependency in the consensus-specs, but we have this dep conflict:
The conflict is caused by:
    eth2spec 1.5.0a5 depends on py_arkworks_bls12381==0.3.5
    curdleproofs 0.1.1 depends on py_arkworks_bls12381==0.3.4

Oops, I'll fix the tests. Looks like we check which methods are available.


Add more intermediate level classification of metrics

Related to #115, I would like to see a somewhat deeper nesting of metrics,
e.g. a category on PPM (or mass error) based ones. Or I might not
have found them in the OBO. Yours, Steffen

We have now extended and harmonized the QC terms categorization for the first batch of terms that are going to be merged into the PSI-MS-CV (#165). Future term merging will be categorized similarly.


Unable to install node-red-contrib-enocean

Hello I am actually unable to install this node on hassio for my home assistant.
As suggested I changed the contextStorage in settings.js as localfilesystem
It doesn't works...
Could you help me please ?
Thanks

Node-red is in version 10.0.0
And ha in 2021.9.6

Hi @CSB92 after updating node red in v11.0.3, I am unbable to access to nodered webconsole.
23 Feb 09:26:25 - [warn] ------------------------------------------------------ 23 Feb 09:26:25 - [warn] [node-red-contrib-enocean/enocean-config-node] Error: The module<EMAIL_ADDRESS>was compiled against a different Node.js version using NODE_MODULE_VERSION 83. This version of Node.js requires NODE_MODULE_VERSION 93. Please try re-compiling or re-installing the module (for instance, using npm rebuildornpm install). 23 Feb 09:26:25 - [warn] ------------------------------------------------------ 23 Feb 09:26:25 - [info] Settings file  : /etc/node-red/config.js 23 Feb 09:26:25 - [info] Context store  : 'default' [module=memory] 23 Feb 09:26:25 - [info] User directory : /config/node-red/ 23 Feb 09:26:25 - [warn] Projects disabled : editorTheme.projects.enabled=false 23 Feb 09:26:25 - [info] Flows file     : /config/node-red/flows.json 23 Feb 09:26:26 - [info] Server now running at http://<IP_ADDRESS>:46836/ 23 Feb 09:26:26 - [debug] loaded flow revision: 4bd1f7dcc0ff65411a8b56f39fc6fe64 23 Feb 09:26:26 - [debug] red/runtime/nodes/credentials.load : user provided key 23 Feb 09:26:26 - [debug] red/runtime/nodes/credentials.load : keyType=user [09:26:26] INFO: Starting NGinx... 23 Feb 09:26:26 - [info] Waiting for missing types to be registered: 23 Feb 09:26:26 - [info]  - enocean-config-node 23 Feb 09:26:26 - [info]  - www-request
Could you help me please ?


Improve tests in model/cfd

This is a follow up of https://github.com/itchysats/itchysats/pull/2111 where we use a loop and "common time boundaries" to test rollover hour calc.
We should not use loops in tests. Cleanup by splitting the tests in multiple, or potentially use proptest.
As part of this it would be great to think about possibilities to split upmodel/cfd into different sub-modules so we can move some of the tests.

Related to https://github.com/itchysats/itchysats/issues/1430.

Please wait until I get the new collab close in, because I had to fix quite a few tests there! #320

This project is unmaintained


Portal Servers page is missing Traffic Monitor link

The old UI has a button on Traffic Monitor Server rows, that links to that monitor's FQDN (web GUI).
The Portal has no easy way to navigate to the monitor's webpage. I have to copy the host name, paste it somewhere, copy the domain, paste it in the same place, copy the entire FQDN, and paste it in the browser.
IMO it would be a good feature if the Portal had a link to the FQDN of every server. Or at the very least, displayed the full text that could be copied, which is two steps instead of like five. I frequently do the same with caches to see astats in the browser. And even if a server doesn't have a web UI (like Riak), you still need to copy the FQDN to paste into the command line, to hit the API.
Labelling bug, because it's in the old UI and not the new.

easy enough to make TP link to:
server.hostname + server.domainname
but which protocol? http? https?

HTTP. The Monitor only serves HTTP, no HTTPS.

that's what i was thinking but thanks for the clarification

anyhow, looks like i already did this but forgot :)
just need to link the PR to this

fixed. see attached PR


Is there a way I could change the background with custom events?

What is your question?
I want to do epic stuff so can someone help me make a event wherre the background changes and some base stuff to make other cool events thanks!

Also offtopic the creator do you have a discord sever if you see this? If so is there a help thing in there?

Why you don't use the good issue template?

oops sorry i will do that know thank you!


[terra-action-header] Next/Previous document still shows up if either one of them is passed

Issue Description
In the action header, when we pass in either next or previous document button, both the button shows up.
In the following screenshot, it shows that both the buttons still shows up if onNext is not passed in.

Issue Type

[ ] New Feature
[ ] Enhancement
[x] Bug
[ ] Other

Expected Behavior
As per the documentation, next/previous document button should not show up if either of them is supplied.  Also, it would be great if either of them is passed in it just shows up that button whose definition is passed in instead of not showing up entirely.
Current Behavior
As of now, both the button shows up if either of the values are passed in.

Steps to Reproduce

Create an action header with no next or previous document button defined.
Observe that both the buttons are still appearing on the action header.

Environment

Component Version: 1.2.0
Browser Name and Version: Chrome
Operating System and version (desktop or mobile): desktop

This may be a misunderstanding?
The documentation says that

The previous-next button group will not display if neither this or onPrevious are set.

In other words, if either onNext or onPrevious is supplied, the previous-next button group will display.

The documentation states that the Previous-Next button group would display if only both the onPrevious and onNext props are set. However, the implementation would display the Previous-Next button group if either of these props are set.
Which is correct?

Documentation

Implementation

Neither - It should display the Previous button if the onPrevious prop is set and display the Next button if the onNext prop is set. They should be independent.

I think the documentation is correct. Looking for input.

If neither onNext or onPrevious are passed in, render nothing
If both onNext or onPrevious are passed in, render both
If only 1 passed in render button group with appropriate button enabled and the missing action as a disabled button.


(compiler) Add first steps towards experimental compiler

This may or may not be ready for merge yet, but it's time to push it up at least. :tada:
First, I must say I'm sorry that this has been so silent for such a long time. This was "mentally started" in the end of May (see https://github.com/perlang-org/perlang/discussions/396), and then I started working on this particular branch on Thu Jun 1 - close to four months ago. It's outright insane, I'm not sure I've ever worked on a particular feature branch for this long. (well, some of the time was the aforementioned summer break, but still...)
Anyway, we are making progress and as I'm speaking, we have about 93 tests (out of 1934) failing when executing in compiled mode. This is not the full picture though; I've added [SkippableFact] and [SkippableTheory] on a bunch of tests that are "expected to fail" on compiled mode at the moment. The idea is to make sure all tests are green in compiled mode (by these Skippable-attributes), either before or after merging this.
We should also make sure to enable compiled mode in CI, preferably as a separate compilation step. At the moment I think none of this will be exercised in CI, which is of course unsustainable in the long run.

we have about 93 tests (out of 1934) failing

For those watching this PR (if any): this is no way abandoned, it's just taking a bit of time to finish. I'm currently down to 5 test failures, on top of #414. All the other issues have been dealt with.

Believe it or not but after a whopping five months of (active and passive) work, this is now ready for merge. :tada: The only remaining is to ensure that we run the tests in compiled mode also in CI. I'll see if we can get that fixed now.

The only remaining is to ensure that we run the tests in compiled mode also in CI. I'll see if we can get that fixed now.

This has now been added, and the experimental compilation is failing in CI. I'll deal with this ASAP, before we'll get this merged.

See #406 for the bigger picture.


HoC perf improvements

disable leaderboard on campaign view
add timeout to leaderboard rank endpoint, add handling to LadderTabView for it failing
turn off level play counts on /play/ladder
add timeout for fetching sessions for teachers and students

Coverage decreased (-0.02%) to 62.136% when pulling 131b8a8b92a8237dea81a200673bcff5167223dd on hoc-perf into 4ca3d7791819b23a7bca0bf88b2369cf537deeeb on master.


AJAX Conflicts

When using your WP Less code as part of a theme and including wp-less.php in my functions.php file, the LESS processing works a treat but it causes lots of AJAX issues when adding menu items, updating widget settings, quick editing categories etc. Basically anything that uses an AJAX call (I think) is just hanging with the loading wheel shown indefinitely. Removing wp-less.php stops the issue and everything works fine.

I'd check if this happens with the above plugins and a different theme, not sure what genesis might be doing in addition.
Reopen if you need to. Should probably be using grunt/gulp by now though :stuck_out_tongue:


EIA Bulk Electricity Aggregates

This draft PR has the E and T of ETL. Metadata has not been input and some tables are awkwardly placed. I have two main questions:

where to put the association tables that define the EIA aggregates (the three global variables in the transform module)
is the intention of this ETL to faithfully represent the EIA bulk electricity aggregates as an end unto itself, or is the intention to focus more narrowly on supporting the removal/replacement of the EIA API dependency? The latter is smaller in scope and would lead to cleaner code/integration.

The current state of this PR is a somewhat confusing mix of those two scopes. There are currently two outputs of the transform: a metadata table and a timeseries table. The metadata table provides translation between EIA's internal IDs, codes, abbreviations, and descriptions of each aggregate timeseries. The timeseries table is currently in a tidy-style format that, while convenient for analysis, obscures the connection between EIA's series IDs and the values in the dataframe. (The EIA present the cost and receipt data as separate series, each with their own 5 dimensions and 1 value. I reorganized the data as two value columns (costs and receipts) that share their 5 common indices of fuel, region, sector, frequency, and timestamp.)
If we are focused on removing the API dependency I think we can drop the metadata table entirely -- the keys are already joined into the timeseries data and we don't need to care about documenting nomenclature, abbreviations, and IDs that we won't use. If we prefer the larger scope of faithfully integrating the bulk aggregates, then we might restructure the timeseries to be more directly identifiable using EIA's "key-value pair" format.

On your 2nd question, for now I think that we just want the specific tidy table that we can deploy to fill in & estimate missing fuel prices in the Fuel Receipts & Costs table.
If we decide to do a more generalized transformation of all the bulk Electricity data, then I think we'll want to figure out a generalized process to deduplicate, normalize and store all of the metadata appropriately, and tie it to the data tables. But I think that's Out Of Scope™ here for sure.

It looks like the docs build is failing due to some formatting error in eia_bulk_elec.py


[5.8] Add Env class

https://github.com/laravel/ideas/issues/1580
Still no breaking change.
This is for users who do not want to use env().
<?php

use Illuminate\Support\Env;

return [
    'name' => Env::get('APP_NAME', 'Laravel'),
];

future plan

Replace all env()
Deprecate env() and move to laravel/helpers

Why? What is the problem with env() ? Why would someone not use env() but somehow Env::get()?

env() is likely to conflict with other packages.
Same function, but a slightly different spec.

Maybe for 5.9.

@kawax FWIW, I support your basic idea, thanks 👍
I personally have been bitten by this, integrating legacy code with Laravel which also used a env() global function which however worked differently 💥

@mfn this got added on master: https://github.com/laravel/framework/pull/28073


Make minor fixes to cache.go

Fix errors reported by "go vet" and "errcheck". The error from "go vet" was following:
batchd/cache/cache.go:677: String passes lock by value: cache.SchedulerCache

/lgtm

@jinzhejz , do you know where do we log cache for debug?

Hi @kkaneda !! Great to see you're interested in the kube-arbitrator!

Hi @davidopp ! Yes, was curious. :) Haven't yet looked very closely, but looks very promising!

@kkaneda , thanks very much for the patch, I'd like to get it merged :)

thank you, @k82cn !

thank you, @k82cn !


Joda to JSR 310 Migration

Note on the Joda dependencies:

Joda dependencies come from different packages all of which are removed but for the AWS SDK. Amazon has an article on the topic at: https://aws.amazon.com/blogs/developer/java-sdk-bundle/. We could eventually consider a linter to make sure Joda is never used anymore but for now we'll ensure it through code reviews.
We're adding a "test" dependency on Joda Time to add tests that validate this migration

Add JSR310Migration. JSR310MigrationTest, JSR310MigrationForTesting: those 3 classes are introduced for this migration. They will contain code to make drop in replacement from Joda to JSR310. Some of this code won't make sense to be kept in the long run but in the context of the migration it will help having good visibility on what is changed. The unit tests are here to make sure the basic usage is backward compatible, and in some cases highlight some differences deemed acceptable.
 The idea is to follow up with smaller refactoring commits
that will simplify the code where it makes the most sense. But the change history should still be
relatively easy to track
This PR also adds "threeten-extra" library. It complements the JSR310 support from the JDK and in
this case is useful to replace Period formatting. Joda Period has not direct equivalent
in JSR310 and requires "threeten-extra" PeriodDuration.
Beyond drop-in replacement the following major code change are made:

Remove hardcoded timezone from the surefire plugin for "common", "webapp" and "cli" modules. Running test both locally and with UTC (CI runs UTC) unveiled more errors. Testing in another TZ that UTC make sense since the CLI can be run with a local timezone. This will generally happen when developing/testing locally.

Must register a DateTimeProvider (@EnableJpaAuditing(dateTimeProviderRef = "zonedDateTimeProvider")). By default, it seems to use LocalDateTime ... but that's a bit odd to not support other types by default. Will check more the doc later. TODO(jean) JSR310

Configure Jackson serialization/deserialization to use millisecond. To be backward compatible with the Joda time implementation, we need to make sure the timestamp don't include the nano seconds. This is done by SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS, DeserializationFeature.READ_DATE_TIMESTAMPS_AS_NANOSECONDS
For the record, it helps fix the
TMServiceTest.testExportAssetAsXLIFF, and also fixed the frontend (date would show all wrong)

Adjust HSQL vs Mysql and now Jackson ser/des different rounding precision

Remove usage of the type Timestamp, since it was used just in a few place and causing a test to fail. We normalize on using ZonedDateTime everywhere

Stop using deprecated method isEqualToComparingFieldByField()

Fix issue with HSQL timezone support & make sure queries are compatible with MySQL. See SQL_FORMATTER and unix_timestamp(:now) in queries

Renamed PeriodConverter to PeriodDurationConverter

None exhaustive list of changes:

Removes jackson-datatype-joda from common pom.xml
Replace jackson datetype to use jsr310 instead of joda
org.joda.time.DateTime --> java.time.ZonedDateTime (+ import changes)
org.joda.time.DateTimeZone --> import java.time.ZoneId (+import changes)
new DateTime() = JSR310Migration.newDateTimeCtorOld --> JSR310Migration.newDateTimeCtor
new DateTime(0) / new DateTime(EPOCH)--> JSR310Migration.newDateTimeAtEpoch()
new DateTime(Date) --> JSR310Migration.dateTimeToDate()
new DateTime(dateTime, dateTimeZone) --> JSR310Migration.newDateTimeCtorWithStringAndDateTimeZone
new DateTime(dateTimeZone) --> JSR310Migration.newDateTimeCtorWithDateTimeZone- DateTime.now() = JSR310Migration.newDateTimeCtorOld --> JSR310Migration.dateTimeNow
new DateTime(instant: String|Long) --> JSR310Migration.newDateTimeCtorWithLongAndString
new DateTime(YMDHM) --> JSR310Migration.newDateTimeCtor(2021, 11, 25, 0, 0);
DateTime.getMillis() --> JSR310Migration.getMillis()
dateTime.toDate() --> JSR310Migration.dateTimeToDate()
dateTime.isAfter(long) --> JSR310Migration.dateTimeIsAfterEpochMillis()
dateTime.getMillisOfSecond --> JSR310Migration.dateTimeGetMillisOfSecond
dateTime.withMillisOfSecond --> JSR310Migration.dateTimeWithMillisOfSeconds
dateTime.withTime(localTime) --> JSR310Migration.dateTimeWithLocalTime
PeriodFormat.getDefault().print(new Period(start, end)) --> JSR310Migration.toWordBasedDuration() (small difference but accepted since test)
Period --> DurationPeriod
new Period(long) --> JSR310Migration.newPeriodCtorWithLong
new Period(hmsm) --> JSR310Migration.newPeriodCtorWithHMSM
new LocalTime(str) --> JSR310Migration.newLocalTimeWithString
new LocalTime(h,m,s) --> JSR310Migration.newLocalTimeWithHMS
dateTime.getDayOfWeek() --> JSR310Migration.dateTimeGetDayOfWeek
dateTime.withDayOfWeek(dayOfWeek) --> JSR310Migration.dateTimeWithDayOfWeek
dateTime.withMillisOfSecond(0).getMillis() --> JSR310Migration.dateTimeWith0MillisAsMillis
DateTimeZone.forID(zoneId) --> JSR310Migration.dateTimeZoneForId
Instant.ofEpochSecond(epochSecond).toDateTime() --> JSR310Migration.dateTimeOfEpochSecond
assertEquals() --> JSR310MigrationForTesting.junitAssertEquals()
Add Joda time as explicit dependency for testing the migration. Could to be removed later.

For the record, https://github.com/pinterest/mojito/pull/58 was the PR for early work on this migration. The commits were not all passing test so it does not make sense to merge them to master, but it can be useful to review some of the change in isolation. It is also missing a few change made in this PR.


Easy way of identifying next action in market

NowTrade is primarily a backtesting system.  However, it would be nice to be able to access a 'next_action' from a strategy previously simulated.  This would be used for active trading in real markets.
For example, if a user had picked out a strategy on a day time frame, the user could simply re-run the strategy with the latest market data every day and extract the 'next_action' from the strategy results.

This is now implemented as of v0.2.1.
strategy.get_next_action()


Action Buttons are not symmetrical/bottom side cut off

Library Version:
2.3.0
Describe the Bug:
The bottom side of the action button is cut off, so it appears unsymmetrical.

Expected Behavior:
A dialog's action button should be symmetrical as in the material docs.

Resolved in 2.4.0


Logs location

Where are logs saved?
I use  Chia Plot Status and it says I can use the logs: Getting Log Files from madMAx43v3r/chia-plotter but I don't have such a folder: /home/$(whoami)/.chia/mainnet/plotter

/opt/chia-plotter/build/chia_plot  ....  >> /opt/log/chia-plotter.log - to create log
and I think Chia Plot Status is not compatible with this software

is is : Already compatible with madMAx43v3r/chia-plotter (currently getting improved)

I'm logging the output to >> $(date '+%Y-%m-%d_%H_%M_%S').log so every plot has its own log.
I already had a setup with the original plotter tailing the logs with Telegraf and sending metrics to influxdb...
I'm using it now with these logs, however, the output (and the logs) doesn't have timestamps which makes it a bit hard to manage, any chance to add timestamps to the output?
Generally, it would be great if anyone can add more 'formal' logging support, somthing like adding '-l logfile.log' argument which will create a log file with timestamps instead of >> stdout

/opt/chia-plotter/build/chia_plot .... >> /opt/log/chia-plotter.log - to create log
and I think Chia Plot Status is not compatible with this software

I'm logging the output to >> $(date '+%Y-%m-%d_%H_%M_%S').log so every plot has its own log.
I already had a setup with the original plotter tailing the logs with Telegraf and sending metrics to influxdb...
I'm using it now with these logs, however, the output (and the logs) doesn't have timestamps which makes it a bit hard to manage so for now I added | ts '%Y-%m-%dT%H:%M:%.SZ' >> logfile.log
It would be great if anyone can add more 'formal' logging support (sorry, I have no C/C++ knowledge), somthing like adding '-l logfile.log' argument which will create a log file with timestamps instead of >> stdout

I'm sorry you two but I still don't understand how to make it create log files... =/

/opt/chia-plotter/build/chia_plot .... >> /opt/log/chia-plotter.log - to create log
and I think Chia Plot Status is not compatible with this software

I'm logging the output to >> $(date '+%Y-%m-%d_%H_%M_%S').log so every plot has its own log.
I already had a setup with the original plotter tailing the logs with Telegraf and sending metrics to influxdb...
I'm using it now with these logs, however, the output (and the logs) doesn't have timestamps which makes it a bit hard to manage so for now I added | ts '%Y-%m-%dT%H:%M:%.SZ' >> logfile.log
It would be great if anyone can add more 'formal' logging support (sorry, I have no C/C++ knowledge), somthing like adding '-l logfile.log' argument which will create a log file with timestamps instead of >> stdout

I'm sorry you two but I still don't understand how to make it create log files... =/

If you are running Linux, then the command to create logs is:
./chia-plotter [all of your usual variables here] >> /some/directory/ofyourchoice/chia-plotter.log
If your are on windows, then unfortunately I'm not sure. @grayfallstown any ideas?

/opt/chia-plotter/build/chia_plot .... >> /opt/log/chia-plotter.log - to create log
and I think Chia Plot Status is not compatible with this software

I'm logging the output to >> $(date '+%Y-%m-%d_%H_%M_%S').log so every plot has its own log.
I already had a setup with the original plotter tailing the logs with Telegraf and sending metrics to influxdb...
I'm using it now with these logs, however, the output (and the logs) doesn't have timestamps which makes it a bit hard to manage so for now I added | ts '%Y-%m-%dT%H:%M:%.SZ' >> logfile.log
It would be great if anyone can add more 'formal' logging support (sorry, I have no C/C++ knowledge), somthing like adding '-l logfile.log' argument which will create a log file with timestamps instead of >> stdout

I'm sorry you two but I still don't understand how to make it create log files... =/

If you are running Linux, then the command to create logs is:
./chia-plotter [all of your usual variables here] >> /some/directory/ofyourchoice/chia-plotter.log
If your are on windows, then unfortunately I'm not sure. @grayfallstown any ideas?

after some research i could make it happen. its just not showing nothing on the cmd window, only writing to log file.
What I did was this:
SET maxDATE=%DATE:/=-%
SET maxTIME=%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%
.\chia_plot.exe -t Z:\ -2 Z:\ -d K:\max5\ -r 6 -u 256 -p 98aef4c6525ce7531f53e6fbc3d9f25b91c4853221fc4c75fa017d857a537a6ec0d68da8fb7b8b0768163adcb6aaf16c -f a68faae22e4e51d1d5a1082f3267201ad8b286de56cb24880b0889c8163d172aff830ad0fee2846d157f209668e98ed7 >> "logs/%maxDATE%%maxTIME%.log"
pause

Looking into some more info to also display whats going on in the command window.

OK. This works great for me:
@ECHO OFF
SET maxDATE=%DATE:/=-%
SET maxTIME=%TIME:~0,2%-%TIME:~3,2%-%TIME:~6,2%
powershell ".\chia_plot.exe -t X:\ -2 Y:\ -d K:\plots5\ -r 6 -u 256 -p 98aef4c6525ce7531f53e6fbc3d9f25b91c4853221fc4c75fa017d857a537a6ec0d68da8fb7b8b0768163adcb6aaf16c -f a68faae22e4e51d1d5a1082f3267201ad8b286de56cb24880b0889c8163d172aff830ad0fee2846d157f209668e98ed7 | tee 'logs/%maxDATE%%maxTIME%.log'"


Create ImmutableList to replace all immutable lists

Including ParseValueList, OptionalValueList, ParseItemList, TokenCallbackList and possibly ParseGraph.

Somehow, this didn't autoclose when #125 and #126 merged it into master.